Thurston Sexton
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Thurston Sexton has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Alden Dima
NIST
1 shared publication - Farhad Ameri
School of Manufacturing Systems and Networks, Arizona State University, Mesa, USA
1 shared publication - Guoxian Xiao
Manufacturing Systems Research Lab, General Motors R&D Center, Warren, MI 48090
1 shared publication - Jorge Arinez
General Motors Global Research and Development, Warren, USA
1 shared publication - Jun Ni
Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, 48109, USA
1 shared publication - Katherine C Morris
Systems Integration Division, National Institute of Standards and Technology 1 , 100 Bureau Dr., Gaithersburg, MD 20899
1 shared publication
Latest Publications
- Open Access
LabelVizier: Interactive Validation and Relabeling for Technical Text Annotations
With the rapid accumulation of text data produced by data-driven techniques, the task of extracting "data annotations"--concise, high-quality data summaries from unstructured raw text--has become increasingly important. The recent advances in weak supervision and crowd-sourcing techniques provide promising solutions to efficiently create annotations (labels) for large-scale technical text data. However, such annotations may fail in practice because of the change in annotation requirements, application scenarios, and modeling goals, where label validation and relabeling by domain experts are required. To approach this issue, we present LabelVizier, a human-in-the-loop workflow that incorporates domain knowledge and user-specific requirements to reveal actionable insights into annotation flaws, then produce better-quality labels for large-scale multi-label datasets. We implement our workflow as an interactive notebook to facilitate flexible error profiling, in-depth annotation validation for three error types, and efficient annotation relabeling on different data scales. We evaluated the efficiency and generalizability of our workflow with two use cases and four expert reviews. The results indicate that LabelVizier is applicable in various application scenarios and assist domain experts with different knowledge backgrounds to efficiently improve technical text annotation quality.
Open AccessLabelVizier: Interactive Validation and Relabeling for Technical Text Annotations
With the rapid accumulation of text data produced by data-driven techniques, the task of extracting "data annotations"--concise, high-quality data summaries from unstructured raw text--has become increasingly important. The recent advances in weak supervision and crowd-sourcing techniques provide promising solutions to efficiently create annotations (labels) for large-scale technical text data. However, such annotations may fail in practice because of the change in annotation requirements, application scenarios, and modeling goals, where label validation and relabeling by domain experts are required. To approach this issue, we present LabelVizier, a human-in-the-loop workflow that incorporates domain knowledge and user-specific requirements to reveal actionable insights into annotation flaws, then produce better-quality labels for large-scale multi-label datasets. We implement our workflow as an interactive notebook to facilitate flexible error profiling, in-depth annotation validation for three error types, and efficient annotation relabeling on different data scales. We evaluated the efficiency and generalizability of our workflow with two use cases and four expert reviews. The results indicate that LabelVizier is applicable in various application scenarios and assist domain experts with different knowledge backgrounds to efficiently improve technical text annotation quality.
- 13 January 2020
- Published by Wiley in Journal of Mechanical Design
- Vol. 142 (3)
Organizing Tagged Knowledge: Similarity Measures and Semantic Fluency in Structure Mining
Recovering a system’s underlying structure from its historical records (also called structure mining) is essential to making valid inferences about that system’s behavior. For example, making reliable predictions about system failures based on maintenance work order data requires determining how concepts described within the work order are related. Obtaining such structural information is challenging, requiring system understanding, synthesis, and representation design. This is often either too difficult or too time consuming to produce. Consequently, a common approach to quickly elicit tacit structural knowledge from experts is to gather uncontrolled keywords as record labels—i.e., “tags.” One can then map those tags to concepts within the structure and quantitatively infer relationships between them. Existing models of tag similarity tend to either depend on correlation strength (e.g., overall co-occurrence frequencies) or on conditional strength (e.g., tag sequence probabilities). A key difficulty in applying either model is understanding under what conditions one is better than the other for overall structure recovery. In this paper, we investigate the core assumptions and implications of these two classes of similarity measures on structure recovery tasks. Then, using lessons from this characterization, we borrow from recent psychology literature on semantic fluency tasks to construct a tag similarity measure that emulates how humans recall tags from memory. We show through empirical testing that this method combines strengths of both common modeling paradigms. We also demonstrate its potential as a preprocessor for structure mining tasks via a case study in semi-supervised learning on real excavator maintenance work orders.
- 13 January 2020
- Published by Wiley in Journal of Mechanical Design
- Vol. 142 (3)
Organizing Tagged Knowledge: Similarity Measures and Semantic Fluency in Structure Mining
Recovering a system’s underlying structure from its historical records (also called structure mining) is essential to making valid inferences about that system’s behavior. For example, making reliable predictions about system failures based on maintenance work order data requires determining how concepts described within the work order are related. Obtaining such structural information is challenging, requiring system understanding, synthesis, and representation design. This is often either too difficult or too time consuming to produce. Consequently, a common approach to quickly elicit tacit structural knowledge from experts is to gather uncontrolled keywords as record labels—i.e., “tags.” One can then map those tags to concepts within the structure and quantitatively infer relationships between them. Existing models of tag similarity tend to either depend on correlation strength (e.g., overall co-occurrence frequencies) or on conditional strength (e.g., tag sequence probabilities). A key difficulty in applying either model is understanding under what conditions one is better than the other for overall structure recovery. In this paper, we investigate the core assumptions and implications of these two classes of similarity measures on structure recovery tasks. Then, using lessons from this characterization, we borrow from recent psychology literature on semantic fluency tasks to construct a tag similarity measure that emulates how humans recall tags from memory. We show through empirical testing that this method combines strengths of both common modeling paradigms. We also demonstrate its potential as a preprocessor for structure mining tasks via a case study in semi-supervised learning on real excavator maintenance work orders.
- 22 July 2019
- Published by ASME International in Journal of Manufacturing Science and Engineering
- Vol. 141 (9)
Where Do We Start? Guidance for Technology Implementation in Maintenance Management for Manufacturing
Recent efforts in smart manufacturing (SM) have proven quite effective at elucidating system behavior using sensing systems, communications, and computational platforms, along with statistical methods to collect and analyze the real-time performance data. However, how do you effectively select where and when to implement these technology solutions within manufacturing operations? Furthermore, how do you account for the human-driven activities in manufacturing when inserting new technologies? Due to a reliance on human problem-solving skills, today’s maintenance operations are largely manual processes without wide-spread automation. The current state-of-the-art maintenance management systems and out-of-the-box solutions do not directly provide necessary synergy between human and technology, and many paradigms ultimately keep the human and digital knowledge systems separate. Decision makers are using one or the other on a case-by-case basis, causing both human and machine to cannibalize each other’s function, leaving both disadvantaged despite ultimately having common goals. A new paradigm can be achieved through a hybridized system approach—where human intelligence is effectively augmented with sensing technology and decision support tools, including analytics, diagnostics, or prognostic tools. While these tools promise more efficient, cost-effective maintenance decisions and improved system productivity, their use is hindered when it is unclear what core organizational or cultural problems they are being implemented to solve. To explicitly frame our discussion about implementation of new technologies in maintenance management around these problems, we adopt well-established error mitigation frameworks from human factors experts—who have promoted human–system integration for decades—to maintenance in manufacturing. Our resulting tiered mitigation strategy guides where and how to insert SM technologies into a human-dominated maintenance management process.
- 22 July 2019
- Published by ASME International in Journal of Manufacturing Science and Engineering
- Vol. 141 (9)
Where Do We Start? Guidance for Technology Implementation in Maintenance Management for Manufacturing
Recent efforts in smart manufacturing (SM) have proven quite effective at elucidating system behavior using sensing systems, communications, and computational platforms, along with statistical methods to collect and analyze the real-time performance data. However, how do you effectively select where and when to implement these technology solutions within manufacturing operations? Furthermore, how do you account for the human-driven activities in manufacturing when inserting new technologies? Due to a reliance on human problem-solving skills, today’s maintenance operations are largely manual processes without wide-spread automation. The current state-of-the-art maintenance management systems and out-of-the-box solutions do not directly provide necessary synergy between human and technology, and many paradigms ultimately keep the human and digital knowledge systems separate. Decision makers are using one or the other on a case-by-case basis, causing both human and machine to cannibalize each other’s function, leaving both disadvantaged despite ultimately having common goals. A new paradigm can be achieved through a hybridized system approach—where human intelligence is effectively augmented with sensing technology and decision support tools, including analytics, diagnostics, or prognostic tools. While these tools promise more efficient, cost-effective maintenance decisions and improved system productivity, their use is hindered when it is unclear what core organizational or cultural problems they are being implemented to solve. To explicitly frame our discussion about implementation of new technologies in maintenance management around these problems, we adopt well-established error mitigation frameworks from human factors experts—who have promoted human–system integration for decades—to maintenance in manufacturing. Our resulting tiered mitigation strategy guides where and how to insert SM technologies into a human-dominated maintenance management process.
- Open Access
- 30 August 2017
- Published by Wiley in Journal of Mechanical Design
- Vol. 139 (10)
Learning an Optimization Algorithm Through Human Design Iterations
Solving optimal design problems through crowdsourcing faces a dilemma: On the one hand, human beings have been shown to be more effective than algorithms at searching for good solutions of certain real-world problems with high-dimensional or discrete solution spaces; on the other hand, the cost of setting up crowdsourcing environments, the uncertainty in the crowd's domain-specific competence, and the lack of commitment of the crowd contribute to the lack of real-world application of design crowdsourcing. We are thus motivated to investigate a solution-searching mechanism where an optimization algorithm is tuned based on human demonstrations on solution searching, so that the search can be continued after human participants abandon the problem. To do so, we model the iterative search process as a Bayesian optimization (BO) algorithm and propose an inverse BO (IBO) algorithm to find the maximum likelihood estimators (MLEs) of the BO parameters based on human solutions. We show through a vehicle design and control problem that the search performance of BO can be improved by recovering its parameters based on an effective human search. Thus, IBO has the potential to improve the success rate of design crowdsourcing activities, by requiring only good search strategies instead of good solutions from the crowd.
- 30 August 2017
- Published by Wiley in Journal of Mechanical Design
- Vol. 139 (10)
Open AccessLearning an Optimization Algorithm Through Human Design Iterations
Solving optimal design problems through crowdsourcing faces a dilemma: On the one hand, human beings have been shown to be more effective than algorithms at searching for good solutions of certain real-world problems with high-dimensional or discrete solution spaces; on the other hand, the cost of setting up crowdsourcing environments, the uncertainty in the crowd's domain-specific competence, and the lack of commitment of the crowd contribute to the lack of real-world application of design crowdsourcing. We are thus motivated to investigate a solution-searching mechanism where an optimization algorithm is tuned based on human demonstrations on solution searching, so that the search can be continued after human participants abandon the problem. To do so, we model the iterative search process as a Bayesian optimization (BO) algorithm and propose an inverse BO (IBO) algorithm to find the maximum likelihood estimators (MLEs) of the BO parameters based on human solutions. We show through a vehicle design and control problem that the search performance of BO can be improved by recovering its parameters based on an effective human search. Thus, IBO has the potential to improve the success rate of design crowdsourcing activities, by requiring only good search strategies instead of good solutions from the crowd.