Michelle Potts Steves
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Michelle Potts Steves has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Brian A. Weiss
National Institute of Standards and Technology, Gaithersburg, Maryland, 20899, USA
1 shared publication - Craig Schlenoff
National Institute of Standards and Technology
1 shared publication - Emile L. Morse
100 Bureau Drive, Stop 8940, National Institute of Standards & Technology, Gaithersburg, Maryland 20899
1 shared publication - Gregory A. Sanders
National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, MD 20899, USA
1 shared publication - Jean Scholtz
Pacific Northwest National Laboratory, Richland, WA, USA
1 shared publication - Sherri Condon
The MITRE Corporation
1 shared publication
Latest Publications
- 7 May 2011
- Published by Elsevier in Computer Speech & Language
- Vol. 27 (2) , 528-553
Evaluation methodology and metrics employed to assess the TRANSTAC two-way, speech-to-speech translation systems
- 7 May 2011
- Published by Elsevier in Computer Speech & Language
- Vol. 27 (2) , 528-553
Evaluation methodology and metrics employed to assess the TRANSTAC two-way, speech-to-speech translation systems
- 1 December 2006
- Published by Oxford University Press (OUP) in Interacting with Computers
- Vol. 18 (6) , 1186-1214
Evaluation metrics and methodologies for user-centered evaluation of intelligent systems
In the past four years, we have worked with several research programs that were developing intelligent software for use by intelligence analysts. Our involvement in these programs was to develop the metrics and methodologies for assessing the impact on users; in this case, on intelligence analysts. In particular, we focused on metrics to evaluate how much the intelligent systems contribute to the users’ tasks and what the cost is to the user in terms of workload and process deviations. In this paper, we describe the approach used. We started with two types of preliminary investigations – first, collecting and analyzing data from analysts working in an instrumented environment for a period of 2 years, and second, developing and conducting formative evaluations of research software. The long-term studies informed our ideas about the processes that analysts use and provided potential metrics in an environment without intelligent software tools. The formative evaluations helped us to define sets of application-specific metrics. Finally, we conducted assessments during and after technology insertions. We describe the metrics and methodologies used in each of these activities, along with the lessons learned.
- 1 December 2006
- Published by Oxford University Press (OUP) in Interacting with Computers
- Vol. 18 (6) , 1186-1214
Evaluation metrics and methodologies for user-centered evaluation of intelligent systems
In the past four years, we have worked with several research programs that were developing intelligent software for use by intelligence analysts. Our involvement in these programs was to develop the metrics and methodologies for assessing the impact on users; in this case, on intelligence analysts. In particular, we focused on metrics to evaluate how much the intelligent systems contribute to the users’ tasks and what the cost is to the user in terms of workload and process deviations. In this paper, we describe the approach used. We started with two types of preliminary investigations – first, collecting and analyzing data from analysts working in an instrumented environment for a period of 2 years, and second, developing and conducting formative evaluations of research software. The long-term studies informed our ideas about the processes that analysts use and provided potential metrics in an environment without intelligent software tools. The formative evaluations helped us to define sets of application-specific metrics. Finally, we conducted assessments during and after technology insertions. We describe the metrics and methodologies used in each of these activities, along with the lessons learned.