Mehrnoosh Sameki
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Mehrnoosh Sameki has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Margrit Betke
Department of Computer Science, College of Arts and Sciences, Boston University, Boston, MA, 02215, USA
2 shared publications - Brian Price
Adobe Research
1 shared publication - Jianming Zhang
Adobe Research
1 shared publication - Kate K. Mays
Boston University
1 shared publication - Lei Guo
Fudan University, Shanghai, China
1 shared publication - Prakash Ishwar
Boston University
1 shared publication
Latest Publications
- Open Access
BUOCA: Budget-Optimized Crowd Worker Allocation
Due to concerns about human error in crowdsourcing, it is standard practice to collect labels for the same data point from multiple internet workers. We here show that the resulting budget can be used more effectively with a flexible worker assignment strategy that asks fewer workers to analyze easy-to-label data and more workers to analyze data that requires extra scrutiny. Our main contribution is to show how the allocations of the number of workers to a task can be computed optimally based on task features alone, without using worker profiles. Our target tasks are delineating cells in microscopy images and analyzing the sentiment toward the 2016 U.S. presidential candidates in tweets. We first propose an algorithm that computes budget-optimized crowd worker allocation (BUOCA). We next train a machine learning system (BUOCA-ML) that predicts an optimal number of crowd workers needed to maximize the accuracy of the labeling. We show that the computed allocation can yield large savings in the crowdsourcing budget (up to 49 percent points) while maintaining labeling accuracy. Finally, we envisage a human-machine system for performing budget-optimized data analysis at a scale beyond the feasibility of crowdsourcing.
Open AccessBUOCA: Budget-Optimized Crowd Worker Allocation
Due to concerns about human error in crowdsourcing, it is standard practice to collect labels for the same data point from multiple internet workers. We here show that the resulting budget can be used more effectively with a flexible worker assignment strategy that asks fewer workers to analyze easy-to-label data and more workers to analyze data that requires extra scrutiny. Our main contribution is to show how the allocations of the number of workers to a task can be computed optimally based on task features alone, without using worker profiles. Our target tasks are delineating cells in microscopy images and analyzing the sentiment toward the 2016 U.S. presidential candidates in tweets. We first propose an algorithm that computes budget-optimized crowd worker allocation (BUOCA). We next train a machine learning system (BUOCA-ML) that predicts an optimal number of crowd workers needed to maximize the accuracy of the labeling. We show that the computed allocation can yield large savings in the crowdsourcing budget (up to 49 percent points) while maintaining labeling accuracy. Finally, we envisage a human-machine system for performing budget-optimized data analysis at a scale beyond the feasibility of crowdsourcing.
- 12 April 2017
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 124 (2) , 169-186
Salient Object Subitizing
- 12 April 2017
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 124 (2) , 169-186
Salient Object Subitizing