Firefly: A Low-Resolution Thermal Trigger for Gated Sensor Fusion in Human–Robot Collaborative Manufacturing
Open Access
Abstract
Automated guided vehicles and mobile robots are present on many Intelligent Manufacturing factory floors. Approximately 7.0 million U.S. workers also work on those same floors. During 2011 through 2017, forklifts caused 614 worker fatalities. Currently, no commercially available collision-avoidance sensor can manage the complexity associated with working within a plant. Therefore, fusing multiple modalities allows a system designer to gate the more expensive ones behind some minimal signal that indicates all systems should be active. Such minimal signals are expected to include body heat that even stationary personnel would have difficulty suppressing. In addition, the signal-capturing sensors ought to operate at milliwatt levels of power while keeping workers’ privacy. One type of device that has been found suitable for these requirements is a coarse thermopile array. Such a device can be fused with a rangefinder that determines distance to form a gated system. Yet the measurement capability frontier of the thermopile arrays in practical scenarios has not been explored. In addition, there is currently no figure of merit to quantify the trade-ff between induced errors and system-level power saving. In order to address these gaps, we built the fusion system, Firefly, together with a training-free detector and tested them against a human and a mannequin 1.2 to 7 m away in four thermally distinct environments, one of which is the floor of an operating factory. We found that the thermal array’s detection range varies with the environment but is predictable in a thermally uniform environment. Based on these experiments, we also proposed the Gated-Fusion Efficiency (GFE), a figure of merit that converts a trigger’s error rates into system-level power savings. Together, the capability boundary and the GFE help guide sensor-fusion system design in industrial manufacturing toward safer working environments.Keywords
Funding Information
- Sensor Energy Automation Laboratory
- Department of Electrical and Computer Engineering, University of Washington
This publication has 79 references indexed in Scilit:
- Design and deployment of a hybrid neural network for real-time GPS correction in slow moving, low SWaP, UAV flight systemsGPS Solutions, 2025
- Image-based anomaly detection in low-light industrial environments with feature enhancementResults in Engineering, 2025
- Cost-effective AIoT-based hazard detection using 2D camera in manufacturing environmentsThe International Journal of Advanced Manufacturing Technology, 2024
- An ethical framework for human-robot collaboration for the future people-centric manufacturing: A collaborative endeavour with European subject-matter experts in ethicsTechnology in Society, 2024
- A review of external sensors for human detection in a human robot collaborative environmentJournal of Intelligent Manufacturing, 2024
- LEGAN: A low-light image enhancement generative adversarial network for industrial internet of smart-camerasInternet of Things, 2023
- Performance analysis of entropy thresholding for successful image segmentationMultimedia Tools and Applications, 2022
- Towards human distance estimation using a thermal sensor arrayNeural Computing and Applications, 2021
- Protection of visual privacy in videos acquired with RGB cameras for active and assisted living applicationsMultimedia Tools and Applications, 2021
- Performance analysis of image thresholding: Otsu techniqueMeasurement, 2018