Poster Abstract: Tasking Heterogeneous Sensor Systems with LLMs | Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Despite the extensive use of sensors enabling intelligent applications, the complementary potential of co-existing sensor systems is often not fully utilized, limiting more advanced applications. This paper introduces a novel solution using Large Language Models (LLMs) to coordinate sensor systems for handling complex user queries. It defines a sensor language for sensor systems, including vocabulary set and grammar rules, analogous to natural language components, enabling LLMs to translate user intentions into sensor coordination plans. Preliminary results show that our approach significantly outperforms the existing solution at plan generation, execution and response generation stages.
Funding Information
  • National Natural Science Foundation of China (62202407)
  • Research Grants Council of Hong Kong (GRF 14207123, GRF 14214022, CRF C4034-21G)