Anil Kanduri
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Anil Kanduri has a total of 31 co-authors. Here are the top co-authors by number of shared publications:
- Amir M. Rahmani
University of California, Irvine, Irvine, California, USA
10 shared publications - Pasi Liljeberg
University of Turku
9 shared publications - Nikil Dutt
University of California, Irvine, Irvine, California, USA
8 shared publications - Antonio Miele
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
5 shared publications - Hamidreza Alikhani
University of California, Irvine, California, USA
4 shared publications Pasi Liljeberg
Department of Computing, University of Turku, Turku, Finland
4 shared publications
Latest Publications
- Open Access
Twill: Scheduling Compound AI Systems on Heterogeneous Mobile Edge Platforms
Compound AI (cAI) systems chain multiple AI models to solve complex problems. cAI systems are typically composed of deep neural networks (DNNs), transformers, and large language models (LLMs), exhibiting a high degree of computational diversity and dynamic workload variation. Deploying cAI services on mobile edge platforms poses a significant challenge in scheduling concurrent DNN-transformer inference tasks, which arrive dynamically in an unknown sequence. Existing mobile edge AI inference strategies manage multi-DNN or transformer-only workloads, relying on design-time profiling, and cannot handle concurrent inference of DNNs and transformers required by cAI systems. In this work, we address the challenge of scheduling cAI systems on heterogeneous mobile edge platforms. We present Twill, a run-time framework to handle concurrent inference requests of cAI workloads through task affinity-aware cluster mapping and migration, priority-aware task freezing/unfreezing, and DVFS, while minimizing inference latency within power budgets. We implement and deploy our Twill framework on the Nvidia Jetson Orin NX platform. We evaluate Twill against state-of-the-art edge AI inference techniques over contemporary DNNs and LLMs, reducing inference latency by 54% on average, while honoring power budgets.
Open AccessTwill: Scheduling Compound AI Systems on Heterogeneous Mobile Edge Platforms
Compound AI (cAI) systems chain multiple AI models to solve complex problems. cAI systems are typically composed of deep neural networks (DNNs), transformers, and large language models (LLMs), exhibiting a high degree of computational diversity and dynamic workload variation. Deploying cAI services on mobile edge platforms poses a significant challenge in scheduling concurrent DNN-transformer inference tasks, which arrive dynamically in an unknown sequence. Existing mobile edge AI inference strategies manage multi-DNN or transformer-only workloads, relying on design-time profiling, and cannot handle concurrent inference of DNNs and transformers required by cAI systems. In this work, we address the challenge of scheduling cAI systems on heterogeneous mobile edge platforms. We present Twill, a run-time framework to handle concurrent inference requests of cAI workloads through task affinity-aware cluster mapping and migration, priority-aware task freezing/unfreezing, and DVFS, while minimizing inference latency within power budgets. We implement and deploy our Twill framework on the Nvidia Jetson Orin NX platform. We evaluate Twill against state-of-the-art edge AI inference techniques over contemporary DNNs and LLMs, reducing inference latency by 54% on average, while honoring power budgets.
- Open Access
- 4 April 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 24 (3) , 1-30
Exploiting Approximation for Run-time Resource Management of Embedded HMPs
Run-time resource management (RTM) of multi-programmed workloads on heterogeneous multi-core platforms is challenging due to (i) fixed power budget of the device, (ii) variable performance requirements of the workloads, and (iii) unknown arrival of the applications. Existing RTM solutions lack power-performance coordination, resulting in performance degradation during power actuation or power violations during performance provisioning. Exploiting inherent error-resilience of the applications can address the performance loss incurred in power actuation, by combining run-time approximation with traditional power knobs (including Dynamic Voltage/Frequency Scaling, Task Migration, Degree of Parallelism, and CPU Quota ). In this work, we present an accuracy-aware resource management framework that jointly actuates run-time approximation and traditional power knobs for efficient power-performance management of multi-programmed and multi-threaded workloads running on heterogeneous mobile platforms. Our strategy configures the accuracy of the applications at run-time to exploit accuracy-performance trade-offs, by considering system-wide power-performance dynamics. We use heuristic estimation models to jointly enforce accuracy configuration and traditional power knobs settings at run-time. We evaluated our framework on real-world embedded mobile platforms, including Odroid XU3 and Asus Tinker Edge R boards to demonstrate the efficiency of our proposed approach across multiple workload scenarios. Our approach achieved 25% lower performance violations against the state-of-the-art run-time resource management policies at the cost of 2.2% accuracy loss across six applications.
- 4 April 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 24 (3) , 1-30
Open AccessExploiting Approximation for Run-time Resource Management of Embedded HMPs
Run-time resource management (RTM) of multi-programmed workloads on heterogeneous multi-core platforms is challenging due to (i) fixed power budget of the device, (ii) variable performance requirements of the workloads, and (iii) unknown arrival of the applications. Existing RTM solutions lack power-performance coordination, resulting in performance degradation during power actuation or power violations during performance provisioning. Exploiting inherent error-resilience of the applications can address the performance loss incurred in power actuation, by combining run-time approximation with traditional power knobs (including Dynamic Voltage/Frequency Scaling, Task Migration, Degree of Parallelism, and CPU Quota ). In this work, we present an accuracy-aware resource management framework that jointly actuates run-time approximation and traditional power knobs for efficient power-performance management of multi-programmed and multi-threaded workloads running on heterogeneous mobile platforms. Our strategy configures the accuracy of the applications at run-time to exploit accuracy-performance trade-offs, by considering system-wide power-performance dynamics. We use heuristic estimation models to jointly enforce accuracy configuration and traditional power knobs settings at run-time. We evaluated our framework on real-world embedded mobile platforms, including Odroid XU3 and Asus Tinker Edge R boards to demonstrate the efficiency of our proposed approach across multiple workload scenarios. Our approach achieved 25% lower performance violations against the state-of-the-art run-time resource management policies at the cost of 2.2% accuracy loss across six applications.
- Open Access
HiDP: Hierarchical DNN Partitioning for Distributed Inference on Heterogeneous Edge Platforms
Edge inference techniques partition and distribute Deep Neural Network (DNN) inference tasks among multiple edge nodes for low latency inference, without considering the core-level heterogeneity of edge nodes. Further, default DNN inference frameworks also do not fully utilize the resources of heterogeneous edge nodes, resulting in higher inference latency. In this work, we propose a hierarchical DNN partitioning strategy (HiDP) for distributed inference on heterogeneous edge nodes. Our strategy hierarchically partitions DNN workloads at both global and local levels by considering the core-level heterogeneity of edge nodes. We evaluated our proposed HiDP strategy against relevant distributed inference techniques over widely used DNN models on commercial edge devices. On average our strategy achieved 38% lower latency, 46% lower energy, and 56% higher throughput in comparison with other relevant approaches.
Open AccessHiDP: Hierarchical DNN Partitioning for Distributed Inference on Heterogeneous Edge Platforms
Edge inference techniques partition and distribute Deep Neural Network (DNN) inference tasks among multiple edge nodes for low latency inference, without considering the core-level heterogeneity of edge nodes. Further, default DNN inference frameworks also do not fully utilize the resources of heterogeneous edge nodes, resulting in higher inference latency. In this work, we propose a hierarchical DNN partitioning strategy (HiDP) for distributed inference on heterogeneous edge nodes. Our strategy hierarchically partitions DNN workloads at both global and local levels by considering the core-level heterogeneity of edge nodes. We evaluated our proposed HiDP strategy against relevant distributed inference techniques over widely used DNN models on commercial edge devices. On average our strategy achieved 38% lower latency, 46% lower energy, and 56% higher throughput in comparison with other relevant approaches.
- 27 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Design & Test
- Vol. 42 (2) , 25-34
ISCA: Intelligent Sense-Compute Adaptive Co-Optimization of Multimodal Machine Learning Kernels for Resilient mHealth Services on Wearables
mHealth services use multi-modal machine learning (MMML) models to process physiological and contextual data for automated decision making. Run-time input data perturbations degrade the prediction accuracy of MMML models, while continuous sensing, transmission, and processing of such noisy data drains the energy resources of wearable devices. Identifying qualitative input data and dropping non-insightful modalities can improve prediction accuracy and energy efficiency simultaneously. We propose a ISCA: a sense-compute adaptive co-optimization framework that employs reinforcement learning to jointly determine sensing and compute configuration settings which minimizes energy consumption while providing accuracy guarantees. Our approach considers run-time noise levels to selectively sense specific modalities, followed by selecting MMML models that are suitable for the chosen modality combination. We demonstrate the effectiveness of our solution using an exemplar mHealth application of pain assessment over various noise levels. Our solution achieves up to 23% improvement in prediction accuracy compared to Noise-agnostic method, and 42% energy savings in comparison with state-of-the-art selective sensing frameworks.
- 27 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Design & Test
- Vol. 42 (2) , 25-34
ISCA: Intelligent Sense-Compute Adaptive Co-Optimization of Multimodal Machine Learning Kernels for Resilient mHealth Services on Wearables
mHealth services use multi-modal machine learning (MMML) models to process physiological and contextual data for automated decision making. Run-time input data perturbations degrade the prediction accuracy of MMML models, while continuous sensing, transmission, and processing of such noisy data drains the energy resources of wearable devices. Identifying qualitative input data and dropping non-insightful modalities can improve prediction accuracy and energy efficiency simultaneously. We propose a ISCA: a sense-compute adaptive co-optimization framework that employs reinforcement learning to jointly determine sensing and compute configuration settings which minimizes energy consumption while providing accuracy guarantees. Our approach considers run-time noise levels to selectively sense specific modalities, followed by selecting MMML models that are suitable for the chosen modality combination. We demonstrate the effectiveness of our solution using an exemplar mHealth application of pain assessment over various noise levels. Our solution achieves up to 23% improvement in prediction accuracy compared to Noise-agnostic method, and 42% energy savings in comparison with state-of-the-art selective sensing frameworks.
- Open Access
- 5 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
EA^2: Energy Efficient Adaptive Active Learning for Smart Wearables
Mobile Health (mHealth) applications rely on supervised Machine Learning (ML) algorithms, requiring end-user-labeled data for the training phase. The gold standard for obtaining such labeled data is by sending queries to users and gathering responses for the corresponding label, which was conventionally done through triggering questions sent at random. Active Learning (AL) methods use intelligent query-sending policies by incorporating users' contextual information to maximize the response rate and informativeness of the collected labeled data. However, wearable devices' substantial battery drainage associated with the sensing of physiological signals underscores the need for developing an efficient sensing policy in addition to a query-sending policy. In this work, we present a co-optimization framework for both sensing and querying strategies within wearable devices, leveraging contextual information and ML model's prediction confidence. We designed a Reinforcement Learning (RL) agent to quantify different contextual parameters combined with model confidence to determine sensing and querying decisions. Our evaluation of an exemplar stress monitoring application showed a 76% reduction in sensing and data transmission energy consumption, with only a 6% drop in user-labeled data.
- 5 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessEA^2: Energy Efficient Adaptive Active Learning for Smart Wearables
Mobile Health (mHealth) applications rely on supervised Machine Learning (ML) algorithms, requiring end-user-labeled data for the training phase. The gold standard for obtaining such labeled data is by sending queries to users and gathering responses for the corresponding label, which was conventionally done through triggering questions sent at random. Active Learning (AL) methods use intelligent query-sending policies by incorporating users' contextual information to maximize the response rate and informativeness of the collected labeled data. However, wearable devices' substantial battery drainage associated with the sensing of physiological signals underscores the need for developing an efficient sensing policy in addition to a query-sending policy. In this work, we present a co-optimization framework for both sensing and querying strategies within wearable devices, leveraging contextual information and ML model's prediction confidence. We designed a Reinforcement Learning (RL) agent to quantify different contextual parameters combined with model confidence to determine sensing and querying decisions. Our evaluation of an exemplar stress monitoring application showed a 76% reduction in sensing and data transmission energy consumption, with only a 6% drop in user-labeled data.
- Open Access
Characterizing Accuracy Trade-offs of EEG Applications on Embedded HMPs
Electroencephalography (EEG) recordings are analyzed using battery-powered wearable devices to monitor brain activities and neurological disorders. These applications require long and continuous processing to generate feasible results. However, wearable devices are constrained with limited energy and computation resources, owing to their small sizes for practical use cases. Embedded heterogeneous multi-core platforms (HMPs) can provide better performance within limited energy budgets for EEG applications. Error resilience of the EEG application pipeline can be exploited further to maximize the performance and energy gains with HMPs. However, disciplined tuning of approximation on embedded HMPs requires a thorough exploration of the accuracy-performance-power trade-off space. In this work, we characterize the error resilience of three EEG applications, including Epileptic Seizure Detection, Sleep Stage Classification, and Stress Detection on the real-world embedded HMP test-bed of the Odroid XU3 platform. We present a combinatorial evaluation of power-performance-accuracy trade-offs of EEG applications at different approximation, power, and performance levels to provide insights into the disciplined tuning of approximation in EEG applications on embedded platforms.
Open AccessCharacterizing Accuracy Trade-offs of EEG Applications on Embedded HMPs
Electroencephalography (EEG) recordings are analyzed using battery-powered wearable devices to monitor brain activities and neurological disorders. These applications require long and continuous processing to generate feasible results. However, wearable devices are constrained with limited energy and computation resources, owing to their small sizes for practical use cases. Embedded heterogeneous multi-core platforms (HMPs) can provide better performance within limited energy budgets for EEG applications. Error resilience of the EEG application pipeline can be exploited further to maximize the performance and energy gains with HMPs. However, disciplined tuning of approximation on embedded HMPs requires a thorough exploration of the accuracy-performance-power trade-off space. In this work, we characterize the error resilience of three EEG applications, including Epileptic Seizure Detection, Sleep Stage Classification, and Stress Detection on the real-world embedded HMP test-bed of the Odroid XU3 platform. We present a combinatorial evaluation of power-performance-accuracy trade-offs of EEG applications at different approximation, power, and performance levels to provide insights into the disciplined tuning of approximation in EEG applications on embedded platforms.
- Open Access
Adaptive Workload Distribution for Accuracy-aware DNN Inference on Collaborative Edge Platforms
DNN inference can be accelerated by distributing the workload among a cluster of collaborative edge nodes. Heterogeneity among edge devices and accuracy-performance trade-offs of DNN models present a complex exploration space while catering to the inference performance requirements. In this work, we propose adaptive workload distribution for DNN inference, jointly considering node-level heterogeneity of edge devices, and application-specific accuracy and performance requirements. Our proposed approach combinatorially optimizes heterogeneity-aware workload partitioning and dynamic accuracy configuration of DNN models to ensure performance and accuracy guarantees. We tested our approach on an edge cluster of Odroid XU4, Raspberry Pi4, and Jetson Nano boards and achieved an average gain of 41.52% in performance and 5.2% in output accuracy as compared to state-of-the-art workload distribution strategies.
Open AccessAdaptive Workload Distribution for Accuracy-aware DNN Inference on Collaborative Edge Platforms
DNN inference can be accelerated by distributing the workload among a cluster of collaborative edge nodes. Heterogeneity among edge devices and accuracy-performance trade-offs of DNN models present a complex exploration space while catering to the inference performance requirements. In this work, we propose adaptive workload distribution for DNN inference, jointly considering node-level heterogeneity of edge devices, and application-specific accuracy and performance requirements. Our proposed approach combinatorially optimizes heterogeneity-aware workload partitioning and dynamic accuracy configuration of DNN models to ensure performance and accuracy guarantees. We tested our approach on an edge cluster of Odroid XU4, Raspberry Pi4, and Jetson Nano boards and achieved an average gain of 41.52% in performance and 5.2% in output accuracy as compared to state-of-the-art workload distribution strategies.
- 7 October 2023
- Published by Springer Nature
- p. 95-125
Edge-Centric Optimization of Multi-modal ML-Driven eHealth Applications
- 7 October 2023
- Published by Springer Nature
- p. 95-125
Edge-Centric Optimization of Multi-modal ML-Driven eHealth Applications
- Open Access
- 25 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Embedded Systems Letters
- Vol. 15 (4) , 222-225
DynaFuse: Dynamic Fusion for Resource Efficient Multimodal Machine Learning Inference
Multi-modal machine learning (MMML) applications combine results from different modalities in the inference phase to improve prediction accuracy. Existing MMML fusion strategies use static modality weight assignment, based on the intrinsic value of sensor modalities determined during the training phase. However, input data perturbations in practical scenarios affect the intrinsic value of modalities in the inference phase, lowering prediction accuracy, and draining computational and energy resources. In this work, we present DynaFuse, a framework for dynamic and adaptive fusion of MMML inference to set modality weights, considering run-time parameters of input data quality and sensor energy budgets. We determine the insightfulness of modalities by combining design-time intrinsic value with the run-time extrinsic value of different modalities to assign updated modality weights, catering to both accuracy requirements and energy conservation demands. The DynaFuse approach achieves up to 22% gain in prediction accuracy and an average energy savings of 34% on exemplary MMML applications of human activity recognition and stress monitoring in comparison with state-of-the-art static fusion approaches.
- 25 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Embedded Systems Letters
- Vol. 15 (4) , 222-225
Open AccessDynaFuse: Dynamic Fusion for Resource Efficient Multimodal Machine Learning Inference
Multi-modal machine learning (MMML) applications combine results from different modalities in the inference phase to improve prediction accuracy. Existing MMML fusion strategies use static modality weight assignment, based on the intrinsic value of sensor modalities determined during the training phase. However, input data perturbations in practical scenarios affect the intrinsic value of modalities in the inference phase, lowering prediction accuracy, and draining computational and energy resources. In this work, we present DynaFuse, a framework for dynamic and adaptive fusion of MMML inference to set modality weights, considering run-time parameters of input data quality and sensor energy budgets. We determine the insightfulness of modalities by combining design-time intrinsic value with the run-time extrinsic value of different modalities to assign updated modality weights, catering to both accuracy requirements and energy conservation demands. The DynaFuse approach achieves up to 22% gain in prediction accuracy and an average energy savings of 34% on exemplary MMML applications of human activity recognition and stress monitoring in comparison with state-of-the-art static fusion approaches.
- Open Access
- 6 April 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (4) , e29535
Digital Health–Enabled Community-Centered Care: Scalable Model to Empower Future Community Health Workers Using Human-in-the-Loop Artificial Intelligence
Digital health–enabled community-centered care (D-CCC) represents a pioneering vision for the future of community-centered care. D-CCC aims to support and amplify the digital footprint of community health workers through a novel artificial intelligence–enabled closed-loop digital health platform designed for, and with, community health workers. By focusing digitalization at the level of the community health worker, D-CCC enables more timely, supported, and individualized community health worker–delivered interventions. D-CCC has the potential to move community-centered care into an expanded, digitally interconnected, and collaborative community-centered health and social care ecosystem of the future, grounded within a robust and digitally empowered community health workforce.
- 6 April 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (4) , e29535
Open AccessDigital Health–Enabled Community-Centered Care: Scalable Model to Empower Future Community Health Workers Using Human-in-the-Loop Artificial Intelligence
Digital health–enabled community-centered care (D-CCC) represents a pioneering vision for the future of community-centered care. D-CCC aims to support and amplify the digital footprint of community health workers through a novel artificial intelligence–enabled closed-loop digital health platform designed for, and with, community health workers. By focusing digitalization at the level of the community health worker, D-CCC enables more timely, supported, and individualized community health worker–delivered interventions. D-CCC has the potential to move community-centered care into an expanded, digitally interconnected, and collaborative community-centered health and social care ecosystem of the future, grounded within a robust and digitally empowered community health workforce.