Jyotibdha Acharya
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Jyotibdha Acharya has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Arindam Basu
City University of Hong Kong
3 shared publications - Andres Ussa
N.1 Institute for Health, National University of Singapore, Queenstown, Singapore
1 shared publication - Bharath Ramesh
National University of Singapore
1 shared publication Chang Song
Duke University, Cary, NC 27519
1 shared publication- Chockalingam Senthil Rajen
N.1 Institute for Health, National University of Singapore, Queenstown, Singapore
1 shared publication Deepak Singla
School of Electrical and Electronic Engineering, Nanyang Technological University, Jurong West, Singapore
1 shared publication
Latest Publications
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 2663-2672
TIRDet: Mono-Modality Thermal InfraRed Object Detection Based on Prior Thermal-To-Visible Translation
Cross-modality images that combine visible-infrared spectra can provide complementary information for object detection. In particular, they are well-suited for autonomous vehicle applications in dark environments with limited illumination. However, it is time-consuming to acquire a large number of pixel-aligned visible-thermal image pairs, and real-time alignment is challenging in practical driving systems. Furthermore, the quality of visible-spectrum images can be adversely affected by complex environmental conditions. In this paper, we propose a novel neural network called TIRDet, which only utilizes Thermal InfraRed (TIR) images for mono-modality object detection. To compensate for the lacked visible-band information, we adopt a prior Thermal-To-Visible (T2V) translation model to obtain the translated visible images and the latent T2V codes. In addition, we introduce a novel attention-based Cross-Modality Aggregation (CMA) module, which can augment the modality-translation awareness of TIRDet by preserving the T2V semantic information. Extensive experiments on FLIR and LLVIP datasets demonstrate that our TIRDet significantly outperforms all mono-modality detection methods based on thermal images, and it even surpasses most State-Of-The-Art (SOTA) multispectral methods using visible-thermal image pairs. Code is available at https://github.com/zeyuwang-zju/TIRDet
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 2663-2672
Open AccessTIRDet: Mono-Modality Thermal InfraRed Object Detection Based on Prior Thermal-To-Visible Translation
Cross-modality images that combine visible-infrared spectra can provide complementary information for object detection. In particular, they are well-suited for autonomous vehicle applications in dark environments with limited illumination. However, it is time-consuming to acquire a large number of pixel-aligned visible-thermal image pairs, and real-time alignment is challenging in practical driving systems. Furthermore, the quality of visible-spectrum images can be adversely affected by complex environmental conditions. In this paper, we propose a novel neural network called TIRDet, which only utilizes Thermal InfraRed (TIR) images for mono-modality object detection. To compensate for the lacked visible-band information, we adopt a prior Thermal-To-Visible (T2V) translation model to obtain the translated visible images and the latent T2V codes. In addition, we introduce a novel attention-based Cross-Modality Aggregation (CMA) module, which can augment the modality-translation awareness of TIRDet by preserving the T2V semantic information. Extensive experiments on FLIR and LLVIP datasets demonstrate that our TIRDet significantly outperforms all mono-modality detection methods based on thermal images, and it even surpasses most State-Of-The-Art (SOTA) multispectral methods using visible-thermal image pairs. Code is available at https://github.com/zeyuwang-zju/TIRDet
- 17 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (8) , 10726-10735
A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems
Deep learning inference that needs to largely take place on the “edge” is a highly computational and memory intensive workload, making it intractable for low-power, embedded platforms such as mobile nodes and remote security applications. To address this challenge, this article proposes a real-time, hybrid neuromorphic framework for object tracking and classification using event-based cameras that possess desirable properties such as low-power consumption (5–14 mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches of using event-by-event processing, this work uses a mixed frame and event approach to get energy savings with high performance. Using a frame-based region proposal method based on the density of foreground events, a hardware-friendly object tracking scheme is implemented using the apparent object velocity while tackling occlusion scenarios. The frame-based object track input is converted back to spikes for TrueNorth (TN) classification via the energy-efficient deep network (EEDN) pipeline. Using originally collected datasets, we train the TN model on the hardware track outputs, instead of using ground truth object locations as commonly done, and demonstrate the ability of our system to handle practical surveillance scenarios. As an alternative tracker paradigm, we also propose a continuous-time tracker with C $++$ implementation where each event is processed individually, which better exploits the low latency and asynchronous nature of neuromorphic vision sensors. Subsequently, we extensively compare the proposed methodologies to state-of-the-art event-based and frame-based methods for object tracking and classification, and demonstrate the use case of our neuromorphic approach for real-time and embedded applications without sacrificing performance. Finally, we also showcase the efficacy of the proposed neuromorphic system to a standard RGB camera setup when simultaneously evaluated over several hours of traffic recordings.
- 17 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (8) , 10726-10735
A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems
Deep learning inference that needs to largely take place on the “edge” is a highly computational and memory intensive workload, making it intractable for low-power, embedded platforms such as mobile nodes and remote security applications. To address this challenge, this article proposes a real-time, hybrid neuromorphic framework for object tracking and classification using event-based cameras that possess desirable properties such as low-power consumption (5–14 mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches of using event-by-event processing, this work uses a mixed frame and event approach to get energy savings with high performance. Using a frame-based region proposal method based on the density of foreground events, a hardware-friendly object tracking scheme is implemented using the apparent object velocity while tackling occlusion scenarios. The frame-based object track input is converted back to spikes for TrueNorth (TN) classification via the energy-efficient deep network (EEDN) pipeline. Using originally collected datasets, we train the TN model on the hardware track outputs, instead of using ground truth object locations as commonly done, and demonstrate the ability of our system to handle practical surveillance scenarios. As an alternative tracker paradigm, we also propose a continuous-time tracker with C $++$ implementation where each event is processed individually, which better exploits the low latency and asynchronous nature of neuromorphic vision sensors. Subsequently, we extensively compare the proposed methodologies to state-of-the-art event-based and frame-based methods for object tracking and classification, and demonstrate the use case of our neuromorphic approach for real-time and embedded applications without sacrificing performance. Finally, we also showcase the efficacy of the proposed neuromorphic system to a standard RGB camera setup when simultaneously evaluated over several hours of traffic recordings.
- 1 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 68 (2) , 617-630
A 0.11–0.38 pJ/cycle Differential Ring Oscillator in 65 nm CMOS for Robust Neurocomputing
This paper presents a low-area and low-power consumption CMOS differential current controlled oscillator (CCO) for neuromorphic applications. The oscillation frequency is improved over the conventional one by reducing the number of MOS transistors thus lowering the load capacitor in each stage. The analysis shows that for the same power consumption, the oscillation frequency can be increased about 11% compared with the conventional one without degrading the phase noise. Alternatively, the power consumption can be reduced 15% at the same frequency. The prototype structures are fabricated in a standard 65 nm CMOS technology and measurements demonstrate that the proposed CCO operates from 0.7-1.2 V supply with maximum frequencies of 80 MHz and energy/cycle ranging from 0.11-0.38 pJ over the tuning range. Further, system level simulations show that the nonlinearity in current-frequency conversion by the CCO does not affect its use as a neuron in a Deep Neural Network if accounted for during training.
- 1 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 68 (2) , 617-630
A 0.11–0.38 pJ/cycle Differential Ring Oscillator in 65 nm CMOS for Robust Neurocomputing
This paper presents a low-area and low-power consumption CMOS differential current controlled oscillator (CCO) for neuromorphic applications. The oscillation frequency is improved over the conventional one by reducing the number of MOS transistors thus lowering the load capacitor in each stage. The analysis shows that for the same power consumption, the oscillation frequency can be increased about 11% compared with the conventional one without degrading the phase noise. Alternatively, the power consumption can be reduced 15% at the same frequency. The prototype structures are fabricated in a standard 65 nm CMOS technology and measurements demonstrate that the proposed CCO operates from 0.7-1.2 V supply with maximum frequencies of 80 MHz and energy/cycle ranging from 0.11-0.38 pJ over the tuning range. Further, system level simulations show that the nonlinearity in current-frequency conversion by the CCO does not affect its use as a neuron in a Deep Neural Network if accounted for during training.
- 15 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
- Vol. 8 (1) , 6-27
Low-Power, Adaptive Neuromorphic Systems: Recent Progress and Future Directions
In this paper, we present a survey of recent works in developing “neuromorphic” or neuro-inspired hardware systems. In particular, we focus on those systems which can either learn from data in an unsupervised or online supervised manner. We present algorithms and architectures developed specially to support on-chip learning. Emphasis is placed on hardware friendly modifications of standard algorithms such as backpropagation as well as novel algorithms such as structural plasticity developed specially for low-resolution synapses. We cover works related to both spike based and more traditional non-spike based algorithms. This is followed by developments in novel devices such as floating-gate MOS, memristors and spintronic devices. CMOS circuit innovations for on-chip learning and CMOS interface circuits for post-CMOS devices such as memristors are presented. Common architectures such as crossbar or island style arrays are discussed along with their relative merits and demerits. Finally, we present some possible applications of neuromorphic hardware such as brain-machine interfaces, robotics, etc. and identify future research trends in the field.
- 15 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
- Vol. 8 (1) , 6-27
Low-Power, Adaptive Neuromorphic Systems: Recent Progress and Future Directions
In this paper, we present a survey of recent works in developing “neuromorphic” or neuro-inspired hardware systems. In particular, we focus on those systems which can either learn from data in an unsupervised or online supervised manner. We present algorithms and architectures developed specially to support on-chip learning. Emphasis is placed on hardware friendly modifications of standard algorithms such as backpropagation as well as novel algorithms such as structural plasticity developed specially for low-resolution synapses. We cover works related to both spike based and more traditional non-spike based algorithms. This is followed by developments in novel devices such as floating-gate MOS, memristors and spintronic devices. CMOS circuit innovations for on-chip learning and CMOS interface circuits for post-CMOS devices such as memristors are presented. Common architectures such as crossbar or island style arrays are discussed along with their relative merits and demerits. Finally, we present some possible applications of neuromorphic hardware such as brain-machine interfaces, robotics, etc. and identify future research trends in the field.