A Wired-Logic and Near-Memory Hybrid Processor for General-Purpose Audio Inference Using a Self-Supervised Encoder
Open Access
- 1 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of the Solid-State Circuits Society
- Vol. 6, 439-449
Abstract
This work presents a wired-logic (WL) and near-memory (NM) hybrid processor that enables general-purpose audio inference by embedding a compact self-supervised learning (SSL) pretrained encoder into an area-efficient WL core while retaining task-level adaptability through an NM-based classifier. To obtain a compact encoder suitable for WL implementation, we develop an NNN-SSL framework that trains a nonlinear neural network (NNN) using ternary weights, 80% pruning, and 14-bit activations within an augmentation-based SSL pipeline. The resulting 64-channel encoder provides unified feature extraction across speech, environmental sound, and musical-instrument domains. A global-pooling block and a frequency-serial fully connected (FC) architecture support variable-length audio inputs and reduce the FC area from 3.56mm2 to 0.76mm2. A 40 nm test chip achieves 48.1 TOPS/W for the convolution layer, while the full-network energy is estimated at 0.87 μJ/inference on GSCD-12. Furthermore, scaling the SSL pretraining dataset and the number of epochs enables accuracy improvements under a fixed-area constraint, showing that SSL can compensate for limited model capacity in WL-based designs. By combining SSL-driven general-purpose feature learning with WL-centric energy efficiency and minimal NM programmability, the proposed processor demonstrates a practical path toward compact, low-power, multi-domain audio AI for edge devices.Keywords
Funding Information
- Japan Science and Technology Agency (JST)/Precursory Research for Embryonic Science and Technology (JPMJPR21B4)
- JST Support for Pioneering Research Initiated by the Next Generation (JPMJSP2108)
This publication has 27 references indexed in Scilit:
- A 250.3μW Versatile Sound Feature Extractor Using 1024-point FFT 64-ch LogMel Filter in 40nm CMOS2024
- 14.8 KASP: A 96.8% 10-Keyword Accuracy and 1.68μJ/Classification Keyword Spotting and Speaker Verification Processor Using Adaptive Beamforming and Progressive Wake-Up2024
- A 0.13mJ/Prediction CIFAR-100 Raster-Scan- Based Wired-Logic Processor Using Non-Linear Neural Network2023
- A 22nm 8Mb STT-MRAM Near-Memory-Computing Macro with 8b-Precision and 46.4-160.1TOPS/W for Edge-AI Devices2023
- A 23μW Solar-Powered Keyword-Spotting ASIC with Ring-Oscillator-Based Time-Domain Feature Extraction2022
- BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation2021
- Activate or Not: Learning Customized Activation2021
- Audio Set: An ontology and human-labeled dataset for audio events2017
- CNN architectures for large-scale audio classification2017
- Where am I? Scene Recognition for Mobile Robots using Audio Features2006