Multiscale Feature Importance-based Bit Allocation for End-to-End Feature Coding for Machines
Open Access
- 1 August 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
Abstract
Feature Coding for Machines (FCM) aims to compress intermediate features effectively for remote intelligent analytics, which is crucial for future intelligent visual applications. In this paper, we propose a Multiscale Feature Importance-based Bit Allocation (MFIBA) for end-to-end FCM. First, we find that the importance of features for machine vision tasks varies with the scales, object size, and image instances. Based on this finding, we propose a Multiscale Feature Importance Prediction (MFIP) module to predict the importance weight for each scale of features. Secondly, we propose a task loss-rate model to establish the relationship between the task accuracy losses of using compressed features and the bitrate of encoding these features. Finally, we develop a MFIBA for end-to-end FCM, which is able to assign coding bits of multiscale features more reasonably based on their importance. Experimental results demonstrate that when combined with a retained Efficient Learned Image Compression (ELIC), the proposed MFIBA achieves an average of 38.202% bitrate savings in object detection compared to the anchor ELIC. Moreover, the proposed MFIBA achieves an average of 17.212% and 36.492% feature bitrate savings for instance segmentation and keypoint detection, respectively. When the proposed MFIBA is applied to the LIC-TCM, it achieves an average of 18.103%, 19.866% and 19.597% bit rate savings on three machine vision tasks, respectively, which validates the proposed MFIBA has good generalizability and adaptability to different machine vision tasks and FCM base codecs.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- Learnt Mutual Feature Compression for Machine Vision2023
- Learned Image Compression with Mixed Transformer-CNN Architectures2023
- Learned Disentangled Latent Representations for Scalable Image Coding for Humans and Machines2023
- Semantics-to-Signal Scalable Image Compression with Learned Revertible RepresentationsInternational Journal of Computer Vision, 2021
- Teacher-Student Learning With Multi-Granularity Constraint Towards Compact Facial Feature Representation2021
- End-to-End optimized image compression for machines, a study2021
- Task-Aware Quantization Network for JPEG Image Compression2020
- Intra Frame Rate Control for Versatile Video Coding with Quadratic Rate-Distortion Modelling2020
- Local keypoint-based Faster R-CNNApplied Intelligence, 2020
- Deep Residual Learning for Image Recognition2016