Non-Gradient Quaternion Training Matrix Modifications for Colored Images Distillation
Preprint
- 13 July 2026
- preprint
- Published by MDPI AG in Preprints.org
Abstract
This paper develops a new multi-stage method for image distillation which involves two well-known methods. At its first stage, our method creates a matrix from all training images. On the next stage, it adapts a modified principal component analysis (M-PCA) approach to transform the training matrix. On the third stage, the Singular Value Decomposition (SVD) further refines the matrix of the training images through low-rank reconstruction and controlled matrix row selection. On the fourth stage, rotation of small 2 × 2 matrix blocks on the entire left singular matrix is conducted. The upper m (user-selected number) rows of the reconstructed matrix are selected and transformed back to images, which we call distilled images. This dataset is significantly smaller yet retains the critical information needed for an accurate classification. We validated the novelties and the advantages of the new method applying the Baseline and ResNet50V2 CNNs and the public image databases Digit-MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, and BloodMNIST. Experimental results show that models trained on images distilled by our new method achieve efficient classification and outperform contemporary competitors, while substantially reducing training time compared to training on the original dataset.Keywords
All Related Versions
- Version 1, 2026-07-13, Preprints
- Published version: Electronics, 15 (18), 4219.
This publication has 11 references indexed in Scilit:
- HFed-MIL: Patch Gradient-Based Attention Distillation Federated Learning for Heterogeneous Multi-Site Ovarian Cancer Whole-Slide Image AnalysisElectronics, 2025
- Automatic Image Distillation with Wavelet Transform and Modified Principal Component AnalysisElectronics, 2025
- Importance-aware adaptive dataset distillationNeural Networks, 2024
- Training Data Augmentation with Data Distilled by Principal Component AnalysisElectronics, 2024
- A Comprehensive Survey of Dataset DistillationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classificationScientific Data, 2023
- Compressed gastric image generation based on soft-label dataset distillation for medical data sharingComputer Methods and Programs in Biomedicine, 2022
- Skin lesion image classification using sparse representation in quaternion wavelet domainSignal, Image and Video Processing, 2022
- Compressing Large-Scale Transformer-Based Models: A Case Study on BERTTransactions of the Association for Computational Linguistics, 2021
- Principal component analysisWIREs Computational Statistics, 2010