STStefan Tomić

Stefan Tomić

*This is an algorithmically generated author record based on publicly available information.

Stefan Tomić has a total of 11 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Bias-Aware FP8 Format for Efficient Weight Representation in Post-Training Quantization

    by Aleksandra Z. Jovanovic, Stefan S. Tomic, Jelena R. Nikolic, Zoran H. Peric, Davide Bertozzi, Michele Favalli

  • Statistical Analysis of Synaptic Weights in Spiking Neural Network Trained on the DVS128 Gesture Dataset

    by Jelena Nikolić, Stefan Tomić, Aleksandra Jovanović, Zoran Perić, Milan Dinčić, Edward Jones, Davide Bertozzi, Riccardo Zese, Marko Andjelković

    The application of compression techniques, such as quantization and pruning, to reduce neural network weight size and complexity is gaining momentum due to the significant savings they provide in both hardware and software resources, particularly important for edge devices with limited capabilities. To fully leverage the advantages of these techniques, their design should account for the amplitude dynamics and the probability distribution of the parameters to be compressed. Therefore, it is essential to analyze the statistical characteristics of these parameters prior to compression. Although this type of analysis has been extensively conducted in the context of Artificial Neural Networks (ANNs), it has received limited attention in the case of Spiking Neural Networks (SNNs). Motivated by this gap, this paper examines the statistical properties of the weights of an SNN trained on the DVS128 Gesture dataset. The results of this analysis are valuable not only for enabling a well-founded selection and effective design of appropriate quantization models, but also for supporting the efficient application of pruning techniques, which could benefit from using synaptic weight histograms as valuable guiding information.


  • Open Access

    Performance Analysis of Two 8-Bit Floating-Point-based Piecewise Uniform Quantizers for a Laplacian Data Source

    by Jelena R. Nikolic, Zoran H. Peric, Aleksandra Z. Jovanovic, Stefan S. Tomic, Sofija Z. Peric

    In this paper, we employ the analogy between the representation of the floating-point (FP) format and the representation level distribution of the piecewise uniform quantizer (PWUQ) to assess the performance of FP-based solutions more thoroughly. We present theoretical derivations to assess the performance of the FP format and the PWUQ determined by this format for input data from the Laplacian source. We also provide a performance comparison of two selected 8-bit FP-based PWUQs. Beyond the typical evaluation of the applied FP format, through the accuracy degradation caused by the application of both FP8 solutions in neural network compression, we also use objective quantization measures. This approach offers insights into the robustness of these 8-bit FP-based solutions with respect to changes in input variance, which can be important when the input variance changes. The results demonstrate that the allocation of bits to encode the exponent and mantissa in the FP8 format is important, as it can significantly impact overall performance.


  • Open Access

    Accuracy degradation aware bit rate allocation for layer-wise uniform quantization of weights in neural network

    by Jelena Nikolić, Stefan Tomić, Zoran Perić, Aleksandra Jovanović, Danijela Aleksić

    Motivated by the fact that optimizing quantization error fails to minimize the accuracy degradation caused by the quantization of Neural Network (NN) weights, in this paper, we highlight the need for a comprehensive analysis of post-training quantization, covering average bit rate, accuracy degradation and SQNR. One such analysis, for the layer-wise uniform quantization and its application is NN weight compression is presented in this paper. We introduce an additional aspect of Uniform Quantizer (UQ) choice, and we allow the selection of one of two UQs (2-bit or 3-bit UQs) for each layer of NN. However, choosing a particular bit rate allocation to minimize average bit rate and accuracy degradation requires finding a trade-off solution. To solve the issue underpinning our motivation, for a post-training layer-wise uniform quantization framework with two UQs, we propose an algorithm for generating bit rate allocation patterns and corresponding pairs of average bit rate and accuracy degradation. We analyze the output of this algorithm and determine a solution of the elaborated two-objective optimization problem. To confirm the validity of the results we also apply the Pareto dominance method. This work can be considered a solid foundation for further analyses of more complex and deeper NNs.


  • Open Access

    Performance of Post-Training Two-Bits Uniform and Layer-Wise Uniform Quantization for MNIST Dataset from the Perspective of Support Region Choice

    by Stefan Tomić, Jelena Nikolić, Zoran Perić, Danijela Aleksić

    This paper contributes to the goal of finding an efficient compression solution for post-training quantization from the perspective of support region choice under the framework of low-bit uniform quantization. The decision to give preference to uniform quantization comes from the fact that support region choice is the most sensitive in the uniform quantization of nonuniform sources (e.g., Laplacian sources). Therefore, in this paper, we analyse in detail how the choice of the support region influences the performance of two-bits uniform quantization, measured with signal to quantization noise ratio (SQNR), and the accuracy of the compressed neural network (NN) model. We provide experimental and theoretical results for a few significant cases of two-bits uniform quantizer design, where we assume that Laplacian source models the distribution of weights in our fully connected NN. We opt for Laplacian distribution since it models well weights of NNs. Specifically, we analyse whether it is possible to apply the simplest uniform quantization in trained NN model weight representation with a bit rate of R=2 bit/sample while preserving the accuracy of the model to a great extent. Also, our goal is to determine whether the choice of the key parameter of two-bits uniform quantizer (support region threshold) equally reflects on both, SQNR and accuracy. Moreover, we extend our analysis to the application of layer-wise two-bits uniform quantization in order to examine whether it is possible to achieve an additional improvement of the accuracy of our NN model for the MNIST dataset. We believe that the detailed analysis of post-training quantization described and conducted in this paper is very useful for all further research studies of this very current topic, especially due to the fact that the problem regarding post-training quantization is addressed from a particularly important perspective of choosing the support region.