PDPierre Dantas

Pierre Dantas

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Pierre Dantas has a total of 6 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Denoising Neural Models with Spectral QUEnching and Eigenvalue ZEroing (SQUEEZE)

    by Pierre Dantas, Waldir Junior, Lucas Cordeiro, Eulanda dos Santos

    \Glspl{llm} built with Transformer technology perform very well but requires substantial computational power. This paper presents \gls{squeeze}, a new framework that treats model compression as a signal-to-noise separation task. Using principles from \gls{rmt}, \gls{squeeze} identifies and preserves structural signals within weight matrices while discarding components that align with random noise. Unlike traditional methods, this approach serves as a post-training transformation that does not require retraining the model. Our analysis of fine-tuned \texttt{BERT-base} models reveals that matrix aspect ratios ($\beta$) significantly influence spectral behavior: rectangular feed-forward (\gls{ffn}) layers ($\beta =$ 0.25) adhere closely to the \gls{mp} and exhibit substantial redundancy, whereas square attention matrices ($\beta =$ 1) and highly rectangular embedding matrices ($\beta \ll$ 1) show significant departures from the null model. We implement a five-step pipeline -- standardization, \gls{svd} decomposition, baseline establishment, \gls{tw} finite-size adjustment, and rank truncation. Testing across three \gls{glue} tasks shows that \gls{squeeze} reduces model size by \SI{8.1}{\percent} with an accuracy loss of less than \SI{2.5}{\percent}. Specifically, targeting \gls{ffn} layers allows for a \SI{64}{\percent} reduction in effective rank while maintaining a cosine similarity of over 0.80. Our results show that spectral geometry is a critical factor in Transformer compressibility, positioning \gls{squeeze} as a high-fidelity alternative to aggressive methods such as quantization when model quality is the primary priority.

  • Open Access

    A review of state-of-the-art techniques for large language model compression

    by Pierre V. Dantas, Lucas C. Cordeiro, Waldir S. S. Junior

    The rapid advancement of large language models (LLMs) has driven significant progress in natural language processing (NLP) and related domains. However, their deployment remains constrained by challenges related to computation, memory, and energy efficiency—particularly in real-world applications. This work presents a comprehensive review of state-of-the-art compression techniques, including pruning, quantization, knowledge distillation, and neural architecture search (NAS), which collectively aim to reduce model size, enhance inference speed, and lower energy consumption while maintaining performance. A robust evaluation framework is introduced, incorporating traditional metrics, such as accuracy and perplexity (PPL), alongside advanced criteria including latency-accuracy trade-offs, parameter efficiency, multi-objective Pareto optimization, and fairness considerations. This study further highlights trends and challenges, such as fairness-aware compression, robustness against adversarial attacks, and hardware-specific optimizations. Additionally, NAS-driven strategies are explored as a means to design task-aware, hardware-adaptive architectures that enhance LLM compression efficiency. Hybrid and adaptive methods are also examined to dynamically optimize computational efficiency across diverse deployment scenarios. This work not only synthesizes recent advancements and identifies open problems but also proposes a structured research roadmap to guide the development of efficient, scalable, and equitable LLMs. By bridging the gap between compression research and real-world deployment, this study offers actionable insights for optimizing LLMs across a range of environments, including mobile devices and large-scale cloud infrastructures.


  • Top Cited Papers
    Open Access

    A comprehensive review of model compression techniques in machine learning

    by Pierre Vilar Dantas, Waldir Sabino da Silva, Lucas Carvalho Cordeiro, Celso Barbosa Carvalho

    This paper critically examines model compression techniques within the machine learning (ML) domain, emphasizing their role in enhancing model efficiency for deployment in resource-constrained environments, such as mobile devices, edge computing, and Internet of Things (IoT) systems. By systematically exploring compression techniques and lightweight design architectures, it is provided a comprehensive understanding of their operational contexts and effectiveness. The synthesis of these strategies reveals a dynamic interplay between model performance and computational demand, highlighting the balance required for optimal application. As machine learning (ML) models grow increasingly complex and data-intensive, the demand for computational resources and memory has surged accordingly. This escalation presents significant challenges for the deployment of artificial intelligence (AI) systems in real-world applications, particularly where hardware capabilities are limited. Therefore, model compression techniques are not merely advantageous but essential for ensuring that these models can be utilized across various domains, maintaining high performance without prohibitive resource requirements. Furthermore, this review underscores the importance of model compression in sustainable artificial intelligence (AI) development. The introduction of hybrid methods, which combine multiple compression techniques, promises to deliver superior performance and efficiency. Additionally, the development of intelligent frameworks capable of selecting the most appropriate compression strategy based on specific application needs is crucial for advancing the field. The practical examples and engineering applications discussed demonstrate the real-world impact of these techniques. By optimizing the balance between model complexity and computational efficiency, model compression ensures that the advancements in AI technology remain sustainable and widely applicable. This comprehensive review thus contributes to the academic discourse and guides innovative solutions for efficient and responsible machine learning practices, paving the way for future advancements in the field. Graphical


  • Energy Disaggregation via Data Mining

    by Pierre Dantas, Waldir Sabino, Maryana Batalha