Evaluating Embedding Representations for Multiclass Code Smell Detection: A Comparative Study of CodeBERT and General-Purpose Embeddings
Preprint
- 20 March 2026
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Code smells are indicators of potential design problems in software systems and are commonly used to guide refactoring activities. Recent advances in representation learning have enabled the use of embedding-based models for analyzing source code, offering an alternative to traditional approaches based on manually engineered metrics. However, the effectiveness of different embedding representations for multiclass code smell detection remains insufficiently explored. This study presents an empirical comparison of embedding models for the automatic detection of three widely studied code smells: Long Method, God Class, and Feature Envy. Using the Crowdsmelling dataset as an empirical basis, source code fragments were extracted from the original projects and transformed into vector representations using two embedding approaches: a general-purpose embedding model and the code-specialized CodeBERT model. The resulting representations were evaluated using several machine learning classifiers under a stratified group-based validation protocol. The results show that CodeBERT consistently outperforms the general-purpose embeddings across multiple evaluation metrics, including balanced accuracy, macro F1-score, and Matthews correlation coefficient. Dimensionality reduction analyses using PCA and t-SNE further indicate that CodeBERT organizes code smell instances in a more structured latent representation space, which facilitates the separation of smell categories. These findings provide empirical evidence that domain-specific pretraining plays an important role in representation learning for software engineering tasks and that embedding choice significantly influences the separability of code smell categories in multiclass detection settings.Keywords
All Related Versions
- Version 1, 2026-03-20, Preprints
- Published version: Applied Sciences, 16 (8), 3622.
This publication has 25 references indexed in Scilit:
- The impact of feature selection and feature reduction techniques for code smell detection: A comprehensive empirical studyAutomated Software Engineering, 2025
- Data preprocessing for machine learning based code smell detection: A systematic literature reviewInformation and Software Technology, 2025
- Evaluating Deep Learning Embedding Techniques for Code Smell Detection2025
- Improving Machine Learning Algorithm’s Accuracy for Detecting Code Smell Using Data Balancing and Parameter Optimization2024
- Beyond the Code: Unraveling the Applicability of Graph Neural Networks in Smell Detection2024
- Automatic detection of Feature Envy and Data Class code smells using machine learningExpert Systems with Applications, 2023
- Method-Level Code Smells Detection Using Machine Learning Models2023
- Automatic detection of Long Method and God Class code smells through neural source code embeddingsExpert Systems with Applications, 2022
- Crowdsmelling: A preliminary study on using collective knowledge in code smells detectionEmpirical Software Engineering, 2022
- A large-scale empirical study on the lifecycle of code smell co-occurrencesInformation and Software Technology, 2018