Andrea Stocco

Large Language Models (LLMs) pre-trained on expansive text and code corpora have revealed promising code generation abilities and have attracted increasing attention in code translation. In this work, we investigate the effectiveness of LLMs in code translation and translation error repair. First, we present CodeTransBenchmark, a framework for evaluating LLM-based translation and repair and devise a post-processing strategy to extract code from inconsistent LLM outputs. Then, we discuss an empirical study evaluates eight models on three datasets and 12 language pairs, in which we categorize incorrect translations by errors to identify weaknesses of existing LLMs. Our work shows that while LLMs specifically trained for multi-lingual coding, like Codestral, correctly translate the majority of code, most general-purpose models struggle with the syntactic rules of the target language. The analysis of erroneous translations reveals the substantial impact of the interrelationship between involved programming languages and training data on the effectiveness. We show that a general post-processing approach must tolerate inconsistencies and leverage the predictability of LLM answers. Further, we show that iterative translation repair via automated feedback significantly improves translation accuracy. While our combined findings highlight the potential of LLMs to automate code translation, an effective deployment of LLM-based code translation in practice would require models with larger context windows.

As deep learning (DL) models are widely used in software systems, test generation plays a crucial role in assessing the quality of such models before deployment. To date, the most advanced test generators rely on generative AI to synthesize inputs; however, these approaches remain limited in providing semantic insight into the causes of misbehaviours and in offering fine-grained semantic controllability over the generated inputs. In this paper, we introduce Detect, a feature-aware test generation framework for vision-based DL models that generates inputs by perturbing disentangled semantic attributes within the latent space. Detect perturbs individual latent features in a controlled way and observes how these changes affect the model’s output. Through this process, it identifies which features lead to behaviour shifts and uses a vision-language model for semantic attribution. Detect enables semantically controlled perturbations in StyleGAN’s disentangled latent space, allowing us to test whether a model generalizes under task-relevant feature changes and remains invariant under task-irrelevant ones. Empirical results across image classification and detection tasks show that Detect generates high-quality test cases with fine-grained control, reveals distinct shortcut behaviours across model architectures (convolutional and transformer-based), and bugs that are not captured by accuracy metrics. Specifically, Detect outperforms a state-of-the-art test generator in decision boundary discovery and provides more interpretable feature-level diagnoses than a leading spurious feature localization method in identifying robustness failures. Our findings show that fully fine-tuned convolutional models are prone to overfitting on localized cues, such as co-occurring visual traits, while weakly supervised transformers tend to rely on global features, such as environmental variances. These findings highlight the value of interpretable and feature-aware testing in improving DL model reliability, and refocus DL testing from class-level failure discovery to feature-level, semantically grounded diagnosis and repair.