Utpal Garain

Data-to-Text Generation (D2T) aims to transform semi-structured data (such as tables and graphs) into natural language text. With the exceptional capability of Large Language Models (LLMs), they have become ubiquitous as foundational models for D2T. This article presents a comprehensive evaluation of LLMs for D2T, focusing on three key qualities: readability (fluency and coherence), informativeness (content preservation), and faithfulness (factual accuracy). We evaluate 12 LLMs from five prominent open source families (BART, T5, BLOOM, OPT, and Llama 2) across five widely used D2T datasets using six established automatic metrics, complemented by human evaluation for deeper insight. Our findings reveal that larger model sizes generally improve readability and informativeness, with Llama 2 showing superior overall performance. However, increased model size does not consistently enhance faithfulness and may sometimes degrade it. Human evaluations indicate that larger models are generally preferred for their readability, informativeness, and faithfulness from the human readers’ perspective, as their minor faithfulness errors are assessed more selectively by automatic evaluation metrics. Through robustness analyses, we confirm that these trends remain stable across different fine-tuning (QLoRA vs. Prefix-Tuning) and decoding (Beam Search vs. Nucleus Sampling) strategies. Furthermore, our experiments show that performance consistently declines as source-reference divergence increases, regardless of model size. To mitigate this, we propose a source-reference divergence-weighted training that adaptively reweights training instances based on their source-reference divergence, achieving consistent improvements across all three key evaluation qualities. This comprehensive study provides practical insights into LLM behavior in D2T and introduces an effective training paradigm for improving performance in D2T.

It is important to investigate whether common or shared weaknesses exist across different cipher families—whether similar or different in structure—that could implicitly be detected/modeled by transfer learning. To explore this, we employ transfer learning between different families of lightweight block ciphers using Convolutional Neural Networks (CNNs) for differential cryptanalysis, where the target domain is represented by only a small number of labeled instances. In 2019, Gohr combined differential cryptanalysis with deep learning to analyze round-reduced SPECK. He demonstrated that the proposed model could effectively perform few-shot learning of a cryptographic distribution by using prior knowledge of a correlated distribution. While our approach is inspired by Gohr’s work, our objectives and experimental design differ. We use neural differential distinguishers trained on round-reduced versions of ciphers from a family, such as SPECK, SIMON, or PRESENT, and transfer the learned knowledge to design successful differential distinguishers for some other member of the same or different family. This suggests that the SIMON, SPECK, and PRESENT families may share common weaknesses, which warrants further investigation. To the best of our knowledge, this is the first attempt to apply transfer learning across different families of block ciphers, with ARX and SPN design structures.

Potency (IC₅₀) prediction of small molecules is pivotal for anticancer drug development. This study benchmarked five deep learning (DL) models for IC₅₀ prediction—DeepCDR, DrugCell, PaccMann, Precily, and tCNN—against a simple mean-based Baseline using standardized GDSC datasets and recently published anticancer compounds. To ensure practicality, conventional error metrics were supplemented with percentage error, log error, three-sigma limit, and a newly proposed Experimental Variability-Aware Prediction Accuracy statistic. The models performed well on randomly split data and unseen cell lines but showed sharply reduced accuracy for unseen compounds. Though all DL models exhibited similar performance trends, DeepCDR, DrugCell, and tCNN held a slight edge in most testing scenarios. Interestingly, several DL algorithms could not significantly outperform the Baseline model in many tests. Assessing prediction error against physicochemical and biological properties of compounds and cell lines revealed weak correlation, highlighting an underexplored aspect of model performance. A user-friendly web server (https://nlplab1.isical.ac.in/ic50.php) was also developed for IC₅₀ prediction of new compounds against cancer cell lines.

Potency (IC ₅₀ ) prediction of anticancer small molecules plays a vital role in developing drug candidates. Five deep learning models for IC ₅₀ prediction--DeepCDR, DrugCell, PaccMann, Precily, and tCNN--were benchmarked using standardized GDSC datasets and recently published anticancer compounds. To ensure practicality, conventional error statistics were supplemented with the absolute percentage error, the three-sigma limit, and a newly proposed statistic named Experimental Variability-Aware Prediction Accuracy. The models performed well on randomly split data and unseen cell lines, but performance declined sharply for unseen compounds. All models exhibited comparable performance trends; however, DrugCell and DeepCDR outperformed the others across several metrics. Assessing prediction accuracy against physicochemical and biological properties of compounds and cell lines revealed its broad independence from these parameters, highlighting an underexplored aspect of machine learning model performance. A user-friendly web server (https://nlplab1.isical.ac.in/ic50.php) was also developed for IC ₅₀ prediction of new compounds against cancer cell lines.