Muhammad Ali Babar

The application of Large Language Models (LLMs) is growing in the productive completion of Software Engineering tasks. Yet, studies investigating productive prompting techniques often employed a limited problem space, focusing primarily on well-known prompting patterns and targeting function-level SE practices. We identify significant gaps in real-world workflows that involve complexities beyond class-level (e.g., multi-class dependencies) and different features that can impact Human-LLM Interaction (HLI) processes in code generation. To address these issues, we designed an experiment to comprehensively analyze HLI features related to code generation productivity. Our study presents two project-level benchmark tasks that extend beyond function-level evaluations. We conducted a user study with 36 participants from diverse backgrounds, asking them to solve the assigned tasks by interacting with the GPT assistant using specific prompting patterns. We also examined participants’ experiences and behavioral features during interactions by analyzing screen recordings and GPT chat logs. Our empirical investigation, based on statistical guidance revealed (1) that three out of 15 HLI features emerged as consistently supported factors for productivity; (2) five primary guidelines for enhancing productivity for HLI processes; and (3) a taxonomy of 29 runtime and logic errors that can occur during HLI processes, along with suggested mitigation plans.

Context: Infrastructure as code (IaC) is the practice of automatically configuring, managing, and provisioning an organization's infrastructure using configuration scripts, e.g., Ansible scripts. While IaC is beneficial for automated software deployment and configuration management, the relationship between certain configuration patterns and performance attributes, such as execution time remain unknown. Configuration patterns are recurring coding patterns in configuration scripts that are used in IaC. A systematic evaluation can quantify the relationship between configuration patterns and performance attributes, which in turn can aid practitioners in their configuration script development process.Objective: The objective of the paper is to help practitioners in obtaining desired performance for IaC-based infrastructure management by deriving a set of configuration patterns that correlate with execution time, CPU consumption, and memory consumption.Methods: We conduct an empirical study with 4,523 and 64 Ansible configuration scripts, collected from open source software (OSS) and proprietary repositories, respectively. We first derive potential configuration patterns related to performance from peer-reviewed publications and grey literature. We execute the collected configuration scripts, collect performance data, and perform statistical analysis to identify the patterns related to performance. Additionally, we develop a linter called PERFector to detect these patterns in configuration scripts.Results: We derive a list of 5 performance-related patterns. Using PERFector we identify 1,873 instances of performance related patterns, where the pattern `needless facts' is the most frequent category. We obtain practitioner feedback for 48 instances of patterns, of which 19 instances have been confirmed as valid.Conclusion: Our paper provides a list of performance-related configuration patterns that practitioners can detect using PERFector and avoid. This provides a solid foundation for future research on techniques to automatically repair performance-related configuration patterns.

Smart contracts underpin modern blockchain ecosystems, enabling decentralized applications in DeFi and digital asset management. However, their immutability exacerbates programming errors and vulnerabilities, leading to unpatchable flaws and significant financial losses, as seen in high-profile exploits. Current vulnerability detection methods including static and dynamic analysis—face critical limitations: (1) static analysis suffers from state-space explosion, control-flow complexity, over-approximation, and high false positives; (2) dynamic fuzzing relies on syntactic/random mutations lacking semantic awareness, inefficiently exploring deep state-dependent paths and complex transaction sequences; (3) both static and dynamic approaches overlook semantic-structural gaps, such as execution order semantics in reentrancy vulnerabilities; (4) fuzzing reports lack explainability, hindering practical auditing. To address these challenges, we propose MAESFuzz1, a multi-agent semantic-guided smart contract fuzzing framework that integrates static structural analysis, dynamic execution feedback, and Large Language Model (LLM)–assisted reasoning within a closed-loop architecture. MAESFuzz employs LLMs as semantic advisors to guide seed generation, transaction sequence construction, and mutation refinement, enabling effective exploration of deep state-dependent behaviors and complex multi-transaction vulnerabilities. Extensive evaluation on the SoliAudit benchmark and a real-world dataset of deployed Ethereum contracts shows that MAESFuzz improves vulnerability detection by up to 28.7% relative to state-of-the-art (SOTA) smart contract fuzzers on SoliAudit. On real-world contracts, MAESFuzz uncovers 25 more expert-validated vulnerabilities than the strongest SOTA baseline while maintaining 94.38% precision and competitive execution coverage. In addition, MAESFuzz generates structured, execution-grounded audit reports to enhance interpretability and support practical security auditing.

As mobile networks evolve toward 6G, the Open Radio Access Network (Open RAN) paradigm has enabled the emergence of the prominent O-RAN framework, which is championed by industry operators and leverages novel technical advantages, with particular emphasis on openness and intelligence. While integrated techniques create unprecedented innovations in large-scale data interconnectivity under 5G/6G networks, these changes inevitably expand new attack surfaces, necessitating the use of robust intrusion detection systems (IDSs) to secure the O-RAN system and positioning Artificial Intelligence (AI)-driven solutions as a critical technical pathway under highly automated, AI-native O-RAN. To comprehensively understand and address Open RAN critical security gaps, this paper introduces related security standardization and proposes systematic guidance for implementing IDSs in O-RAN, covering typical IDS categories, main O-RAN components, interfaces, and relevant network activities. This paper summarizes Open RAN potential threats in terms of architecture and operational scenarios, presenting a multifaceted security landscape for disaggregated O-RAN, cloudified infrastructure, virtualized network functions, multi-vendor openness, and built-in intelligent platforms across diverse industrial verticals in 5G/6G. The work explores various AI techniques, along with studying their applications in monitoring O-RAN anomalies, spanning classical machine learning paradigms, deep learning, collaborative frameworks, and innovative generative AI incorporating emerging transformer-based models. This paper also discusses the challenges of securing Open RAN in the fast-evolving AI era, highlighting opportunities for addressing limited evaluation datasets, the balance between privacy protection overheads and stringent QoS requirements, threats targeting IDS intelligence, and increasing heterogeneity.

The Internet of Things (IoT) has rapidly emerged as a ubiquitous and pervasive paradigm in software development, significantly impacting both social life and business environments. However, this growth has also led to a corresponding increase in the number and sophistication of threats and attacks targeting IoT devices and services. The vulnerability of IoT software to security breaches has become a significant concern for the research community. Managing software vulnerabilities in IoT is a hugely challenging process involving several socio-technical decisions. Despite the rapid increase in primary studies focusing on Software Vulnerability Management (SVM) in IoT systems, no secondary studies specifically identify and analyse the socio-technical challenges, solutions, and state-of-the-art evaluation studies in SVM in IoT systems. This paper aims to address this gap by systematically identifying, classifying, comparing, and evaluating state of the art of SVM in IoT systems from a socio-technical point of view. We conducted a systematic mapping study (SMS) based on 73 qualitatively selected studies to i) classify the types, frequency, and demography of published research; ii) identify the socio-technical challenges in this regard; iii) classify the reported solutions; and iv) understand the rigour of the evaluation, including real-world application. In summary, our results point to 32 socio-technical challenges in IoT vulnerability management, where most are practice-related. In terms of the solutions, we found a maximum number of solutions proposed for the software vulnerability identification stage, with 22 frameworks. The software vulnerability disclosure stage has the least amount of solutions reported. This SMS also reveals that there needs to be more rigorous evaluation using more mature forms of evaluations like simulation with real data and case studies. Based on the findings that highlight the important concerns in this domain, we recommend a list of future research directions.

Artificial Intelligence (AI) has revolutionized software development, particularly by automating repetitive tasks and improving developer productivity. While these advancements are well-documented, the use of AI-powered tools for Software Vulnerability Management (SVM), such as vulnerability detection and repair, remains underexplored in industry settings. To bridge this gap, our study aims to determine the extent of the adoption of AI-powered tools for SVM, identify barriers and facilitators to the use, and gather insights to help improve the tools to meet industry needs better. We conducted a survey study involving 60 practitioners from diverse industry sectors across 27 countries. The survey incorporates both quantitative and qualitative questions to analyze the adoption trends, assess tool strengths, identify practical challenges, and uncover opportunities for improvement. Our findings indicate that AI-powered tools are used throughout the SVM life cycle, with 69% of users reporting satisfaction with their current use. Practitioners value these tools for their speed, coverage, and accessibility. However, concerns about false positives, missing context, and trust issues remain prevalent. We observe a socio-technical adoption pattern in which AI outputs are filtered through human oversight and organizational governance. To support safe and effective use of AI for SVM, we recommend improvements in explainability, contextual awareness, integration workflows, and validation practices. We assert that these findings can offer practical guidance for practitioners, tool developers, and researchers seeking to enhance secure software development through the use of AI.

Transformer-based Text Classifiers (TTC), exemplified by models like BERT, Roberta, T5, and GPT, have showcased impressive proficiency in Natural Language Processing (NLP). Nonetheless, their vulnerability to adversarial examples (AEs) presents a substantial security concern. Existing efforts to fortify model robustness either entail high computational costs or lack transparency and interpretability. This paper introduces a holistic framework called Explainability-driven Detection, Identification, and Transformation (EDIT) to address the shortcomings of prevailing inference-time defenses. (i) EDIT employs explainability methodologies (e.g., attention maps, integrated gradients) and frequency features for automated detection and identification of perturbed words while providing insights into the model's decision-making process. (ii) Building upon detection, EDIT purifies detected adversarial examples through an optimal transformation mechanism that utilizes pre-trained embeddings and model feedback to find the best replacements for identified perturbed words. (iii) To integrate human intelligence, the framework automates the process and triggers alerts for a security analyst to ensure secure safeguards and enable proactive human feedback. We comprehensively evaluate our framework on BERT and ROBERTA, trained on four SOTA text classification datasets: IMDB, YELP, AGNEWS, and SST2. We also assess our framework against seven SOTA Word Substitution Attacks (WSA) at character, word, and multi-levels of granularity. The results show that EDIT achieves an average F-score and balanced accuracy of 89.69% and 89.70%, respectively, across all datasets and classifiers. Compared to four SOTA inference defenses, our detector is 1.22 times better in balanced accuracy (BAL_ACC) and 1.33 times better in F1-Score. It also excels in operational efficiency, being about 83 times faster in feature extraction. The identification module outperforms techniques like ReplaceScore, FreqScore, and ExplainScore, achieving 1.125 times better BAL_ACC and Recall, 1.142 times higher AUC, 1.120 times greater Precision, and 1.149 times improved F1-Score. It significantly reduces false positives and negatives by approximately 1.537 and 1.172 times, respectively. Our transformation module effectively converts adversarial examples into non-adversarial counterparts with an accuracy of 91%. EDIT identifies the necessity for human intervention with an average median accuracy of 89%. With alerts integrated, EDIT achieves an average median accuracy of 90% across all evaluated datasets, models, and types of attacks, all while maintaining computational efficiency, averaging 6.94 seconds.