Ramtin Ehsani
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Ramtin Ehsani has a total of 39 co-authors. Here are the top co-authors by number of shared publications:
- Preetha Chatterjee
Drexel University, Philadelphia, USA
9 shared publications - Preetha Chatterjee
Drexel University Philadelphia PA USA
5 shared publications - Rezvaneh Rezapour
Drexel University, Philadelphia, USA
4 shared publications - Esteban Parra
Belmont University, Nashville, USA
2 shared publications - Kostadin Damevski
Virginia Commonwealth University, Richmond, USA
2 shared publications - Layla Bouzoubaa
College of Computing & Informatics, Drexel University, Philadelphia, PA, United States
2 shared publications
Latest Publications
- Open Access
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 1741-1753
How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
Large Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent. Even when provided with the same contextual information, an LLM may generate a correct patch for one bug but fail on another closely related bug. Why this happens remains poorly understood, and it is unclear how LLMs prioritize the diverse information in bug reports and whether model attention affects repair success. In this paper, we present the first empirical study of attention patterns in LLM-based program repair, providing interpretable insights into how models process bug reports and where their attention is concentrated during repair. We analyze 319 real-world Python and Java bugs from SWE-bench Verified and Multi-SWE-bench to study (RQ1) how model attention is distributed across bug report sections, (RQ2) how attention patterns within each section differ between successful and unsuccessful repairs, and (RQ3) how these patterns compare to information developers consider important for bug fixing. We find that successful repairs are characterized by diffused attention across multiple diagnostic components such as bug descriptions, stacktraces, and test cases, while failures often exhibit over-localized attention toward metadata such as version information. We further observe that stronger alignment between model attention and developer-identified key sections and phrases is associated with higher repair success. Our results provide the first empirical evidence that attention misallocation is a key factor in LLM-based APR failures, and offer actionable insights for designing more interpretable and reliable future APR systems.
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 1741-1753
Open AccessHow Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
Large Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent. Even when provided with the same contextual information, an LLM may generate a correct patch for one bug but fail on another closely related bug. Why this happens remains poorly understood, and it is unclear how LLMs prioritize the diverse information in bug reports and whether model attention affects repair success. In this paper, we present the first empirical study of attention patterns in LLM-based program repair, providing interpretable insights into how models process bug reports and where their attention is concentrated during repair. We analyze 319 real-world Python and Java bugs from SWE-bench Verified and Multi-SWE-bench to study (RQ1) how model attention is distributed across bug report sections, (RQ2) how attention patterns within each section differ between successful and unsuccessful repairs, and (RQ3) how these patterns compare to information developers consider important for bug fixing. We find that successful repairs are characterized by diffused attention across multiple diagnostic components such as bug descriptions, stacktraces, and test cases, while failures often exhibit over-localized attention toward metadata such as version information. We further observe that stronger alignment between model attention and developer-identified key sections and phrases is associated with higher repair success. Our results provide the first empirical evidence that attention misallocation is a key factor in LLM-based APR failures, and offer actionable insights for designing more interpretable and reliable future APR systems.
- Open Access
- 9 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Faster Code, Deeper Debt? A Multivocal Literature Review on Technical Debt and Its Early Signs in LLM-Assisted Software Development
With the rapid adoption of LLM-assisted coding, the need to manage the technical debt these systems introduce has become urgent. In this paper, we conduct a multivocal literature review of 104 sources (31 formal, 73 grey) to examine how LLM-assisted development contributes to technical debt and what strategies, metrics, and benchmarks exist to mitigate it. We find that LLMs often amplify traditional forms of technical debt, particularly code, design, and documentation debts, while also introducing new LLM-specific debts. Notably, we identify fast-integration debt, where rapidly generated code prioritizes speed over quality, triggering a domino effect that leads to governance debt and increased long-term maintenance costs. Additional emerging categories include prompt, ethical, data, and provenance debt, reflecting new challenges unique to LLM adoption. To address these, strategies suggested in the literature include human-in-the-loop frameworks, prompt engineering, and data quality alignment. In practice, tools such as SonarQube are commonly used to detect technical debt indicators, while research prototypes such as CodeSmellEval are emerging to assess how LLMs contribute to debts. However, no standardized benchmarks or LLM-specific metrics yet exist, leaving an important gap. Based on findings, we outline insights and future directions to ensure reliable integration of LLMs into software engineering workflows.
- 9 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Open AccessFaster Code, Deeper Debt? A Multivocal Literature Review on Technical Debt and Its Early Signs in LLM-Assisted Software Development
With the rapid adoption of LLM-assisted coding, the need to manage the technical debt these systems introduce has become urgent. In this paper, we conduct a multivocal literature review of 104 sources (31 formal, 73 grey) to examine how LLM-assisted development contributes to technical debt and what strategies, metrics, and benchmarks exist to mitigate it. We find that LLMs often amplify traditional forms of technical debt, particularly code, design, and documentation debts, while also introducing new LLM-specific debts. Notably, we identify fast-integration debt, where rapidly generated code prioritizes speed over quality, triggering a domino effect that leads to governance debt and increased long-term maintenance costs. Additional emerging categories include prompt, ethical, data, and provenance debt, reflecting new challenges unique to LLM adoption. To address these, strategies suggested in the literature include human-in-the-loop frameworks, prompt engineering, and data quality alignment. In practice, tools such as SonarQube are commonly used to detect technical debt indicators, while research prototypes such as CodeSmellEval are emerging to assess how LLMs contribute to debts. However, no standardized benchmarks or LLM-specific metrics yet exist, leaving an important gap. Based on findings, we outline insights and future directions to ensure reliable integration of LLMs into software engineering workflows.
- Open Access
- 14 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 61-65
Towards A Sustainable Future for Peer Review in Software Engineering
- 14 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 61-65
Open AccessTowards A Sustainable Future for Peer Review in Software Engineering
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 807-811
Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
AI coding agents are now submitting pull requests (PRs) to software projects, acting not just as assistants but as autonomous contributors. As these agentic contributions are rapidly increasing across real repositories, little is known about how they behave in practice and why many of them fail to be merged. In this paper, we conduct a large-scale study of 33k agent-authored PRs made by five coding agents across GitHub. (RQ1) We first quantitatively characterize merged and not-merged PRs along four broad dimensions: 1) merge outcomes across task types, 2) code changes, 3) CI build results, and 4) review dynamics. We observe that tasks related to documentation, CI, and build update achieve the highest merge success, whereas performance and bug-fix tasks perform the worst. Not-merged PRs tend to involve larger code changes, touch more files, and often do not pass the project’s CI/CD pipeline validation. (RQ2) To further investigate why some agentic PRs are not merged, we qualitatively analyze 600 PRs to derive a hierarchical taxonomy of rejection patterns. This analysis complements the quantitative findings in RQ1 by uncovering rejection reasons not captured by quantitative metrics, including lack of meaningful reviewer engagement, duplicate PRs, unwanted feature implementations, and agent misalignment. Together, our findings highlight key socio-technical and human-AI collaboration factors that are critical to improving the success of future agentic workflows.
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 807-811
Open AccessWhere Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
AI coding agents are now submitting pull requests (PRs) to software projects, acting not just as assistants but as autonomous contributors. As these agentic contributions are rapidly increasing across real repositories, little is known about how they behave in practice and why many of them fail to be merged. In this paper, we conduct a large-scale study of 33k agent-authored PRs made by five coding agents across GitHub. (RQ1) We first quantitatively characterize merged and not-merged PRs along four broad dimensions: 1) merge outcomes across task types, 2) code changes, 3) CI build results, and 4) review dynamics. We observe that tasks related to documentation, CI, and build update achieve the highest merge success, whereas performance and bug-fix tasks perform the worst. Not-merged PRs tend to involve larger code changes, touch more files, and often do not pass the project’s CI/CD pipeline validation. (RQ2) To further investigate why some agentic PRs are not merged, we qualitatively analyze 600 PRs to derive a hierarchical taxonomy of rejection patterns. This analysis complements the quantitative findings in RQ1 by uncovering rejection reasons not captured by quantitative metrics, including lack of meaningful reviewer engagement, duplicate PRs, unwanted feature implementations, and agent misalignment. Together, our findings highlight key socio-technical and human-AI collaboration factors that are critical to improving the success of future agentic workflows.
- Open Access
Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
AI coding agents are now submitting pull requests (PRs) to software projects, acting not just as assistants but as autonomous contributors. As these agentic contributions are rapidly increasing across real repositories, little is known about how they behave in practice and why many of them fail to be merged. In this paper, we conduct a large-scale study of 33k agent-authored PRs made by five coding agents across GitHub. (RQ1) We first quantitatively characterize merged and not-merged PRs along four broad dimensions: 1) merge outcomes across task types, 2) code changes, 3) CI build results, and 4) review dynamics. We observe that tasks related to documentation, CI, and build update achieve the highest merge success, whereas performance and bug-fix tasks perform the worst. Not-merged PRs tend to involve larger code changes, touch more files, and often do not pass the project's CI/CD pipeline validation. (RQ2) To further investigate why some agentic PRs are not merged, we qualitatively analyze 600 PRs to derive a hierarchical taxonomy of rejection patterns. This analysis complements the quantitative findings in RQ1 by uncovering rejection reasons not captured by quantitative metrics, including lack of meaningful reviewer engagement, duplicate PRs, unwanted feature implementations, and agent misalignment. Together, our findings highlight key socio-technical and human-AI collaboration factors that are critical to improving the success of future agentic workflows.
Open AccessWhere Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
AI coding agents are now submitting pull requests (PRs) to software projects, acting not just as assistants but as autonomous contributors. As these agentic contributions are rapidly increasing across real repositories, little is known about how they behave in practice and why many of them fail to be merged. In this paper, we conduct a large-scale study of 33k agent-authored PRs made by five coding agents across GitHub. (RQ1) We first quantitatively characterize merged and not-merged PRs along four broad dimensions: 1) merge outcomes across task types, 2) code changes, 3) CI build results, and 4) review dynamics. We observe that tasks related to documentation, CI, and build update achieve the highest merge success, whereas performance and bug-fix tasks perform the worst. Not-merged PRs tend to involve larger code changes, touch more files, and often do not pass the project's CI/CD pipeline validation. (RQ2) To further investigate why some agentic PRs are not merged, we qualitatively analyze 600 PRs to derive a hierarchical taxonomy of rejection patterns. This analysis complements the quantitative findings in RQ1 by uncovering rejection reasons not captured by quantitative metrics, including lack of meaningful reviewer engagement, duplicate PRs, unwanted feature implementations, and agent misalignment. Together, our findings highlight key socio-technical and human-AI collaboration factors that are critical to improving the success of future agentic workflows.
- Open Access
- 7 January 2026
- Published by Oxford University Press (OUP) in Annals of Behavioral Medicine
- Vol. 60 (1)
Feasibility study of an artificial intelligence chatbot-powered cognitive-behavioral lifestyle modification program
Background: Approximately 2 billion adults seek lifestyle modification to improve their health and well-being. However, because of cost and access barriers, only a tiny fraction receive gold-standard lifestyle modification programs led by trained coaches. Digital programs without human coaches are more disseminable but have limited efficacy. Recent advances in large language model (LLM) AI chatbots have the potential to revolutionize digital programs by emulating human coaches, offering a safe, low-cost, and scalable lifestyle modification intervention. Yet, no studies have developed and tested such a program. Purpose: This study aims to evaluate the feasibility, safety, and user engagement of a new AI chatbot-powered lifestyle modification program named Lyra, which utilizes recent advances in LLM technology to potentially emulate the benefits of human coaching. Methods: We developed Lyra, which integrates structured psychoeducational programming, digital tracking of health behaviors, and text messaging, all powered by an LLM engine. We tested the 8-week program, which included interactive training in behavioral strategies, customized meal plans, tailored feedback on goal progress, and on-demand coaching, with 17 participants in a sample of participants with overweight/obesity seeking lifestyle modification. Results: The program was successfully implemented at a cost of $1.78/participant/week. Coders rated 2288 messages from and 475 conversations with Lyra. All were deemed safe, and the mean expertise was 8.5/10. Participants (mean BMI = 31.5) rated Lyra as helpful (69%), empathic (88%), and easy to use (94%); 88% valued having 24-h availability, and 75% developed a connection to Lyra. Participants initiated an average of 6.2 conversations, ranging from coping with cravings to unprompted “confessions” of having consumed unhealthy food. Conclusion: This new form of intervention proved feasible, acceptable, safe, clinically adept, and capable of fostering a connection. However, rigorous testing of Lyra’s effectiveness is necessary.
- 7 January 2026
- Published by Oxford University Press (OUP) in Annals of Behavioral Medicine
- Vol. 60 (1)
Open AccessFeasibility study of an artificial intelligence chatbot-powered cognitive-behavioral lifestyle modification program
Background: Approximately 2 billion adults seek lifestyle modification to improve their health and well-being. However, because of cost and access barriers, only a tiny fraction receive gold-standard lifestyle modification programs led by trained coaches. Digital programs without human coaches are more disseminable but have limited efficacy. Recent advances in large language model (LLM) AI chatbots have the potential to revolutionize digital programs by emulating human coaches, offering a safe, low-cost, and scalable lifestyle modification intervention. Yet, no studies have developed and tested such a program. Purpose: This study aims to evaluate the feasibility, safety, and user engagement of a new AI chatbot-powered lifestyle modification program named Lyra, which utilizes recent advances in LLM technology to potentially emulate the benefits of human coaching. Methods: We developed Lyra, which integrates structured psychoeducational programming, digital tracking of health behaviors, and text messaging, all powered by an LLM engine. We tested the 8-week program, which included interactive training in behavioral strategies, customized meal plans, tailored feedback on goal progress, and on-demand coaching, with 17 participants in a sample of participants with overweight/obesity seeking lifestyle modification. Results: The program was successfully implemented at a cost of $1.78/participant/week. Coders rated 2288 messages from and 475 conversations with Lyra. All were deemed safe, and the mean expertise was 8.5/10. Participants (mean BMI = 31.5) rated Lyra as helpful (69%), empathic (88%), and easy to use (94%); 88% valued having 24-h availability, and 75% developed a connection to Lyra. Participants initiated an average of 6.2 conversations, ranging from coping with cravings to unprompted “confessions” of having consumed unhealthy food. Conclusion: This new form of intervention proved feasible, acceptable, safe, clinically adept, and capable of fostering a connection. However, rigorous testing of Lyra’s effectiveness is necessary.
- Open Access
- 18 November 2025
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (1) , 22
What characteristics make ChatGPT effective for software issue resolution? An empirical study of task, project, and conversational signals in GitHub issues
Conversational large-language models (LLMs), such as ChatGPT, are extensively used for issue resolution tasks, particularly for generating ideas to implement new features or resolve bugs. However, not all developer-LLM conversations are useful for effective issue resolution and it is still unknown what makes some of these conversations not helpful. In this paper, we analyze 686 developer-ChatGPT conversations shared within GitHub issue threads to identify characteristics that make these conversations effective for issue resolution. First, we empirically analyze the conversations and their corresponding issue threads to distinguish helpful from unhelpful conversations. We begin by categorizing the types of tasks developers seek help with (e.g., code generation , bug identification and fixing , test generation ), to better understand the scenarios in which ChatGPT is most effective. Next, we examine a wide range of conversational, project, and issue-related metrics to uncover statistically significant factors associated with helpful conversations. Finally, we identify common deficiencies in unhelpful ChatGPT responses to highlight areas that could inform the design of more effective developer-facing tools. We found that only 62% of the ChatGPT conversations were helpful for successful issue resolution. Among different tasks related to issue resolution, ChatGPT was most helpful in assisting with code generation, and tool/library/API recommendations, but struggled with generating code explanations. Our conversational metrics reveal that helpful conversations are shorter, more readable, and exhibit higher semantic and linguistic alignment. Our project metrics reveal that larger, more popular projects and experienced developers benefit more from ChatGPT’s assistance. Our issue metrics indicate that ChatGPT is more effective on simpler issues characterized by limited developer activity and faster resolution times. These typically involve well-scoped technical problems such as compilation errors and tool feature requests. In contrast, it performs less effectively on complex issues that demand deep project-specific understanding, such as system-level code debugging and refactoring. The most common deficiencies in unhelpful ChatGPT responses include incorrect information and lack of comprehensiveness. Our findings have wide implications including guiding developers on effective interaction strategies for issue resolution, informing the development of tools or frameworks to support optimal prompt design, and providing insights on fine-tuning LLMs for issue resolution tasks.
- 18 November 2025
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (1) , 22
Open AccessWhat characteristics make ChatGPT effective for software issue resolution? An empirical study of task, project, and conversational signals in GitHub issues
Conversational large-language models (LLMs), such as ChatGPT, are extensively used for issue resolution tasks, particularly for generating ideas to implement new features or resolve bugs. However, not all developer-LLM conversations are useful for effective issue resolution and it is still unknown what makes some of these conversations not helpful. In this paper, we analyze 686 developer-ChatGPT conversations shared within GitHub issue threads to identify characteristics that make these conversations effective for issue resolution. First, we empirically analyze the conversations and their corresponding issue threads to distinguish helpful from unhelpful conversations. We begin by categorizing the types of tasks developers seek help with (e.g., code generation , bug identification and fixing , test generation ), to better understand the scenarios in which ChatGPT is most effective. Next, we examine a wide range of conversational, project, and issue-related metrics to uncover statistically significant factors associated with helpful conversations. Finally, we identify common deficiencies in unhelpful ChatGPT responses to highlight areas that could inform the design of more effective developer-facing tools. We found that only 62% of the ChatGPT conversations were helpful for successful issue resolution. Among different tasks related to issue resolution, ChatGPT was most helpful in assisting with code generation, and tool/library/API recommendations, but struggled with generating code explanations. Our conversational metrics reveal that helpful conversations are shorter, more readable, and exhibit higher semantic and linguistic alignment. Our project metrics reveal that larger, more popular projects and experienced developers benefit more from ChatGPT’s assistance. Our issue metrics indicate that ChatGPT is more effective on simpler issues characterized by limited developer activity and faster resolution times. These typically involve well-scoped technical problems such as compilation errors and tool feature requests. In contrast, it performs less effectively on complex issues that demand deep project-specific understanding, such as system-level code debugging and refactoring. The most common deficiencies in unhelpful ChatGPT responses include incorrect information and lack of comprehensiveness. Our findings have wide implications including guiding developers on effective interaction strategies for issue resolution, informing the development of tools or frameworks to support optimal prompt design, and providing insights on fine-tuning LLMs for issue resolution tasks.
- Open Access
AI-Powered Commit Explorer (APCE)
Commit messages in a version control system provide valuable information for developers regarding code changes in software systems. Commit messages can be the only source of information left for future developers describing what was changed and why. However, writing high-quality commit messages is often neglected in practice. Large Language Model (LLM) generated commit messages have emerged as a way to mitigate this issue. We introduce the AI-Powered Commit Explorer (APCE), a tool to support developers and researchers in the use and study of LLM-generated commit messages. APCE gives researchers the option to store different prompts for LLMs and provides an additional evaluation prompt that can further enhance the commit message provided by LLMs. APCE also provides researchers with a straightforward mechanism for automated and human evaluation of LLM-generated messages. Demo link https://youtu.be/zYrJ9s6sZvo
Open AccessAI-Powered Commit Explorer (APCE)
Commit messages in a version control system provide valuable information for developers regarding code changes in software systems. Commit messages can be the only source of information left for future developers describing what was changed and why. However, writing high-quality commit messages is often neglected in practice. Large Language Model (LLM) generated commit messages have emerged as a way to mitigate this issue. We introduce the AI-Powered Commit Explorer (APCE), a tool to support developers and researchers in the use and study of LLM-generated commit messages. APCE gives researchers the option to store different prompts for LLMs and provides an additional evaluation prompt that can further enhance the commit message provided by LLMs. APCE also provides researchers with a straightforward mechanism for automated and human evaluation of LLM-generated messages. Demo link https://youtu.be/zYrJ9s6sZvo
- Open Access
- 13 May 2025
- Published by JMIR Publications Inc. in JMIR Infodemiology
- Vol. 5, e56004
Shifting Narratives in Media Coverage Across a Decade of Drug Discourse in the Philadelphia Inquirer: Qualitative Sentiment Analysis
Background: The media has immense power in shaping public narratives surrounding sensitive topics such as substance use. Its portrayals can unintentionally fuel harmful stereotypes and stigma, negatively impacting individuals struggling with addiction, influencing policy decisions, and hindering broader public health efforts. Objective: This study aimed to examine how the regional newspaper, The Philadelphia Inquirer, covered events related to illicit drug use between 2013 and 2022, focusing on linguistic patterns and themes associated with specific types of substances. Methods: We collected a dataset of 157,476 articles published in The Philadelphia Inquirer between 2013 and 2022 and categorized mentioned substances into 8 classes: stimulants, narcotics, cannabis, hallucinogens, depressants, designer drugs, drugs of concern, and treatment medications. From these 157,476 articles, we identified 3661 (2.32%) that mentioned at least 1 substance with potential for misuse. Using dynamic topic modeling, we analyzed thematic evolution in coverage across different drug classes. We then applied aspect-based sentiment analysis to extract the most significant phrases mentioned in each distinct drug class annually and examined the sentiments around these aspects to understand shifting discourse patterns. Results: Cannabis (1575/3661, 43.02%) and narcotics (1361/3661, 37.17%) dominated the coverage, with 2018 showing peak drug-related reporting (666/3661, 18.19%). Our substance co-occurrence analysis revealed that heroin was most frequently discussed alongside treatment medications (methadone, naloxone, and buprenorphine), reflecting evolving approaches to opioid use disorder. Topic modeling revealed distinct themes across drug classes: legislative and medical aspects dominated cannabis coverage, while narcotics coverage focused heavily on overdose deaths and safe injection sites, particularly during 2017 to 2018. Stimulant coverage centered on feature news and crime-related reporting, while treatment coverage showed an increasing focus on overdose prevention by 2021. The aspect-based sentiment analysis showed that 74.3% (165/222) of extracted aspects were portrayed negatively across all drug classes, with narcotics maintaining consistently negative sentiment throughout the period. However, some drug classes showed notable evolution: hallucinogens demonstrated a marked shift in sentiment score (SS) from negative coverage in 2013 (–0.79 SS) to positive coverage of therapeutic applications by 2021 (+0.47 SS), while cannabis coverage reflected complex societal debates, with industry and business aspects showing strong positive sentiment score peaks (0.64 SS in 2019) even as legislation and policy aspects remained volatile (–0.76 SS in 2013 to 0.61 SS in 2019 and declining to –0.31 SS by 2022). Conclusions: Our analysis revealed a predominance of negative and punitive language in drug-related news coverage, with limited representation of harm reduction principles. While some drug classes, particularly cannabis and hallucinogens, saw evolving narratives toward medical applications and policy reform, coverage of narcotics remained primarily focused on crime and overdose. These findings suggest a need for more balanced reporting that incorporates harm reduction perspectives and avoids potentially stigmatizing language when covering substance use disorders.
- 13 May 2025
- Published by JMIR Publications Inc. in JMIR Infodemiology
- Vol. 5, e56004
Open AccessShifting Narratives in Media Coverage Across a Decade of Drug Discourse in the Philadelphia Inquirer: Qualitative Sentiment Analysis
Background: The media has immense power in shaping public narratives surrounding sensitive topics such as substance use. Its portrayals can unintentionally fuel harmful stereotypes and stigma, negatively impacting individuals struggling with addiction, influencing policy decisions, and hindering broader public health efforts. Objective: This study aimed to examine how the regional newspaper, The Philadelphia Inquirer, covered events related to illicit drug use between 2013 and 2022, focusing on linguistic patterns and themes associated with specific types of substances. Methods: We collected a dataset of 157,476 articles published in The Philadelphia Inquirer between 2013 and 2022 and categorized mentioned substances into 8 classes: stimulants, narcotics, cannabis, hallucinogens, depressants, designer drugs, drugs of concern, and treatment medications. From these 157,476 articles, we identified 3661 (2.32%) that mentioned at least 1 substance with potential for misuse. Using dynamic topic modeling, we analyzed thematic evolution in coverage across different drug classes. We then applied aspect-based sentiment analysis to extract the most significant phrases mentioned in each distinct drug class annually and examined the sentiments around these aspects to understand shifting discourse patterns. Results: Cannabis (1575/3661, 43.02%) and narcotics (1361/3661, 37.17%) dominated the coverage, with 2018 showing peak drug-related reporting (666/3661, 18.19%). Our substance co-occurrence analysis revealed that heroin was most frequently discussed alongside treatment medications (methadone, naloxone, and buprenorphine), reflecting evolving approaches to opioid use disorder. Topic modeling revealed distinct themes across drug classes: legislative and medical aspects dominated cannabis coverage, while narcotics coverage focused heavily on overdose deaths and safe injection sites, particularly during 2017 to 2018. Stimulant coverage centered on feature news and crime-related reporting, while treatment coverage showed an increasing focus on overdose prevention by 2021. The aspect-based sentiment analysis showed that 74.3% (165/222) of extracted aspects were portrayed negatively across all drug classes, with narcotics maintaining consistently negative sentiment throughout the period. However, some drug classes showed notable evolution: hallucinogens demonstrated a marked shift in sentiment score (SS) from negative coverage in 2013 (–0.79 SS) to positive coverage of therapeutic applications by 2021 (+0.47 SS), while cannabis coverage reflected complex societal debates, with industry and business aspects showing strong positive sentiment score peaks (0.64 SS in 2019) even as legislation and policy aspects remained volatile (–0.76 SS in 2013 to 0.61 SS in 2019 and declining to –0.31 SS by 2022). Conclusions: Our analysis revealed a predominance of negative and punitive language in drug-related news coverage, with limited representation of harm reduction principles. While some drug classes, particularly cannabis and hallucinogens, saw evolving narratives toward medical applications and policy reform, coverage of narcotics remained primarily focused on crime and overdose. These findings suggest a need for more balanced reporting that incorporates harm reduction perspectives and avoids potentially stigmatizing language when covering substance use disorders.
- Open Access
- 15 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 515-519
Incivility in Open Source Projects: A Comprehensive Annotated Dataset of Locked GitHub Issue Threads
In the dynamic landscape of open source software (OSS) development, understanding and addressing incivility within issue discussions is crucial for fostering healthy and productive collaborations. This paper presents a curated dataset of 404 locked GitHub issue discussion threads and 5961 individual comments, collected from 213 OSS projects. We annotated the comments with various categories of incivility using Tone Bearing Discussion Features (TBDFs), and, for each issue thread, we annotated the triggers, targets, and consequences of incivility. We observed that Bitter frustration, Impatience, and Mocking are the most prevalent TBDFs exhibited in our dataset. The most common triggers, targets, and consequences of incivility include Failed use of tool/code or error messages, People, and Discontinued further discussion, respectively. This dataset can serve as a valuable resource for analyzing incivility in OSS and improving automated tools to detect and mitigate such behavior.
- 15 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 515-519
Open AccessIncivility in Open Source Projects: A Comprehensive Annotated Dataset of Locked GitHub Issue Threads
In the dynamic landscape of open source software (OSS) development, understanding and addressing incivility within issue discussions is crucial for fostering healthy and productive collaborations. This paper presents a curated dataset of 404 locked GitHub issue discussion threads and 5961 individual comments, collected from 213 OSS projects. We annotated the comments with various categories of incivility using Tone Bearing Discussion Features (TBDFs), and, for each issue thread, we annotated the triggers, targets, and consequences of incivility. We observed that Bitter frustration, Impatience, and Mocking are the most prevalent TBDFs exhibited in our dataset. The most common triggers, targets, and consequences of incivility include Failed use of tool/code or error messages, People, and Discontinued further discussion, respectively. This dataset can serve as a valuable resource for analyzing incivility in OSS and improving automated tools to detect and mitigate such behavior.