Pranav Narayanan Venkit
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Pranav Narayanan Venkit has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Shomir Wilson
Pennsylvania State University, University Park, Pennsylvania, USA
4 shared publications - Clyde Lee Giles
Pennsylvania State University
2 shared publications - Mukund Srinath
Pennsylvania State University, University Park, PA, USA
2 shared publications - Sanjana Gautam
University of Texas, Austin, USA
2 shared publications - Avijit Ghosh
Hugging Face, New York, USA
1 shared publication - Chien-Sheng Wu
Salesforce AI, United States
1 shared publication
Latest Publications
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 13387-13391
Agents in the Wild: Where Research Meets Deployment
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 13387-13391
Open AccessAgents in the Wild: Where Research Meets Deployment
- Open Access
- 25 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 792-815
What if AI systems weren't chatbots?
- 25 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 792-815
Open AccessWhat if AI systems weren't chatbots?
- Open Access
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1325-1340
Search Engines in the AI Era: A Qualitative Understanding to the False Promise of Factual and Verifiable Source-Cited Responses in LLM-based Search
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1325-1340
Open AccessSearch Engines in the AI Era: A Qualitative Understanding to the False Promise of Factual and Verifiable Source-Cited Responses in LLM-based Search
- Open Access
- 7 November 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 8 (CSCW2) , 1-26
Race and Privacy in Broadcast Police Communications
Radios are essential for the operations of modern police departments, and they function as both a collaborative communication technology and a sociotechnical system. However, little prior research has examined their usage or their connections to individual privacy and the role of race in policing, two growing topics of concern in the US. As a case study, we examine the Chicago Police Department's (CPD's) use of broadcast police communications (BPC) to coordinate the activity of law enforcement officers (LEOs) in the city. From a recently assembled archive of 80,775 hours of BPC associated with CPD operations, we analyze human-generated text transcripts of radio transmissions broadcast 9:00 AM to 5:00 PM on August 10th, 2018 in one majority Black, one majority White, and one majority Hispanic area of the city (24 hours of audio) to explore four research questions: (1) Do BPC reflect reported racial disparities in policing? (2) How and when is gender, race/ethnicity, and age mentioned in BPC? (3) To what extent do BPC include sensitive information, and who is put at most risk by this practice? (4) To what extent can large language models (LLMs) heighten this risk? We explore the vocabulary and speech acts used by police in BPC, comparing mentions of personal characteristics to local demographics, the personal information shared over BPC, and the privacy concerns that it poses. Analysis indicates (a) policing professionals in the city of Chicago exhibit disproportionate attention to Black members of the public regardless of context, (b) sociodemographic characteristics like gender, race/ethnicity, and age are primarily mentioned in BPC about event information, and (c) disproportionate attention introduces disproportionate privacy risks for Black members of the public. This study shows BPC can provide a novel window into disproportionate attention (i.e., via radio communications) by law enforcement officers to specific racial groups, leading to increased privacy vulnerability for those groups, particularly Black males.
- 7 November 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 8 (CSCW2) , 1-26
Open AccessRace and Privacy in Broadcast Police Communications
Radios are essential for the operations of modern police departments, and they function as both a collaborative communication technology and a sociotechnical system. However, little prior research has examined their usage or their connections to individual privacy and the role of race in policing, two growing topics of concern in the US. As a case study, we examine the Chicago Police Department's (CPD's) use of broadcast police communications (BPC) to coordinate the activity of law enforcement officers (LEOs) in the city. From a recently assembled archive of 80,775 hours of BPC associated with CPD operations, we analyze human-generated text transcripts of radio transmissions broadcast 9:00 AM to 5:00 PM on August 10th, 2018 in one majority Black, one majority White, and one majority Hispanic area of the city (24 hours of audio) to explore four research questions: (1) Do BPC reflect reported racial disparities in policing? (2) How and when is gender, race/ethnicity, and age mentioned in BPC? (3) To what extent do BPC include sensitive information, and who is put at most risk by this practice? (4) To what extent can large language models (LLMs) heighten this risk? We explore the vocabulary and speech acts used by police in BPC, comparing mentions of personal characteristics to local demographics, the personal information shared over BPC, and the privacy concerns that it poses. Analysis indicates (a) policing professionals in the city of Chicago exhibit disproportionate attention to Black members of the public regardless of context, (b) sociodemographic characteristics like gender, race/ethnicity, and age are primarily mentioned in BPC about event information, and (c) disproportionate attention introduces disproportionate privacy risks for Black members of the public. This study shows BPC can provide a novel window into disproportionate attention (i.e., via radio communications) by law enforcement officers to specific racial groups, leading to increased privacy vulnerability for those groups, particularly Black males.
- Open Access
- 22 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-4
Privacy Now or Never
The General Data Protection Regulation (GDPR) and other recent privacy laws require organizations to post their privacy policies, and place specific expectations on organisations' privacy practices. Privacy policies take the form of documents written in natural language, and one of the expectations placed upon them is that they remain up to date. To investigate legal compliance with this recency requirement at a large scale, we create a novel pipeline that includes crawling, regex-based extraction, candidate date classification and date object creation to extract updated and effective dates from privacy policies written in English. We then analyze patterns in policy dates using four web crawls and find that only about 40% of privacy policies online contain a date, thereby making it difficult to assess their regulatory compliance. We also find that updates in privacy policies are temporally concentrated around passage of laws regulating digital privacy (such as the GDPR), and that more popular domains are more likely to have policy dates as well as more likely to update their policies regularly.
- 22 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-4
Open AccessPrivacy Now or Never
The General Data Protection Regulation (GDPR) and other recent privacy laws require organizations to post their privacy policies, and place specific expectations on organisations' privacy practices. Privacy policies take the form of documents written in natural language, and one of the expectations placed upon them is that they remain up to date. To investigate legal compliance with this recency requirement at a large scale, we create a novel pipeline that includes crawling, regex-based extraction, candidate date classification and date object creation to extract updated and effective dates from privacy policies written in English. We then analyze patterns in policy dates using four web crawls and find that only about 40% of privacy policies online contain a date, thereby making it difficult to assess their regulatory compliance. We also find that updates in privacy policies are temporally concentrated around passage of laws regulating digital privacy (such as the GDPR), and that more popular domains are more likely to have policy dates as well as more likely to update their policies regularly.
- Open Access
- 22 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Privacy Lost and Found
Legal jurisdictions around the world require organisations to post privacy policies on their websites. However, in spite of laws such as GDPR and CCPA reinforcing this requirement, organisations sometimes do not comply, and a variety of semi-compliant failure modes exist. To investigate the landscape of web privacy policies, we crawl the privacy policies from 7 million organisation websites with the goal of identifying when policies are unavailable. We conduct a large-scale investigation of the availability of privacy policies and identify potential reasons for unavailability such as dead links, documents with empty content, documents that consist solely of placeholder text, and documents unavailable in the specific languages offered by their respective websites. We estimate the frequencies of these failure modes and the overall unavailability of privacy policies on the web and find that privacy policies URLs are only available in 34% of websites. Further, 1.37% of these URLs are broken links and 1.23% of the valid links lead to pages without a policy. Further, to enable investigation of privacy policies at scale, we use the capture-recapture technique to estimate the total number of English language privacy policies on the web and the distribution of these documents across top level domains and sectors of commerce. We estimate the lower bound on the number of English language privacy policies to be around 3 million. Finally, we release the CoLIPPs Corpus containing around 600k policies and their metadata consisting of policy URL, length, readability, sector of commerce, and policy crawl date.
- 22 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Open AccessPrivacy Lost and Found
Legal jurisdictions around the world require organisations to post privacy policies on their websites. However, in spite of laws such as GDPR and CCPA reinforcing this requirement, organisations sometimes do not comply, and a variety of semi-compliant failure modes exist. To investigate the landscape of web privacy policies, we crawl the privacy policies from 7 million organisation websites with the goal of identifying when policies are unavailable. We conduct a large-scale investigation of the availability of privacy policies and identify potential reasons for unavailability such as dead links, documents with empty content, documents that consist solely of placeholder text, and documents unavailable in the specific languages offered by their respective websites. We estimate the frequencies of these failure modes and the overall unavailability of privacy policies on the web and find that privacy policies URLs are only available in 34% of websites. Further, 1.37% of these URLs are broken links and 1.23% of the valid links lead to pages without a policy. Further, to enable investigation of privacy policies at scale, we use the capture-recapture technique to estimate the total number of English language privacy policies on the web and the distribution of these documents across top level domains and sectors of commerce. We estimate the lower bound on the number of English language privacy policies to be around 3 million. Finally, we release the CoLIPPs Corpus containing around 600k policies and their metadata consisting of policy URL, length, readability, sector of commerce, and policy crawl date.
- Open Access
- 8 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 554-565
Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articles
- 8 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 554-565
Open AccessUnmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articles
- Open Access
- 8 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1004-1005
Towards a Holistic Approach: Understanding Sociodemographic Biases in NLP Models using an Interdisciplinary Lens
The rapid growth in the usage and applications of Natural Language Processing (NLP) in various sociotechnical solutions has highlighted the need for a comprehensive understanding of bias and its impact on society. While research on bias in NLP has expanded, several challenges persist that require attention. These include the limited focus on sociodemographic biases beyond race and gender, the narrow scope of analysis predominantly centered on models, and the technocentric implementation approaches. This paper addresses these challenges and advocates for a more interdisciplinary approach to understanding bias in NLP. The work is structured into three facets, each exploring a specific aspect of bias in NLP. The first facet focuses on identifying sociodemographic bias in various NLP architectures, emphasizing the importance of considering both the models themselves and human computation to comprehensively understand and identify bias. In the second facet, we delve into the significance of establishing a shared vocabulary across different fields and disciplines involved in NLP. By highlighting the potential bias stemming from a lack of shared understanding, this facet emphasizes the need for interdisciplinary collaboration to bridge the gap and foster a more inclusive and accurate analysis of bias. Finally, the third facet investigates the development of a holistic solution by integrating frameworks from social science disciplines. This approach recognizes the complexity of bias in NLP and advocates for an interdisciplinary framework that goes beyond purely technical considerations, involving social and ethical perspectives to address bias effectively. The first facet includes the following of my published works [6, 7, 8, 9] to provide results into how the importance of understanding the presence of bias in various minority group that has not been in focus in the prior works of bias in NLP. The work also shows the need to create a method that considers both human and AI indicators of bias, showcasing the importance of the first facet of my research. In my study [9], I delve into sentiment analysis and toxicity detection models to identify explicit bias against race, gender, and people with disabilities (PWDs). Through statistical exploration of conversations on social media platforms such as Twitter and Reddit, I gain insights into how disability bias permeates real-world social settings. To quantify explicit sociodemographic bias in sentiment analysis and toxicity analysis models, I create the Bias Identification Test in Sentiment (BITS) corpus1. Applying BITS, I uncover significant biases in popular AIaaS sentiment analysis tools, including TextBlob, VADER, and Google Cloud Natural Language API, as well as toxicity analysis models like Toxic-BERT. Remarkably, all of these models exhibit statistically significant explicit bias against disability, underscoring the need for comprehensive understanding and mitigation of biases affecting such groups. The work also demonstrates the utility of BITS as a model-independent method of identifying bias by focusing on social groups instead. Expanding on this, my next work [8] delves into the realm of implicit bias in NLP models. While some models may not overtly exhibit bias, they can unintentionally perpetuate harmful stereotypes [4]. To measure and identify implicit bias in commonly used embedding and large language models, I propose a methodology to measure social biases in various NLP architectures. Focusing on people with disabilities (PWD) as a group with complex social dynamics, I analyze various word embedding-based and transformer-based LLMs, revealing significant biases against PWDs in all tested models. These findings expose how models trained on extensive corpora tend to favor ableist language, underscoring the urgency of detecting and addressing implicit bias. The above two works look at both the implicit and explicit nature of bias in NLP, showcasing the need to distinguish the efforts placed in understanding them. The results also demonstrate the utility of identifying such biases as it provides context to the black-box nature of such public models. As the field of NLP evolved from embedding-based models to large language models, the way these models are constructed underwent significant changes [5]. However, the concern arises from the fact that these models often reflect a populist viewpoint [1] that perpetuates majority-held ideas rather than objective truths. This difference in perception can lead to biases perpetuated by the majority’s worldview. To explore this aspect, I investigate how LLMs represent nationality and their impact on societal stereotypes [6]. By examining LLM-generated stories for various nationalities, I establish a correlation between sentiment and the population of internet users in a country. The study reveals the unintentional implicit and explicit nationality biases exhibited by GPT-2, with nations having lower internet representation and economic status generating negative sentiment stories and employing a greater number of negative adjectives. Additionally, I explore potential debiasing methods such as adversarial triggering and prompt engineering, demonstrating their efficacy in mitigating stereotype propagation through LLM models. While prior work predominantly relies on automatic indicators like sentiment scores or vector distances to identify bias [3], the next phase of my research emphasizes the importance of understanding biases through the lens of human readers [7], bringing to light the need for a human lens in understanding bias through human-aided indicators and mixed-method identification. By incorporating concepts of social computation, using human evaluation, we gain a better understanding of biases’ potential societal impact within the context of language models. To achieve this, I conduct open-ended interviews and employ qualitative coding and thematic analysis to comprehend...
- 8 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 1004-1005
Open AccessTowards a Holistic Approach: Understanding Sociodemographic Biases in NLP Models using an Interdisciplinary Lens
The rapid growth in the usage and applications of Natural Language Processing (NLP) in various sociotechnical solutions has highlighted the need for a comprehensive understanding of bias and its impact on society. While research on bias in NLP has expanded, several challenges persist that require attention. These include the limited focus on sociodemographic biases beyond race and gender, the narrow scope of analysis predominantly centered on models, and the technocentric implementation approaches. This paper addresses these challenges and advocates for a more interdisciplinary approach to understanding bias in NLP. The work is structured into three facets, each exploring a specific aspect of bias in NLP. The first facet focuses on identifying sociodemographic bias in various NLP architectures, emphasizing the importance of considering both the models themselves and human computation to comprehensively understand and identify bias. In the second facet, we delve into the significance of establishing a shared vocabulary across different fields and disciplines involved in NLP. By highlighting the potential bias stemming from a lack of shared understanding, this facet emphasizes the need for interdisciplinary collaboration to bridge the gap and foster a more inclusive and accurate analysis of bias. Finally, the third facet investigates the development of a holistic solution by integrating frameworks from social science disciplines. This approach recognizes the complexity of bias in NLP and advocates for an interdisciplinary framework that goes beyond purely technical considerations, involving social and ethical perspectives to address bias effectively. The first facet includes the following of my published works [6, 7, 8, 9] to provide results into how the importance of understanding the presence of bias in various minority group that has not been in focus in the prior works of bias in NLP. The work also shows the need to create a method that considers both human and AI indicators of bias, showcasing the importance of the first facet of my research. In my study [9], I delve into sentiment analysis and toxicity detection models to identify explicit bias against race, gender, and people with disabilities (PWDs). Through statistical exploration of conversations on social media platforms such as Twitter and Reddit, I gain insights into how disability bias permeates real-world social settings. To quantify explicit sociodemographic bias in sentiment analysis and toxicity analysis models, I create the Bias Identification Test in Sentiment (BITS) corpus1. Applying BITS, I uncover significant biases in popular AIaaS sentiment analysis tools, including TextBlob, VADER, and Google Cloud Natural Language API, as well as toxicity analysis models like Toxic-BERT. Remarkably, all of these models exhibit statistically significant explicit bias against disability, underscoring the need for comprehensive understanding and mitigation of biases affecting such groups. The work also demonstrates the utility of BITS as a model-independent method of identifying bias by focusing on social groups instead. Expanding on this, my next work [8] delves into the realm of implicit bias in NLP models. While some models may not overtly exhibit bias, they can unintentionally perpetuate harmful stereotypes [4]. To measure and identify implicit bias in commonly used embedding and large language models, I propose a methodology to measure social biases in various NLP architectures. Focusing on people with disabilities (PWD) as a group with complex social dynamics, I analyze various word embedding-based and transformer-based LLMs, revealing significant biases against PWDs in all tested models. These findings expose how models trained on extensive corpora tend to favor ableist language, underscoring the urgency of detecting and addressing implicit bias. The above two works look at both the implicit and explicit nature of bias in NLP, showcasing the need to distinguish the efforts placed in understanding them. The results also demonstrate the utility of identifying such biases as it provides context to the black-box nature of such public models. As the field of NLP evolved from embedding-based models to large language models, the way these models are constructed underwent significant changes [5]. However, the concern arises from the fact that these models often reflect a populist viewpoint [1] that perpetuates majority-held ideas rather than objective truths. This difference in perception can lead to biases perpetuated by the majority’s worldview. To explore this aspect, I investigate how LLMs represent nationality and their impact on societal stereotypes [6]. By examining LLM-generated stories for various nationalities, I establish a correlation between sentiment and the population of internet users in a country. The study reveals the unintentional implicit and explicit nationality biases exhibited by GPT-2, with nations having lower internet representation and economic status generating negative sentiment stories and employing a greater number of negative adjectives. Additionally, I explore potential debiasing methods such as adversarial triggering and prompt engineering, demonstrating their efficacy in mitigating stereotype propagation through LLM models. While prior work predominantly relies on automatic indicators like sentiment scores or vector distances to identify bias [3], the next phase of my research emphasizes the importance of understanding biases through the lens of human readers [7], bringing to light the need for a human lens in understanding bias through human-aided indicators and mixed-method identification. By incorporating concepts of social computation, using human evaluation, we gain a better understanding of biases’ potential societal impact within the context of language models. To achieve this, I conduct open-ended interviews and employ qualitative coding and thematic analysis to comprehend...