Arjun Arunasalam

E-commerce websites are targets of abusive individuals, though it is difficult to understand the methods and tools these individuals employ. We introduce a methodology and case study that leverage online communities as data sources to identify and analyze harmful activities.

Mental wellbeing technologies are prevalent in digital spaces, such as content creation websites (e.g., YouTube) and mobile apps. Many users leverage such technologies and thus develop expectations for what they should provide. However, tools to verify whether these technologies conform to user expectations remain largely unexplored. We investigate this problem in the domain of binaural beats - a popular mental wellbeing technology. Using results from preliminary research, where we establish commonly held user expectations and introduce a method to measure expectation conformance, we explore the design of a tool to be used by binaural beats listeners. Through a pre-design survey with 43 participants and a user study with 28 participants, we demonstrate how tools leveraging commonly held user expectations can help users of mental wellbeing applications make informed decisions.

Users seek security & privacy (S&P) advice from online resources, including trusted websites and content-sharing platforms. These resources help users understand S&P technologies and tools and suggest actionable strategies. Large Language Models (LLMs) have recently emerged as trusted information sources. However, their accuracy and correctness have been called into question. Prior research has outlined the shortcomings of LLMs in answering multiple-choice questions and user ability to inadvertently circumvent model restrictions (e.g., to produce toxic content). Yet, the ability of LLMs to provide reliable S&P advice is not well-explored. In this paper, we measure their ability to refute popular S&P misconceptions that the general public holds. We first study recent academic literature to curate a dataset of over a hundred S&P-related misconceptions across six different topics. We then query two popular LLMs (Bard and ChatGPT) and develop a labeling guide to evaluate their responses to these misconceptions. To comprehensively evaluate their responses, we further apply three strategies: query each misconception multiple times, generate and query their paraphrases, and solicit source URLs of the responses. Both models demonstrate, on average, a 21.3% non-negligible error rate, incorrectly supporting popular S&P misconceptions. The error rate increases to 32.6% when we repeatedly query LLMs with the same or paraphrased misconceptions. We also expose that models may partially support a misconception or remain noncommittal, refusing a firm stance on misconceptions. Our exploration of information sources for responses revealed that LLMs are susceptible to providing invalid URLs ( for Bard and for ChatGPT) or point to unrelated sources ( returned by Bard and by ChatGPT). Our findings highlight that existing LLMs are not completely reliable for S&P advice and motivate future work in understanding how users can better interact with this technology.