Anna Squicciarini
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Anna Squicciarini has a total of 193 co-authors. Here are the top co-authors by number of shared publications:
Elisa Bertino
Purdue University, West Lafayette, IN, USA
34 shared publications- Federica Paci
University of Verona
17 shared publications Dan Lin
Department of Computer Science, Missouri University of Science & Technology, Rolla, MO
16 shared publicationsSmitha Sundareswaran
College of Information Sciences and Technology, Pennsylvania State University
15 shared publicationsChristopher Griffin
Applied Research Laboratory, Pennsylvania State University, State College, Pennsylvania, USA
11 shared publicationsAbhilasha Bhargav-Spantzel
Intel Corporation, Santa Clara, CA
10 shared publications
Latest Publications
- Open Access
- 1 April 2025
- Published by Privacy Enhancing Technologies Symposium Advisory Board in Proceedings on Privacy Enhancing Technologies
- Vol. 2025 (2) , 5-22
Time-Efficient Locally Relevant Geo-Location Privacy Protection
Geo-obfuscation serves as a location privacy protection mechanism (LPPM), enabling mobile users to share obfuscated locations with servers, rather than their exact locations. This method can protect users’ location privacy when data breaches occur on the server side since the obfuscation process is irreversible. To reduce the utility loss caused by data obfuscation, linear programming (LP) is widely employed, which, however, might suffer from a polynomial explosion of decision variables, rendering it impractical in largescale geo-obfuscation applications. In this paper, we propose a new LPPM, called Locally Relevant Geo-obfuscation (LR-Geo), to optimize geo-obfuscation using LP in a time-efficient manner. This is achieved by confining the geoobfuscation calculation for each user exclusively to the locally relevant (LR) locations to the user’s actual location. Given the potential risk of LR locations disclosing a user’s actual whereabouts, we enable users to compute the LP coefficients locally and upload them only to the server, rather than the LR locations. The server then solves the LP problem based on the received coefficients. Furthermore, we refine the LP framework by incorporating an exponential obfuscation mechanism to guarantee the indistinguishability of obfuscation distribution across multiple users. Based on the constraint structure of the LP formulation, we apply Benders’ decomposition to further enhance computational efficiency. Our theoretical analysis confirms that, despite the geo-obfuscation being calculated independently for each user, it still meets geo-indistinguishability constraints across multiple users with high probability. Finally, the experimental results based on a real-world dataset demonstrate that LR-Geo outperforms existing geo-obfuscation methods in computational time, data utility, and privacy preservation.
- 1 April 2025
- Published by Privacy Enhancing Technologies Symposium Advisory Board in Proceedings on Privacy Enhancing Technologies
- Vol. 2025 (2) , 5-22
Open AccessTime-Efficient Locally Relevant Geo-Location Privacy Protection
Geo-obfuscation serves as a location privacy protection mechanism (LPPM), enabling mobile users to share obfuscated locations with servers, rather than their exact locations. This method can protect users’ location privacy when data breaches occur on the server side since the obfuscation process is irreversible. To reduce the utility loss caused by data obfuscation, linear programming (LP) is widely employed, which, however, might suffer from a polynomial explosion of decision variables, rendering it impractical in largescale geo-obfuscation applications. In this paper, we propose a new LPPM, called Locally Relevant Geo-obfuscation (LR-Geo), to optimize geo-obfuscation using LP in a time-efficient manner. This is achieved by confining the geoobfuscation calculation for each user exclusively to the locally relevant (LR) locations to the user’s actual location. Given the potential risk of LR locations disclosing a user’s actual whereabouts, we enable users to compute the LP coefficients locally and upload them only to the server, rather than the LR locations. The server then solves the LP problem based on the received coefficients. Furthermore, we refine the LP framework by incorporating an exponential obfuscation mechanism to guarantee the indistinguishability of obfuscation distribution across multiple users. Based on the constraint structure of the LP formulation, we apply Benders’ decomposition to further enhance computational efficiency. Our theoretical analysis confirms that, despite the geo-obfuscation being calculated independently for each user, it still meets geo-indistinguishability constraints across multiple users with high probability. Finally, the experimental results based on a real-world dataset demonstrate that LR-Geo outperforms existing geo-obfuscation methods in computational time, data utility, and privacy preservation.
- Top Cited PapersOpen Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 29680-29697
The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
Feiran Jia, Tong Wu, Xin Qin, Anna Squicciarini. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 29680-29697
Top Cited PapersOpen AccessThe Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
Feiran Jia, Tong Wu, Xin Qin, Anna Squicciarini. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- Open Access
The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex real-world tasks through tool integration. This enhanced ability to interact with external systems and process various data sources, while powerful, introduces significant security vulnerabilities. In particular, indirect prompt injection attacks pose a critical threat, where malicious instructions embedded within external data sources can manipulate agents to deviate from user intentions. While existing defenses based on rule constraints, source spotlighting, and authentication protocols show promise, they struggle to maintain robust security while preserving task functionality. We propose a novel and orthogonal perspective that reframes agent security from preventing harmful actions to ensuring task alignment, requiring every agent action to serve user objectives. Based on this insight, we develop Task Shield, a test-time defense mechanism that systematically verifies whether each instruction and tool call contributes to user-specified goals. Through experiments on the AgentDojo benchmark, we demonstrate that Task Shield reduces attack success rates (2.07\%) while maintaining high task utility (69.79\%) on GPT-4o.
Open AccessThe Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex real-world tasks through tool integration. This enhanced ability to interact with external systems and process various data sources, while powerful, introduces significant security vulnerabilities. In particular, indirect prompt injection attacks pose a critical threat, where malicious instructions embedded within external data sources can manipulate agents to deviate from user intentions. While existing defenses based on rule constraints, source spotlighting, and authentication protocols show promise, they struggle to maintain robust security while preserving task functionality. We propose a novel and orthogonal perspective that reframes agent security from preventing harmful actions to ensuring task alignment, requiring every agent action to serve user objectives. Based on this insight, we develop Task Shield, a test-time defense mechanism that systematically verifies whether each instruction and tool call contributes to user-specified goals. Through experiments on the AgentDojo benchmark, we demonstrate that Task Shield reduces attack success rates (2.07\%) while maintaining high task utility (69.79\%) on GPT-4o.
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1039-1042
ACCORD: Constraint-driven Mediation of Multi-user Conflicts in Cloud Services
When multiple users adopt collaborative cloud services like Google Drive to work on a shared resource, incorrect or missing permis- sions may cause conflicting or inconsistent access or use privileges. These issues (or conflicts) compromise resources confidentiality, integrity, or availability leading to a lack of trust in cloud services. An example conflict is when a user with editor permissions changes the permissions on a shared resource without consent from the orig- inal resource owner. In this demonstration, we introduce ACCORD, a web application built on top of Google Drive able to detect and resolve multi-user conflicts. ACCORD employs a simulator to help users preemptively identify potential conflicts and assists them in defining action constraints. Using these constraints, ACCORD can automatically detect and resolve any future conflicts.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1039-1042
Open AccessACCORD: Constraint-driven Mediation of Multi-user Conflicts in Cloud Services
When multiple users adopt collaborative cloud services like Google Drive to work on a shared resource, incorrect or missing permis- sions may cause conflicting or inconsistent access or use privileges. These issues (or conflicts) compromise resources confidentiality, integrity, or availability leading to a lack of trust in cloud services. An example conflict is when a user with editor permissions changes the permissions on a shared resource without consent from the orig- inal resource owner. In this demonstration, we introduce ACCORD, a web application built on top of Google Drive able to detect and resolve multi-user conflicts. ACCORD employs a simulator to help users preemptively identify potential conflicts and assists them in defining action constraints. Using these constraints, ACCORD can automatically detect and resolve any future conflicts.
- 3 January 2024
- Published by Springer Nature in Social Network Analysis and Mining
- Vol. 14 (1) , 1-23
A longitudinal dataset and analysis of Twitter ISIS users and propaganda
- 3 January 2024
- Published by Springer Nature in Social Network Analysis and Mining
- Vol. 14 (1) , 1-23
A longitudinal dataset and analysis of Twitter ISIS users and propaganda
- Open Access
- 21 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 619-628
RoCourseNet: Robust Training of a Prediction Aware Recourse Model
Counterfactual (CF) explanations for machine learning (ML) models are preferred by end-users, as they explain the predictions of ML models by providing a recourse (or contrastive) case to individuals who are adversely impacted by predicted outcomes. Existing CF explanation methods generate recourses under the assumption that the underlying target ML model remains stationary over time. However, due to commonly occurring distributional shifts in training data, ML models constantly get updated in practice, which might render previously generated recourses invalid and diminish end-users trust in our algorithmic framework. To address this problem, we propose RoCourseNet, a training framework that jointly optimizes predictions and recourses that are robust to future data shifts. This work contains four key contributions: (1) We formulate the robust recourse generation problem as a tri-level optimization problem which consists of two sub-problems: (i) a bi-level problem that finds the worst-case adversarial shift in the training data, and (ii) an outer minimization problem to generate robust recourses against this worst-case shift. (2) We leverage adversarial training to solve this tri-level optimization problem by: (i) proposing a novel virtual data shift (VDS) algorithm to find worst-case shifted ML models via explicitly considering the worst-case data shift in the training dataset, and (ii) a block-wise coordinate descent procedure to optimize for prediction and corresponding robust recourses. (3) We evaluate RoCourseNet's performance on three real-world datasets, and show that RoCourseNet consistently achieves more than 96% robust validity and outperforms state-of-the-art baselines by at least 10% in generating robust CF explanations. (4) Finally, we generalize the RoCourseNet framework to accommodate any parametric post-hoc methods for improving robust validity.
- 21 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 619-628
Open AccessRoCourseNet: Robust Training of a Prediction Aware Recourse Model
Counterfactual (CF) explanations for machine learning (ML) models are preferred by end-users, as they explain the predictions of ML models by providing a recourse (or contrastive) case to individuals who are adversely impacted by predicted outcomes. Existing CF explanation methods generate recourses under the assumption that the underlying target ML model remains stationary over time. However, due to commonly occurring distributional shifts in training data, ML models constantly get updated in practice, which might render previously generated recourses invalid and diminish end-users trust in our algorithmic framework. To address this problem, we propose RoCourseNet, a training framework that jointly optimizes predictions and recourses that are robust to future data shifts. This work contains four key contributions: (1) We formulate the robust recourse generation problem as a tri-level optimization problem which consists of two sub-problems: (i) a bi-level problem that finds the worst-case adversarial shift in the training data, and (ii) an outer minimization problem to generate robust recourses against this worst-case shift. (2) We leverage adversarial training to solve this tri-level optimization problem by: (i) proposing a novel virtual data shift (VDS) algorithm to find worst-case shifted ML models via explicitly considering the worst-case data shift in the training dataset, and (ii) a block-wise coordinate descent procedure to optimize for prediction and corresponding robust recourses. (3) We evaluate RoCourseNet's performance on three real-world datasets, and show that RoCourseNet consistently achieves more than 96% robust validity and outperforms state-of-the-art baselines by at least 10% in generating robust CF explanations. (4) Finally, we generalize the RoCourseNet framework to accommodate any parametric post-hoc methods for improving robust validity.
- Open Access
Consensus in Complex Networks with Noisy Agents and Peer Pressure
In this paper we study a discrete time consensus model on a connected graph with monotonically increasing peer-pressure and noise perturbed outputs masking a hidden state. We assume that each agent maintains a constant hidden state and a presents a dynamic output that is perturbed by random noise drawn from a mean-zero distribution. We show consensus is ensured in the limit as time goes to infinity under certain assumptions on the increasing peer-pressure term and also show that the hidden state cannot be exactly recovered even when model dynamics and outputs are known. The exact nature of the distribution is computed for a simple two vertex graph and results found are shown to generalize (empirically) to more complex graph structures.
Open AccessConsensus in Complex Networks with Noisy Agents and Peer Pressure
In this paper we study a discrete time consensus model on a connected graph with monotonically increasing peer-pressure and noise perturbed outputs masking a hidden state. We assume that each agent maintains a constant hidden state and a presents a dynamic output that is perturbed by random noise drawn from a mean-zero distribution. We show consensus is ensured in the limit as time goes to infinity under certain assumptions on the increasing peer-pressure term and also show that the hidden state cannot be exactly recovered even when model dynamics and outputs are known. The exact nature of the distribution is computed for a simple two vertex graph and results found are shown to generalize (empirically) to more complex graph structures.
- Open Access
User Customizable and Robust Geo-Indistinguishability for Location Privacy
Location obfuscation functions generated by existing systems for ensuring location privacy are monolithic and do not allow users to customize their obfuscation range. This can lead to the user being mapped in undesirable locations (e.g., shady neighborhoods) to the location-requesting services. Modifying the obfuscation function generated by a centralized server on the user side can result in poor privacy as the original function is not robust against such updates. Users themselves might find it challenging to understand the parameters involved in obfuscation mechanisms (e.g., obfuscation range and granularity of location representation) and therefore struggle to set realistic trade-offs between privacy, utility, and customization. In this paper, we propose a new framework called, CORGI, i.e., CustOmizable Robust Geo-Indistinguishability, which generates location obfuscation functions that are robust against user customization while providing strong privacy guarantees based on the Geo-Indistinguishability paradigm. CORGI utilizes a tree representation of a given region to assist users in specifying their privacy and customization requirements. The server side of CORGI takes these requirements as inputs and generates an obfuscation function that satisfies Geo-Indistinguishability requirements and is robust against customization on the user side. The obfuscation function is returned to the user who can then choose to update the obfuscation function (e.g., obfuscation range, granularity of location representation). The experimental results on a real dataset demonstrate that CORGI can efficiently generate obfuscation matrices that are more robust to the customization by users.
Open AccessUser Customizable and Robust Geo-Indistinguishability for Location Privacy
Location obfuscation functions generated by existing systems for ensuring location privacy are monolithic and do not allow users to customize their obfuscation range. This can lead to the user being mapped in undesirable locations (e.g., shady neighborhoods) to the location-requesting services. Modifying the obfuscation function generated by a centralized server on the user side can result in poor privacy as the original function is not robust against such updates. Users themselves might find it challenging to understand the parameters involved in obfuscation mechanisms (e.g., obfuscation range and granularity of location representation) and therefore struggle to set realistic trade-offs between privacy, utility, and customization. In this paper, we propose a new framework called, CORGI, i.e., CustOmizable Robust Geo-Indistinguishability, which generates location obfuscation functions that are robust against user customization while providing strong privacy guarantees based on the Geo-Indistinguishability paradigm. CORGI utilizes a tree representation of a given region to assist users in specifying their privacy and customization requirements. The server side of CORGI takes these requirements as inputs and generates an obfuscation function that satisfies Geo-Indistinguishability requirements and is robust against customization on the user side. The obfuscation function is returned to the user who can then choose to update the obfuscation function (e.g., obfuscation range, granularity of location representation). The experimental results on a real dataset demonstrate that CORGI can efficiently generate obfuscation matrices that are more robust to the customization by users.
- Open Access
- 31 August 2022
- Published by Association for Computing Machinery (ACM) in ACM Transactions on the Web
- Vol. 16 (3) , 1-23
An Extended Ultimatum Game for Multi-Party Access Control in Social Networks
In this article, we aim to answer an important set of questions about the potential longitudinal effects of repeated sharing and privacy settings decisions over jointly managed content among users in a social network. We model user interactions through a repeated game in a network graph. We present a variation of the one-shot Ultimatum Game, wherein individuals interact with peers to make a decision on a piece of shared content. The outcome of this game is either success or failure, wherein success implies that a satisfactory decision for all parties is made and failure instead implies that the parties could not reach an agreement. Our proposed game is grounded in empirical data about individual decisions in repeated pairwise negotiations about jointly managed content in a social network. We consider both a “continuous” privacy model as well the “discrete” case of a model wherein privacy values are to be chosen among a fixed set of options. We formally demonstrate that over time, the system converges toward a “fair” state, wherein each individual’s preferences are accounted for. Our discrete model is validated by way of a user study, where participants are asked to propose privacy settings for own shared content from a small, discrete set of options.
- 31 August 2022
- Published by Association for Computing Machinery (ACM) in ACM Transactions on the Web
- Vol. 16 (3) , 1-23
Open AccessAn Extended Ultimatum Game for Multi-Party Access Control in Social Networks
In this article, we aim to answer an important set of questions about the potential longitudinal effects of repeated sharing and privacy settings decisions over jointly managed content among users in a social network. We model user interactions through a repeated game in a network graph. We present a variation of the one-shot Ultimatum Game, wherein individuals interact with peers to make a decision on a piece of shared content. The outcome of this game is either success or failure, wherein success implies that a satisfactory decision for all parties is made and failure instead implies that the parties could not reach an agreement. Our proposed game is grounded in empirical data about individual decisions in repeated pairwise negotiations about jointly managed content in a social network. We consider both a “continuous” privacy model as well the “discrete” case of a model wherein privacy values are to be chosen among a fixed set of options. We formally demonstrate that over time, the system converges toward a “fair” state, wherein each individual’s preferences are accounted for. Our discrete model is validated by way of a user study, where participants are asked to propose privacy settings for own shared content from a small, discrete set of options.
- Open Access
Automated Detection of Doxing on Twitter
Doxing refers to the practice of disclosing sensitive personal information about a person without their consent. This form of cyberbullying is an unpleasant and sometimes dangerous phenomenon for online social networks. Although prior work exists on automated identification of other types of cyberbullying, a need exists for methods capable of detecting doxing on Twitter specifically. We propose and evaluate a set of approaches for automatically detecting second- and third-party disclosures on Twitter of sensitive private information, a subset of which constitutes doxing. We summarize our findings of common intentions behind doxing episodes and compare nine different approaches for automated detection based on string-matching and one-hot encoded heuristics, as well as word and contextualized string embedding representations of tweets. We identify an approach providing 96.86% accuracy and 97.37% recall using contextualized string embeddings and conclude by discussing the practicality of our proposed methods.
Open AccessAutomated Detection of Doxing on Twitter
Doxing refers to the practice of disclosing sensitive personal information about a person without their consent. This form of cyberbullying is an unpleasant and sometimes dangerous phenomenon for online social networks. Although prior work exists on automated identification of other types of cyberbullying, a need exists for methods capable of detecting doxing on Twitter specifically. We propose and evaluate a set of approaches for automatically detecting second- and third-party disclosures on Twitter of sensitive private information, a subset of which constitutes doxing. We summarize our findings of common intentions behind doxing episodes and compare nine different approaches for automated detection based on string-matching and one-hot encoded heuristics, as well as word and contextualized string embedding representations of tweets. We identify an approach providing 96.86% accuracy and 97.37% recall using contextualized string embeddings and conclude by discussing the practicality of our proposed methods.