DKDaniel Kang

Daniel Kang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Daniel Kang has a total of 28 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    REPRO-Bench: Can Agentic AI Systems Assess the Reproducibility of Social Science Research?

    by Chuxuan Hu, Liyun Zhang, Yeji Lim, Aum Wadhwani, Austin Peters, Daniel Kang

    Assessing the reproducibility of social science papers is essential for promoting rigor in research processes, but manual assessment is costly. With recent advances in agentic AI systems (i.e., AI agents), we seek to evaluate their capability to automate this process. However, existing benchmarks for reproducing research papers (1) focus solely on reproducing results using provided code and data without assessing their consistency with the paper, (2) oversimplify real-world scenarios, and (3) lack necessary diversity in data formats and programming languages. To address these issues, we introduce REPRO-Bench, a collection of 112 task instances, each representing a social science paper with a publicly available reproduction report. The agents are tasked with assessing the reproducibility of the paper based on the original paper PDF and the corresponding reproduction package. REPRO-Bench features end-to-end evaluation tasks on the reproducibility of social science papers with complexity comparable to real-world assessments. We evaluate three representative AI agents on REPRO-Bench, with the best-performing agent achieving an accuracy of only 21.4%. Building on our empirical analysis, we develop REPRO-Agent, which improves the highest accuracy achieved by existing agents by 71%. We conclude that more advanced AI agents should be developed to automate real-world reproducibility assessment. REPRO-Bench is publicly available at https://github.com/uiuc-kang-lab/REPRO-Bench.

  • Open Access

    Analytical Queries for Unstructured Data

    by Daniel Kang

    Publishers of Foundations and Trends, making research accessible

  • Open Access

    REPRO-Bench: Can Agentic AI Systems Assess the Reproducibility of Social Science Research?

    by Chuxuan Hu, Liyun Zhang, Yeji Lim, Aum Wadhwani, Austin Peters, Daniel Kang

    Chuxuan Hu, Liyun Zhang, Yeji Lim, Aum Wadhwani, Austin Peters, Daniel Kang. Findings of the Association for Computational Linguistics: ACL 2025. 2025.


  • Top Cited Papers
    Open Access

    Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents

    by Qiusi Zhan, Richard Fang, Henil Shalin Panchal, Daniel Kang

    Qiusi Zhan, Richard Fang, Henil Shalin Panchal, Daniel Kang. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.


  • Open Access

    Removing RLHF Protections in GPT-4 via Fine-Tuning

    by Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, Daniel Kang

    As large language models (LLMs) have increased in their capabilities, so does their potential for dual use. To reduce harmful outputs, produces and vendors of LLMs have used reinforcement learning with human feedback (RLHF). In tandem, LLM vendors have been increasingly enabling fine-tuning of their most powerful models. However, concurrent work has shown that fine-tuning can remove RLHF protections. We may expect that the most powerful models currently available (GPT-4) are less susceptible to fine-tuning attacks. In this work, we show the contrary: fine-tuning allows attackers to remove RLHF protections with as few as 340 examples and a 95% success rate. These training examples can be automatically generated with weaker models. We further show that removing RLHF protections does not decrease usefulness on non-censored outputs, providing evidence that our fine-tuning strategy does not decrease usefulness despite using weaker models to generate training data. Our results show the need for further research on protections on LLMs.


  • Open Access

    Trustless Audits without Revealing Data or Models

    by Suppakit Waiwitlikhit, Ion Stoica, Yi Sun, Tatsunori Hashimoto, Daniel Kang

    There is an increasing conflict between business incentives to hide models and data as trade secrets, and the societal need for algorithmic transparency. For example, a rightsholder wishing to know whether their copyrighted works have been used during training must convince the model provider to allow a third party to audit the model and data. Finding a mutually agreeable third party is difficult, and the associated costs often make this approach impractical. In this work, we show that it is possible to simultaneously allow model providers to keep their model weights (but not architecture) and data secret while allowing other parties to trustlessly audit model and data properties. We do this by designing a protocol called ZkAudit in which model providers publish cryptographic commitments of datasets and model weights, alongside a zero-knowledge proof (ZKP) certifying that published commitments are derived from training the model. Model providers can then respond to audit requests by privately computing any function F of the dataset (or model) and releasing the output of F alongside another ZKP certifying the correct execution of F. To enable ZkAudit, we develop new methods of computing ZKPs for SGD on modern neural nets for simple recommender systems and image classification models capable of high accuracies on ImageNet. Empirically, we show it is possible to provide trustless audits of DNNs, including copyright, censorship, and counterfactual audits with little to no loss in accuracy.


  • Open Access

    LLM Agents can Autonomously Hack Websites

    by Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, Daniel Kang

    In recent years, large language models (LLMs) have become increasingly capable and can now interact with tools (i.e., call functions), read documents, and recursively call themselves. As a result, these LLMs can now function autonomously as agents. With the rise in capabilities of these agents, recent work has speculated on how LLM agents would affect cybersecurity. However, not much is known about the offensive capabilities of LLM agents. In this work, we show that LLM agents can autonomously hack websites, performing tasks as complex as blind database schema extraction and SQL injections without human feedback. Importantly, the agent does not need to know the vulnerability beforehand. This capability is uniquely enabled by frontier models that are highly capable of tool use and leveraging extended context. Namely, we show that GPT-4 is capable of such hacks, but existing open-source models are not. Finally, we show that GPT-4 is capable of autonomously finding vulnerabilities in websites in the wild. Our findings raise questions about the widespread deployment of LLMs.


  • Top Cited Papers
    Open Access

    InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

    by Qiusi Zhan, Zhixiang Liang, Zifan Ying, Daniel Kang

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, Daniel Kang. Findings of the Association for Computational Linguistics ACL 2024. 2024.


  • Top Cited Papers
    Open Access

    Removing RLHF Protections in GPT-4 via Fine-Tuning

    by Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori B. Hashimoto, Daniel Kang

    Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, Daniel Kang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). 2024.


  • Open Access

    Do hospitals that participate in COVID-19 research differ from non-trial hospitals? A cross-sectional study of US hospitals

    by Daniel Kang, Cher X. Huang, Alexander D. Yuen, Keith C. Norris, Tara Vijayan

    Objectives: To compare hospitals that did and did not participate in clinical trials evaluating potential inpatient COVID-19 therapeutics. Methods: We conducted a cross-sectional study of hospitals participating in trials that were registered on clinicaltrials.gov between April and August 2020. Using the 2019 RAND Hospital Dataset and 2019 American Community Survey, we used logistic regression modeling to compare hospital-level traits including demographic features between trial and non-trial hospitals. Results: We included 488 hospitals that were participating in 298 interventional trials and 4232 non-participating hospitals. After controlling for demographic and other hospital traits, we found that teaching status (OR 2.11, 95% CI 1.52–2.95), higher patient acuity (OR 7.48, 4.39, 13.1), and location in the Northeast (OR 1.83, 95% CI 1.18, 2.85) and in wealthier counties (OR: 1.32, 95% CI 1.16–1.51) were associated with increased odds of trial participation, while being in counties with more White residents was associated with reduced odds (OR 0.98, 95% CI 0.98–0.99). Conclusions: Hospitals participating and not participating in COVID-19 inpatient treatment clinical trials differed in many ways, resulting in important implications for the generalizability of trial data.