Typology of Carrots, Sticks, and Sermons: Tracing Its Influence on the Discipline of Public Policy
Open Access
- 25 May 2025
- journal article
- Published by The Evaluation Center at Western Michigan University in Journal of Multidisciplinary Evaluation
- Vol. 21 (50) , 63-74
Abstract
Background: The 1998 book "Carrots, Sticks & Sermons: Policy Instruments and Their Evaluation" introduced a seminal typology categorizing policy tools by their mechanisms of influence. Despite its potential to shape policy theory and practice, its broader disciplinary impact and citation patterns remain underexplored. This article evaluates the book's legacy in public policy literature while testing bibliometric methods, including emerging generative artificial intelligence (GAI) tools for research synthesis. Purpose: The study aims to (1) assess the influence of the book on the discourse in the public policy literature and (2) explore the utility and limits of AI-driven tools for tracing scholarly impact. It addresses four questions: the book's citation trends, its presence in major disciplinary texts, citation intent, and GAI's ability to reference the work accurately. Setting: Not applicable. Intervention: Not applicable. Research Design: A mixed-methods approach consisted of: (i) quantitative bibliometrics (citation counts, trends, and keyword analysis), (ii) qualitative content analysis by human researchers (major disciplinary handbooks/textbooks and articles), and (iii) generative AI semantic search and analysis. Data Collection and Analysis: The quantitative analysis covered a corpus of interdisciplinary public policy literature from 1998 to 2024, extracted from citation databases (Scopus, Web of Science, Google Scholar). Qualitative analysis focused on 20 major handbooks and textbooks from public policy discipline (occurrence of references to book and policy instruments typology) and coding of 20 highly cited articles (classification of citation types in terms of intent). Work with GAI focused on semantic search and analysis with various AI platforms. Findings: The findings underscore the typology's enduring relevance while advocating for methodological rigor in impact assessments and GAI-aided research. The book has achieved "classic" status in public policy, evidenced by 1,369 citations on Google Scholar, 924 citations on Scopus, 627 citations on Web of Science, as well as presence in 16 out of 20 major disciplinary handbooks/textbooks. Its typology is foundational but often superficially referenced; only 7 of 20 top articles applied it directly. GAI tools also referred to the book, although frequently misattributed or diluted the source, highlighting risks in automated literature reviews. The authors propose updating the typology of policy instruments to include behavioral tools (e.g., nudges) and stress the need for human verification of GAI outputs.Keywords
This publication has 41 references indexed in Scilit:
- Encyclopedia of Public Policy2023
- Global Encyclopedia of Public Administration, Public Policy, and Governance2022
- Does Environmental Information Disclosure Benefit Waste Discharge Reduction? Evidence from ChinaJournal of Business Ethics, 2014
- Towards a Systematic Framework for the Analysis of Environmental Policy IntegrationEnvironmental Policy and Governance, 2014
- Visualizing energy consumption activities as a tool for making everyday life more sustainableApplied Energy, 2011
- The devil is in the details: Household electricity saving behavior and the role of informationEnergy Policy, 2010
- The role of governments in corporate social responsibility: characterising public policies on CSR in EuropePolicy Sciences, 2009
- Governance modes, policy regimes and operational plans: A multi-level nested model of policy instrument choice and policy designPolicy Sciences, 2009
- The Tools of Government in the Digital Age2007
- Sustainability indicators, policy and governance: Issues for ecological economicsEcological Economics, 2006