Typology of Carrots, Sticks, and Sermons: Tracing Its Influence on the Discipline of Public Policy

StatusVoR
Simple item page
cris.lastimport.scopus2026-02-21T04:17:38Z
dc.abstract.enBackground: The 1998 book "Carrots, Sticks & Sermons: Policy Instruments and Their Evaluation" introduced a seminal typology categorizing policy tools by their mechanisms of influence. Despite its potential to shape policy theory and practice, its broader disciplinary impact and citation patterns remain underexplored. This article evaluates the book's legacy in public policy literature while testing bibliometric methods, including emerging generative artificial intelligence (GAI) tools for research synthesis. Purpose: The study aims to (1) assess the influence of the book on the discourse in the public policy literature and (2) explore the utility and limits of AI-driven tools for tracing scholarly impact. It addresses four questions: the book's citation trends, its presence in major disciplinary texts, citation intent, and GAI's ability to reference the work accurately. Setting: Not applicable. Intervention: Not applicable. Research Design: A mixed-methods approach consisted of: (i) quantitative bibliometrics (citation counts, trends, and keyword analysis), (ii) qualitative content analysis by human researchers (major disciplinary handbooks/textbooks and articles), and (iii) generative AI semantic search and analysis. Data Collection and Analysis: The quantitative analysis covered a corpus of interdisciplinary public policy literature from 1998 to 2024, extracted from citation databases (Scopus, Web of Science, Google Scholar). Qualitative analysis focused on 20 major handbooks and textbooks from public policy discipline (occurrence of references to book and policy instruments typology) and coding of 20 highly cited articles (classification of citation types in terms of intent). Work with GAI focused on semantic search and analysis with various AI platforms. Findings: The findings underscore the typology's enduring relevance while advocating for methodological rigor in impact assessments and GAI-aided research. The book has achieved "classic" status in public policy, evidenced by 1,369 citations on Google Scholar, 924 citations on Scopus, 627 citations on Web of Science, as well as presence in 16 out of 20 major disciplinary handbooks/textbooks. Its typology is foundational but often superficially referenced; only 7 of 20 top articles applied it directly. GAI tools also referred to the book, although frequently misattributed or diluted the source, highlighting risks in automated literature reviews. The authors propose updating the typology of policy instruments to include behavioral tools (e.g., nudges) and stress the need for human verification of GAI outputs.
dc.affiliationInstytut Nauk Społecznych
dc.contributor.authorOlejniczak, Karol
dc.contributor.authorKupiec, Tomasz
dc.contributor.authorWojtowicz, Dominika
dc.date.access2025-05-25
dc.date.accessioned2025-08-01T11:03:48Z
dc.date.available2025-08-01T11:03:48Z
dc.date.created2025-05
dc.date.issued2025-05-25
dc.description.abstract<jats:p>Background: The 1998 book "Carrots, Sticks &amp; Sermons: Policy Instruments and Their Evaluation" introduced a seminal typology categorizing policy tools by their mechanisms of influence. Despite its potential to shape policy theory and practice, its broader disciplinary impact and citation patterns remain underexplored. This article evaluates the book's legacy in public policy literature while testing bibliometric methods, including emerging generative artificial intelligence (GAI) tools for research synthesis.  Purpose: The study aims to (1) assess the influence of the book on the discourse in the public policy literature and (2) explore the utility and limits of AI-driven tools for tracing scholarly impact. It addresses four questions: the book's citation trends, its presence in major disciplinary texts, citation intent, and GAI's ability to reference the work accurately.  Setting: Not applicable. Intervention: Not applicable. Research Design: A mixed-methods approach consisted of: (i) quantitative bibliometrics (citation counts, trends, and keyword analysis), (ii) qualitative content analysis by human researchers (major disciplinary handbooks/textbooks and articles), and (iii) generative AI semantic search and analysis. Data Collection and Analysis: The quantitative analysis covered a corpus of interdisciplinary public policy literature from 1998 to 2024, extracted from citation databases (Scopus, Web of Science, Google Scholar). Qualitative analysis focused on 20 major handbooks and textbooks from public policy discipline (occurrence of references to book and policy instruments typology) and coding of 20 highly cited articles (classification of citation types in terms of intent). Work with GAI focused on semantic search and analysis with various AI platforms. Findings: The findings underscore the typology's enduring relevance while advocating for methodological rigor in impact assessments and GAI-aided research. The book has achieved "classic" status in public policy, evidenced by 1,369 citations on Google Scholar, 924 citations on Scopus, 627 citations on Web of Science, as well as presence in 16 out of 20 major disciplinary handbooks/textbooks. Its typology is foundational but often superficially referenced; only 7 of 20 top articles applied it directly. GAI tools also referred to the book, although frequently misattributed or diluted the source, highlighting risks in automated literature reviews. The authors propose updating the typology of policy instruments to include behavioral tools (e.g., nudges) and stress the need for human verification of GAI outputs.</jats:p>
dc.description.accesstimeat_publication
dc.description.grantnumber2021/43/B/HS5/01935
dc.description.granttitleInformacja zwrotna jako interwencja publiczna. Analiza porównawcza polityk przeciwdziałających marnowaniu żywności i testowanie ich mechanizmów z wykorzystaniem poważnych gier.
dc.description.issue50
dc.description.physical63-74
dc.description.versionfinal_published
dc.description.volume21
dc.identifier.doi10.56645/jmde.v21i50.1171
dc.identifier.issn1556-8180
dc.identifier.urihttps://share.swps.edu.pl/handle/swps/1631
dc.identifier.weblinkhttps://journals.sfu.ca/jmde/index.php/jmde_1/article/view/1171
dc.languageen
dc.pbn.affiliationnauki o polityce i administracji
dc.rightsCC-BY-NC
dc.rights.questionYes_rights
dc.share.articleOPEN_JOURNAL
dc.subject.enpolicy tools
dc.subject.enpolicy instruments
dc.subject.engenerative AI
dc.subject.enbibliometric analysis
dc.subject.enpolicy instruments typology
dc.swps.sciencecloudsend
dc.titleTypology of Carrots, Sticks, and Sermons: Tracing Its Influence on the Discipline of Public Policy
dc.title.journalJournal of MultiDisciplinary Evaluation
dc.typeJournalArticle
dspace.entity.typeArticle