ChatGPT as a Competent Enough Judge in Validating Responses from a Divergent Thinking Task

StatusVoR
Dublin Core metadata
Alternative title
Authors
Kucwaj, Hanna
Kroczek, Bartłomiej
Monograph
Monograph (alternative title)
Date
2025-07
Publisher
Journal title
Proceedings of the Annual Meeting of the Cognitive Science Society
Issue
Volume
47
Pages
Pages
446-452
DOI
ISSN
1069-7977
ISSN of series
Weblink
https://escholarship.org/uc/item/0fs5t29x
Access date
2025-07
Abstract PL
Abstract EN
The validation of responses in divergent thinking tasks is a critical yet understandardized step that should precede creativity scoring. However, inconsistencies related to human judges in this step may compromise the reliability of the results. This study introduces a systematic approach using ChatGPT to validate responses in the Alternate Uses Task (AUT) and compares its performance against six human judges. Analyzing 1245 AUT responses for common objects, we evaluated validity based on precisely defined criteria. Human judges exhibited significant variability, achieving unanimous agreement for only 58% of responses, while ChatGPT demonstrated significant alignment with human assessments, reflecting a capacity to replicate aggregated human judgment. These findings underscore the potential of Large Language Models to enhance objectivity and reproducibility in creativity research by automating response validation. We advocate for integrating AI-driven validation protocols into divergent thinking response evaluation and emphasize transparent reporting of criteria to advance methodological rigor in the field.
Abstract other
Keywords PL
Keywords EN
Altenrate Uses Task
divergent thinking
creativity
Large Language Models
ChatGPT
Keywords other
Exhibition title
Place of exhibition (institution)
Exhibition curator
URI
https://share.swps.edu.pl/handle/swps/1677
Type
License type
cc-by
Except as otherwise noted, this item is licensed under the Attribution licence | Permitted use of copyrighted works
Funder
Time range from
Time range to
Contact person name
Related publication
Related publication
Grant/project name
Files
pdf
ChatGPT as a Competent Enough Judge in Validating Responses from a Divergent Thinking Task.pdf(449.22 KB)
Download
Views
Views11
Acquisition Date16.12.2025
Downloads
Downloads0
Acquisition Date20.10.2022
Google Scholar
Google Scholar