ChatGPT as a Competent Enough Judge in Validating Responses from a Divergent Thinking Task

StatusVoR
Simple item page
dc.abstract.enThe validation of responses in divergent thinking tasks is a critical yet understandardized step that should precede creativity scoring. However, inconsistencies related to human judges in this step may compromise the reliability of the results. This study introduces a systematic approach using ChatGPT to validate responses in the Alternate Uses Task (AUT) and compares its performance against six human judges. Analyzing 1245 AUT responses for common objects, we evaluated validity based on precisely defined criteria. Human judges exhibited significant variability, achieving unanimous agreement for only 58% of responses, while ChatGPT demonstrated significant alignment with human assessments, reflecting a capacity to replicate aggregated human judgment. These findings underscore the potential of Large Language Models to enhance objectivity and reproducibility in creativity research by automating response validation. We advocate for integrating AI-driven validation protocols into divergent thinking response evaluation and emphasize transparent reporting of criteria to advance methodological rigor in the field.
dc.affiliationWydział Psychologii w Krakowie
dc.conference47th Annual Conference of the Cognitive Science Society
dc.conference.countryStany Zjednoczone
dc.conference.coverageinternational
dc.conference.datefinish2025-08-02
dc.conference.datestart2025-07-30
dc.conference.placeSan Francisco
dc.conference.seriesAnnual Conference of the Cognitive Science Society
dc.conference.seriesshortcutCOGSCI
dc.conference.seriesweblinkhttps://cognitivesciencesociety.org/
dc.conference.shortcutCOGSCI 2025
dc.conference.weblinkhttps://cognitivesciencesociety.org/cogsci-2025/
dc.contributor.authorKucwaj, Hanna
dc.contributor.authorKroczek, Bartłomiej
dc.date.access2025-07
dc.date.accessioned2025-08-19T09:56:55Z
dc.date.available2025-08-19T09:56:55Z
dc.date.created2025-04-04
dc.date.issued2025-07
dc.description.accesstimeat_publication
dc.description.physical446-452
dc.description.versionfinal_published
dc.description.volume47
dc.identifier.eissn1069-7977
dc.identifier.issn1069-7977
dc.identifier.urihttps://share.swps.edu.pl/handle/swps/1677
dc.identifier.weblinkhttps://escholarship.org/uc/item/0fs5t29x
dc.languageen
dc.pbn.affiliationpsychologia
dc.rightsCC-BY
dc.rights.questionYes_rights
dc.share.articleOPEN_JOURNAL
dc.subject.enAltenrate Uses Task
dc.subject.endivergent thinking
dc.subject.encreativity
dc.subject.enLarge Language Models
dc.subject.enChatGPT
dc.swps.sciencecloudsend
dc.titleChatGPT as a Competent Enough Judge in Validating Responses from a Divergent Thinking Task
dc.title.journalProceedings of the Annual Meeting of the Cognitive Science Society
dc.typeJournalArticleConference
dspace.entity.typeArticle