生成式人工智能对批判性思维的影响调查报告

TheImpactofGenerativeAIonCriticalThinking:Self-Reported

ReductionsinCognitiveEffortandConfidenceEffectsFroma

SurveyofKnowledgeWorkers

Hao-Ping(Hank)Lee

CarnegieMellonUniversity

Pittsburgh,Pennsylvania,USA

haopingl@

AdvaitSarkar

MicrosoftResearch

Cambridge,UnitedKingdom

advait@

LevTankelevitch

MicrosoftResearch

Cambridge,UnitedKingdom

levt@

IanDrosos

MicrosoftResearch

Cambridge,UnitedKingdom

t-iandrosos@

SeanRintel

MicrosoftResearch

Cambridge,UnitedKingdom

serintel@

RichardBanks

MicrosoftResearchCambridge

Cambridge,UnitedKingdom

rbanks@

NicholasWilson

MicrosoftResearch

Cambridge,UnitedKingdom

niwilson@

Abstract

TheriseofGenerativeAI(GenAI)inknowledgeworkflowsraisesquestionsaboutitsimpactoncriticalthinkingskillsandpractices.Wesurvey319knowledgeworkerstoinvestigate1)whenandhowtheyperceivetheenactionofcriticalthinkingwhenusingGenAI,and2)whenandwhyGenAIafectstheireforttodoso.Participantsshared936first-handexamplesofusingGenAIinworktasks.Quantitatively,whenconsideringbothtask-anduser-specificfactors,auser’stask-specificself-confidenceandconfidenceinGenAIarepredictiveofwhethercriticalthinkingisenactedandtheefortofdoingsoinGenAI-assistedtasks.Specifically,higherconfidenceinGenAIisassociatedwithlesscriticalthinking,whilehigherself-confidenceisassociatedwithmorecriticalthinking.Qualitatively,GenAIshiftsthenatureofcriticalthinkingtowardinformationverification,responseintegration,andtaskstewardship.OurinsightsrevealnewdesignchallengesandopportunitiesfordevelopingGenAItoolsforknowledgework.

CCSConcepts

•Human-centeredcomputing→EmpiricalstudiesinHCI.

Keywords

Criticalthinking,GenerativeAItools,Knowledgeworker,Bloom’staxonomy,Survey

ACMReferenceFormat:

Hao-Ping(Hank)Lee,AdvaitSarkar,LevTankelevitch,IanDrosos,SeanRintel,RichardBanks,andNicholasWilson.2025.TheImpactofGenerativeAIonCriticalThinking:Self-ReportedReductionsinCognitiveEfortand

ThisworkislicensedunderaCreativeCommonsAttribution4.0InternationalLicense.

CHI’25,Yokohama,Japan

®2025Copyrightheldbytheowner/author(s).

ACMISBN979-8-4007-1394-1/25/04

/10.1145/3706598.3713778

ConfidenceEfectsFromaSurveyofKnowledgeWorkers.InCHIConferenceonHumanFactorsinComputingSystems(CHI’25),April26–May01,2025,Yokohama,Japan.ACM,NewYork,NY,USA,

23

pages.

/10

.

1145/3706598.3713778

1Introduction

GenerativeAI(GenAI)tools,definedasany“endusertool[...]whosetechnicalimplementationincludesagenerativemodelbasedondeeplearning”,

1

arethelatestinalonglineoftechnologiesthatraisequestionsabouttheirimpactonthequalityofhumanthought,alinethatincludeswriting(objectedtobySocrates),printing(objectedtobyTrithemius),calculators(objectedtobyteachersofarithmetic),andtheInternet.

Suchconsternationisnotunfounded.Usedimproperly,technolo-giescananddoresultinthedeteriorationofcognitivefacultiesthatoughttobepreserved.AsBainbridge[

7

]noted,akeyironyofautomationisthatbymechanisingroutinetasksandleavingexception-handlingtothehumanuser,youdeprivetheuseroftheroutineopportunitiestopracticetheirjudgementandstrengthentheircognitivemusculature,leavingthematrophiedandunpre-paredwhentheexceptionsdoarise.

Inresponse,researchhasbegunlookingcloselyathowdiferentactivitiesareimpactedbyGenAIandtheextenttowhichcognitiveoffloading[

8

]occurs,andwhetherthismaybeanundesirablething.Someworkhasfocused,forinstance,onstudyingtheefectsofGenAIuseonmemory(e.g.,[

1

,

106

])andoncreativity(e.g.,[

28

,

100

]).Moreover,designresearchhasalsobeendevelopinginterventionsthatimprovetheabilityofpeopletothinkincertainways(e.g.,[

24

]).WereviewtheselinesofworkinSection

2

.

Inthispaper,wefocusonahigher-levelconceptthatcapturesan-otheraspectofthoughtconsidereddesirableandworthyofpreser-vation:criticalthinking(definedinSection

2

).Theefectoftheuse

1Whilethereisnobroadconsensusonhowtodefinethisnow-commonterm,forclarityweadoptthisdefinition,arationaleforwhichisgivenin

[115

].

©Owner/Author2025.Thisistheauthor'sversionofthework.Itispostedhereforyourpersonaluse.Notforredistribution.ThedefinitiveversionwaspublishedinCHI2025,

/10.1145/3706598.3713778

.

CHI’25,April26–May01,2025,Yokohama,JapanLeeetal.

ofGenAItoolsoncriticalthinking,asadirectobjectofinquiry,hasnotyetbeenexplored.

Moreover,wefocusoncriticalthinkingforknowledgework(asconceptualisedbyDrucker[

30

]andKidd[

67

]).MuchresearchontheefectofGenAIonthinkingskillsisfocusedoneducationalsettings,whereconcernforskillcultivationismostacute(e.g.,theefectofGenAIcodecompletiontoolsonprogrammingandcomputerscienceeducation[

107

]).Aspreviouslynoted[

116

,

119

],criticalthinkinghasbeenoperationalisedindetailincertainspe-cificdisciplines,suchasacademichistory,clinicalpsychology,andnursing.ButtheostensibleshiftsincriticalthinkingbehavioursbroughtaboutbyGenAIextendtoabroadsetofprofessionsandknowledgeworkflows—GenAItoolsarenowwidelyusedinknowl-edgework[

13

]—andlittleisknownaboutthecriticalthinkingdemandsofthese.Welackbroad-basedempiricalexamplesofwhatkindsofknowledgeworkactivitiesareconsideredbyprofessionalstorequirecriticalthinking.

RecentworkhasmotivatedtheneedforcriticalthinkingsupportinAI-assistedknowledgework[

116

,

119

].ItismotivatedprimarilybytheobservationofthetendencyofAI-assistedknowledgework-flowstobesubjectto“mechanisedconvergence”[

114

],i.e.,thatuserswithaccesstoGenAItoolsproducealessdiversesetofout-comesforthesametask,comparedtothosewithout.Thistendencyforconvergencereflectsalackofpersonal,contextualised,criticalandreflectivejudgementofAIoutputandthuscanbeinterpretedasadeteriorationofcriticalthinking.

However,welackdirectempiricalevidenceforaninterpretationthatpositsaconnectionbetweenmechanisedconvergenceandcriticalthinking.Outputdiversityisaproxyforcriticalthinking,andaflawedone.Forinstance,userswhoreuseGenAIoutputwith-outeditingitmayhavenonethelessperformedacritical,reflectivejudgementinformingthedecisionnottoeditit.Suchreflectivethinkingisinvisibletomeasuresthatfocusonlyontheultimateartefactproduced.WithoutknowinghowknowledgeworkersenactcriticalthinkingwhenusingGenAIandtheassociatedchallenges,weriskcreatinginterventionsthatdonotaddressworkers’realneeds.

Inthispaper,weaimtoaddressthisgapbyconductingasurvey

assettingcleargoals,refiningprompts,andassessingAI-generatedcontenttomeetspecificcriteriaandstandards.Theirreflectiveapproachinvolvesverifyingoutputsagainstexternalsourcesandtheirownexpertise,especiallyintasksthatrequirehigheraccuracy.

Thedataidentifykeymotivatorsforcriticalthinking:thedesiretoenhanceworkquality,avoidnegativeoutcomes,anddevelopskills.However,severalbarriersinhibitthisreflectiveprocess,in-cludinglackofawareness,limitedmotivationduetotimepressureorjobscope,anddifficultyimprovingAIresponsesinunfamiliardomains.Surprisingly,whileAIcanimproveefficiency,itmayalsoreducecriticalengagement,particularlyinroutineorlower-stakestasksinwhichuserssimplyrelyonAI,raisingconcernsaboutlong-termrelianceanddiminishedindependentproblem-solving.

RegardingRQ2(Section

5

),GenAItoolsappeartoreducetheper-ceivedefortrequiredforcriticalthinkingtasksamongknowledgeworkers,especiallywhentheyhavehigherconfidenceinAIcapa-bilities.However,workerswhoareconfidentintheirownskillstendtoperceivegreaterefortinthesetasks,particularlywhenevaluatingandapplyingAIresponses.

ThedatashowsashiftincognitiveefortasknowledgeworkersincreasinglymovefromtaskexecutiontooversightwhenusingGenAI.Whilethisshift“frommaterialproductiontocriticalin-tegration”hasbeenobservedinpriorstudies[

114

],suchstudiesaretypicallycontrolledstudiesinnarrowdomainswithsmallpar-ticipantsamples.Ourdataprovidescomplementaryevidencethatthisalsooccursinreal-worlduseofGenAItools,acrossawidevarietyoftasksandprofessions.Fortaskslikeknowledgeretrieval,AIreducesefortbyautomatinginformationgathering,butwork-ersmustnowinvestmoreinverifyingtheaccuracyofAIoutputs.Similarly,whileAIsimplifiescontentcreation,workersstillneedtospendtimealigningoutputswithspecificneedsandqualitystandards.

Ourpapermakesthefollowingcontributions:

ofaprofessionallydiversesetofknowledgeworkers(n=319),elicitingdetailedreal-worldexamplesoftasks(936)forwhichtheyuseGenAI,anddirectlymeasuringtheirperceptionsofcriticalthinkingduringthesetasks:wheniscriticalthinkingnecessary,howiscriticalthinkingenacted,whetherGenAItoolsafecttheefortofcriticalthinking,andtowhatextent(Section

3

).Wefocuson“enaction”(i.e.,actionsthataresignalsormanifestations)ofcriticalthinkingratherthancriticalthinkingperse,becausecriticalthinkingitselfasapurementalphenomenonisdifficultforpeopletoself-observe,reflecton,andreport.

Concretely,weaimtoanswertworesearchquestions:

RQ1WhenandhowdoknowledgeworkersperceivetheenactionofcriticalthinkingwhenusingGenAI?

RQ2Whenandwhydoknowledgeworkersperceiveincreased/decreasedefortforcriticalthinkingduetoGenAI?

WithrespecttoRQ1(Section

4

),thestudyrevealsthatknowledgeworkersengageincriticalthinkingwhenusingGenAItoolsprimar-ilytoensurethequalityoftheirwork.Theydefinecriticalthinking

•Wereviewtheliteratureoninteractiondesigninterventionsforcriticalthinking,andstudiesoftheefectsofautomationonknowledgeworkflows(Section

2

).

•Wedescribethedevelopmentanddeploymentofasurveyforgatheringempiricalevidenceforknowledgeworkers’experiencesandperceptionsoftheefectofGenAIoncrit-icalthinking(Section

3

).WefindthatGenAItoolsreducetheperceivedefortofcriticalthinkingwhilealsoencour-agingover-relianceonAI,withconfidenceinthetooloftendiminishingindependentproblem-solving.AsworkersshiftfromtaskexecutiontoAIoversight,theytradehands-onengagementforthechallengeofverifyingandeditingAIoutputs,revealingboththeefficiencygainsandtherisksofdiminishedcriticalreflection(Sections

4

and

5

).

•Drawingfromoursurveyinsights,wehighlighthowtheuseofGenAItoolscreatesnewchallengesforcriticalthink-ing.WeoutlineimplicationsfordesigningGenAItosupportknowledgeworkerstoenhancetheirawareness,motivation,andabilitytothinkcritically(Section

6

).

TheImpactofGenerativeAIonCriticalThinkingCHI’25,April26–May01,2025,Yokohama,Japan

2RelatedWork

2.1Criticalthinking

WeadoptthedefinitionofcriticalthinkingdevelopedbyBloometal.[

12

,

54

],ahierarchicaltaxonomythatcharacterisesstudentlearningobjectivesintosixtypes:knowledge(recallofideas),comprehension(demonstratingunderstandingofideas),application(puttingideasintopractice),analysis(contrastingandrelatingideas),synthesis(combiningideas),andevaluation(judgingideasthroughcriteria).

Thisdefinitionofcriticalthinkingisnotuncontested.Therearemultiplealternativeframeworks[

36

–

38

,

104

],andcriticalthinkingissometimesalsoreferredtoasreflectivethinking[

26

],thoughnotallscholarsconflatethem.Therehavebeenmultipleproposalsforconnectingandreconcilingthismultiplicityofframeworks[

32

,

74

,

96

].

WeadopttheBloometal.frameworkformultiplereasons.First,asoneoftheearliestframeworks,ithasstrongsupportinthere-searchliteratureandwideadoptionineducationsystems—itsdefinitionofcriticalthinkinghasbeenwidelyinfluential,andhaswithstoodseverecriticismandscrutiny[

40

].Second,itisrelativelysimple,havingonlysixcoredimensions(asopposedto,forinstance,thenuancedPaul-Elderframework[

104

]whichconsistsofeight“elementsofthought”,ten“intellectualstandards”,andeight“intel- lectualvirtues”).ThesimplicityoftheBloometal.framework— itssmallsetofdimensionswithcleardefinitions—rendersitmoresuitableasthebasisofasurveyinstrument.

Criticalthinkingskillscanbedevelopedinsequentialstages[

70

,

98

,

104

].Despiteconcernsaboutwhethercriticalthinkingcanbetaught[

138

],researchineducationhasdevelopedanumberofapproachestoteachingcriticalthinking[

104

,

139

],suchasstruc-turedargumentationexercises[

25

,

70

,

72

,

133

].Criticalthinkingcanbemeasuredthroughself-,peer-,orexpertevaluation[

66

],us-ingarangeofquestionnaires[

35

,

65

,

73

,

145

,

146

],justifiedmultiplechoicequestions,structuredessays,protocolsforwhole-portfolioassessment,taskobservation,andpeerinteraction[

34

,

105

].Inourstudy,weapplyaone-itemfive-pointscaleassessmentforeachofthesixcognitiveactivitiesassociatedwithcriticalthinking(sixitemsintotal,seeSection

3.1.3

),similartopreviouswork(e.g.,AlaoutinenandSmolander[

3

]).

2.2Designresearchforcriticalandreflectivethinking

Previousresearchhasinvestigatedhowinteractiondesigncanencouragecriticalorreflectivethinking.Variousdimensionsofthespaceofcriticalthinkinginterventionshavebeenexplored.Forinstance,whetherthesystemshouldbeproactive,i.e.,intro-ducecriticalthinkingpromptswithoutanexplicituserrequest[

69

,

109

].Ortheextenttowhichuserparticipationandengage-mentisimportantincreatingcriticalthinkingoutcomes,e.g.,pre-sentingAIexplanationsasquestionsratherthanstatementsim-proveslogicaldiscernment[

24

],questionsalsoimprovecriticalreading[

110

,

142

],attentioncheckspromotesystematicthinking[

49

],conflict-filleddiscussioninducescriticalthinking[

78

],andingeneralincreasedengagementresultsinbehaviouralchanges[

82

,

92

].Researchhasexploredtheefectivenessofgamificationofcriticalthinking[

31

,

91

,

129

].Researchhasalsoexploredtheextent

towhichinterventionsoughttobepresentedinanagentisedoranthropomimeticmanner[

99

,

131

,

141

].

Therearedomainsandactivities,someofwhicharerelevanttocommonknowledgeworkflows,wherecriticalthinkinginterven-tionshavebeenheavilystudied.Forexample,designforcriticalthinkingcanaidinthepreventionandverificationofmisinfor-mation,e.g.,throughstructuredthinkingaids[

50

,

51

],analyticalthinkingnudges[

143

],worksheetsandgroupdiscussion[

136

],andgamification[

129

].Orinwriting,ideationandargumentationtools,suchasthroughvisualisingargumentstructure[

126

,

133

],reflect-ingonfuturescenarios[

132

],ideationandevaluationsupport[

45

],assessingrisksinresearchimpactstatements[

94

].Anothercom-monareaforreflectivethinkinginterventionsisinmentalhealthandwellbeing,e.g.,tosupportcognitivereappraisal[

71

],reducecompulsivesmartphoneuse[

80

,

81

],improvetimemanagement[

55

],createjournalingprompts[

97

],encouragereflectiononbookhighlights[

61

],supportprayer[

75

],coachingforleadershipgrowth[

4

],andreflectiononcherishedobjects[

57

].Criticalthinkinginter-ventionshavealsobeenexploredindataanalysis[

44

,

48

].

Overreliance,definedas“usersacceptingincorrectrecommen-dations,i.e.,makingerrorsofcommission”[

102

],iscloselyrelatedto(thelackof)criticalthinking.Buçincaetal.[

17

]foundthat“cog-nitiveforcingfunctions”suchasrequiringtheusertowaitbeforereceivingAIoutput,ortomakeinteractiveupdatestoAIoutput,sig-nificantlyreduceoverreliancecomparedtosimplerAIexplanations.Thoughthereisoverlap,overrelianceisnotstrictlythesameprob-lemas(andisperhapsaspecialcaseof)alackofcriticalthinking.Alackofcriticalthinkingmayalsomanifestthroughacceptingasolutionthatmerelymeetsabaselineaspirationalthreshold[

6

,

119

]

—insuchcases,theAIsolutioniscorrect(albeitpotentiallyofpoorquality)andthereforenotoverreliance,strictlyspeaking.

Collectively,thesecaninformdesigninterventionstosupportcriticalthinkingforknowledgeworkers.Still,thesesystemsandtoolsdonotengagewithhowtheneedforcriticalthinkingsupportchangesduetoshiftsinworkflowcausedspecificallybytheuseofGenAI.Wealsolackempiricalfoundationsforunderstandinghowknowledgeworkersenactcriticalthinkinginreal-worldGenAIworkflows.

2.3Efectsofautomationonthinkingand

knowledgeworkflows:writingandmemory

Effectsonwriting.GenerativeAItoolslikeCopilotandChat-GPTcanboostwritingproductivitybyassistingwithtaskssuchascontentgeneration,ideacreation,andstylisticediting,helpingbothexpertandnovicewriters[

18

,

84

,

112

,

135

].However,thereareconcernsthatnovicewritersmaybecomeoverlyreliantonthesetools,potentiallyimpairingtheirlong-termskilldevelopmentbybypassingcriticalwritingprocessessuchasconstructinglog-icalargumentsandunderstandingsubjectmatter[

14

,

53

,

63

,

64

].Tomitigatethis,usingGenAIforindividualised,content-focusedfeedbackmayhelpnovicewritersdevelopwritingskillswhileim-provingproductivity[

58

,

86

,

144

].Althoughhumanfeedbackhastraditionallybeennecessaryforefectiveself-improvement,theintegrationofAIintotoolslikeMicrosoftWordcoulddemocra-tiseaccesstowritingskilldevelopmentbyprovidingconsistent,low-costfeedback[

2

,

123

].EarlystudiessuggestthatAI-generated

CHI’25,April26–May01,2025,Yokohama,JapanLeeetal.

feedbackcanimprovewritingqualityandlogicalstructure,espe-ciallyforlower-performingstudentsandlessconfidentEnglishlearners[

79

,

101

,

128

,

135

].Thus,equippingAItoolswithbetterfeedbackmechanismscouldfosterlong-termwritingskilldevelop-mentwhileaddressinginequalitiesinaccesstowritingeducation[

2

,

79

],andenablehumansandAItointeractovertimetomaximisebothproductivityandlearningoutcomes[

128

,

135

].

Effectsonmemory.WhileGenAIandconversationalsearchen-ginescanstreamlinetaskslikeliteraturereviews,somefearthatout-sourcingthisworkcouldharmourabilitytolearnandremember,inwhatissometimesreferredtoas“digitalamnesia”[

47

,

111

],thoughevidenceforthisefectislargelyinconclusive[

19

,

21

,

27

,

127

].Re-searchshowsthatsummarisingmaterialandfollow-upwritingpracticeenhancememorybyintegratingnewknowledgewithex-istingknowledge[

62

,

93

,

134

],butreal-worldsummarywritingisoftenpassiveandinefective[

15

,

16

,

41

,

121

,

140

],andthusmaynotimproverecallincomparisontosimplyre-readingthetext[

124

].GenAItoolslikeChatGPTandCopilotcanmitigatethesedrawbacks,especiallyforlessexperiencedlearners,byprovidinghigh-qualitysummariesuponwhichcollaborative,self-monitoredwritingtaskscanbeconducted[

120

,

125

].Cognitivescienceshowsthatefectivelearningrequires“grounding”informationthroughmultipleperspectivesandexamples[

10

,

11

,

68

],andGenAIcanoferpersonalisedanalogiestoaidthisprocess[

77

,

90

].

Insummary,previousworkhasdefinedcriticalthinkingandinvestigatedwaystodevelopandmeasurethisskillineducationalsettings.Separately,designresearchhasinvestigatedwaysofdevel-opingtechnologythatinducescriticalreflection.IthasalsobeenfoundthatAItoolscansignificantlyimpactcommonknowledgeworkflows,suchaswriting.However,thereisagapinunderstand-ingknowledgeworkers’perceptionsofhowGenAIafectstheirenactionofcriticalthinking,andtheefortofdoingso,acrossabroadrangeofusecases.Thisisthegapweaddresswithoursurvey.

3Method

Toanswerourresearchquestions—whenandhowknowledgeworkersperceivetheenactionofcriticalthinkingwhenusingGenAI(RQ1),andwhenandwhydoknowledgeworkersperceiveincreased/decreasedefortforcriticalthinkingduetoGenAI(RQ2)

—weconductedanonlinesurveyontheProlificplatform

2

tostudyknowledgeworkers’experienceswithcriticalthinkingwhenusingGenAItoolsfortheirwork.

Toensureparticipantsfullyunderstoodthescopeandmeaningofourquestionsoncriticalthinking,aspartofthesurveystudyonboarding,theywereintroducedtotheconceptofcriticalthink-inginthecontextofusingGenAIthroughconcreteexamplesofhowcriticalthinkingcouldbeappliedatvariouslevelsofBloom’staxonomy(e.g.,checkingthetoneofgeneratedemails,verifyingtheaccuracyofcodesnippets,andassessingpotentialbiasesindatainsights).Theseexamplesservedtosensitiseparticipantstothevariousdimensionsofcriticalthinkingwhileavoidingconcep-tualisingcriticalthinkingtoonarrowly.Theseactedas“cognitive

2

https://prolific.co/

priming”,helpingparticipantsbetterunderstandtheconceptofcrit-icalthinking,thussolicitingbetterrecognitionofcriticalthinkingbehavioursinparticipants’dailyGenAIuse.

Intotal,wereceived319surveyresponses,inwhichparticipantssharedatotalof936real-worldexampleswheretheyusedaGenAItoolfortheirwork,andsharedhowcriticalthinkingplayedaroleinthesetasks.

ToanswerRQ1,wecreatedanexplanatoryregressionmodelwithadependentvariablemeasuringwhetherparticipantsperceivedtheenactionofcriticalthinkingwhenusingGenAItoolsforthetaskstheyshared,andindependentvariablescorrespondingtotwosetsoffactorsthatwehypothesisedmightcorrelatewiththetendencytoengagewithtaskscritically:1)taskfactors:measuresaboutthetaskathand—e.g.,tasktype,confidenceindoingthetask.2)Userfactors:measuresaboutusers—e.g.,age,gender,occupation,tendencytoreflectinwork,andtrustinGenAI.Inaddition,weanalysedparticipants’motivatorsandinhibitorsforcriticalthinkingfromtheirfree-textresponses.

ToanswerRQ2,wecreateexplanatoryregressionmodelswithdependentvariablesmeasuringwhetherparticipantsperceiveddif-ferentcognitiveactivitiesconstitutingcriticalthinking(e.g.,breakingdownaproblem,puttingtogetherideas)tobemoreorlesseffortfulwhenusingaGenAItoolforthetaskscomparedtowhennotusingone.IndependentvariablesincludedthesamesetoffactorsasforRQ1above.Wealsoanalysedparticipants’free-textresponsestounderstandwhytheyperceivedthesecognitiveactivitiesasmoreorlessefortfulduetoGenAI.

3.1SurveyDesign

Tomodeltherelationshipbetweentaskanduserfactorsastheyrelatetocriticalthinkingactivities,wedesignedasurveyasfollows(seeAppendix

A.1

forthecompletesurvey).

3.1.1Task-RelatedFactors.Priorstudieshaveshownthatknowl-edgeworkersapplyGenAItoolsforarangeoftasksandexpressdiferentneedswhiledoingthesetasks[

13

],andthattheirperceivedconfidenceinthemselvesandAIdoingthetaskscaninfluencetheiruseandrelianceonthetool[

20

,

22

,

83

,

130

].Wehypothesisedthatfactorsrelatingtotheuser’stask,includingtasktype,confidenceinthemselves,andAIdoingthetask,couldafecttheircriticalthinking.

Tasktype.Brachmanetal.[

13

]classifyknowledgeworkers’cur-rentusageofGenAItoolsintoninetypes(SeeTable

1

),groupedintothreemajorcategories:1)forcreation,2)tofindorworkwithinformation,3)togetadvice.Thistaxonomyoferscleardistinctionsamongthemajorcategoriesoftasktype,whichwehypothesisedwouldcorrelatewithusers’criticalthinkingduetodiferingobjectivesandrequirements.WefollowBrachmanetal.

[

13

]’staxonomyandoperationalisetheirtasktypecategorisationinoursurvey,focusingonthemajorcategories.ForeachGenAItooluseexample,participantswerefirstaskedtodescribeindetailthetasktheydid(i.e.,Pleasetellus:1)whatyouweretryingtoachieve,2)inwhatGenAItool,and3)howyouusedtheGenAItool,includinganyprompts.).Then,theywereaskedtopickoneoftheninetasktypesthatbestdescribedtheirtask.Usingthisinformation,we

TheImpactofGenerativeAIonCriticalThinkingCHI’25,April26–May01,2025,Yokohama,Japan

Table1:Categoriesandsub-categoriesforGenAItoolusage

[13

].

Category

Sub-category

Description

Creation

Artefact

Generateanewartefacttobeuseddirectlyorwithsomemodification

Idea

Generateanidea,tobeusedindirectly

Information

Search

Seekafactorpieceofinformation

Learn

Learnaboutanewtopicmorebroadly

Summarise

Generateashorterversionofapieceofcontentthatdescribestheimportantelements

Analyse

Discoveranewinsightaboutinformationordata

Advice

Improve

Generateabetterversion

Guidance

Getguidanceabouthowtomakeadecision

Validation

Checkwhetheranartefac