2025验证教学观察工具：基于内容的小学游戏与教学方法（英）

PublicDisclosureAuthorized

PublicDisclosureAuthorized

PublicDisclosureAuthorized

PublicDisclosureAuthorized

EDUCATION

WORKINGPAPER

No.10|February2024

ValidatingTeachingObservationTools:

AContent-BasedApproachforPLAYandTeachPrimary

DiegoLunaBazalduaandEmmaCarter

©2025InternationalBankforReconstructionandDevelopment/TheWorldBank

1818HStreetNWWashingtonDC20433

Telephone:202-473-1000Email:

AskEd@

Internet:

/en/topic/education

ThisworkisaproductofthestaffofTheWorldBankwithexternalcontributions.Thefindings,interpretations,andconclusionsexpressedinthisworkdonotnecessarilyreflecttheviewsofTheWorldBank,itsBoardofExecutiveDirectors,orthegovernmentstheyrepresent.

TheWorldBankdoesnotguaranteetheaccuracy,completeness,orcurrencyofthedataincludedinthisworkanddoesnotassumeresponsibilityforanyerrors,omissions,ordiscrepanciesintheinformation,orliabilitywithrespecttotheuseoforfailuretousetheinformation,methods,processes,orconclusionssetforth.Theboundaries,colors,denominations,links/footnotesandotherinformationshowninthisworkdonotimplyanyjudgmentonthepartofTheWorldBankconcerningthelegalstatusofanyterritoryortheendorsementoracceptanceofsuchboundaries.ThecitationofworksauthoredbyothersdoesnotmeantheWorldBankendorsestheviewsexpressedbythoseauthorsorthecontentoftheirworks.

NothinghereinshallconstituteorbeconstruedorconsideredtobealimitationuponorwaiveroftheprivilegesandimmunitiesofTheWorldBank,allofwhicharespecificallyreserved.

RightsandPermissions

Thematerialinthisworkissubjecttocopyright.BecauseTheWorldBankencouragesdisseminationofitsknowledge,thisworkmaybereproduced,inwholeorinpart,fornoncommercialpurposesaslongasfullattributiontothisworkisgiven.

Anyqueriesonrightsandlicenses,includingsubsidiaryrights,shouldbeaddressedtoWorldBankPublications,TheWorldBankGroup,1818HStreetNW,Washington,DC20433,USA;fax:202-522-2625;e-mail:

pubrights@.

Coverdesign:MarianneSiblini

ValidatingTeachingObservationTools:AContent-BasedApproachforPLAYandTeachPrimary

DiegoLunaBazalduaandEmmaCarter

TableofContents

Abstract 3

Keywords 3

Acknowledgements 4

ExecutiveSummary 5

Studyobjective 6

Descriptionofobservationtools 6

Method 10

Results 12

Contentvalidityresults:ContentmatchingofTeachPrimarybehaviorsandPLAYitems 12

Concurrentvalidityresults:RelationshipamongTeachPrimaryandPLAYscores 19

ConstructvalidityresultsforPLAY:InternalstructureforthePLAYitems 19

Conclusion 22

References 23

Annexes 25

AnnexA:PLAYConstructsofsupportforchildren’sengagementinlearning 25

AnnexB:TablecomparingthemainfeaturesoftheTeachPrimaryclassroomobservationtooland

PLAYtoolkit 30

AnnexC:Overviewofexperts 31

AnnexD:FullmappingresultsforPhase1 32

AnnexE:MappingresultsforPhase2 47

AnnexF:CorrelationofTeachPrimaryandPLAYscores 52

AnnexG:CorrelationofPLAYitems,item-totalcorrelationsandvisualrepresentationsofinternal

structure 53

AnnexH:ExploratoryFactorAnalysissolution 58

Abstract

Thisstudycomparestwotoolsusedtoobserveandassesshowteacherssupportstudentlearning:theLEGOFoundation’sEngagetool,formerlyknownasPLAY,andtheWorldBank’sTeachPrimarytool.Thegoalwastoevaluatetheirsimilaritiesandoverlapintheirmeasurementofteachingpractices.Agroupofnineexpertsreviewedbothtoolsindetail,carefullymatchingitemsfromPLAYtothoseinTeachPrimary.Resultsindicatethataboutone-thirdoftheitemscloselymatched,showingthatPLAYisbuiltonasolidframeworktocaptureengagingteachingpractices.Anotherthirdoftheitemsdidnothaveaclearmatch,meaningthateachtoolseparatelycapturesuniqueaspectsofteachingthattheotherdoesnot.Thestudyalsolookedathowwellthetwotools’scoresalignusingrealclassroomdatafromEthiopia,Peru,andSierraLeone.Theresultsshowedasmallpositiverelationshipbetweenthetwotools,meaningtheymeasurerelatedbutdifferentaspectsofteaching.ThestudyalsofoundthatthewayPLAYisstructuredmayneedsomeadjustmentstobetterreflecttheskillsitisdesignedtomeasure.Bycombiningexpertopinionswithdataanalysis,thisstudyprovidesstrongevidencethatbothtoolsarevaluablebutservedifferentpurposes.Thefindingshighlighttheimportanceofusingmultipleapproachestoassessteachingquality.Thestudyalsooffersrecommendationsforimprovingtoolimprovementandsuggestsnextstepsonhowclassroomobservationtoolscanhelpsupportimprovedteachingandlearningworldwide.

JEL:I25-EducationandEconomicDevelopment

Keywords

Classroomobservations,teachingquality,engagingteaching,socio-emotionallearning,teachingpractices,contentvalidity,constructvalidity.

Acknowledgements

ThisdocumentwasdevelopedbyDiegoLunaBazaldua(SeniorEducationSpecialist;HEDGE),EmmaCarter(Consultant;HEDGE).TheteamworkedundertheoverallguidanceofHalilDundar(PracticeManager,HEDGE).LauraGregory(SeniorEducationSpecialist;HEDGE),SergioVenegas(Economist,HEDGE)andMaryamAkmal(Economist,HEDGE)providedoverallsupportinthedifferentstagesofthisstudy.

PeerreviewersincludeAliHasanAnsari(SeniorEconomist,HAWE3),HumaKidwai(SeniorEducationSpecialist,HAEE1),EzequielMolina(SeniorEconomist,HLCED),andLauriPynnonen(SeniorEducationSpecialist,HEAED).AdditionalvaluableinputswerereceivedfromAmerHasanandHarshilKumarSahai,andothermembersoftheEducationGlobalPracticewhoparticipatedindiscussionsonthepaper.Thenextlistofinternationalexpertscontributedtocontentreviewsofthetools:AbbieRaikes,CarolinaMeloHurtado,CarolinaMoreiraVasquez,DawitTiruneh,ElizabethHentschel,JonathanSeiden,LenkaJanikBlaskova,RabeaMalik,andTadesseTeferi.

ThisworkissponsoredbytheFoundationalLearningCompact(FLC)TrustFund.

ExecutiveSummary

Engagingteachinghasemergedasakeyapproachtofosteringchildren’ssocio-emotionalandcognitiveskills.Becauseoftheimportanceofembeddedengagingactivitiesintheteachingandlearningprocess,LEGOFoundationandtheWorldBankhavecollaboratedtoadvanceamoreevidence-basedapproachtoenhancingholisticteachingandlearning.Theseeffortsemphasizepracticesthatfocusonengagement-basedandsocio-emotionallearning.Thisreportpresentsfindingsfromasignificantcomponentofthepartnership,focusingoncomparingtheLEGOFoundation'sEngage,formerlyknownPlayfulLearningAcrosstheYears(PLAY)

1

,classroomobservationtoolwiththeWorldBank’sTeachPrimaryclassroomobservationinstrument.Thegoalwastoassesstheextentofoverlapbetweenthesetools,bothqualitativelyandquantitatively,providingcontentandconcurrentvalidityevidenceforthenewlydesignedPLAYinstrument.Thisstudyalsoaimedtoenhanceunderstandingofthetechnicalpropertiesofbothtools,emphasizingsynergiesanddistinctions.

Methodologically,thequalitativeaspectofthestudyfollowedtheDelphimethod,involvingnineexpertsfromdiverseinternationalbackgrounds.Theyconductedtwophasesofcontentreview.Phase1includedadetailedcontentcomparisonanditem-to-itemmatching,withrationalesprovidedformatchingdecisionsmade.Phase2focusedonreviewingambiguouscases—itemsthatdidnotclearlymatchonPhase1.Inaddition,quantitativeanalysisassessedthecorrelationbetweenTeachPrimaryandPLAYaggregatescorestoevaluateconcurrentvalidity,usingdatafromEthiopia,Peru,andSierraLeone.

KeyqualitativeresultsshowedthatoverathirdoftheitemsinthePLAYtoolstronglycorrespondedwithbehavioralitemsfromtheTeachPrimaryinstrument,providingevidenceoftheempiricalbasisoftheformer.Additionally,thestudyidentifiedasimilarproportionofitemswithminimalornocorrespondencetoTeachPrimarybehaviors,highlightingthePLAYtool'suniqueabilitytocaptureotheraspectsofteachingconducivetoqualityengagement-basedinstructionandlearning.Quantitativefindingsincludedatotalscorecorrelationof0.15betweentheTeachPrimaryandPLAYinstruments,supportingconcurrentvalidityandreinforcingthetools’measurementofseparatebutrelatedconstructsofqualityteacherpracticeandengagement-basedteachingandlearning,respectively.Takentogether,thesefindingsrevealedthatcombiningqualitativelearningsfromtheDelphimethodwithrobustquantitativevalidationresultscreatesacomprehensivevalidationframework,strengtheningconfidenceintheseclassroomobservationtoolstomeasurequalityengagement-basedandsocio-emotionallearning.

Inadditiontotheanalyticalfindings,valuablelessonswerelearnedfromtheapplicationoftheDelphitechnique,whichcouldimproveitseffectivenessforconsensus-buildinganddecision-makinginfutureresearch.Amongtheseweretheprovisionofsufficienttimefortaskstofacilitatedataaccuracyandtheimportanceofconsistentcommunicationwithexperts.Finally,clearincentivesalongwithstreamlinedprocessesforlabor-intensiveexerciseswererecommendedashelpfulstrategiesforenhancingparticipants’engagement.

1ThenamePLAYisusedthroughoutthispaperasitcorrespondstothenameusedinpubliclyavailablematerialsatthetimeofwriting.TheauthorsacknowledgethatthetoolisnowcalledEngage.

Studyobjective

TheLEGOFoundationandtheWorldBankhavecollaboratedtosupporteducationsystemsinadoptingamoreevidence-informedapproachtoimproveholisticteachingandlearning,particularlythroughengagement-basedandsocio-emotionalteachingpractices.Acrucialaspectofthisobjectiveisenhancingthemeasurementofthesepractices,especiallywithinlow-andmiddle-incomecountries(LMICs).OneactivityincludedinthiscollaborationinvolvedadetailedcontentcomparisonoftheLEGOFoundation'sPlayfulLearningAcrosstheYears(PLAY)andtheWorldBank'sTeachPrimaryclassroomobservationtools.Thiscomparisonaimedtounderstandtheextentofcontentandscoreoverlap,aswellastheirconnectionwithstudents'learningexperiencesandoutcomes.Additionally,theexercisefocusedonidentifyingimportantcontentdistinctionsbetweentheinstrumentsandhowtheymayprovidecomplementaryinformationtousers.

Thiscomparisonhasalsoservedasaformofcontentvalidityevidenceandfacilitatessubsequentquantitativeanalysisaimedatunderstandingandcomparingthetechnicalpropertiesofthetools.Forinstance,thiscontentmatchinganalysishighlightsthedegreeofconcordancebetweenitems,subscales,andunderlyinglatentconstructsmeasuredbyPLAYandTeachPrimary.Italsoenhancesourunderstandingoftheassociationbetweenconstructsindicatingthequalityofengagement-basedteachingpracticesandthosereflectinggeneralteachingpracticesandteacher-studentinteractions.ThiscanbeachievedbyimplementingasimilarmethodologytothatusedbytheWorldBank’sTeachteaminrelationtoinclusiveteachingclassroompractices(see

Molinaetal.,2024

),examiningtherelationshipbetweentheunderlyingconstruct—engagingteachingpractices—andTeachPrimarybehaviors.ThisapproachenablesmutuallearningandincreasesawarenessofthesynergiesbetweenthePLAYandTeachPrimarymeasurementapproaches,facilitatingmeasurementscale-upandpromotingengagement-basedandsocio-emotionallearningintheclassroom.

Thispaperisstructuredasfollows:Section1beginswithadescriptionoftheLEGOFoundation’sPLAYandtheWorldBank’sTeachPrimaryclassroomobservationtools.Section2detailsthestudy’smethodologicalapproach,includingthestudy’sdesign,profilesofexpertsinvolved,andanalyticalmethods.Section3providesanoverviewoftheresultsfromthetwokeyphasesofthisstudy.Finally,Section4presentskeyconclusionsanddiscussesthelimitationsrelatedtothemethodused.

Descriptionofobservationtools

PLAYClassroomObservationTool

TheLEGOFoundationhascollaboratedwithRTIInternationalandNewYorkUniversity(NYU)GlobalTIESforChildrentodevelopaculturallyresponsivetoolkitthatincludesclassroomobservationandsurveymeasurementtoolsfocusedoncapturingthequalityofengagingteachingandlearninginclassrooms.ThePLAYtoolkitaimstomeasurehowadultssupportchildren'sengagementinlearning,whichleadstoabroadrangeoflearningoutcomes.

PLAYincludestoolsformultipleagegroupsacrossdifferentsettings.Forstudentsaged6to12,themeasurementtoolsaredesignedspecificallyforclassroomuse.Thesetoolsincludeaclassroominventory,teacher-childobservation,ateachersurvey,andachildsurvey.Theteacher-childclassroomobservation

tool,whichisthefocusofthecomparisontaskdescribedinthenextsectionofthisdocument,encompassesfourkeydimensionsessentialforsupportingchildren'sengagementinlearning.Thesedimensions,alsoreferredtoasconstructs,aresummarizedbelowanddetailedfurtherinAnnexA.

Supportforexploration:Adultsupportforchildren'sinteractionwiththingsandideastoexpandthought.

Supportforagency:Adultsupportforchildren'sabilitytoinfluencehowandwhattheylearn.

Supportforpersonalandsocialconnection:Adultsupportfor(1)relatinglearningtochildren'spersonalexperiences,(2)childrenlearningthroughsocialinteraction,and/or(3)childrenfeelingasenseofbelongingandbeingsociallyconnected.

Supportforemotionalclimate:Adultsfosteranenvironmentwhereinteractionsbetweenadult(s),child(ren),andpeersarewarm,respectful,andpositive.

Thesefourconstructsarecapturedin25itemswithinthetool.Eachitemisscoredona3-pointscale,measuringthefrequencyandqualityofteacherandstudentbehaviors,aswellasparticipationintheclassroom.

TheWorldBank'sTeachPrimary(2ndEdition)ClassroomObservationTool

TheTeachPrimarytoolcapturesteachingpracticesthatsupportqualitylearningandnurturechildren'scognitiveandsocio-emotionalskills.Initiallylaunchedin2019afterarigorousdevelopmentandvalidationprocess,TeachPrimaryunderwentarevisionin2020-2021toenhanceitsmeasurementofinclusiveteachingpracticesthatareresponsiveandfacilitatewhole-childdevelopment.SimilartoPLAY,theTeachPrimarytoolisdesignedforclassroomswithstudentsaged6to12.

TheTeachPrimaryobservationtoolassessestwomainaspects:(i)thetimeteachersspendonlearningandtheextenttowhichstudentsremainontask,and(ii)thequalityofteachingpracticesthatdevelopstudents'socio-emotionalandcognitiveskills.TheTimeonTaskcomponentusesthree"snapshots"of1–10secondstorecordtheteacher'sactionsandthenumberofstudentswhoareontask.Observersnotewhethertheteacherprovidesalearningactivityformoststudents.Ifso,theyscantheclassroomtodetermineifstudentsareontask.Off-taskbehaviorisratedasfollows:0-1studentsbeingoff-task(High);2-5studentsbeingoff-task(Medium);6ormorestudentsbeingoff-task(Low).

TheQualityofTeachingPracticescomponentisorganizedintothreeprimaryareas:

ClassroomCulture:Theteachercreatesacultureconducivetolearning.Thisfocusesontheextenttowhichtheteacher:

createsasupportivelearningenvironmentbytreatingallstudentsrespectfully,consistentlyusingpositivelanguage,respondingtostudents'needs,andbothchallengingstereotypesandnotexhibitingbiasintheclassroom.

establishespositivebehavioralexpectationsbysettingclearbehavioralexpectations,acknowledgingpositivestudentbehavior,andeffectivelyredirectingmisbehavior.

Instruction:Theteacherfacilitateslessonsthatdeepenunderstandingandencouragecriticalthinkingandanalysis.Thisfocusesontheextenttowhichtheteacher:

facilitatesthelessonbyexplicitlyarticulatinglessonobjectivesthatarealignedtothelearningactivity,explainingcontentusingmultipleformsofrepresentation,andconnectingthelearningactivitytoothercontentknowledgeorstudents'dailylives,andbymodelingthelearningactivitythroughenactingorthinkingaloud.

doesnotsimplymovefromonetopictothenextbutchecksforunderstandingbyusingquestions,prompts,orotherstrategiestodeterminestudents'levelofunderstanding,bymonitoringstudentsduringgroupandindependentwork,andbyadjustinghis/herteachingtothelevelofstudents.

givesfeedbackbyprovidingspecificcommentsorpromptstohelpclarifystudents'misunderstandingsoridentifytheirsuccesses.

encouragesstudentstothinkcriticallybyaskingopen-endedquestionsandprovidingthemwiththinkingtasksthatrequirethemtoanalyzecontentactively.Studentsexhibitcriticalthinkingabilitybyaskingopen-endedquestionsorperformingthinkingtasks.

SocioemotionalSkills:Theteacherfosterssocio-emotionalskillstohelpstudentssucceedinsideandoutsidetheclassroom.Thisfocusesontheextenttowhichtheteacher:

instillsautonomybyprovidingstudentswithopportunitiestomakechoicesandtakeonmeaningfulrolesintheclassroom.Studentsexhibittheirautonomybyvolunteeringtoparticipateinclassroomactivities.

promotesperseverancebyacknowledgingstudents'effortsratherthanfocusingsolelyontheirintelligenceornaturalabilities,havingapositiveattitudetowardstudents'challenges,framingfailureandfrustrationsaspartofthelearningprocess,andencouragingstudentstosetshort-andlong-termgoals.

fosterssocialandcollaborativeskillsbyencouragingcollaborationthroughpeerinteractionandbypromotinginterpersonalskills,suchasperspective-taking,empathizing,emotionregulation,andsocialproblem-solving.Studentsexhibitsocialandcollaborativeskillsbycollaboratingwithoneanotherthroughpeerinteraction.

Thesethreeareasencompassnineelements,summarizedabove,capturedin28itemslinkedtospecificteachingbehaviors.Basedonthequalityofobservedteachingpracticescapturedinobservedbehaviors,theseitemsareratedonarubricaslow,medium,orhigh.Thescoresarethentranslatedintoa5-pointscalederivedfromtwo15-minutelessonobservationsofthesameteacher.FurtherdetailsabouteachbehaviorandelementcanbefoundintheTeachPrimary

manual

(Molinaetal.,2021).

Tooladaptation

TeachPrimaryObservationTool

TheTeachPrimarytoolwasoriginallydevelopedinEnglishbuthasbeenadaptedandimplementedacrossvariouslinguisticandculturalcontextstoensureitsapplicability.ForSpanish-speakingcountries,thetoolhasbeentranslatedintoSpanishandusedextensively,requiringminimalcontentadaptationduetoitsbroadalignmentwithteachingpracticesintheseregions.InAfricancountries,wheremultipleofficiallanguagesofinstructioncoexist,Teachhasbeentranslatedintoseverallocallanguagestoensureaccessibilityforenumeratorsandrelevancefordiverseclassroomsettings,butoveralltheimplementationreliedonEnglishinSierraLeoneandEthiopia.TheseadaptationsofTeachPrimaryhavefocusedprimarilyonlinguistictranslationwhilemaintainingfidelitytotheoriginalconstructs,ensuringthetool'svalidityacrossdifferentcontextsandisunderstoodbyenumerators.

PLAYClassroomObservationTool

ThePLAYtool,developedinEnglish,requiredtranslationintoSpanishforitsimplementationinPeru.ThistranslationfacilitatedthescoringofteachersinSpanish-speakingclassroomswhilemaintainingthetheoreticalconstructsandscoringframeworkoftheoriginaltool.Whilenoadditionaladaptationsweremadeforothercountriesincludedinthestudy,theflexibilityofthePLAYtoolallowsforfurthercustomizationshouldfuturecontextsrequireit.

Enumeratortraining

Trainingenumeratorsinclassroomobservationtoolsisessentialforensuringthatdatacollectedisaccurate,consistent,andreliable.Ithelpsenumeratorsunderstandhowtousethetoolcorrectly,observeandrecordbehaviorsobjectively,andavoidpersonalbiases.Propertrainingalsoensuresethicalobservations,protectingtherightsofstudentsandteachers,andhelpsenumeratorsadapttodifferentclassroomsettings.Byaligningallenumeratorsonhowtoapplythetool,thequalityofthedataimproves,makingitmoreusefulfordecision-makingandcreatingbetterinsightsintoclassroompractices.

TeachPrimaryObservationTool

EnumeratortrainingfortheTeachPrimarytoolfollowsastructuredandrigorousprocessfacilitatedbycertifiedmastertrainers.Publiclyavailableresources,includingimplementationguidelinesandtrainingmaterials,provideacomprehensiveframeworkforpreparingenumeratorsintheTeachPrimarytooladministrationandscoring.Trainingtypicallyspansseveraldaysandinvolvesmultiplecomponents:

ConceptualUnderstanding:TraineesareguidedthroughthetheoreticalunderpinningsoftheTeachtool,includingitsconstructsandscoringframework.

PracticalExposure:Enumeratorsengagewithcontext-relevantvideosofteachersinclassrooms,gainingpracticalexperienceinscoringclassroombehaviors.

SkillPractice:TraineespracticeadministeringandscoringTeachPrimarytobuildconfidenceandaccuracy.

Certification:Enumeratorsmustpassacertificationexamtodemonstratemasteryofthetool.Thisincludesachievingahighlevelofagreementintheirratingsofstandardizedvideos,ensuringreliabilityandreadinessforfielddatacollection.

Thisrigorousapproachensuresconsistencyandreliabilityindatacollectionacrossdiverseimplementationcontexts.

PLAYClassroomObservationTool

TrainingforthePLAYtoolwasdesignedtobuildcapacityamongenumeratorsthroughacascadingtrainingmodel.Acertifiedmastertrainerconductedacomprehensivetrainingsessionforaselectgroupoftrainers,whothenreplicatedthetrainingwithenumerators.KeycomponentsofthePLAYtrainingincluded:

Orientation:Enumeratorswereintroducedtothetool’sconstructsandthemethodsforobservingandscoringengagingteachingandlearning.

PracticeOpportunities:Sessionsincludedopportunitiestopracticescoringbasedonobservationscenariosandexamples.

UnlikeTeach,thePLAYtrainingdidnotrequireenumeratorstoreachapredeterminedlevelofinter-rateragreementforcertification.However,thetrainingensuredthatenumeratorswerefamiliarwiththetoolandconfidentintheirabilitytoapplyitduringdatacollection.

Method

Studydescription

ThisstudycomparedtheitemcontentofthePLAYandTeachPrimaryclassroomobservationtools.Specifically,itinvolvedmappingeachofthe25itemsfromPLAYtooneormoreofthe30itemsfromTeachPrimarytodeterminetheextentofoverlapanddifferencebetweentheinstruments.AtableoutliningthemainfeaturesofthesetoolsisprovidedasAnnexB.

Participantsinthisstudycomprisedagroupofexpertswithbackgroundsinthedevelopmentand/oruseofclassroomobservationmeasures,particularlywithinlow-andmiddle-incomecountrycontextsandacrossmultipleworldregions.Initially,15expertswereidentifiedforpotentialinvolvement.Afterbeingcontactedviaemailandprovidedwithabriefoverviewofthestudy,nineconsentedtoparticipate.Followingthisconfirmation,participantsreceivedamoredetailedoutlineofthetaskandTermsofReferenceconcerningtheirconsultancyrole.Participantsengagedincontentreviewandcomparisonofthetwotools,item-to-itemmatchingtasks,andparticipatedinanyfollow-updiscussionsortasks.AnoverviewoftheexpertsisprovidedasAnnexC.

TheDelphimethodforcontentvalidity

ThisstudystronglydrewupontheprinciplesoftheDelphimethod.TheDelphimethod,utilizedinmultiplefields,includingeducation,health,andpsychology,allowsexpertstoprovidejudgmentsonatopic,whichcanthenbemodifiedorrefinedbasedonreactionstothecollectiveviewsofthegroup(

Green,2014;

Hyatt

etal.,2024;

ManizadeandMason,2010

).Unlikeface-to-facetechniques,theDelphimethodpromotesindependenceofthoughtandallowsresearcherstoleveragetheexpertiseofnumerousexpertsfromdifferentlocationswhilefosteringanenvironmentthatencouragesgroupacceptanceandsharedresponsibilityfortheoutcome.TheDelphimethodtypicallyinvolvesseveralkeycharacteristicsforobtaininggroupconsensusonatopic,including(1)participantanonymity,(2)anonymityofresponses,(3)multipleandstructurediterationsofconsultations,(4)groupconsensus,(5)controlledfeedbackofresponsestoparticipants,and(6)statisticalanalysisofdata(

GrantaandKinney,1992,pp.13-14

).StudiesutilizingtheDelphimethodhavebeenconductedwithasfewassevenandasmanyas1,000participants,thoughsamplesbetween10and50aretypicallyadvisedduetotheextensivedatageneratedandtheanalysesrequiredforeachindividualresponse(

IqbalandPipon-Young,2009

).

TheDelphimethodhasemergedasapopularapproachfordeterminingcontentvalidity(

Green,2014;

ManizadeandMason,2010

).Contentvaliditydenotestheextenttowhichtheitemsofameasurementinstrumentarerelevanttoandrepresentativeofthetargetedconstructitisintendedtomeasure.Itconsiderstheimportance,relevance,andclarityofinstrumentitems,domains,anddefinitions,aswellastheappropriatenessandadequacyofitemresponsescales(

Roebiantoetal.,2023

).Contentvalidityis

particularlyrelevantasitincreasestheaccuracyofmeasuringconstructs.Forexample,inclassroomobservationtools,contentvalidityensuresthatalldevelopeditemscoveraspectsofclassroompracticethatlearningtheoriesdefineasrelevantforimprovingstudentlearning.Additionally,contentvalidityhelpsreducemeasurementbiasbycarefullyselectingandreviewingitemstocoverallrelevantaspectsofaconstruct.

Therearemanywaystoestablishcontentvalidityforclassroomobservationmeasures.Thecurrentstudyisparticularlyrelevanttoprovideevidenceonwhethertheinstrumenthasanempiricalfoundationreflectingkeypracticesandprocessesofothervalidatedobservationalmethodsdesignedtocaptureteachingqualitywithinsimilarcontexts(

Molinaetal.,2020;

Seidmanetal.,2018

).Equallyimportan