Language-Testing-and-uation.ppt

LanguageTestingandEvaluation IntroductiontoLanguageTesting Testingisanindispensablepartofeveryteachingandlearningexperience Rightfromthecommencementofformaleducation weareconfrontedwithamyriadoftests fromsmallquizzes mid termandfinalexaminations torigidmatriculationexaminations nottomentionmanyothertestsafterourgraduation onthegradesofwhichhangdecisionsconcerningourcareer Asamatteroffact wearehauntedbyteststhroughoutourlifetime Whatisa test Apsychologicaloreducationaltestisaproceduredesignedtoelicitcertainbehaviorfromwhichonecanmakeinferenceaboutcertaincharacteristicsofanindividual Carroll 考试是由一定数量的题目组成 用于测定人的某一心理品质 或学习掌握程度的工具 教师出试题实际上是创设一个语言情景 学生的答题过程则是他的语言能力和知识掌握程度的一种表现 考试的目的就是要测定学生在某一方面的能力或学习情况 舒运祥2001 AssessmentToputitsimply assessmentmeanstodiscoverwhatthelearnersknowandcandoatacertainstageofthelearningprocess Withdifferentassessmentpurposesinmind wemayadoptdifferentassessmentmethods Aclosestudyontheassessmentpurposeswillmakeitclearthatallthepeopleinvolvedineducationhavesomereasonstoconsiderassessmentnecessary Theyareadministrators teachers parentsandstudents Administrators Administratorsprovidemoneyandpersonnelforeducation Theyneedtoknowwhethertheprogrammestheyhaveplannedareworkingwell Theonlywaytodothisistodiscoverhowwellthepupilsaredoingwiththeircourses Ifassessmentresultsaredifferentfromwhattheyhaveexpected theyneedtoreplantheprogrammessothatbetteraccomplishmentcanbeachievedlater Teachers Teachersputtheadministrators plansintopractice Inthecontinuumoflearning teachersneedtoknowwhathasbeendoneandwhatneedstobedonenext whatthepupilsalreadyknoworcandoandwhattheydonotknoworcannotdoyet Iftheprogramiswellplanned itprettymuchdependsontheteachers performancewhethertheprogrameventuallyleadstosuccess Parents Nobodyismoreanxiousthanparentstoknowhowtheirchildrenaredoinginschool Unabletowatchtheirkidsintheclass parentsvaluethefeedbackabouttheirchildren sperformancefromtheteachersandtheschool Students Finally studentsneedtoknowwhatthey veaccomplished beawareofwhattheyneedtoworkontext andbuilduptheirconfidenceandsatisfactionfromwhattheyhaveachieved Reasonsfortest assessmentuse Toknowpeople slanguageability Todiscoverhowsuccessfulstudentshavebeeninachievingtheobjectiveofacourseofstudy Toprovidefeedbackforlearners Toevaluatetheeffectivenessofteaching Todiagnosestudents strengthsandweaknesses Toplacestudentsinanappropriateprogram 语言能力 兴趣 态度等 考试 观察 面谈 数值 价值 判断 测量 标准 量化 评价过程 考试 测量和评价三者之间既有联系又有区别 评价过程实际上包含了测量 如果测量以考试为手段 那么评价中也包含了考试 外语教学中为了了解学生的语言能力 兴趣 知识掌握得情况 就要对其进行评价 但能力 兴趣等心理品质是一个抽象概念 为了有利于评价 我们通过具体做题目 考试 等方式 对抽象的概念进行定性 定量的描述 最后还需借助一定的标准解释考试成绩 以确定学生的语言水平 这样一个过程就是评价 Test Measurement evaluation Assessment 外语测试 语言学 心理学与教学法 理论语言学 应用语言学 心理语言学 社会语言学 学习心理学 外语教学法 测量学 教育统计学 教育测量学 计算机科学 外语测试与有关学科 TeachingandTestingTestingandteachingarecloselyinterrelated Languagetestingbothservesandisservedbyresearchinlanguagelearningandlanguageteaching Theeffectoftestingonteachingandlearningisknownasbackwash Backwashcanbeharmfulorbeneficial Well madetestsofEnglishcanbringaboutbeneficialbackwashwhileill designedoneswillleadtomisguidedteachingandlearning Goodtestscanhelpcreatepositiveattitudetowardslearning Goodtestscanhelpfosterpositiveclassroomatmosphereandhelpthemmasterthelanguage TeachingObjectivesTeachingContentTeachingMethodsTesting Notalltests however aresufficienttomeetourrequirement i e toprovideanaccuratemeasurementofthecandidates languageproficiencyfortheirspecificuses Moreoftenthannot acandidate sperformanceonalanguagetestdoesnotmatchhisperformanceinrealuse Ifblindfaithisplacedinthetestresultregardlessofthevalidityandreliabilityofthetestforaspecificsituation itwillgiverisetosevereproblems Whenthetestscoreisregardedastheonlyyardstickofaperson slanguageproficiency studentswilltendtostudytesttakingskillsratherthanthelanguageitself Theybecomefiendsattestinsteadofmastersofthelanguage Thenweareboundtofallintotheexamination centeredteaching whichshouldbeavoided Clearly thisisundesirable Teachingcontents TeachingMethods Testing KindsofTestsandTestingTestscanbecategorizedfollowingdifferentcriteria suchastheintendeduseofthetest theapproachtotesting testingcontent thewaytointerpretthescoreandscoringmethods First testsareusedtoobtaininformation Givingatestislikeaskingaquestion itisarequestforinformation Ifyouasktherightsortofquestion yougettherightsortofanswer Ifyouaskasillyquestion yougetasillyanswer Inthiswaytheycanbecategorizedintoproficiencytests achievementtests diagnostictests placementtestsandaptitudetests Proficiencytests 水平测试 Proficiencytestsaredesignedtomeasurepeople soverallmasteryofEnglishortomeasurehowsuitablecandidateswillbeforperformingacertaintaskorfollowaspecificcourse TOEFL EPT NMETetc Forexample doesthestudentknowenoughEnglishtofollowacertainuniversitylecturegiveninthemediumofEnglish DoesthestudentknowenoughEnglishinordertofunctionefficientlyinaparticulartypeofemployment Thecontentofaproficiencytestisnotbasedonthecontentorobjectivesoflanguagecourseswhichpeopletakingthetestmayhavefollowed Itisinnowayrelatedtoanysyllabusorteachingprogram indeed manyproficiencytestsareintendedforstudentsfromseveraldifferentschools countriesandevenlanguagebackgrounds Achievementtests 成绩测试 Achievementtestsaredirectlyrelatedtolanguagecourses theirpurposebeingtoestablishhowsuccessfulindividualstudents groupsofstudents orthecoursesthemselveshavebeeninachievingobjectives Theyareoftwokinds finalachievementtestsandprogressachievementtests Finalachievementtestsareusuallyformalexaminations givenattheendofacoursestudyorattheendoftheschoolyear Oftenittakestheformofanexternaltestwhichissetbyofficialexaminingboards ministriesofeducation orbymembersofteachinginstitutions Progressachievementtestsareintendedtomeasuretheprogressthatstudentsaremaking Theyareusuallydesignedbyteacherstochecktheextenttowhichthestudentshavemasteredthematerialtaughtintheclassroom Beingthemostwidelyusedtypesoftests progresstestsareusuallyaneffectiveteachingdevicetomotivatelearningandreinforcewhathasbeentaught Diagnostictests 诊断测试 Diagnostictestsareusedtoidentifystudents strengthsandweaknesses Theyareintendedprimarilytoascertainwhatfurtherteachingisnecessary Inourteaching wemaysometimesbetemptedtoconcentrateonfollowingthesyllabusandignoretheneedsofourstudents However ifwedothis alotofourstudentsmayfail howeverthoroughlywehavecoveredthesyllabus Itisveryimportanttotakeintoaccounttheneedsofourstudentsateverystageinourteaching Justasitisnecessaryfordoctorstodiagnoseanillnessinordertocuretheirpatients soteachersmustdiagnoseproblemsinordertoteacheffectively Veryfewtestsareconstructedforpurelydiagnosticpurposes Usuallyadiagnostictestformspartofanothertypeoftest especiallyaclassroomprogresstest Achievementandproficiencytestsarealsofrequentlyusedfordiagnosticpurposes areasofdifficultyarediagnosedinsuchtestssothatappropriateremedialactioncanbetakenlater Placementtests 摸底测试 Placementtestsareintendedtoprovideinformationwhichwillhelptoplacestudentsatthestage orinthepart oftheteachingprogrammostappropriatetotheirabilities Typicallytheyareusedtoassignstudentstoclassesatdifferentlevels PlacementtestsareusuallygeneralandconcentrateontestingawideandrepresentativerangeofabilityinEnglishinordertolookforwardtothelanguagedemandswhichwillbemadeonstudentsduringthecourse AptitudeTests 学能测试 AptitudeTestisdesignedtopredictastudent slikelihoodofsuccessandeaseinlearningaforeignlanguage Languagelearningaptitudedoesnotrefertowhetherornotanindividualcanorcannotlearnaforeignlanguage itisassumedthatvirtuallyeveryonecanlearnaforeignlanguagegivenadequateopportunity Languagelearningaptitudereferstothe predictionofhowwell relativetootherindividuals anindividualcanlearnaforeignlanguageinagivenamountoftimeandundergivenconditions 例如 本子在盒子上 Pasyhojtzov 盒子在桌子上 Pasydeupthoj 椅子在桌子下 Pasydeupfgrip 笔在盒子里 Pasyhojcmak 书在椅子下 Pasygripftjav 1 书在桌子上 A Pasydeupttjav B PasydeupftjavC Pasygripfdeupt D Pasytjavtgrip 2 笔在本子下 A Pasyhojtmak B Pasyzovtmak C Pasyzovfmak D Pasyhojfmak 3 本子在盒子里 A Pasyhojczov B Pasyhojctjav C Pasyhojtzov D Pasyhojttjav HistoryofTestingModernlanguagetestinghasahistoryofabout100years Broadlyspeaking thedevelopmentoftestingcanbedividedintothreestages thepre scientificstage thescientificstageandthecommunicativestage ThePre scientificStage Thenineteenthcenturyandtheearlydecadesofthetwentiethcentury basically intuitive orsubjective dependingonthepersonalinclinationoflanguageteachers Studentsevaluatedinavarietyofways notonlyonskillsinusingthelanguagebutalsoonfactsaboutEnglish Languageknowledgeoftentakingupheavyweighting studentshadtolabelpartsofasentenceandmemorizenumerouslistsoflanguagepatterns Abundantwritinginvariousformsincludingtranslation essay dictation precis andopen endedanswersbasedonreadingcomprehension Westillseesomeofthesetestingformsinuseandtheyturnouttobequitesoundforadvancedstudents TheScientificStage Subjectivetestsinthepre scientificstageattackedforlackofreliability astrongneedforteststhatwouldbeobjectiveandfairer Testingspecialistswithlinguistictrainingenteredthescene Languagetheoriessuggestedthatlanguagecouldbebrokenintosub skillsandcomponentsandcouldbeevaluated scientifically bitbybit Objectivetestsweredevisedthatmeasuredperformanceorrecognitionofseparatesounds specificgrammaticalfeatures orvocabularyitems Subjectivewrittentestsbegantobereplacedbyobjectivetests Answersbeingstandardized scoredevenbyuntrainedpeople Specialistsstartedtoevaluatetestsstatistically lookingattheeffectivenessofeachquestionandmeasuringtheexamination sreliabilityandvalidity TheCommunicativeStage Teststodaymainlyconcernedwithevaluatingrealcommunication abroaderviewoflanguageability Notonlymeasurelanguageskillsbutalsocommunicativecompetence Thebestexams combinevarioussub skills Tomeasuremorethanisolatedlanguageskills howwellapersoncouldfunctioninhissecondlanguage Tousethebestfeaturesofsubjectiveandobjectivetesting Theclozetestreflectsthiscompromise Fromastoryoressay wordsareremovedatregularintervals Afterconsideringthecontext theexamineehastofillinthemissingwords Thus thetaskisholistic thatis grammarandvocabularyandoverallmeaningaretestedsimultaneously Butthescoringisquiteobjective Inresponsetothesebroaderviewsofcommunicativelanguageabilityandcommunicativelanguageuse mucheffortisbeingdirectedtowarddevelopingtests thatnotonlymeasureawiderangeoflanguageabilities includinggrammatical discourse sociolinguistic andstrategiccompetencies butthatarealso authentic inthattheyrequiretesttakerstointeractwithandprocessboththeexplicitlinguisticinformationandtheimplicitillocutionaryorfunctionalmeaningofthetestmaterial CharacteristicsofDiscretepointtests 分离式测试 Integrativetests 综合性测试 andCommunicativetests 交际性测试 StructuralisttestingapproachCorrespondingtothetraditionalviewonthenatureoflanguageandlanguagelearning conventionaltestsadoptthestructuralistapproachtotesting Whatistobetestedisrevealedbyastructuralcontrastiveanalysisbetweenthetargetlanguageandthelearner smothertongue Morrow Basedonthecontrastiveanalysescarriedoutatallthelevelsofstructure grammatical syntactic phonological testitemsareconstructed Thesameapproachisadoptedastohowtotest Languageisbrokendownintoseparateelementsanddiscreteitemsareconstructed eachofwhichideallyrevealsthetesttaker smasteryofonelevelofthelanguageintermsofoneofthefourskills listening speaking readingandwriting Norm referencedConventionaltestsrelyonquantitativeassessment Followingthebehavioristviewoflanguagelearningthroughhabitformation testsposequestionstoelicitresponseswhichshowwhetherornotcorrecthabitshavebeenestablished Correctanswersarerewardedandnegativeonespenalizedinsomeway Passingatestinvolvesmakingaspecifiedproportionofcorrectresponses Consequently scoresofconventionalteststendtorevealsimplythequantityofitemsansweredcorrectly Nothingcanbeinferredaboutthequalityofthetesttaker slanguageperformance 分离式测试的理论基础 1 结构主义语言学 教学内容可被分解成语音 词法和句法 词汇等方面 语言能力可分解为听说读写等方面 2 行为主义学习论 复杂的学习行为可分解为最基本的单元 刺激 反应 S R 的联结 新行为主义代表斯金纳提出 刺激 反应 强化 3 听说教学法 听说领先 反复实践形成习惯 以句型为中心 排斥或限制母语 对比语言结构 确定教学难点 及时纠正错误 培养正确的语言习惯和广泛利用现代化教学手段 分离式测试的特点和问题 1 个人的语言技能发展是不平衡的 语言测试可以从听说读写四个方面分别进行 2 语言是有结构并按层次构成的 测试可按层次由低到高进行 如对听能的测试可分为 1 音素 元音 辅音 语调 重音等 2 词汇 3 句子 4 小段对话或短文等 3 语言情景是无限的 语言成分是有限的 测试成分更有效 4 母语和外语在相互转换时 其形式 结构和意义等方面由一定的不同之处 也是测试重点 5 试题采用只有一个正确答案的题型 以保证客观性与科学性 6 评分采用客观打分 任何人阅卷结果都一致 分离式测试以行为主义学习理论 结构主义语言学和听说教学法的理论为依据分别对考生的语音 词汇 语法等语言知识和听说过读写等方面的语言技能进行测试 题量大 每题的测试目标明确 有助于教师了解学生在学习中的长处和不足 测试数据能量化分析 打开科学测试的一页 它的最大弱点是重语言形式而轻语言应用 把现实中的语言分解成无上下文的单句 其测试结果很难反映学生实际应用语言能力 正是由于这些弱点 使综合测试和交际性测试应运而生 He sgoingtocallusback A won theB didn theC doesn theD isn the Didn tyougooutyesterday A Yes Ididn tB Yes IdidC No IdidD No Iwentout IntegrativetestingIntegrativetesting bycontrast requiresthecandidatetocombinemanylanguageelementsinthecompletionofatask Thismightinvolvewritingacomposition makingnoteswhilelisteningtoalecture takingadictation orcompletingaclozepassage 综合性测试的理论基础 1 语用期待语法假设 美国语言学家奥勒 JohnW Oller jr 认为交际过程中 语境对于确定语音或词义有着至关重要的作用 正常的语言使用过程中存在两种语境 语言自身语境 linguisticcontext 和语言外语境 extralinguisticcontext 言语和体势语构成语言自身语境 语言知识 语言外的现实世界构成语言外语境 语境中语言加工能力为 语用期待语法 规则 奥勒认为一个人的语言能力是由这样一个期待语法生成系统 expectancygeneratingsystem 构成的 完型填空和听写是最有效的语用测试形式 e g 1 Hellen willyoubeatthepartytonight Yes butIhavesomuchhomeworktodothatIreally A can tB mustn tC won tD shouldn t2 Wasitthere youwereawaytoanswerthephone There snodoubtaboutit A thatB whichC whileD where context A Whatareyoulookingfor B Oh mygod Ican tfindmywalletanywhere IputitjustherebeforeIwentintoanswerthephone A Wasittherewhileyouawaytoanswerthephone B Thereisnodoubtaboutit Lookatthefollowingdialogue 1 A Canyoutellmethetime B Well themilkmanhasjustcome 2 A Thereisthetelephone B Iaminthebath A OK 3 A Whatarethepolicedoing B I vejustarrived 4 A CanIbuyyouadrink B Actually I dratherhavethemoney 5 A IthinkIcouldmakeyouveryhappy B Why Areyouleaving 2 格式塔心理学 强调感知的整体性 认为整体不等于部分之和 整体是先于部分而存在 并制约着部分的性质和意义 综合性测试特点 综合性测试也称语用测试 pragmatictest 其测试方法正好与分离式相反 综合性测试将人的语言能力视为一个整体 每次对考生的多种能力的多个方面同时进行测试 这类考试始终在语境中测试学生使用语言的能力 因此结果更接近于考生实际使用语言能力 以完型和听写为主要形式的综合性测试仍属间接测试 无法直接测出考生的语言表达能力 需与其它测试结合使用 才能取得满意效果 ThecommunicativetestingThecommunicativeapproachtolanguagetestingissometimeslinkedtotheintegrativeapproach However althoughbothapproachesemphasizetheimportanceofthemeaningofutteranceratherthantheirformandstructure thereareneverthelessfundamentaldifferencesbetweenthetwoapproaches Communicativetestsareconcernedprimarily ifnottotally withhowlanguageisusedincommunication Successisjudgedintermsoftheeffectivenessofthecommunicationratherthanformallinguisticaccuracy Inpractice however sometestsofacommunicativenatureincludethetestingofusageandalsoassessabilitytohandletheformalpatternsofthetargetlanguage Indeed fewsupportersofthecommunicativeapproachwouldarguethatcommunicativecompetencecouldeverbeachievedwithoutaconsiderablemasteryofthegrammarofalanguage 交际性测试的理论依据是社会语言学和交际法 也称意念 功能法 以社会语言学为理论基础的交际教学法提出以语言功能项目为纲 以培养交际能力为本的教学法 海姆斯 D H hymes 认为 交际能力应体现在四个方面即 形式正确 合理可行 内容得体 实际使用 语言使用时应遵循 得体法则 ruleofappropriateness 因此交际能力强的人的语言应该语法正确 符合人的感知特点 内容得体 语言表达的是人们生活中真实的文化行为 分离式和综合测试无法测定真实语言交际能力 交际性测试的特点 是行为测试 在真实或接近于真实情景中 通过完成任务来测试语言掌握程度 因此又称 基于任务的 task basedtests 考试 如 考生听一段录音后写一份电传回电 将听与写结合起来 虽然有其优点 但考试材料 语境和任务的真实性仍然有争议 同时作为主观测试 评分的客观性问题阻碍它在大规模考试中的应用 但目前 有人将多项选择题引入到交际测试 如 Youwereapplyingtoauniversityandneededaletterofrecommendation Youwenttoaprofessor whowasalsoyourfriend andsaid A I dappreciateitifyoucouldwritealetterofrecommendationforme B Iwanttoaskyoutowritealetterofrecommendationforme C Iwonderifyoucouldwritealetterrecommendationme D Hey givemerecommendationletter 正确 得2分 语法正确 不够得体 得1分 较得体 语言有误 得1分 语言有误 内容不得体 不得分 ContrastofCommunicativeandTraditionalApproacheswithTesting 1 Whatislanguage C asystemforcommunicatingmeaning T asystemofknowledge2 Whatislanguagelearning C learntocommunicatebyusingT learngrammarrulesandstructures3 pedagogicconcerns C communicativecompetence meaning contentT linguisticcompetence form 4 Pedagogictechniques C analytic top down roughly tunedinput interactionT synthetic bottom up finely tunedinput transmission5 Affective C Learner centered initiator performer problem solver T Teacher centered controller knowledge imparter assessor 6 testing C Integrative Holistic Criterion referenced Production Communicativecompetence Function UseRawdata SurfacestructureT Analytical Discretepoint Norm referenced Reception Linguisticcompetence Form Usage Idealization Deepstructure Norm referencedtestingNorm referenced NR testsaredesignedtoenablethetestusertomake normative interpretationsoftestresults Thatis testresultsareinterpretedwithreferencetotheperformanceofagivengroup ornorm Soastudent sperformanceiscomparedwiththeperformancesoftheotherstudentsinthesamegroup Forexample isthestudentinthetoporinthebottompartofthegrouporinthemiddleofthegroup Howmanystudentsinthegrouparebetterthanthestudentandhowmanyareworse Theperformancecharacteristics ornorms mosttypicallyusedasreferencepointsaretheaveragescoreofthegroup Criterion referencedtestingCriterion referenced CR testsaredesignedtoenablethetestusertointerpretatestscorewithreferencetoacriterionlevelofabilityordomainofcontent Anexamplewouldbethecaseinwhichstudentsareevaluatedintermsoftheirrelativedegreeofmasteryofcoursecontent ratherthanwithrespecttotheirrelativerankingintheclass Thus allstudentswhomasterthecoursecontentmightreceivean A irrespectiveofhowmanystudentsachievethisgrade Inotherwords weuseCRteststofindoutwhetherastudentcanperformaparticulartaskornot Forexample canthestudentwritealetteraskingforinformationaboutsomething 常模参照考试 Norm referencedtests 主要用于甄别 分类 选拔等 例如编班考试 外语竞赛 中考和高考等 标准参照考试 criterion referencedtests 常用于各类证书考试 如高中会考 期末考试 大学四 六级等 QualitiesofTestsToevaluateatest wemustinthefirstplacebeclearofthebasicqualitiesthatmakeatestuseful BachmanandPalmer Testusefulnessincludessixtestqualities reliability validity authenticity interactiveness impact andpracticality Reliability 信度 Reliabilityisoftendefinedasconsistencyofmeasurement Areliablescoreisfreefromerrorsofmeasurementandconsistentacrossdifferentcharacteristicsofthetestingsituation Forexample ifthesametestweretobeadministeredtothesamegroupofindividualsontwodifferentoccasions intwodifferentsettings its