




已阅读5页,还剩1页未读, 继续免费阅读
版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领
文档简介
EvaluationofanAuthenticExaminationSystem(AES)forProgrammingCoursesTorbjrnJonsson,PouriaLoghmaniandSiminNadjm-TehraniDepartmentofComputerandInformationScienceLinkpingUniversity,Swedentorjo,poulo,siminida.liu.seAbstractThispaperdescribesourexperiencewithanauthenticexaminationsystemforprogrammingcourses.Webrieflydescribethearchitectureofthesystem,andpresentresultsofevaluatingthesysteminrealexaminationsituations.Someofthefactorsstudiedindetailaretheon-lineinteractionsbetweenthestudentsandexaminers,theresponsetimesandtheireffectsonthepressureexperiencedbystudent,theacceptanceofthemethodamongthestudents,andwhethertheexaminationformisgender-neutral.IntroductionAsexperiencedteachersinprogrammingcourseswehavenoticedthedrawbacksinthetraditionalexaminationformusedinprogrammingcourses.Thestudentslearntoprogramvialaboratoryexercises,butthefinalevaluationoftheirabilitiesandthegradingoftheexaminationareinaformthatusespaperandpeninsteadofcomputers.Consideringthatthestudentwillneverusethismodeforproducingaprogramthroughtheprofessionallife,weconsiderthistobenotasuitablemethod.AttheDepartmentofComputerScienceatLinkpingUniversity12fundamentalprogrammingcoursesforapproximately1000studentsindifferenteducationalprogramsaretaughtannually.Thispaperdealswithanewpedagogicalviewintheseprogrammingcourses,whichcanbeappliedtoanyprogramminglanguage,typeofstudentandeducationalprogram.Theideaisbasedonextensivestudiesarounddifferentexaminationforms,whereindividualgrading,efficientandusefulfeedbackandtheauthenticityoftheexaminationformareusedasbasiccriteriaforthechoiceofexaminationmethod.Webelievethatthechoiceofmethodtogetherwiththeaddedefficiencyintheassessmentprocessimprovesthequalityofourstudyprogrammes.Inparticular,webelievethatitwillchangetheexaminationprocessfromasummativetoanormativeassessmentoccasion1.Foranumberofyearswehaveexperimentedwithtestingthestudentsviacomputer-aidedexaminationsinsomepilotcoursesanauthenticexaminationformforthistypeofcourse.However,thisexaminationformhasnotbecomemorewidespreadduetoinsufficientsupportforthecomputerenvironmentnecessaryforthiskindofexamination.Duringthepastyearanewauthenticexaminationsystem(AES)hasbeendeveloped,whereallthestudentsandtheexaminingteachersareconnectedtothesamesystem.Theprocess,includingcommunicationandgrading,issupportedbythisenvironment.Inthispaperwedescribetheexaminationsystemandourinitialevaluationsofthissysteminanumberofrelativelylargeexaminationsessions.ThecoursesinquestioncoveredprogramminginAdaandweretakenbyfirstandsecondyearstudents.DuringthepastyearwehaveevaluatedtheAES.Theinstrumentsusedfortheevaluationconsistedofquestionnairesfilledby231studentsoveraperiodof3monthsand4examinations.Thepaperisorganisedasfollows.Insection1wedescribewhythetypeofexaminationweproposeisthemostappropriateforprogrammingcoursesandcomparetosomerelatedsystems.Section2includesabrieftechnicaldescriptionoftheexaminationsystems,includingitsarchitecturaldesign.Insection3wedescribehowthecomputersystem,thatmanagestheexaminationprocesson-line,hastobeaugmentedbyrulessetupineachparticularcourse.Section4coversourevaluationmethodsandisfollowedbyevaluationresultsinsection5.Section6concludesthepaper.1ExaminationformsEveryexaminationmethodhasspecificcharacteristicsthatmakeitmoreorlessappropriatetoaparticularcoursesetting.HkanOswaldssonstudiedtherangeofpossibleexaminationformsforatypicalprogrammingcoursepriortothedevelopmentofthecurrentexaminationsysteminourdepartment5.Whileseveralmodesofexaminationcanbeconsideredaseffectivemeansforenhancedlearning(e.g.homeassignments,oralexaminationsfollowingadesignassignment,etc),therearenotmanyexaminationtypesthatcombinetheneedforasummativeassessment,withadequatefeedbacktoinducelearning.Combinedwiththelargenumberofstudentsthatwearecurrentlyteaching,designofanidealexaminationsettingisatrulychallengingtask.TheworkbyDawson-Howeisanearlyattempttobringcomputersupportintotheprocessofprogrammingassignmentevaluationandadministration2.Theneedforautomatedexaminationsystemshasbecomemorepertinentduringthelate90swiththeadventofdistanceandlifelonglearning.Forexample,attheOpenUniversityinUKtherehavebeenattemptstoexchangestudentassignments,andtheir(subsequent)correctionandassessmentbyexaminersviaMSWorddocuments8.However,theavailablereports(e.g.theworkbyPriceandPetre)concentrateontheeaseofadministrationforcourseassignmentandgrading,ratherthanthepedagogicalfeedbackinanon-lineauthenticexamination.Inrecentyearsseveralauthorsreportonautomaticassessmentsystems,mostlyconcentratingonpresentationofthetechnicalaspectsofthesystemandtheresultsofthestudentsintermsofgrading4,5,7,8.Whilewesharetheaspirationoftheseresearchteamsandconductsimilarstudies,ourfocushasbeenontheformalevaluationofhowthestudentsperceivedtheexaminationenvironment.Inadditionwehavestudiedhowtheywereaffectedbyfactorsspecifictoauthenticexaminations,howthesystemperformanceandtheexaminerson-linebehaviouraffectstheperceivedloadonthestudent,andothersuchaspects.2TechnicaldescriptionoftheAESAEShasbeendevelopedusingtheJ2EEplatform.Thisrepresentsasinglestandardforimplementinganddeployingcomplexenterpriseapplications.Havingbeendesignedthroughanopenprocess,J2EEmeetsawiderangeofenterpriseapplicationrequirements,includingdistribution-specificmechanismssuchasmessagingsystem,scalabilityandmodularity.TheclientsarebasedontheModel-View-Controller(MVC)applicationarchitecture,whichseparatesthreedistinctformsoffunctionalitywithintheapplication:TheModelrepresentsthestructureofthedataintheapplication,aswellasapplication-specificoperationondata.TheViewaccessesdatafromthemodelandspecifieshowthatdatashouldbepresented.ViewsintheAESconsistofstand-aloneapplicationsthatprovideviewfunctionality.TheControllertranslatesuseractionsonthemodelandselectstheappropriateviewbasedonuserpreferences.TheAESisdesignedasasetoflooselycoupledmodules,whicharetightlycoupledinternally.Groupingfunctionalityintomodulesprovidesintegrationbetweenclassesthatcooperate,yetdecouplesclassesthatrefertoeachotheroccasionally.Modulardesignsupportsthedesigngoalthatsoftwarewillbereusable.Eachmodulehasaninterfacethatdefinesthemodulesfunctionalrequirementsandprovidesaplacewherelatercomponentsmaybeintegrated.TheAESincludesmodulesfor:StudentaccountsTeacheraccountsExamsExaminationProcessingMessagingStatisticsTheAESdesignisdividedintomultipletiers:theClienttier,theMiddletier(consistingofoneormoresub-tiers),andtheBackendtier(seefigure2.1).Partitioningthedesignintotiersallowsustochoosetheappropriatetechnologyforagivensituation.Multipletechnologiescanevenbeusedtoprovidethesameserviceindifferentsituations.Forexample,HTMLpages,JSPpages,andstand-aloneapplicationscanallbeusedintheclienttier.Eachofthethreetiersplaysaspecificroleinthedesign.TheClienttierisresponsibleforpresentingdatatotheuser,interactingwiththeuser,andcommunicatingwiththeothertiersofthesystem.InthiscasetheClienttieristheonlypartofthesystemvisibletotheuser.TheAESClienttierconsistsmainlyofastand-aloneapplicationthatcommunicateswiththeothertiersthroughwell-definedinterfaces.Amessage-orientedapproachbasedonJMS(JavaMessagingSystem)hasbeenchosentotakecareofthecommunicationbetweentheClienttierandtheMiddletier.TheMiddletierisresponsibleforanyprocessinginvolvingEnterpriseJavaBeans.EnterpriseJavaBeansaresoftwarecomponentsthatextendserverstoperformapplicationspecificfunctionality.TheinterfacebetweenthesecomponentsandtheircontainersisdefinedintheEnterpriseJavaBeansspecification.ThecontainersprovideservicestotheEnterpriseJavaBeansinstancestheycontain,suchascontrollingtransactions,managingsecurity,threadorotherresourcepooling,andhandlingpersistence,amongotherhigh-levelsystemtasks.TheBackendtieristhesysteminformationinfrastructure.Thistierincludesoneormorerelationaldatabasemanagementsystemsandpotentiallyotherinformationassetsthatcouldbeuseful,e.g.thecentraluniversitycourseresultsadministrationsystem(LADOK).TheEIStieralsoenforcessecurityandoffersscalability.TheBackendtierprovidesalayerofsoftwarethatmapsexistingdataandMiddleTierJNDIJavaNamingandDirectoryInterfaceBackendTierClientTierEJBContainerEnterpriseBeansMessageDrivenBeansJMSJavaMessagingServiceClientStandaloneSwingapplicationRDBMSFigure2.1:TheAESdesign.applicationresourcesintothedesignofAESinanimplementation-neutralway.Thesystemisseparatedintofivedifferentfunctionallayers,eachwithitsownresponsibilitiesanditsownAPI.Theselayersarephysicallysplitacrossthethreedifferenttiers.Thepersistencelayer,forexample,providesthemechanismsnecessarytopermanentlysaveobjectstate.ItprovidesbasicCRUD(create,read,update,delete)servicesandalsodealswiththeobject-to-relationalmappingissues.Thisleadstoamoreflexibleandmaintainablesystem,e.g.layerscanbechangedwithnoeffectonotherlayers,aslongastheAPIremainsconstant.3Examinationset-upTheexaminationsystemisonlyonepartoftheexaminationprocess.Thesecondpartistheset-up(therules)wehaveforthestudents.Wehavetriedafewset-upsoveranumberofyears(usingaprototypeforthesystemfor5-6years).3.1Thefirstset-upThefirstversionallowedthestudentstowritetheprogramsusingacomputerinsteadofwritingonpaper.Wefoundthismethodtobeanimprovementbecausewedidnothavetoread“illegible”textsandthesubmittedsolutionscouldbetestedafterwards.Gradeswerebasedonthenumberofcorrectlysolvedexercises.Aproblemwiththisset-upwasthatallthegradingstillhadtobedoneaftertheexamwasfinished.Mostofthestudentswaitedtosendinthesolutionsuntilthelastminuteoftheexam.3.2Thesecondset-upOurintentionwastohaveanexaminationwherethestudentsshouldhavearesponsefromtheexaminer(s)withinafewminutesandwheregradesweregiventothestudentswhentheylefttheexam.Wealsointendedtoprovidethestudentwiththepossibilityofgettingaresponseforeachexercisewithinafewminutes,sotheycouldcorrectanearlycorrectsolution.Thesecondset-up(whichweusetoday)isbasedonbothnumberofcorrectlysolvedexercisesandtheamountoftimetakentosolvethem.Anumberofdeadlinesaregiven.Ifthestudentwantsahighgradehe/shehastosolveanumberofexerciseswithinapre-specifiedtimelimit.Thecurrentexaminationprocessfollowsafewsteps:1.Thestudentsendsanexaminationrequestforanexercisetotheexaminer(s).2.Theexaminerscanreturnoneofthefollowingresults.Passed-thesolutioniscorrect.Incomplete-thesolutionhaserrors,andmustbecorrected.Itspossibletomakeanewattemptlater.Fail-thesolutionisincorrectandthestudentisnotallowedtocontinuetoworkonthisexercise.3.Everyexaminationattemptandtheresultwillcontributetothefinalexamgrade,andthestudentisinformedofhis/hercurrentgrade.Ifthestudentsubmitsanewexaminationrequestonanadditionalexercisehe/shecanreachahighergrade.ThisexaminationprocessisbuiltintoourcurrentAES,buttherules(timelimitsetc.)canbechangedforseparatecourses.Thismakesthesystemflexible.TimelimitsandgradingInthecoursesthissystemwastestedtherewerethreeexercisesineachexamandtherequirementsfordifferentgradeswere:Forthegrade5(excellent)thestudentmustcomplete:o3exercisescorrectin3hoursoro2exercisescorrectin2hoursForthegrade4(verygood)thestudentmustcomplete:o2exercisescorrectin3hoursoro1exercisecorrectin1.5hoursForthegrade3(passed)thestudentmustcomplete:o1exercisecorrectin4hoursTheaboveset-uptogetherwiththeAESsupportgivesustheopportunitytogradethestudentsduringtheexam.Studentswhohavesolvedanexerciseareinformedofthegradetheyhavereached.Iftheyaresatisfiedwiththatgradetheycanleavetheexam(manystudentsleaveafteronetotwohourswhentheyhavegrade4or5).StudentquestionsInanordinarycomputer-aidedexam,anumberofquestionsaresubmittedbythestudents,wheretheanswercaneitherbeclassifiedaspersonalorasinterestingforallstudents.Theexaminercandecideifhe/shewillsendtheanswertothewholegroupofstudentsorjusttoaspecificstudent.Thenumberofquestionsseemstoberelativelyconstantduringtheexam(approximately2-5questionsper5minutes).Mostquestionsaresentinduringthebeginningoftheexam,whichcanbeexplainedbythefactthatthestudentsaskaboutspecificthingspertainingtotheexercisesandthattherearemorestudentsinthebeginningoftheexam.Submission/approvalattemptsInanordinarycomputer-aidedexamwehavealargenumberofexaminationrequestsfromthestudents.Aswecanseeinfigure3.1wehavearelativelyhighfrequencyintheperiodfrom30minutesto3hours.Afterthat,mostofthestudentsleave(theycantgetahighergradethan3afterthattime).Aroundthedeadlineswecanseethattheexaminationattemptsappearmoreoften,butnotsignificantlymoreoften.Still,theincreaseofexaminationrequestsleadstomoreworkfortheexaminers.Thiscanresultinanincreaseintheresponsetime(waitingtimeforthestudent).4EvaluationmethodsThedevelopmentofthecurrentsystemstartedinsummer2001andcontinuedthroughwinter2001/2002.Whenwebegantestingthissystemwewantedasatestexampleacoursewithalargenumberofstudents.Oneofourintroductorycoursesinprogramminghasaround270studentseachyear,sothatwasourfirstchoice.Approximately180ofthesestudentsareIndustrialManagementEngineeringstudentsandtherestareTechnicalBiologystudents.Ourstatisticsarebasedontheirfirstexaminationinthiscourse,whichtookplaceinMarch2002.Wealsousedaretakeexaminthiscoursetodoanewstudywithanewsetofquestions.ThisevaluationwasdoneinMay2002.Inthesetwostudies,studentsfilledinquestionnairesdirectlyaftertheexam.Thefinalquestionnairehadtwoparts.Thefirstpartwasmainlyquestionswhereanswersareinfreetextformat.Thesecondpartincludedquestionswithscaledanswers(gradeontofive,disagree-agree,worse-better).Thefirstpartwasusedinthreeevaluations.Themoreextensivequestionnairewithtwopartswasusedonlyforthelastevaluation(i.e.forthetwolastexams).Theappendixshowsthefinalquestionnaire.Bothtypesofquestionnaireswereanonymousandthequestionnaireswerefilledinafterthegradingwasdonefortheexams.Thestudentshadalreadyreceivedtheirgradeswhentheyfilledinthequestionnaires.Webelievethatthisprovidesameasureofobjectivityonthestudentside.WealsousedthelogfilesfromtheAESfortheexamstogetstatisticaltrendsaboutgrades,gender,responsetimesforquestionsrespectivelyapprovalattemptsamongothers(seesection5).5EvaluationresultsUnfortunatelyalmostallstudentshadnopreviousexperiencewithpaperbasedprogrammingexaminations,sotherepliescouldnotbeusedforcomparisonswiththatexaminationform.However,weusedtheresponsetostudyotherquestionsindetail(speciallythepartrelatedtothetime/stressfactor).First,howoftenthestudentssentinarequest(questionsorapprovalattempts),andhowlongthetimeforaresponsewas?Secondly,howwellwastheexaminationsystemacceptedbythestudents?Athirdquestionwasacomparisonbygradesbetweenthegenders.Theresponserateofthequestionnaireswasquitegood.Wehadfourexamsduringtheevaluationperiodwiththefollowingresponserates:Exam1:76answersof112students(67.8%)Exam2:87answersof105students(82.8%)Exam3:50answersof66students(75.7%)Exam4:18answersof22students(81.8%)Thefirstthreequestionnairesweredoneatthefirstexaminationoccasionforthestudentsandthefourthonewasdoneinaretakeexaminationwhereallthestudentswerestudentswithnogradefr
温馨提示
- 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
- 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
- 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
- 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
- 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
- 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
- 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
最新文档
- 肝癌介入术后护理课件
- 二三河南数学试卷
- 高二北京数学试卷
- 高一金太阳湖南数学试卷
- 肛门闭锁护理课件
- 定州四年级期中数学试卷
- 2025至2030大理石行业市场深度调研及发展趋势与投资报告
- 2025至2030船舶机电设备行业市场深度研究及发展前景投资可行性分析报告
- 攀枝花市市直机关遴选公务员考试真题2024
- 关键期中考试数学试卷
- 广西燃气安全检查标准 DBJ T45-1472-2023(2023年7月1日实施)
- 湖南省长沙2024年七年级下册生物期末试卷附答案
- 2023-2024学年第二学期期末学业质量检测八年级语文试卷
- 统编版(2024)一年级语文上册拼音1《a o e》精美课件
- 农村房子继承人放弃继承协议书
- WS 329-2024 麻醉记录单标准
- 2024-2029全球及中国福利管理系统行业市场发展分析及前景趋势与投资发展研究报告
- 新标准英语小学五年级下各模块习题
- 开票税点自动计算器
- 人教版二年级数学下册乘除法专项考点练习
- JCT 2768-2024 木塑制品行业绿色工厂评价要求(正式版)
评论
0/150
提交评论