已阅读5页,还剩1页未读, 继续免费阅读
版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领
文档简介
EvaluationofanAuthenticExaminationSystem(AES)forProgrammingCoursesTorbjrnJonsson,PouriaLoghmaniandSiminNadjm-TehraniDepartmentofComputerandInformationScienceLinkpingUniversity,Swedentorjo,poulo,siminida.liu.seAbstractThispaperdescribesourexperiencewithanauthenticexaminationsystemforprogrammingcourses.Webrieflydescribethearchitectureofthesystem,andpresentresultsofevaluatingthesysteminrealexaminationsituations.Someofthefactorsstudiedindetailaretheon-lineinteractionsbetweenthestudentsandexaminers,theresponsetimesandtheireffectsonthepressureexperiencedbystudent,theacceptanceofthemethodamongthestudents,andwhethertheexaminationformisgender-neutral.IntroductionAsexperiencedteachersinprogrammingcourseswehavenoticedthedrawbacksinthetraditionalexaminationformusedinprogrammingcourses.Thestudentslearntoprogramvialaboratoryexercises,butthefinalevaluationoftheirabilitiesandthegradingoftheexaminationareinaformthatusespaperandpeninsteadofcomputers.Consideringthatthestudentwillneverusethismodeforproducingaprogramthroughtheprofessionallife,weconsiderthistobenotasuitablemethod.AttheDepartmentofComputerScienceatLinkpingUniversity12fundamentalprogrammingcoursesforapproximately1000studentsindifferenteducationalprogramsaretaughtannually.Thispaperdealswithanewpedagogicalviewintheseprogrammingcourses,whichcanbeappliedtoanyprogramminglanguage,typeofstudentandeducationalprogram.Theideaisbasedonextensivestudiesarounddifferentexaminationforms,whereindividualgrading,efficientandusefulfeedbackandtheauthenticityoftheexaminationformareusedasbasiccriteriaforthechoiceofexaminationmethod.Webelievethatthechoiceofmethodtogetherwiththeaddedefficiencyintheassessmentprocessimprovesthequalityofourstudyprogrammes.Inparticular,webelievethatitwillchangetheexaminationprocessfromasummativetoanormativeassessmentoccasion1.Foranumberofyearswehaveexperimentedwithtestingthestudentsviacomputer-aidedexaminationsinsomepilotcoursesanauthenticexaminationformforthistypeofcourse.However,thisexaminationformhasnotbecomemorewidespreadduetoinsufficientsupportforthecomputerenvironmentnecessaryforthiskindofexamination.Duringthepastyearanewauthenticexaminationsystem(AES)hasbeendeveloped,whereallthestudentsandtheexaminingteachersareconnectedtothesamesystem.Theprocess,includingcommunicationandgrading,issupportedbythisenvironment.Inthispaperwedescribetheexaminationsystemandourinitialevaluationsofthissysteminanumberofrelativelylargeexaminationsessions.ThecoursesinquestioncoveredprogramminginAdaandweretakenbyfirstandsecondyearstudents.DuringthepastyearwehaveevaluatedtheAES.Theinstrumentsusedfortheevaluationconsistedofquestionnairesfilledby231studentsoveraperiodof3monthsand4examinations.Thepaperisorganisedasfollows.Insection1wedescribewhythetypeofexaminationweproposeisthemostappropriateforprogrammingcoursesandcomparetosomerelatedsystems.Section2includesabrieftechnicaldescriptionoftheexaminationsystems,includingitsarchitecturaldesign.Insection3wedescribehowthecomputersystem,thatmanagestheexaminationprocesson-line,hastobeaugmentedbyrulessetupineachparticularcourse.Section4coversourevaluationmethodsandisfollowedbyevaluationresultsinsection5.Section6concludesthepaper.1ExaminationformsEveryexaminationmethodhasspecificcharacteristicsthatmakeitmoreorlessappropriatetoaparticularcoursesetting.HkanOswaldssonstudiedtherangeofpossibleexaminationformsforatypicalprogrammingcoursepriortothedevelopmentofthecurrentexaminationsysteminourdepartment5.Whileseveralmodesofexaminationcanbeconsideredaseffectivemeansforenhancedlearning(e.g.homeassignments,oralexaminationsfollowingadesignassignment,etc),therearenotmanyexaminationtypesthatcombinetheneedforasummativeassessment,withadequatefeedbacktoinducelearning.Combinedwiththelargenumberofstudentsthatwearecurrentlyteaching,designofanidealexaminationsettingisatrulychallengingtask.TheworkbyDawson-Howeisanearlyattempttobringcomputersupportintotheprocessofprogrammingassignmentevaluationandadministration2.Theneedforautomatedexaminationsystemshasbecomemorepertinentduringthelate90swiththeadventofdistanceandlifelonglearning.Forexample,attheOpenUniversityinUKtherehavebeenattemptstoexchangestudentassignments,andtheir(subsequent)correctionandassessmentbyexaminersviaMSWorddocuments8.However,theavailablereports(e.g.theworkbyPriceandPetre)concentrateontheeaseofadministrationforcourseassignmentandgrading,ratherthanthepedagogicalfeedbackinanon-lineauthenticexamination.Inrecentyearsseveralauthorsreportonautomaticassessmentsystems,mostlyconcentratingonpresentationofthetechnicalaspectsofthesystemandtheresultsofthestudentsintermsofgrading4,5,7,8.Whilewesharetheaspirationoftheseresearchteamsandconductsimilarstudies,ourfocushasbeenontheformalevaluationofhowthestudentsperceivedtheexaminationenvironment.Inadditionwehavestudiedhowtheywereaffectedbyfactorsspecifictoauthenticexaminations,howthesystemperformanceandtheexaminerson-linebehaviouraffectstheperceivedloadonthestudent,andothersuchaspects.2TechnicaldescriptionoftheAESAEShasbeendevelopedusingtheJ2EEplatform.Thisrepresentsasinglestandardforimplementinganddeployingcomplexenterpriseapplications.Havingbeendesignedthroughanopenprocess,J2EEmeetsawiderangeofenterpriseapplicationrequirements,includingdistribution-specificmechanismssuchasmessagingsystem,scalabilityandmodularity.TheclientsarebasedontheModel-View-Controller(MVC)applicationarchitecture,whichseparatesthreedistinctformsoffunctionalitywithintheapplication:TheModelrepresentsthestructureofthedataintheapplication,aswellasapplication-specificoperationondata.TheViewaccessesdatafromthemodelandspecifieshowthatdatashouldbepresented.ViewsintheAESconsistofstand-aloneapplicationsthatprovideviewfunctionality.TheControllertranslatesuseractionsonthemodelandselectstheappropriateviewbasedonuserpreferences.TheAESisdesignedasasetoflooselycoupledmodules,whicharetightlycoupledinternally.Groupingfunctionalityintomodulesprovidesintegrationbetweenclassesthatcooperate,yetdecouplesclassesthatrefertoeachotheroccasionally.Modulardesignsupportsthedesigngoalthatsoftwarewillbereusable.Eachmodulehasaninterfacethatdefinesthemodulesfunctionalrequirementsandprovidesaplacewherelatercomponentsmaybeintegrated.TheAESincludesmodulesfor:StudentaccountsTeacheraccountsExamsExaminationProcessingMessagingStatisticsTheAESdesignisdividedintomultipletiers:theClienttier,theMiddletier(consistingofoneormoresub-tiers),andtheBackendtier(seefigure2.1).Partitioningthedesignintotiersallowsustochoosetheappropriatetechnologyforagivensituation.Multipletechnologiescanevenbeusedtoprovidethesameserviceindifferentsituations.Forexample,HTMLpages,JSPpages,andstand-aloneapplicationscanallbeusedintheclienttier.Eachofthethreetiersplaysaspecificroleinthedesign.TheClienttierisresponsibleforpresentingdatatotheuser,interactingwiththeuser,andcommunicatingwiththeothertiersofthesystem.InthiscasetheClienttieristheonlypartofthesystemvisibletotheuser.TheAESClienttierconsistsmainlyofastand-aloneapplicationthatcommunicateswiththeothertiersthroughwell-definedinterfaces.Amessage-orientedapproachbasedonJMS(JavaMessagingSystem)hasbeenchosentotakecareofthecommunicationbetweentheClienttierandtheMiddletier.TheMiddletierisresponsibleforanyprocessinginvolvingEnterpriseJavaBeans.EnterpriseJavaBeansaresoftwarecomponentsthatextendserverstoperformapplicationspecificfunctionality.TheinterfacebetweenthesecomponentsandtheircontainersisdefinedintheEnterpriseJavaBeansspecification.ThecontainersprovideservicestotheEnterpriseJavaBeansinstancestheycontain,suchascontrollingtransactions,managingsecurity,threadorotherresourcepooling,andhandlingpersistence,amongotherhigh-levelsystemtasks.TheBackendtieristhesysteminformationinfrastructure.Thistierincludesoneormorerelationaldatabasemanagementsystemsandpotentiallyotherinformationassetsthatcouldbeuseful,e.g.thecentraluniversitycourseresultsadministrationsystem(LADOK).TheEIStieralsoenforcessecurityandoffersscalability.TheBackendtierprovidesalayerofsoftwarethatmapsexistingdataandMiddleTierJNDIJavaNamingandDirectoryInterfaceBackendTierClientTierEJBContainerEnterpriseBeansMessageDrivenBeansJMSJavaMessagingServiceClientStandaloneSwingapplicationRDBMSFigure2.1:TheAESdesign.applicationresourcesintothedesignofAESinanimplementation-neutralway.Thesystemisseparatedintofivedifferentfunctionallayers,eachwithitsownresponsibilitiesanditsownAPI.Theselayersarephysicallysplitacrossthethreedifferenttiers.Thepersistencelayer,forexample,providesthemechanismsnecessarytopermanentlysaveobjectstate.ItprovidesbasicCRUD(create,read,update,delete)servicesandalsodealswiththeobject-to-relationalmappingissues.Thisleadstoamoreflexibleandmaintainablesystem,e.g.layerscanbechangedwithnoeffectonotherlayers,aslongastheAPIremainsconstant.3Examinationset-upTheexaminationsystemisonlyonepartoftheexaminationprocess.Thesecondpartistheset-up(therules)wehaveforthestudents.Wehavetriedafewset-upsoveranumberofyears(usingaprototypeforthesystemfor5-6years).3.1Thefirstset-upThefirstversionallowedthestudentstowritetheprogramsusingacomputerinsteadofwritingonpaper.Wefoundthismethodtobeanimprovementbecausewedidnothavetoread“illegible”textsandthesubmittedsolutionscouldbetestedafterwards.Gradeswerebasedonthenumberofcorrectlysolvedexercises.Aproblemwiththisset-upwasthatallthegradingstillhadtobedoneaftertheexamwasfinished.Mostofthestudentswaitedtosendinthesolutionsuntilthelastminuteoftheexam.3.2Thesecondset-upOurintentionwastohaveanexaminationwherethestudentsshouldhavearesponsefromtheexaminer(s)withinafewminutesandwheregradesweregiventothestudentswhentheylefttheexam.Wealsointendedtoprovidethestudentwiththepossibilityofgettingaresponseforeachexercisewithinafewminutes,sotheycouldcorrectanearlycorrectsolution.Thesecondset-up(whichweusetoday)isbasedonbothnumberofcorrectlysolvedexercisesandtheamountoftimetakentosolvethem.Anumberofdeadlinesaregiven.Ifthestudentwantsahighgradehe/shehastosolveanumberofexerciseswithinapre-specifiedtimelimit.Thecurrentexaminationprocessfollowsafewsteps:1.Thestudentsendsanexaminationrequestforanexercisetotheexaminer(s).2.Theexaminerscanreturnoneofthefollowingresults.Passed-thesolutioniscorrect.Incomplete-thesolutionhaserrors,andmustbecorrected.Itspossibletomakeanewattemptlater.Fail-thesolutionisincorrectandthestudentisnotallowedtocontinuetoworkonthisexercise.3.Everyexaminationattemptandtheresultwillcontributetothefinalexamgrade,andthestudentisinformedofhis/hercurrentgrade.Ifthestudentsubmitsanewexaminationrequestonanadditionalexercisehe/shecanreachahighergrade.ThisexaminationprocessisbuiltintoourcurrentAES,buttherules(timelimitsetc.)canbechangedforseparatecourses.Thismakesthesystemflexible.TimelimitsandgradingInthecoursesthissystemwastestedtherewerethreeexercisesineachexamandtherequirementsfordifferentgradeswere:Forthegrade5(excellent)thestudentmustcomplete:o3exercisescorrectin3hoursoro2exercisescorrectin2hoursForthegrade4(verygood)thestudentmustcomplete:o2exercisescorrectin3hoursoro1exercisecorrectin1.5hoursForthegrade3(passed)thestudentmustcomplete:o1exercisecorrectin4hoursTheaboveset-uptogetherwiththeAESsupportgivesustheopportunitytogradethestudentsduringtheexam.Studentswhohavesolvedanexerciseareinformedofthegradetheyhavereached.Iftheyaresatisfiedwiththatgradetheycanleavetheexam(manystudentsleaveafteronetotwohourswhentheyhavegrade4or5).StudentquestionsInanordinarycomputer-aidedexam,anumberofquestionsaresubmittedbythestudents,wheretheanswercaneitherbeclassifiedaspersonalorasinterestingforallstudents.Theexaminercandecideifhe/shewillsendtheanswertothewholegroupofstudentsorjusttoaspecificstudent.Thenumberofquestionsseemstoberelativelyconstantduringtheexam(approximately2-5questionsper5minutes).Mostquestionsaresentinduringthebeginningoftheexam,whichcanbeexplainedbythefactthatthestudentsaskaboutspecificthingspertainingtotheexercisesandthattherearemorestudentsinthebeginningoftheexam.Submission/approvalattemptsInanordinarycomputer-aidedexamwehavealargenumberofexaminationrequestsfromthestudents.Aswecanseeinfigure3.1wehavearelativelyhighfrequencyintheperiodfrom30minutesto3hours.Afterthat,mostofthestudentsleave(theycantgetahighergradethan3afterthattime).Aroundthedeadlineswecanseethattheexaminationattemptsappearmoreoften,butnotsignificantlymoreoften.Still,theincreaseofexaminationrequestsleadstomoreworkfortheexaminers.Thiscanresultinanincreaseintheresponsetime(waitingtimeforthestudent).4EvaluationmethodsThedevelopmentofthecurrentsystemstartedinsummer2001andcontinuedthroughwinter2001/2002.Whenwebegantestingthissystemwewantedasatestexampleacoursewithalargenumberofstudents.Oneofourintroductorycoursesinprogramminghasaround270studentseachyear,sothatwasourfirstchoice.Approximately180ofthesestudentsareIndustrialManagementEngineeringstudentsandtherestareTechnicalBiologystudents.Ourstatisticsarebasedontheirfirstexaminationinthiscourse,whichtookplaceinMarch2002.Wealsousedaretakeexaminthiscoursetodoanewstudywithanewsetofquestions.ThisevaluationwasdoneinMay2002.Inthesetwostudies,studentsfilledinquestionnairesdirectlyaftertheexam.Thefinalquestionnairehadtwoparts.Thefirstpartwasmainlyquestionswhereanswersareinfreetextformat.Thesecondpartincludedquestionswithscaledanswers(gradeontofive,disagree-agree,worse-better).Thefirstpartwasusedinthreeevaluations.Themoreextensivequestionnairewithtwopartswasusedonlyforthelastevaluation(i.e.forthetwolastexams).Theappendixshowsthefinalquestionnaire.Bothtypesofquestionnaireswereanonymousandthequestionnaireswerefilledinafterthegradingwasdonefortheexams.Thestudentshadalreadyreceivedtheirgradeswhentheyfilledinthequestionnaires.Webelievethatthisprovidesameasureofobjectivityonthestudentside.WealsousedthelogfilesfromtheAESfortheexamstogetstatisticaltrendsaboutgrades,gender,responsetimesforquestionsrespectivelyapprovalattemptsamongothers(seesection5).5EvaluationresultsUnfortunatelyalmostallstudentshadnopreviousexperiencewithpaperbasedprogrammingexaminations,sotherepliescouldnotbeusedforcomparisonswiththatexaminationform.However,weusedtheresponsetostudyotherquestionsindetail(speciallythepartrelatedtothetime/stressfactor).First,howoftenthestudentssentinarequest(questionsorapprovalattempts),andhowlongthetimeforaresponsewas?Secondly,howwellwastheexaminationsystemacceptedbythestudents?Athirdquestionwasacomparisonbygradesbetweenthegenders.Theresponserateofthequestionnaireswasquitegood.Wehadfourexamsduringtheevaluationperiodwiththefollowingresponserates:Exam1:76answersof112students(67.8%)Exam2:87answersof105students(82.8%)Exam3:50answersof66students(75.7%)Exam4:18answersof22students(81.8%)Thefirstthreequestionnairesweredoneatthefirstexaminationoccasionforthestudentsandthefourthonewasdoneinaretakeexaminationwhereallthestudentswerestudentswithnogradefr
温馨提示
- 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
- 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
- 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
- 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
- 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
- 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
- 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
最新文档
- 2023年化妆品销售的总结与计划
- 2023年化学教育教学工作总结
- 太阳能干燥装置相关行业投资规划报告范本
- 医用高值耗材相关项目投资计划书
- 人教版二年级数学下册课件《用7、8的乘法口诀求商》
- 【语文】《在马克思墓前的讲话》课件 2023-2024学年统编版高中语文必修下册
- 云游戏时代的来临及其影响
- 中药材项目可行性研究报告及运营方案|瑞克咨询|2024年编|
- 政府招商项目市场研究报告及运营管理方案|瑞克咨询|2024年编制|
- 《到实验室去:粗盐中难溶性杂质的去除》优教教学课件
- 部编版六年级下册道德与法治全册优秀课件
- 2022年普通高级中学美术课程标准
- 南京师范大学科研项目绩效考核表
- 物业服务项目领导参观接待方案
- 小学生防溺水安全教育主题班会ppt市公开课一等奖省名师优质课赛课一等奖课件
- DB41-T 1960.2-2021公共机构能耗定额 第2部分:医疗机构-(高清现行)
- DB31T 1210-2020 非居住物业管理服务规范
- IATF16949:2016标准理解和解析试题含答案
- 《通分》-完整版课件
- (完整版)医学统计学题库
- 螺栓腐蚀分析及防腐措施
评论
0/150
提交评论