会员注册 | 登录 | 微信快捷登录 支付宝快捷登录 QQ登录 微博登录 | 帮助中心 人人文库renrendoc.com美如初恋!
站内搜索 百度文库

热门搜索: 直缝焊接机 矿井提升机 循环球式转向器图纸 机器人手爪发展史 管道机器人dwg 动平衡试验台设计

   首页 人人文库网 > 资源分类 > PDF文档下载

外文资料--Evaluation of an Authentic Examination System (AES).pdf

  • 资源星级:
  • 资源大小:71.51KB   全文页数:6页
  • 资源格式: PDF        下载权限:注册会员/VIP会员
您还没有登陆,请先登录。登陆后即可下载此文档。
  合作网站登录: 微信快捷登录 支付宝快捷登录   QQ登录   微博登录
友情提示
2:本站资源不支持迅雷下载,请使用浏览器直接下载(不支持QQ浏览器)
3:本站资源下载后的文档和图纸-无水印,预览文档经过压缩,下载后原文更清晰   

外文资料--Evaluation of an Authentic Examination System (AES).pdf

EvaluationofanAuthenticExaminationSystemAESforProgrammingCoursesTorbjörnJonsson,PouriaLoghmaniandSiminNadjmTehraniDepartmentofComputerandInformationScienceLinköpingUniversity,Sweden{torjo,poulo,simin}ida.liu.seAbstractThispaperdescribesourexperiencewithanauthenticexaminationsystemforprogrammingcourses.Webrieflydescribethearchitectureofthesystem,andpresentresultsofevaluatingthesysteminrealexaminationsituations.Someofthefactorsstudiedindetailaretheonlineinteractionsbetweenthestudentsandexaminers,theresponsetimesandtheireffectsonthepressureexperiencedbystudent,theacceptanceofthemethodamongthestudents,andwhethertheexaminationformisgenderneutral.IntroductionAsexperiencedteachersinprogrammingcourseswehavenoticedthedrawbacksinthetraditionalexaminationformusedinprogrammingcourses.Thestudentslearntoprogramvialaboratoryexercises,butthefinalevaluationoftheirabilitiesandthegradingoftheexaminationareinaformthatusespaperandpeninsteadofcomputers.Consideringthatthestudentwillneverusethismodeforproducingaprogramthroughtheprofessionallife,weconsiderthistobenotasuitablemethod.AttheDepartmentofComputerScienceatLinköpingUniversity12fundamentalprogrammingcoursesforapproximately1000studentsindifferenteducationalprogramsaretaughtannually.Thispaperdealswithanewpedagogicalviewintheseprogrammingcourses,whichcanbeappliedtoanyprogramminglanguage,typeofstudentandeducationalprogram.Theideaisbasedonextensivestudiesarounddifferentexaminationforms,whereindividualgrading,efficientandusefulfeedbackandtheauthenticityoftheexaminationformareusedasbasiccriteriaforthechoiceofexaminationmethod.Webelievethatthechoiceofmethodtogetherwiththeaddedefficiencyintheassessmentprocessimprovesthequalityofourstudyprogrammes.Inparticular,webelievethatitwillchangetheexaminationprocessfromasummativetoanormativeassessmentoccasion1.Foranumberofyearswehaveexperimentedwithtestingthestudentsviacomputeraidedexaminationsinsomepilotcourses–anauthenticexaminationformforthistypeofcourse.However,thisexaminationformhasnotbecomemorewidespreadduetoinsufficientsupportforthecomputerenvironmentnecessaryforthiskindofexamination.DuringthepastyearanewauthenticexaminationsystemAEShasbeendeveloped,whereallthestudentsandtheexaminingteachersareconnectedtothesamesystem.Theprocess,includingcommunicationandgrading,issupportedbythisenvironment.Inthispaperwedescribetheexaminationsystemandourinitialevaluationsofthissysteminanumberofrelativelylargeexaminationsessions.ThecoursesinquestioncoveredprogramminginAdaandweretakenbyfirstandsecondyearstudents.DuringthepastyearwehaveevaluatedtheAES.Theinstrumentsusedfortheevaluationconsistedofquestionnairesfilledby231studentsoveraperiodof3monthsand4examinations.Thepaperisorganisedasfollows.Insection1wedescribewhythetypeofexaminationweproposeisthemostappropriateforprogrammingcoursesandcomparetosomerelatedsystems.Section2includesabrieftechnicaldescriptionoftheexaminationsystems,includingitsarchitecturaldesign.Insection3wedescribehowthecomputersystem,thatmanagestheexaminationprocessonline,hastobeaugmentedbyrulessetupineachparticularcourse.Section4coversourevaluationmethodsandisfollowedbyevaluationresultsinsection5.Section6concludesthepaper.1ExaminationformsEveryexaminationmethodhasspecificcharacteristicsthatmakeitmoreorlessappropriatetoaparticularcoursesetting.HåkanOswaldssonstudiedtherangeofpossibleexaminationformsforatypicalprogrammingcoursepriortothedevelopmentofthecurrentexaminationsysteminourdepartment5.Whileseveralmodesofexaminationcanbeconsideredaseffectivemeansforenhancedlearninge.g.homeassignments,oralexaminationsfollowingadesignassignment,etc,therearenotmanyexaminationtypesthatcombinetheneedforasummativeassessment,withadequatefeedbacktoinducelearning.Combinedwiththelargenumberofstudentsthatwearecurrentlyteaching,designofanidealexaminationsettingisatrulychallengingtask.TheworkbyDawsonHoweisanearlyattempttobringcomputersupportintotheprocessofprogrammingassignmentevaluationandadministration2.Theneedforautomatedexaminationsystemshasbecomemorepertinentduringthelate90swiththeadventofdistanceandlifelonglearning.Forexample,attheOpenUniversityinUKtherehavebeenattemptstoexchangestudentassignments,andtheirsubsequentcorrectionandassessmentbyexaminersviaMSWorddocuments8.However,theavailablereportse.g.theworkbyPriceandPetreconcentrateontheeaseofadministrationforcourseassignmentandgrading,ratherthanthepedagogicalfeedbackinanonlineauthenticexamination.Inrecentyearsseveralauthorsreportonautomaticassessmentsystems,mostlyconcentratingonpresentationofthetechnicalaspectsofthesystemandtheresultsofthestudentsintermsofgrading4,5,7,8.Whilewesharetheaspirationoftheseresearchteamsandconductsimilarstudies,ourfocushasbeenontheformalevaluationofhowthestudentsperceivedtheexaminationenvironment.Inadditionwehavestudiedhowtheywereaffectedbyfactorsspecifictoauthenticexaminations,howthesystemperformanceandtheexaminersonlinebehaviouraffectstheperceivedloadonthestudent,andothersuchaspects.2TechnicaldescriptionoftheAESAEShasbeendevelopedusingtheJ2EEplatform.Thisrepresentsasinglestandardforimplementinganddeployingcomplexenterpriseapplications.Havingbeendesignedthroughanopenprocess,J2EEmeetsawiderangeofenterpriseapplicationrequirements,includingdistributionspecificmechanismssuchasmessagingsystem,scalabilityandmodularity.TheclientsarebasedontheModelViewControllerMVCapplicationarchitecture,whichseparatesthreedistinctformsoffunctionalitywithintheapplicationTheModelrepresentsthestructureofthedataintheapplication,aswellasapplicationspecificoperationondata.TheViewaccessesdatafromthemodelandspecifieshowthatdatashouldbepresented.ViewsintheAESconsistofstandaloneapplicationsthatprovideviewfunctionality.TheControllertranslatesuseractionsonthemodelandselectstheappropriateviewbasedonuserpreferences.TheAESisdesignedasasetoflooselycoupledmodules,whicharetightlycoupledinternally.Groupingfunctionalityintomodulesprovidesintegrationbetweenclassesthatcooperate,yetdecouplesclassesthatrefertoeachotheroccasionally.Modulardesignsupportsthedesigngoalthatsoftwarewillbereusable.Eachmodulehasaninterfacethatdefinesthemodulesfunctionalrequirementsandprovidesaplacewherelatercomponentsmaybeintegrated.TheAESincludesmodulesforStudentaccountsTeacheraccountsExamsExaminationProcessingMessagingStatisticsTheAESdesignisdividedintomultipletierstheClienttier,theMiddletierconsistingofoneormoresubtiers,andtheBackendtierseefigure2.1.Partitioningthedesignintotiersallowsustochoosetheappropriatetechnologyforagivensituation.Multipletechnologiescanevenbeusedtoprovidethesameserviceindifferentsituations.Forexample,HTMLpages,JSPpages,andstandaloneapplicationscanallbeusedintheclienttier.Eachofthethreetiersplaysaspecificroleinthedesign.TheClienttierisresponsibleforpresentingdatatotheuser,interactingwiththeuser,andcommunicatingwiththeothertiersofthesystem.InthiscasetheClienttieristheonlypartofthesystemvisibletotheuser.TheAESClienttierconsistsmainlyofastandaloneapplicationthatcommunicateswiththeothertiersthroughwelldefinedinterfaces.AmessageorientedapproachbasedonJMSJavaMessagingSystemhasbeenchosentotakecareofthecommunicationbetweentheClienttierandtheMiddletier.TheMiddletierisresponsibleforanyprocessinginvolvingEnterpriseJavaBeans.EnterpriseJavaBeansaresoftwarecomponentsthatextendserverstoperformapplicationspecificfunctionality.TheinterfacebetweenthesecomponentsandtheircontainersisdefinedintheEnterpriseJavaBeansspecification.ThecontainersprovideservicestotheEnterpriseJavaBeansinstancestheycontain,suchascontrollingtransactions,managingsecurity,threadorotherresourcepooling,andhandlingpersistence,amongotherhighlevelsystemtasks.TheBackendtieristhesysteminformationinfrastructure.Thistierincludesoneormorerelationaldatabasemanagementsystemsandpotentiallyotherinformationassetsthatcouldbeuseful,e.g.thecentraluniversitycourseresultsadministrationsystemLADOK.TheEIStieralsoenforcessecurityandoffersscalability.TheBackendtierprovidesalayerofsoftwarethatmapsexistingdataandMiddleTierJNDIJavaNamingandDirectoryInterfaceBackendTierClientTierEJBContainerEnterpriseBeansMessageDrivenBeansJMSJavaMessagingServiceClientStandaloneSwingapplicationRDBMSFigure2.1TheAESdesign.applicationresourcesintothedesignofAESinanimplementationneutralway.Thesystemisseparatedintofivedifferentfunctionallayers,eachwithitsownresponsibilitiesanditsownAPI.Theselayersarephysicallysplitacrossthethreedifferenttiers.Thepersistencelayer,forexample,providesthemechanismsnecessarytopermanentlysaveobjectstate.ItprovidesbasicCRUDcreate,read,update,deleteservicesandalsodealswiththeobjecttorelationalmappingissues.Thisleadstoamoreflexibleandmaintainablesystem,e.g.layerscanbechangedwithnoeffectonotherlayers,aslongastheAPIremainsconstant.3ExaminationsetupTheexaminationsystemisonlyonepartoftheexaminationprocess.Thesecondpartisthesetuptheruleswehaveforthestudents.Wehavetriedafewsetupsoveranumberofyearsusingaprototypeforthesystemfor56years.3.1ThefirstsetupThefirstversionallowedthestudentstowritetheprogramsusingacomputerinsteadofwritingonpaper.Wefoundthismethodtobeanimprovementbecausewedidnothavetoreadillegibletextsandthesubmittedsolutionscouldbetestedafterwards.Gradeswerebasedonthenumberofcorrectlysolvedexercises.Aproblemwiththissetupwasthatallthegradingstillhadtobedoneaftertheexamwasfinished.Mostofthestudentswaitedtosendinthesolutionsuntilthelastminuteoftheexam.3.2ThesecondsetupOurintentionwastohaveanexaminationwherethestudentsshouldhavearesponsefromtheexaminerswithinafewminutesandwheregradesweregiventothestudentswhentheylefttheexam.Wealsointendedtoprovidethestudentwiththepossibilityofgettingaresponseforeachexercisewithinafewminutes,sotheycouldcorrectanearlycorrectsolution.Thesecondsetupwhichweusetodayisbasedonbothnumberofcorrectlysolvedexercisesandtheamountoftimetakentosolvethem.Anumberofdeadlinesaregiven.Ifthestudentwantsahighgradehe/shehastosolveanumberofexerciseswithinaprespecifiedtimelimit.Thecurrentexaminationprocessfollowsafewsteps1.Thestudentsendsanexaminationrequestforanexercisetotheexaminers.2.Theexaminerscanreturnoneofthefollowingresults.Passedthesolutioniscorrect.Incompletethesolutionhaserrors,andmustbecorrected.Itspossibletomakeanewattemptlater.Failthesolutionisincorrectandthestudentisnotallowedtocontinuetoworkonthisexercise.3.Everyexaminationattemptandtheresultwillcontributetothefinalexamgrade,andthestudentisinformedofhis/hercurrentgrade.Ifthestudentsubmitsanewexaminationrequestonanadditionalexercisehe/shecanreachahighergrade.ThisexaminationprocessisbuiltintoourcurrentAES,buttherulestimelimitsetc.canbechangedforseparatecourses.Thismakesthesystemflexible.TimelimitsandgradingInthecoursesthissystemwastestedtherewerethreeexercisesineachexamandtherequirementsfordifferentgradeswereForthegrade5excellentthestudentmustcompleteo3exercisescorrectin3hoursoro2exercisescorrectin2hoursForthegrade4verygoodthestudentmustcompleteo2exercisescorrectin3hoursoro1exercisecorrectin1.5hoursForthegrade3passedthestudentmustcompleteo1exercisecorrectin4hoursTheabovesetuptogetherwiththeAESsupportgivesustheopportunitytogradethestudentsduringtheexam.Studentswhohavesolvedanexerciseareinformedofthegradetheyhavereached.Iftheyaresatisfiedwiththatgradetheycanleavetheexammanystudentsleaveafteronetotwohourswhentheyhavegrade4or5.StudentquestionsInanordinarycomputeraidedexam,anumberofquestionsaresubmittedbythestudents,wheretheanswercaneitherbeclassifiedaspersonalorasinterestingforallstudents.Theexaminercandecideifhe/shewillsendtheanswertothewholegroupofstudentsorjusttoaspecificstudent.Thenumberofquestionsseemstoberelativelyconstantduringtheexamapproximately25questionsper5minutes.Mostquestionsaresentinduringthebeginningoftheexam,whichcanbeexplainedbythefactthatthestudentsaskaboutspecificthingspertainingtotheexercisesandthattherearemorestudentsinthebeginningoftheexam.Submission/approvalattemptsInanordinarycomputeraidedexamwehavealargenumberofexaminationrequestsfromthestudents.Aswecanseeinfigure3.1wehavearelativelyhighfrequencyintheperiodfrom30minutesto3hours.Afterthat,mostofthestudentsleavetheycantgetahighergradethan3afterthattime.Aroundthedeadlineswecanseethattheexaminationattemptsappearmoreoften,butnotsignificantlymoreoften.Still,theincreaseofexaminationrequestsleadstomoreworkfortheexaminers.Thiscanresultinanincreaseintheresponsetimewaitingtimeforthestudent.4EvaluationmethodsThedevelopmentofthecurrentsystemstartedinsummer2001andcontinuedthroughwinter2001/2002.Whenwebegantestingthissystemwewantedasatestexampleacoursewithalargenumberofstudents.Oneofourintroductorycoursesinprogramminghasaround270studentseachyear,sothatwasourfirstchoice.Approximately180ofthesestudentsareIndustrialManagementEngineeringstudentsandtherestareTechnicalBiologystudents.Ourstatisticsarebasedontheirfirstexaminationinthiscourse,whichtookplaceinMarch2002.Wealsousedaretakeexaminthiscoursetodoanewstudywithanewsetofquestions.ThisevaluationwasdoneinMay2002.Inthesetwostudies,studentsfilledinquestionnairesdirectlyaftertheexam.Thefinalquestionnairehadtwoparts.Thefirstpartwasmainlyquestionswhereanswersareinfreetextformat.Thesecondpartincludedquestionswithscaledanswersgradeontofive,disagreeagree,worsebetter.Thefirstpartwasusedinthreeevaluations.Themoreextensivequestionnairewithtwopartswasusedonlyforthelastevaluationi.e.forthetwolastexams.Theappendixshowsthefinalquestionnaire.Bothtypesofquestionnaireswereanonymousandthequestionnaireswerefilledinafterthegradingwasdonefortheexams.Thestudentshadalreadyreceivedtheirgradeswhentheyfilledinthequestionnaires.Webelievethatthisprovidesameasureofobjectivityonthestudentside.WealsousedthelogfilesfromtheAESfortheexamstogetstatisticaltrendsaboutgrades,gender,responsetimesforquestionsrespectivelyapprovalattemptsamongothersseesection5.5EvaluationresultsUnfortunatelyalmostallstudentshadnopreviousexperiencewithpaperbasedprogrammingexaminations,sotherepliescouldnotbeusedforcomparisonswiththatexaminationform.However,weusedtheresponsetostudyotherquestionsindetailspeciallythepartrelatedtothetime/stressfactor.First,howoftenthestudentssentinarequestquestionsorapprovalattempts,andhowlongthetimeforaresponsewasSecondly,howwellwastheexaminationsystemacceptedbythestudentsAthirdquestionwasacomparisonbygradesbetweenthegenders.Theresponserateofthequestionnaireswasquitegood.WehadfourexamsduringtheevaluationperiodwiththefollowingresponseratesExam176answersof112students67.8Exam287answersof105students82.8Exam350answersof66students75.7Exam418answersof22students81.8Thefirstthreequestionnairesweredoneatthefirstexaminationoccasionforthestudentsandthefourthonewasdoneinaretakeexaminationwhereallthestudentswerestudentswithnogradefromanearlierexam.5.1EventsduringanexaminationThenumberofevents,questionsandexaminationrequests,spreadoveranexaminationsessionof4hourscanbeaninterestingmetrictolookat.Themajornegativefactorthatwasindicatedinthequestionnaireswasthefeelingoftimepressureorstress.17ofthefreetextanswershadsomeconnectiontothisfactor.Fromatechnicalpointofviewwewerealsointerestedinfindingthatthecapacityofthesystemwasadequate.Thereforewehavesummarisedthenumberofinteractionstakingplaceineveryexam.Infigure5.1wecanseethatthenumberofquestionsishigherinthebeginningofanexamination,butwehavequestioneventsoverthewholeexaminationtime.Thenumberofexaminationrequestsisrelativetotime.Therewereafewrequestsinthefirsthalfanhourandthatthefirsttwohoursarebusyfortheexaminers.Therequestrateisquitehighwhenwereachthetimelimitsforthegradesespeciallythe4hourlimit.Fromatechnicalpointofviewthesystemperformanceundertheaboveloadshasbeenadequate.Tostudythestudentexperienceofstressduetowaitingtimewehavecalculatedtheaveragewaitingfortheanswertoaquestionandanapprovalofanexaminationrequestrespectively.Wehavealsolookedattheextremevalues.Itturnsoutthatforaquestiontheshortestansweringtimewas30secondsandthelongest6minutes.Thecorrespondingfiguresforapprovalattemptswere1minuteand10minutesrespectively.Thefirsttypeofinteractiontook2minutesand42seconds,andthesecondtype3minutesand31secondsonaverageforoneparticularexam.Thestudentresponses,fromthequestionnaires,onthisamountoftimeisthatitisacceptabletowaitaminuteorEventsduringexamsin10minuteintervals0510152025TimefromstartQuestionsExaminationsFigure5.1Studenteventsquestionsandexaminationrequestsduringanexam.

注意事项

本文(外文资料--Evaluation of an Authentic Examination System (AES).pdf)为本站会员(淘宝运营)主动上传,人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对上载内容本身不做任何修改或编辑。 若此文所含内容侵犯了您的版权或隐私,请立即通知人人文库网([email protected]),我们立即给予删除!

温馨提示:如果因为网速或其他原因下载失败请重新下载,重复下载不扣分。

copyright@ 2015-2017 人人文库网网站版权所有
苏ICP备12009002号-5