翻译以.原文和在同一文件中前

基于声望机制的大规模 大规模开放课堂（简称MOOC）主要基于供学生相互交流的。我们通过一个有1101名参与者的在edX平台上的现场实验来的设计如何影响学生的活动和学习成果。介绍一种为发布有益内容的学生提供点数的声望机制。我们发现，在MOOC中的使用与更高的成绩和更好的力相关联，这点和在其他环境的影响。这一点表明了对于MOOC来说是必不可少的，声望机制可以优化的中，我们也给出了一套对于在MOOC上进行试验的指导方案。：大规模开放课堂，MOOC，，声望机ACM分类：H.5.3信息界面和演示（如：人机互动：群组和组织界面 计算机和教育：计算机在教育中的大规模开放课堂，比如最近由Coursera，edX和Udacity运营的MOOCs，通常在开放网络[36]的众多学习社区中这种的模式是广泛而成功的。然而，并不是所有一点已经在MOOC的实践中被注意到了。Yahoo!AnswersStackOverflow这样的大型的，受欢迎的解答问题的网络平样的额外。我们着重一般中的声望机制能否在MOOC的环境中帮助促进1)的参与度和学生的学习成果是否有关系，比如学生的力和最终的成绩。对学生的传统课程有益，在MOOC中是否也有类似的效果呢？声望机制是否可以提高的参与度，比如减少待回答问题的数和缩短问题被StackOverflow的声望机制帮助问题被解答的时我们在edX平台上一个独立课程的跨领域的实验中了这些问题。1101名我们发现的参与程度与更高的成绩和更强的力相关。声望特性的出现显著MOOC的内容上并没有实质上地帮助提高学习评估的结果和促进社区的建的措辞用法；当前一代的MOOC；依次探讨这些问题。邮件列表和Usenet网[7]。有效的通过对社区的建设和互相协作的来促进学生我们发现自发参与到中的学生取得更好的考试成绩并且综合来说表现得更好，相关：被察觉到的学习的重要性和同龄人带来的压力——学生感受到同龄人向中投问答和声望机在用于信息性目的的同时也用于交流的目的[18]Yahoo!Answers用来讨论大众类型的问题，StackOverflow用来讨论软件发展的问题。对于现有的调如Adamic的团队通过收集基于文本和用户特征的不同类别的问题来分析用户在Yahoo!出可信赖用户的机制[29]，现在已经被应用于支持学生[6]和社会协作的学习系统[32]。在Tausczik的对MathOverflow（一个用于数学研究的问答）[34]的中这声望机制作为外在的刺激因素也可能导致消极的。Dearman发现Answers的用户无形之中用声望值抑制了回答——如果提问者被发现较低的分数来市场的和一个带有前十名榜的系统[19]的比较，基于声望特性的系统拥有处的却几乎没有。研究者已经系统性地研究了社区其他显著特性的影响。Harper[16,17]和Ludford[21]了在MovieLens推荐上的被用来提高参与度的并且在像可汗学院这样的教学拥有先驱[37]在MOOC中是普遍的并且被参与到中的学生。相比之下，我们提供了一个对设计替代品以及他们对MOOC学作为代替，我们志在评估对现有的容易在目前的MOOC中展开的机制进行增强领域实验“CS169.1x:用作服务的软件”来自加州大学伯克利分校）包含了对刚刚出现材料测试的简短评估。这个课程已经分别在Coursera平台上和edX序。其他MOOC供应商像Coursera也有类似的设计。期课程的。老师讲授内容但是并不参与到中。我们构软用个测性的个变来替标的edX生和社区助教第一次的时候他们根据被打乱的用户名被随机分配到两个实验使测试出来。两个都是基于开源的OSQA软体[27]，这和StackOverflow类似（见图1。像edX一样，OSQA被设计用于回答特定实际问题使用至多三个根特。这个模式鼓励成员提出对于一般课程相关讨论个人存在的具体问题。在其中一个实验性——基本!用户可以好评或者差评问题、回答和评论来找到最有用的信息。收到好评!用户会基于他们收到多少好评得到声望值。声望值会给用户带来好处并且那些!anchor是一个可以自然而然促进讨论的课程材料，比如一次家庭作业或是一存在大量的遗漏[3]2013年三月份的课程呈现出了5985得更晚，开始于1572名学员结束语443名通过的学员（见表1 pletion-RateFCCR)Russell设计的用来测量最后通过课程的学员数占听第一节课学员数比例度量方式，这个比例是11%，低于给他们的PowerSearching课程测量的19%36%，这表明存在更高的损耗。1101名学生被挑选参与到这项试验中，有548名学生在全功能中，另外553们是否参加了实验都同意允许者研究他们的edX数据在任意一周内13%--获取到。对于同意参加到实验中的1101名学生来说，关于他们使用情况的详细信详细的行为和他们的edX成绩关联起来。图2：每周的和课程的猜测二：参与者会比未参与者有更好的力声望特性可以通过鼓励的参与和提供经验丰富用户在讨论中的内容来增 是说，学生们在基本中是否使用言语上的暗示比如“有人可以帮我吗”来说课程成果：最终成绩和的任何都和课程有关。我们从edX和活动提供的点击数据量由我们自己的0-100分制，50分及格。但是由于较低的完成率，MOOC上的最终成绩有不同的解释：100%25%我们通过如下的先前分析的模式来使的参与度可控[24]。我们问题回答的数量。有大量没有被回答的问题象征着一个和较低社区感相关联的不活跃的[10]。我们使用Rovai的课堂社区度量[30]来测量每个用户的社区感。在这个自我评定中，参与者会回答20个关于对社区感受的问题，这些问题是以五分的利克示的等级中，从0到80。Rovai度量在这里被应用，比如，将特定数值的数据和社区感联系在一起[10]。社区感作为课程结束的一部分进行评定。Fisher精确的测试表明全功能上的用户相比而言不太可能内容（p约为的问题和回答。基本同样有的问题被提出，并且基本上的内容比全功能论坛上的内容更可能成为问题，就是说有更少的回答被发布（p10的-6次幂。尽管是在全功能中第一名和第二名分别发布的23%和9%的内容。而非前十名用户发布境，中所有的内容对于所有的用户都是开放的，并且这回影响其他用户的行为，并用户平均将两个应用于每个问题（要求每个问题至少应用一个，有非参数的Mann-WhitneyUrank-sum测试来确定一个用户是否比其他用户更具价值。全站内容的23%和9%脱颖而出。使用与更高的成绩和更强的力相Cheng的团队[5]已经说明志愿加入的学生表现得更好并且取得更高的考试成过至少一次测试的用户定义为非用于（共计3221名用户这将1192名了课0%的非用户，就像图4展示的那样。这个差异是显著的（Mann-WhitneyU=701305.5，n11321n22622,p0.001U480786.0p0001这个差异是明显的(Mann-WhitneyU=5609,n1=115,n2=144,p<0.001)，支撑了猜测2。 图5：用户和非用户的百分比，相对地，还有每周放弃课程的人数，最后超过20%的非用户在第一周退出了课程，有将近50%的用户坚持到课程结束 庭作业（没有显示出差异）之外评估的高分有很大的关系。例如，在家庭作业3中用户和非用户的中值成绩都是500，并没有发现十分显著的差异(Mann-‐‐WhitneyU=9109.5,n1=350,n2=56,p>0.1)。表4总结了所有评估的结果。再一次，因为用声望特性改善回复数在两组中参与到中的情况是类似的：在课程期间大多数学员只了几个个被用户的明显差异(Mann-‐‐WhitneyU=150996,n1=546,n2=555,p>0.4)。图6中的直方图显示了惊人相似的分布。而在之间主动参与的情况则显示出很明显的不同。在全功能中的问题收到了的回复并且收到第一条回复的时间更短。在全功能中一个问题平均收到3.5条回复而在基本中只收到平均2.4条回复，这个差距是显著的。为了测量收到第一条回复的时间没有收到回复的问题排除，但是这种情况在全功能中很少见。这个差异依旧十分明显(Mann-‐‐WhitneyU4626n1110n2120p0.001)，支撑了猜测3。见图7和图8。 声望机制对学生的成果没有显著的在全功能和基本中学生的最终成绩中值分别为73分和71分（见图9。没有发现显著的差异（Mann-WhitneyU4235.5n189n297,p044没小团体拥有极高的保持里（两个都有超过85%的用户参与到了课程的结束，但提况；即从他们开始课程到他们最后一次进入经过的天数，在图10可以看到。在全功能和基本中保留天数的中值分别为24.5天和26.1天，并没有显著的差异。(Mann-WhitneyU=147315,n1=546,n2=555,p>0.2)，并没有为猜测5提供。有收到至少一条回复的问题第一条远超基本给出有力的证明（全功能49-50%的比率对基本30-35%的比率，12小时或以上时间才得到回复的问题所占比率不到一半。声望机制对社区感并无显著的omnibusnormaltest31]k2=0.252.8,p02Rovai度量问题被包含在定性的之中，全功能和基本份额别有43和40条回复。平均分数相近：全功能和基本分别在总分80分的度量中得到48.5和49.8的平均分，表明了一种适中的社区感。见图11。间没有发现明显的差异（t(81)=--‐:,p>0.7猜测6没有被支持。声望特性减少了对帮助目地评估（不使用内容所属的知识。评估由三个独立的评估人来决定内容是否在在全功能中819条内容中的24（2.9%和546名不同用户中的1931名用户（5.6%）使用了寻求帮助的措辞。这个差异很显著（Fisher’sexacttest,0.0002性质上的回复率在8%。所有的问题都是选择的。83%的学生使用化名；这种可能会在一个从0到4的度量上，对于两个的益处评估的平均得分都是2.0，我们没有发现显著的差异。(Mann-WhitneyU=629.0,n1=37,n2=39,p>0.1)。类似的，当被p>0.3)。如图12显示的那样，学生自我陈述说他们回复内容的主要原因是为了帮助其相比较而言，MathOverflow的用户倾向于低估他们增加声望值的。尤其是当 我们在整个环境下回顾我们的结果，列出重要的限制，并且提出建议供MOOC运声望机制为MOOC提供了具体但有限的好处。它们可以加快回复时间并且增所以这些内容并不是富有成效的。在全功能上发布内容或是点赞的用户数量和基本上的数量相仿（见表2），表明了点赞也许正在取代全功能上的一些内容。的MOOC平台中被广泛应用也就不足为奇了。的理想的平均分数难度很低。非用户在个人评估中的全部劣势表明了低的保留度并限制和 比例。在全功能中，43%的内容是由前5名用户贡献的，在基本中21%的内容页 间信息的流通，影响课程结果和活动。MOOC是，这样的实验也着在传统课堂中不会遇到的独特。在这一部分我们概述一些给MOOC研究者和MOOC平台运营者的建议。给MOOC者的建将保留度和表现分开：在MOOC中，大量人员流失很普遍。像成绩这样的度不要依赖后期的：因为人员的流失，后期的趋向于捕捉一小群有倾向性的的目的以及作为预备的功能，并使课程中的进步测量变得可能。入了可以引导学生到由实验者掌控的外部，牺牲了完整的体验来换取的操控性。我们认证与edX的OpenID接口重合的edX用户名，简化了过MOOC平台的建平台上复杂的扩展和客户端的。这些显然可以替换或扩展现有的课程功能，并且易于与他人。持联合，并且使外部请求有关用户信息或代表他们采取行动（用他们的权使数据易于：在可能的范围内，MOOC平台应该事先从所有学生那里得到清晰广泛的，以研究为目的并且在不扩散的前提下将他们不敏感的数据无偿地向公我们给出了一个针对和声誉机制对MOOC学生影响的领域实验的结果。探讨的长期复杂的讨论展开的学生是否有效。和像StackOverflow这样的成功其稳定的贡献相。在这项工作中的使用的代表一个单独的设计，和保留在其他设计中的模式和标准模式从像StackOverflow这样的其他有不同的点数，点数的另我们感谢SamJoseph和WorldTAsofCS169.1x协调对中学生的支持。这项工作作为国家科学的IIS- 和国家人文HK-50011名下ShouldYourMOOCForumUseaReputation Armando MartiA. ComputerScienceDivisionandSchoolofInformationUniversityofCalifornia,Berkeley,CA94720,USAMassiveopenonlinecourses(MOOCs)relyprimarilyondis-cussionforumsforinteractionamongstudents.Weinves-tigatehowforumdesignaffectsstudentactivityandlearn- esthroughafieldexperimentwith1101partici-pantsontheedXplatform.Weintroduceareputationsys-tem,whichgivesstudentspointsformakingusefulposts.Weshowthat,asinothersettings,useofforumsinMOOCsiscorrelatedwithbettergradesandhigherretention.Reputationsystemsadditionallyproducefasterresponsetimesandlargernumbersofresponsesperpost,aswellasdifferencesinhowstudentsaskquestions.However,reputationsystemshavenosignificantimpactongrades,retention,orthestudents’sub-jectivesenseofcommunity.Thissuggeststhatforumsarees-sentialforMOOCs,andreputationsystemscanimprovetheforumexperience,butothertechniquesareneededtoimprove esandcommunityformation.WealsogiveasetofguidelinesforrunningfieldexperimentsonMOOCs.Massiveopenonlinecourse;MOOC;forum;reputationACMH.5.3.InformationInterfacesandPresentation(e.g.HCI):GroupandOrganizationInterfaces;K.3.1.ComputersandEducation:ComputerUsesinEducationMassiveopenonlinecourses(MOOCs),suchasthosecur-rentlyoperatedbyCoursera,edX,andUdacity,typicallyin-cludehundredstothousandsofstudentsineachcourse.Tra-ditionalsocialstructuresofon-campuscourses,suchasdirectinteractionofteacherswitheachstudent, einfeasiblyexpensiveatthisscale.AsChamberlinandParishlamented,“Thenumberofpeopleinacoursecanbeoverwhelming!Howdoessomeoneconnectwith2000-pluspeople?”[4].∗TheauthorshavereleasedthisworkundertheCreativeCom-monsAttribution3.0License Permissiontomakedigitalorhardcopiesofpartorallofthisworkfor alorclassroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributedforprofitorcommercialadvantageandthatcopiesbearthisnoticeandthefullcita-tiononthefirstpage.Copyrightsforthird-partycomponentsofthisworkmustbehonored.Forallotheruses,contacttheowner/author(s).Copyrightisheldbytheau-CSCW’14,February15–19,2014,Baltimore,Maryland,USA.ACM978-1-4503-2540-0/14/02.

Ascalablealternativeisinteractionamongstudentsinanon-linediscussionforum,inwhichstudentscanassistonean-other.Thismodelhasbeenpervasiveandsuccessfulinlearn-ingcommunitiesontheopenweb[36].However,somefo-rumsareeffectiveincreatinglearningcommunitieswhileothersarenot,andapoorsenseofcommunityhasbeenlinkedtohighstudentattrition[39],whichhasbeenobservedinpracticeinMOOCs[8].Large,popularquestionansweringforumsontheweb,suchasYahoo!AnswersandStackOverflow,havebeensuccess-fulingeneratingcommunitieswithhighparticipation,quickresponses,andqualitycontent[1,24].Theseforumsgener-allyrelyonreputationsystems[29],whichprovideextrinsicrewardssuchas“reputationpoints”fortimely,high-qualitycontributions.Weinvestigatewhetherforumsingeneral—andreputationsystemsinparticular—canhelptopromotelearningandasuccessfulstudentcommunityintheMOOCcontext.Wetargetfivemainresearchquestions:Isforumparticipationcorrelatedwithstudentlearninges,suchasstudentretentionandfinalgrades?On-lineforumsbenefitstudentsintraditionalcourses[5]—dotheyhavesimilareffectsinaMOOC?likenumberofunansweredquestionsandresponsetime?PriorworkshowedthatStackOverflow’sreputationsystemhelpstokeepresponsetimesclosetotheminimumpossi-ble[24].CanareputationsystemachievesimilarbenefitsinaMOOCforum?Doreputationsystemsaffectthetypeofcontentpostedonclassforums?Differentincentivesystemschangethekindsofquestionsthatareasked[19].Willthereputationsystemalterhowstudentsaskquestions?Doreputationsystemsaffectstudentlearning es?Ifreputationsystemsincreaseforumparticipation,studentsmayhaveaccesstoalternativeexplanationsofconceptsandspendmoretimeexplainingmaterialtoothers(asinreciprocalteaching[28]),whichcouldleadtoimprovedlearningasmeasuredbycourseassessments.Doreputationsystemsaffectthestudent’ssubjectivesenseofcommunity,theirsensethattheybelongtoacommunityoflearnerswhotheycanrelyupon?Theincentivespro-videdbyreputationsystemsmayhaveapositiveeffectiftheyleadtoincreasedengagement,oranegativeeffect,iftheyinterferewithstudents’intrinsicmotivationsandpro-motecompetitionovercooperation[19,12]..Weinvestigatethesequestionsinabetween-subjectsfieldex-perimentonasinglecourseontheedXplatform.N=1101participantswereassignedtoforumswithandwithoutrep-utationfeatures.Inlinewithpreviousworkontraditionalhighergradesandhigherretention.Presenceofforumrep-utationfeaturessignificantlyaffectsforumparticipation,in-creasingthenumberofresponsesanddecreasingthetimetofirstresponse.Reputationfeaturesalsocurtailcertainlan-guageusagepatternslikeappealsforhelp.However,repu-tationfeaturesdidnothaveasignificantimpactonstudents’ esortheirsenseofcommunity.Overall,theseresultssuggestthatreputationsystemscanimprovetheforumexperience,butdonotcontributesubstantiallytoas- esorcommunityformationinthecurrentMOOCcontext.Relevantpriorworkincludesresearchonforumsintradi-tionalclasses;answeringsitesforinformallearningcommu-nities;languageuseinonlineforums;andthecurrentgenera-tionsofMOOCs.Wediscusseachareainturn.StudentsandOnlinediscussionforumshavebeenusedtosupporton-campuscoursessincetheearly1990s,whentheywerebasedonmailinglistsandUsenet[7].Effectiveforumscontributetolearningthroughcommunitybuildingandcollaborativedi-alogues.Thecognitiveapprenticeshipmodelinlearningthe-ory,whereinmoreexperiencedpeersandexpertsprovidetheminimalassistanceneededtocompletetasksuntilautonomyisachieved,helpstoexplainthebenefitsofforumsforprob-lemsolving[20].Studentswhovoluntarilyparticipateinforumshavebeenfoundtoachievebetterexamgradesandperformbetterover-all,evenwhenonlyviewingcontentandwithminimalin-structorinteraction[5].Weexpectthatforumusewillalsoimprovestudent esinMOOCs.Intenttoparticipateinforumshasbeenlinkedtotwomotivationalfactors:theper-ceivedimportanceoflearning,andpeerpressure,thesensethatpeersareputtingeffortintotheforum[41].Theimpor-tanceofpeerpressuresuggeststhateveninterventionsthatgeneratesmallchangestoparticipationcanhavealargerin-directeffect.Q&ASitesandReputationQuestionandanswer(Q&A)sitesprovideresponsestospe-cificquestionsandoftendiscouragediscussionnotrelatedtoaquestion.Nevertheless,theyareusedbothforinforma-tionalpurposes(asintended)aswellasforconversationalpurposes[18].PopularexamplesincludeYahoo!Answers,forgeneraltopics,andStackOverflow,forsoftwaredevelop-ment.Investigationsofexistingsitestypicallyfocusonchar-acterizingusagepatternsthroughlargescaledataysis,withoutcomparingdesignalternatives,asresearchersareun-abletomodifythesystem.Forexample,Adamicetal.char-acterizeactivityonYahoo!Answersbyclusteringcategoriesofquestionsbasedontextanduserfeatures[1];Mamykinaet

al.measureresponsetimesonStackOverflowandcategorizeusersbasedontativebehaviormetrics[24].Q&Asitesrelyonreputationsystems,inwhichparticipantsreceiveextrinsicrewardssuchas“reputationpoints”whentheircontributionsareratedasusefulbyothers.Reputa-tionsystemsarethoughttoencourageparticipation,increasecontributionquality,andidentifyreliableinformation.Theyoriginateonelectronicmarketplaceswheretheyhelpidentifytrustworthyparticipants[29],andhavebeenusedtosupportbothstudentdiscussionforums[6]andsocialcollaborativelearningsystems[32].InTausczik’sinvestigationofMathOverflow[34],aQ&Asiteformathematicsresearch,“[u]serbehaviorsuggeststhatbuildingreputationisanimportantin-centive,eventhoughusersdonotreportthisinthesurvey.”Reputationsystemsasextrinsicmotivatorsmayalsohavenegativeconsequences.DearmanfoundthatYahoo!Answersusersemployedreputationasacuetowithholdanswers–ifthequestionerwasknownforassigninglowscorestoin-creasetheirownpoints,thenotherswouldbereluctanttore-spond[11].Tangibleextrinsicmotivatorssuchasmoneyorcandycanalsodiminishfeelingsofcommunityandbond-ing[12].Ontheotherhand,inacomparisonofafor-feemarket-basedanswerforumtoasystemwithatop-10leaderboard[19],thereputation-basedsystemhadmoresocially-conducivecommunitybuildingquestions,suggestingthatthediscouragingeffectsofreputationpointsarenotassevereasthoseoftangibleextrinsicmotivators.Despitetheirubiquity,therehasbeenlittlesystematicinvesti-gationintowhetherandhowreputationsystemsbenefitsiteswhichusethem.Researchershavehoweversystematicallystudiedtheimpactofotherdesignfeaturesononlinecom-munities.Harper[16,17]andLudford[21]reportononlinefieldexperimentsmotivatedbysocialdesigntoincreasepar-ticipationattheMovieLensfilm mendersite.Ourworksimilarlyisafieldexperimentdesignedtostudytheimpactofspecificdesignfeatures(reputationsystems)oncoursepartic-ipationandassessment Ourworkinvestigateslanguagedifferencesbetweenthetwoexperimentalforums.Inparticular,intheabsenceoftheex-trinsicincentivesofareputationsystem,studentsmayhavetotakeextrastepstoencourageresponses.Studiesfromso-cialpsychologyhaveshownthatdemonstrationsofgratitudecanincreaseeffortstoassistsomeoneevenwhenthoseeffortsarecostly,andthatgratitudecanincreaseassistanceprovidedtostrangers[2,14].Otherlanguageusagesuchasendinganinformationneedwithaquestionmark,explicitlyscotheaudience,andbeingsuccinct,canalsoincreasethelikelihoodofquicklyreceivingmanyhigh-qualityanswersonsocialnet-works[35].Asdescribedintheresults,wefocusonappealsforhelpsuchas“thanksforanyhelp,”whichcanbeinter-pretedaspre-emptiveexpressionsofgratitude.Massiveopenonlinecourses(MOOCs)areonlinecourseswithopenenrollmentthattypicallyhavethousandsofac-tivestudents.TheyoriginatedwiththeUniversityofMan-itoba’sConnectivismandConnectiveKnowledgeMOOCinKhanAcademy[37].ForumsarepervasiveinMOOCsandhavebeencharacterizedas“anessentialingredientofanef-forumsshowstrugglestoretainforumusersovertime.Inoneexample,halfofforumusersceasedparticipationduetofac-torssuchaslackofforumfacilitation,anoverwhelmingnum-berofmessages,andrudebehaviorofotherstudents[23].PublishedresearchonMOOCshasthusfarfocusedonre-flectionsontheMOOCexperienceandthegreatnumberanddiversityofstudentswhoparticipate[25,9].Incontrast,weandtheirimpactonMOOCstudents.technologytofacilitatelearninginMOOCs.ARTful[33]usesEEGtechnologytotrackstudentattentionandlaterusesthisinformationtosuggestmaterialforreview,whilePex4Fun[38]usesautomatedtestgenerationtechniquestoassistinautomatedgradinginprogrammingcourses.Al-thoughthesetechniquesarevaluable,theyarepresentlylim-itedbyspecialhardwarerequirementsandarestricted,respectively.Weaiminsteadtoevaluateenhance-mentstoexistingforumsystemsthatareeasytodeployincurrentMOOCsettings.Weconductedabetween-subjectsfieldexperimentonaseven-week,open-enrollmentsoftwareengineeringcourseofferedontheedXplatform(“CS169.1x:SoftwareasaSer-vice”fromUniversityofCalifornia,Berkeley).Thecoursefeatures11 lecturesrecordedduringacor-respondingon-campuscourse,4sand4quizzes.Lecturesincorporateshortassessmentsthattestthematerialjustpresented.ThecoursehasbeenofferedthreetimesontheCourseraplatformandthreetimesontheedXplatform,eachtimewithsimilarcontent.AlledXcoursesincludeasimpleforumdesignedforanswer-ingspecificfactualquestionswithlimitedthreaddepth(atmostthree).Postscanbeupvotedorreportedformisuse,butnotdownvoted.Postsarelistedintheordertheywereposted.Eachpostshowstheauthor’sname,whenitwasposted,andthenumberofupvotes.Userscanbemanuallymarkedasmoderators.Eachuserhasauserpagelistingalltheircon-tributions;therearenoreputationscoresorbadges.Threadsaresearchedusingfull-textsearch,ratherthantags,andcanbeorderedeitherbydateornumberofvotes.OtherMOOCproviderssuchasCourserahavesimilarforumdesigns.Intheparticularcourseunderstudy,theforumisself-moderatedbystudentsandasetofcommunityteachingas-sistants(communityTAs),volunteerswhotookthepreviousofferingsofthecourse.Instructorsgeneratecontentbutdonotparticipateintheforum.WemodifiedthecoursesoftwaretoreplacethestandardedXforumwithtwovariantsofanexperimentalforum.Whenstu-

dentsandcommunityTAsfirstvisittheforum,theyareran-domlyassignedtooneofthetwoexperimentalforumweb-sitesbasedonahashoftheirusername,andareunabletoaccesstheother.Theassignmentofparticipantstoforumsre-mainsstablethroughoutthecourse,andthetwoforumssharenocontent.Althoughusingseparatecontentdatabasesleadstolessavailablecontentforeachforum,anddividestheef-fortsofthecommunityTAs,itavoidscomplexinteractionsbetweenstudentsusingdifferentfeatures,andallowsdiffer-encesinpassiveparticipationtobeexamined.Bothsiteswerebasedontheopen-sourceOSQAsoft-ware[27],whichissimilartoStackOverflow(seeFigure1).LiketheedXforum,OSQAisdesignedforansweringspe-cificfactualquestionswiththreaddepthatmostthree,ratherthanforextendedcomplexdiscussions.Thismodelencour-agesaskingconcretequestionsonindividualassignmentsovergeneralclass-relateddiscussions.Ononeoftheex-perimentalforums,thebasicforum,studentscanpostques-tions,answers,andcomments,searchcontentusingtagsandfull-textsearch,andfolloworsendprivatemessagestootherusers.Thefull-featuredforumadditionallyprovidesareputa-tionsystemwiththefollowingfeatures:Userscanupvoteanddownvotequestions,answers,andcommentstoindicatethemostusefulones.Answerswithmoreupvotesarelistedfirst.Usersreceiveareputationscorebasedprimarilyonhowmanyupvotestheirpostsreceive.Reputationpointsrewardusersandhighlightwhichusersarereliabletoothers.Asusersgainreputationpoints,theygraduallyacquireaccesstomoderationfeaturesontheforum.Userswhoaskquestionscanselectabestansweramongtheanswers.Ananchorisapieceofcoursematerialthatnaturallystim-ulatesdiscussion,suchasaorlecture.[15]Wefollowthebestpracticeofmakingrelevantdiscussioneasilyaccessiblefromanchorsbyinsertingforumlinksintoanchorpageswhichshowpostswithrelatedtags.Afterthecoursecompleted,wesentallparticipatingstudentsusefulnessoftheforum,usefulnessofthereputationsystemfeatures,theirmotivationforcontributingtotheforum,andtheirsenseofcommunity.MOOCsarecharacterizedbyvaryingparticipationoverthedurationofthecourse,withconsiderableattritionbetweentones[3].TheMarch2013courseweexaminedex-hibitedthispattern,startingwith5985enrolledstudentsandendingwith532passingstudents;studentswhousethefo-rumdepartsomewhatmoreslowly,with1572studentsstart-ingthecourseand442passing(seeTable1).First-Class-to-pletion-Rate(FCCR),ametricdevisedbyRus-sellmeasuringtheratioofthenumberofstudentswhore-ceivedapassinggradetothenumberwhoviewedthefirstlecture,was11%,lowerthanthe19%and36%reported fortheirPowerSearchingcourse,[13]suggestinghigherattrition.Non-ViewedanyCompletedfirstStartedWatchedfinalStartedCompletedfinalReceivedpassingcourse.Attritionoccursrapidlyatthebeginningandcontinuesovertime.Forumusers(usingeithertheexperimentaloredXforumsatanyofstudentsandendingat64%.Mostpassingstudents(83%)usedtheCourseSiteFull-FeaturedForumVisitorsBasicForumVisitorstures;thebasicforumdoesnot.1101studentsoptedintoparticipateintheexperiment,with548inthefull-featuredforumand553inthebasicforum;471studentswhooptedoutinsteadusedthestandardedXforum.3391students(68%)viewedatleastonelectureoras-sessment,butnevervisitedanyforum.Studentsoptinwhentheyfirstvisittheforum,andallsubjectsretainedaccesstothestandardedXforumtoenablethemtooptoutoftheex-perimentatanytime.Allstudents,regardlessofwhethertheyparticipatedintheforumexperiment,agreedtotheedXtermsofservicewhichallowresearcherstostudytheiredXdata.Inanygivenweek,13%–17%ofstudentswhoaccessedthecoursewebsitealsoaccessedtheexperimentalforumwebsite,asshowninFigure2.Differentinformationwasavailablefordifferentcoursepar-ticipants.Aggregategradeandparticipationinformationwasavailableforallstudents.Forthe1101studentswhocon-sentedtotheexperiment,detailedinformationabouttheirfo-rumusageisavailable.Additionally,216studentsoptedintoamorepermissiveconsentformthatallowedustocorrelatetheirdetailedforumactivitywiththeiredXgrades.Priorworkindicatesthatstudentswhouseforumsachievebettergrades.OurfirstgoalistoestablishwhetherintheMOOCsettingthereisaconnectionbetweenmereuseoftheforumandcourse es.Becauseforumusewasself-selectedbystudentsandnotmanipulatedinourexperiment,

Figure2.Forumandclassvisitorsbywecanonlyshowcorrelation,butsuchacorrelationwouldsupportthepotentialusefulnessofperformingforuminter-ventionsandprovidecontextforourotherresults.Thisgivestwohypotheses:H1.ForumparticipantswillhavehighercoursegradesthanH2.Forumparticipantswillhavehigherretentionthannon-participants(theydropoutlater).Next,weexploitourcontrolledforuminterventiontoseekstrongconnectionsbetweenforumreputationfeaturesandimportantstudent es:coursegradesandcoursereten-tion.Inaddition,wealsostudyfactorsdirectlylinkedtofo-rumparticipationsuchastimetofirstresponseandnumberofresponses.Suchfactorscanhelpexplainchangesin (orthelackthereof).Wealsoinvestigatethestudents’subjec-tivesenseofcommunity.Reputationfeaturescouldpromotesenseofcommunitybyencouragingmoreparticipationandbyprovidingmoreinformationaboutwhichusersareexperi-encedinthecontextofaforumdiscussion.H3.Reputationfeatureswillimproveforumparticipationmetrics,suchastimetofirstresponse.H4.ReputationfeatureswillimproveH5.ReputationfeatureswillimprovestudentH6.Reputationfeatureswillaffectthestudent’ssenseofAlthoughuseful,simpletativeforummetricsareun-abletodistinguishforumthreadsthathavesimilarstructures.Subjectivemetricslikesenseofcommunityaresubjecttobiasastheyarebasedonself-reportedand pletedata.Forthisreasonwealsosoughttoinvestigatedifferencesinthecontentoftheforumsusingtextysistechniques.Al-thoughmanyquestionscanbeinvestigatedwithtexty-sis,forthepurposesofthisworkwefocusononeexamplethatisstraightforwardtoyze,namely,whetherstudentsinthebasicforumusemorelinguisticcuessuchas“canany-onehelpme”topersuadeotherstoassistthem.Asnotedinrelatedwork,weexpectthisbecausereputationsystemsofferextrinsicrewardstouserspostinganswers,reducingtheneedforsocialrewards.H7.Usersofthebasicforumwillusemoreappealsforhelpintheirquestions. es:FinalgradesandWeoperationalizeretentionasdaysfromthebeginningofthecourseuntilthelaststudentactivity,definedasanyaccesstoanywgeassociatedwiththecourse.WeextractstudentactivityfromclickstreamdataprovidedbyedXandforumactivity,recordedbyourownforumserver.Similarlytoon-campuscourses,astudent’sfinalgradeisasummativeassessmentoftheirworkthroughoutthecourseona0-100scale,andstudentswithascoreof50pass.However,duetolowcompletionrates,finalgradesinaMOOChaveadifferentinterpretation:theyareacombinedmeasureoftheportionofthecoursethatthestudentcompletedandthestudent’sperformanceinthatportion.Forexample,astudentwhoscored100%onthefirstassignmentandthendroppedout,andastudentwhoscored25%onallfourassignmentswouldreceivethesamefinalgrade.alsowishtoeffectivelyseparateperformanceoncompletedworkfromattrition.Onestraightforwardandflexiblewaytodothisistocompareresultsonindividualassessments,includingonlystudentswhocompletedtheassessment.ofprioryses,e.g.[24].Inparticular,weinvestigatenum-berofresponsestoquestions.Ahighnumberofquestionswithoutanyresponsesindicatesaninactiveforumandhasbeenlinkedtoalowsenseofcommunity[10],whileques-tionsthatreceivemanyanswersareindicativeofmultiplesolutionsordiscussions.Wealsoinvestigatetimetofirstresponse–theintervalbetweenthetimeanewquestionisposted,andthetimethefirstanswertothatquestionisposted.

Full-BasicTotalQuestionsTotalPostersor—PostQuestionAnswerCommentTable2.Basicdescriptivestatisticsforeachforum.Deletedcontentisabouteachother.”Theresultsareaggregatedintoanumeri-calratingfrom0to80.Rovai’sScalehasbeenused,forex-ample,toconnectcertaintativeforummetricstosenseofcommunity[10].Weassessedsenseofcommunityaspartoftheend-of-coursesurvey.Thetwoexperimenta