RSIAgent：面向新环境递归自我改进的自主探索

¹AetherAI,2UniversityofCaliforniaSanDiego,³Universityof*Correspondingauthorandprojectleader;Equalcoframeworkforrecursiveself-impcoordinatescurriculum,actoractions,conditions,andconcombiningparallelbroadrecursiandpreviouslyunknowncausaldependencies.TheresultingmemoryisfCorrespondence:KunZhou(fraWebsite:aetherlabsai.github.io/RSIAgent/(a)Recursiveself-improvement(a)Recursiveself-improvementNear-termVerifierVerifierAgentktAstra,ontheOSWorld2.0(0808offline)andAgents'LRSIAgentenvironmentswhoseinterfacbytheirpretrainedknditionalinteractiondataforfurthertraining,oftenwithhumanassistancshipsbetweenactions,environmentconditions,andoutcomes,aHumanlearningofnewsoftwusefulknowledgefromobservedoutcomes.Importantly,thistheirconsequences,humansgraduallyinferwhichfactorsdeterminesutransitions.Inspiredbythisprdigitalagentsinnewenvironments.Toinstantiatethisloop,wcausalknowledge.Thecurritheenvironmentandupdatesthememory,andtheverifieragentgroundBuildingonthismulti-agentfocusesonimportantdiThiscoarse-to-fineprocthepretraining-then-posttrainingparadigminEmpirically,RSIAgentenablesstrongrecursiveself-improvementacrossresultsdemonstratethateffectiveagent-levelself-improvementcansigreverse,thecapabilitygapbetweenopen-andtoautonomouslyacquire,verify,andreuseknowledgeinnewenvironmentswithoutupdatingdiverseenvironmentknowledgeandthenrefinehardcases,hiddenconstraints,andbou·WeshowthatRSIAgentcanrecursivInthispaper,weformalizethedigitalagentanddefienvironment&throughasequencinteractionhistoryht=(o₀,a₁,Afterexecutinga,theenvironmenttransitionsat~πe(·|q,ht,M),(St,Ot)～8(·|st-1,at).(1)andretainsreusableknowledgeacquiredsoftwaresystems,includingGnewenvironmentwithoutupdatinfirstperformsautonomousexplorationtoconstructapersistentmemoryMthattesttime.AccordinRSIAgentEnvironment:FreeCADmAddsupportgeometrMemoryBankFigure2.OverviewofRSIAgent,illustratedwithaFreeCADtask.BRSacquiresdiveparallelexploration;DRSiterativelyrefinesmemorythroughgoingdInthissection,wepresentRSIAagentstoautonomouslyagentharnessframework,whichdecomposestheagentsystemupdatingthememoryandreusingitattesttime.Figure2providesanoverviewofourmethod.3.1Multi-AgentHarnessFrTobettercontrolautonomousexplorationamulti-agentharnessframeworkthatcanoperatethroughacoordinatedrecagentsystem,responsibleforunderstandingtheenvironmentandgener5subsequentactoragentinstancesuccessfullysatisfiedthetaskrequirements.Tomakethisjudgmentreliable,itdirectlyinspectstheongroundedevidence,itreturnsasuccwhichisthenusedtodeterminewhetherthecorrespondingexperienceConcretely,itgeneratessuitablepracticetasksfortheatarget,accumulatedmemory,andprevicontinuesuntilthegeneratedtasksareunlikelytocontributesubstant3.2Multi-stageAutonomouswhichrefinestheaccumulatedknowledgethroughtargebuildabroadunderstandingofanewenvireusableprocedures,andfaconditionsrevealedduringtaskeincreasesexplorationdifficulty.Thecurriculumagentfirsexposeunpredictableissues,hiddenconstraintsagentthenattemptsthetaskusingtheaccumulatedmemory,whiletheverinextiteration.Eachverifiedexperienceisconsolidatedintomemorybeforproposed,allowingDRStocolessonsstoredinmemorytoguideitsactions,whiletagainstthetaskrequirements.Thisaction-verificationloopctermtasks.Wereportpartialscore,themeantaskscore,andbinaryaccuracy,theproportionoftasksreceivingfullcredit.Both5.3servesasthedefaprojects,withuptofourprojectsexecutedconcurrently.Thebudgetischeckedbetweencomplewaves,withoutinterruptinganongoingwave.Foisworthwhile.Afterexploration,theaccumulatedmemoryisfrozenanexistingagentharnefrom83.75to84.82,andbinaryaccuracyincreasescleargainsextendingbothprocedurecorrectnessandfulltaskcompletRSIAgent一一一一一一—一一一一一——一RSIAgentmemoryreusetostrengthenopen-sourcemodelswithoutupdatingmodelpT044(videoediting),T049(presenta-ing).Figure3presentstheasthebaseline.Thethebaseline.Bystep8,accumulatesdiverseproceduresT065coveratargettask'scriticalrequirements,resolvingaremainingbottlensheetrepair),T085(REAPERaudioeditmemory.FullRSIcombinrepetitioncounts,andexplorationbudgetsaredetailedinFullRSIreachesameanscorwhereasdeep-onlyexplorationfallsbelowthebaselRSIAgent36.0120T080T085T089SpreadsheetrepairAudioe54.06100WefurtherevaluatewhetherRSIAgentgeneralizesbeyondstandarpareRSIAgentagainstPlay2Code[11],acontinualgame-improvementbagamebecomesstrongeranditcanevendegradealreadyhinsufficientlytargetedexploration,incompleteverifInsufficientlyTargetedExploration.Additionalpracticemayleavetarget-specificweaknessessubmissionpersistence,whileobtainingunavailableuserinformationremaineduntested.Theprob-lematicmissing-informationrulesconsequentlyremainedinmemory.Theseobsertargetrequirements,beyondexpandingthediversityofpracticetasks.Table2.Performanceon40GameCraft-Benchtasks.Rowsaregroupedbythegeneratorofttaskrequirementshavebeensatisfied.Intheauditedcases,toutputwasproduceddoesnotestablishthattheunderlyinginterpretationorrequirementhatnegativelyinfluagentretainedrulesthattreatedmissing-datamarkersasvalidanswersorinterpretedunavaiofanexperience,ratherthantreatinglocalacceptanceasevidenceofgeneralvalidity.(a)Insufficientlytargeted(b)Incomplete(c)Unrexplorationverificationco1/3(33.3%)softwareenvironments[10,55].Benchmarksevalandtoolenvironments[24,38,40,47],includingchanginguserrequirementsandthequalityofknowledgeneededfordownstreamtasks,withoutfurthermodeltraining.RecursiveSelf-Improvemeallowsameta-agenttomodifybothitselfandthetaskagent,enablingtheimprovementprocedureexperience[16,39,49]oraccumulateexperienceinreflections,skills,manuals,andpmemory[3,13,22,26].Recentworkimprretrieved[32,33,36,44,51,54].HyMEMcombinessymbolicinahybridmemorygraph,supportingmulti-hopretrievalandinference-timemdownstreamtaskexecutionwithoutupdatingmodelparameters.improvementinnewdigRSIAgentautonomouslyacquires,vthroughbroadrecursiveself-exploration(BRS),andthenfocusesonhardcases,hidenablethemtooutperformfrontierclosed-sourcemodelswithoutupdatingtools,workflows,andscientificprevolvingenvironmentsforstudyinglong-horizonexploration,adaptation,andRSIAgenthasseverallimitations.First,recursiveself-improvementrequiresadditprograms,andretainreusablememory,introducinRSIAgent[1]SaaketAgashe,KyleWong,VincentTu,JiachenYang,AngLi,andXinEricWang.Agents2:Acompositionalgeneralist-specialistframeworkforcompute/abs/25[2]LakshyaAAgrawal,ShangyinTan,DilaraSoylu,NoahZiems,RArnavSinghvi,HerumbShandilya,MichaelJRyan,MengJiang,etal.Gepa:Reflecti[3]MinghaoChen,YihangLi,Y[4]KanzhiCheng,QiushiSun,YougangChu,FangzhiXu,YantWu.Seeclick:Harnessingguigroundingforadvancedvisualguiagents,2024.U///abs/2401.109rewardsbeyondmathandcode:Lightweightcorpus-groundedprocesssupervisionfo[6]ShichengFan,MingdaiYang,DuohaoWang,CLi,JulianMcAuley,KunZhang,PhilipSYu,etal.Agenticcom[7]QijunHan,HaoqinTu,ZYuhuiXu,RanXu,CaimingXiong,ZeyuZheng,HVlaa-gui:Knowingwhentostop,recover,a2026.URL/abs/[8]HongliangHe,WenlinYao,Kaimodels,2024.URLhttps[9]ShengranHu,CongLu,andJeffClune.Automateddesignofagenticsystems.InInternational[10]XueyuHu,TaoXiong,BiaoYi,ZTao,XiangxinZhou,ZiyuZhao,YuhuaiLi,ShengzeQiao,ZhaokaiWang,KunKuang,TieyoWangchunshuZhou,GuoyinWang,KetingYin,ZhouZhaoZhang,andFeiWu.Osagents:Asurveyonmllm-buse,2025.URL/abs/2508.04482.RSIAgent[12]JinhaoJiang,KunZhou,ZicanDong,KemingYe,Xinofthe2023conferenceonempiricalmethodsinnaturallanguageprocessing,pages9237-9251,2023.[13]SethKarten,AlexLZhang,Kimprovingrlmharness.arXivpreprintarXiv:2608.23552,2026.[14]RunzeLi,YuwenZhai,BoXu,LiWuEchotrail-gui:Buildingactionablememoryforguiagentsviacritic-guideds9347-9356,2026.[15]YangLiu,ShiweiHou,Xiyofflineapiself-exploration.arXivpreprintarXiv:2608.07925,2026.[16]ShuqiLu,ChaofanLi,KunLuo,ZhangZhang,HuiWangresearch.arXivpreprintarXiv:2607.21461,2026.[17]TongxuLuo,RongshengWang,Liang,KeJi,ShuqiGuo,YuhaoDu,etal.Gamecraft-bench:Canagentsbuildplayablegamesend-to-endinarealgameengine?arXivpreprintarXiv:2606.17861,2026.[18]ChunyuMiao,HenrLi,WooseongYang,BoweiHe,Xinndevelopmentwithinteractivehumanfeedback.arXivpreprintarXiv:2510.06186,2025.[19]DehaiMin,KailinZhang,TongtongWu,andLuCheng.Quco-rag:fromthepre-trainingcorpusfordynamicretri[20]AntonRazzhigaev,AndreiGritsaev,AndreiKaznacheev,Niki[22]NoahShinn,FedericoCassano,AshwinGopinath,KarthikNarflexion:Languageagentswithverbalreinforcement[23]XueqiaoSun,XiarXiv:2606.31270,2026.Wen,MingyangXu,XiaoyuanLiuChen,PatrickBryant,CarlBoettiArvindRao,TapioSchneider,GeorgiosYannakakis,LaureZanna,KaanOZohdi,GeorgeEmKarniadakis,JackGallaArasBacho,ShengcaoCao,ZengyiQin,YixiongChen,HengduanFan,HaoLiu,LinZeng,ZongzhengZhang,HonglinBao,ShuoLu,JianhongTu,ZhonghuaWang,ZhengZhang,ZijiaoGu,HaoyueHua,HaoyangLi,WIshaanSinghChandok,LeiDing,JingxuanFan,AndrewGlover,JiamingHu,YiranHHuang,ZixinJiang,HaoraXu,ZhongxingXu,ZhilingYan,BoqinYuan,RuiqiZhang,YifanZhang,ZiboZhao,Liana,Chen,XuewenChen,YipuChen,ChenyuZhu,ChenDai,StefanoDeCastro,YunfKaustubhDhole,JiayuanDing,ChenchenDu,ZhehangDu,HaoFan,Run-ZeFan,HengyuFu,ShiGu,YifanGu,CharlieGuo,BaiheHuang,BaixiangHuang,RimikaJaiswal,ZhihanJiang,RanJin,ErinKasson,XinLan,JosephLee,DerenLei,ChenyuLi,DaofengLLi,HongweiLi,JingyanLi,XLongtaiLiao,KevinQinLiu,PanLu,WenboLv,YichengLyu,QiuyangMangNing,JorinOverwiening,XuPan,LaynaParaboschi,CoreFranciscoPark,JustinPurnomo,SwatiRajwal,ScottRankin,BixuanHannahSzlyk,HaochengWang,JiaChenlongZhang,ChiZhang,HanningZhang,HaSongZhang,WenjinZhang,WenshuoZhang,YingZhang,YizhiZhang,BrianZhao,QijiZhao,YiminZhao,YuhaohuaZheng,LiweiZhou,TianyueZhou,SicZhu,YishuZhu,JieruiZuoNawenDeng,RaoFu,TianfuFu,YifanHan,HeRen,ZhenyuHe,QiaoJin,LanglangLi,YuetaiLi,SylviaLiu,LuLu,LuqiHaoranWu,ZhiyueWu,HannahYao,ZhuoranYi,JZhu,JunfanZhu,AlanYuille,YangLiu,RussellTao,JingHuang,WenqiShi,CostasSpanos,LichaoDong,HectorGomez,AylinCaliskanRSIAgentZhu,andDawnSong.Agents'lastexam,2026.URL/abs/2606.05405.[25]WeihaoTan,WentaoZhang,XinrunXu,HaochongXia,ZiluoDing,BoyuLJunpengYue,JiechuanJiang,YewenLi,YujieWu,XiaoqiangChai,YifeiBiLu.Cradle:Empoweringfoundationagentstowardsgeneralcomputercontrol,2024.URL/abs/2403.0[26]GuanzhiWang,YuqiXie,Yunfa[27]HaomingWang,HaoyangZou,HuatongSong,JiazhanFeng,JunjieFang,JuntingLiu,QinyuLuo,ShihaoLiang,ShijueHuanagentwithmulti-turnreinforcementlearning.arXivpreprintarXiv:2509.02544,[28]WenyiWang,PiotrPiękosMingchenZhuge,andJürgenSchmidhuber.Huxley-g\"odelmachine:Human-levelcodingagentdevelopmentbyanapproximationoftheoptimalself-improvingmachine.InInter[30]XinyuanWang,BowenWang,DunjieXiaoleGuo,YihengXu,ChenWu,etal.Opencua:OpenfoundationsforcAdvancesinNeuralInformationProce[31]ZefengWang,MinxiYan,JinheBi,SikuanYan,VolkerTresp,andYunpuMa.MRecursiveself-improvementofllmag[32]WenyiWu,KunZhou,RuoxinYuan,VivianYu,StephenWang,ZhitingHu,andBiweiHuAuto-scalingcontinuousmemoryforguiagent.arXivpreprintarXiv:2510.09038,2025.[33]WenyiWu,ZixuanSong,KunZhou,YifeiShao,Zhicontinuousmemoryforvision-languagemodels.AdvancesinNeuralInformatSystems,38:128685-128710,2026.[34]WenyiWu,SiboZhu,KunZhou,andBiweiHuang.Plannerbalancedmulti-agentcollaborationframeworkforlong-horizonplanning.ar[35]WenyiWu,SiboZhu,KunZhou,AayushS2026.[36]ZhaofenWu,HanrongZhang,Fuliforllmagents.InProceedingsofthe64thAnnualMeetingoftheAssociationforComputationalLinguistics(Volume1:LongPapers),pages34647-34664,2026.2024.URL/abs/[38]ZiqiaoXi,ShuangLiang,QiLiu,JiaqingZhang,LetianPeng,FangNan,MeshalNayim,TianhuiZhang,RishikaMundada,LianhuiQin,et[39]PengXia,KaideZeng,JiaqiLiu,CanQin,FangWu,YiyangZhou,CaimingXiong,andYao.Agent0:Unleashingself-evolvingagentsfromzerodataviatool-integratedreasarXivpreprintarXiv:2511[40]TianbaoXie,DanyangZhang,JixuanChen,XiaochuanLi,SihenHua,ZhoujunCheng,DongchanSforopen-endedtasksinrJunliWang,DunjieLu,HaoHu,andTaoYu.Introducingosworld-verified[42]TaofengXue,ChongPeng,MianqiuHuang,LinsenGuo,TianchengHan,HaozheWang,JianingWang,XiaochengZhang,XinYang,DengchangZhao,eta[43]MingdaiYang,ShichengFan,KZhiweiLiu.Payingforhonestywithoutknowingthetruth:Reputation-penaltydesignforllmmarketplaceagents.arXivpreprintarXiv:2607.2[44]ShuYang,JunchaoWu,agents.arXivpreprintarXiv:2607.03726,2026.[45]YifanYang,ZiyangGong,WpreprintarXiv:2605.23904,2026.[46]ZhaochenYu,YingchengWu,ZhenfeiYin,KaiyuanChen,ZheZhao,MengdiWang,Shuichengharnesses.arXivprepri[47]MengqiYuan,ZilongZhouCui,BowenWang,HaoyuanWu,YitonRSIAgentJiaqiDeng,YuhaoYang,ChengCOsworld2.0:Benchmarkingcomhttps:///abs/2606.29537.[48]EricZelikman,Elian(stop):Recursivelyself-improvingcodegeneration,2024.URL02304.[49]HanrongZhang,YankaiChen,ShichengFan,etsynthesis:Acomprehensivesurvey,2026.URL/publication/[50]HanrongZhang,ShichengFan,HenryPengZou,YankaiChen,ZhentingWang,JiayuZhoWenyaWang.Memskill:Learningandevolvingmemoryskillsforself[52]JennyZhang,ShengranHu,CongLu,RobeOpen-endedevolutionofself-improvingagents,2026.URLhttps[53]JennyZhang,BingchenZhao,WannanYang,JakobFoerster,JeffClune,MinqiDevlin,andTatianaShavrina.Hyperagents.arXivpreprintarXiv:2603.19461,2026.[54]QizhengZhang,ChangranHu,ShubhangiUpasani,BoyKamanuru,JayRainton,ChenWuRepresentations,volume[55]WayneXinZhao,KunBeichenZhang,JunjieZhang,ZicanDong,YifanDu,ChenYang,YushuoChen,ZJinhaoJiang,RuiyangRen,YifanLi,XinyuTang,ZikangLiu,Peiyuagent,ifgrounded,2024.URL/abs/2401.01614.[57]SiboZhu,WenyiWU,KunZhou,StephenWang,andstructuredmemoryforcomputer-useagents.InMariaLZhang,andDavidJurgens,editors,FindingsoftheAssociationforComputationalLinguistics:ACL2026,pages11287-11304,SanDiego,California,UnitedStates,July2026.AssoComputationalLinguistics.ISBN979-8-89176-395-1.doi:10.18653/v1/202URL/2026.fin[58]HenryPengZou,ChunYaozuWu,LianchengFang,ZhengyaoGu,ZhenZhang,etal.Whenuserschangetheirmind:Evaluatinginterruptibleagentsinlong-horizonwebnavigation.arXi2026.RSIAgentAImplementationandAgentInterfacesimplementation:thetargetqueryguidesexploration,whilepersistentmemorexperiencebetweenattempts.Modelconfiguratinterfaces.Thecurriculumagent'scontextpersistswithinanexplorationlcontextpersistsacrosscandidaterevisionswithinatargetaprojectsortargetatTableA1.Agentinterfacesinexecutionfeedback,andavailafterverification,memoryupdatesandTargetquery,completedexplorEvidence-groundedloProgram-BasedActions.Theactorinterfaceexpoprogramsusingexecutionfeedbadoesnotrelaxrequirementsconcerningnamedapplications,editableartifacts,rForBRS,thecurriculumagentpubliandalistofprojectswithuniquForDRS,aPROJECThandoffsimilarlycontainsthepracticerequestandpracticeinterfaceproducesonegroundedPASSorFAILperproject,withoutasame-projectExperience-OwnedLearning.Theactoragentthatproducedanexreportandenterstwolearningsteps.Distillationidentifiesusqualifyingunsupportedconclusions.Afailedprojectcancontributeusefulerecordedasaverifiedsuccess.TheCanonicalMemoryOwnership.Onlycompletedactorlearningpersistentmemorybank.Work-pthecurriculumagentnortheverifieragentapprovesmemorywording.Anactor-theactoragent'sprivatereasoningtranscript.waveisselectedonlyafterthesecommitsfinish,allowinverifiedoutcomesandtheknowledgeactuallyrA.3SequentialRefinemelearning,afterwhichthecurricprocedure.EachpracticeprojectisverRSIAgentnessofthetargetcandidate.UnderthedefaultcFrozen-MemoryEvaluation.Finalevaluatienvironment,anddisableresultingcandidateoutsideallagentcontexts.Therunnerchecksprotocol,stoppingpolicy,terminalstatus,andevaluationresultsethisreferencelifecycle;historicalandselected-runvariantsretaintheirownprotAlgorithmA1.Target-conditionedRSIreferenceprocedure.Inputs:targetqueryq,resettableenvironment,initialmemoryMo,anddeclaredstoppiOutputs:frozenmemory,finalcandidate,t2.BRS:repeatcompletewavesuntilarecordedcurriculumstoporthecomplete-wave(a)Thecurriculumagentselectsindependentp(b)SnapshotM.Executeprojectsandobtainverifierverdictsinparallelinisolate3.DRS:attemptqafteranenvironmentre4.Ifthepolicyisverifier_passandthetargetpassed,proceedagenttoselectthenex5.Executeeachselectedpracticeprojectsequentially:actorexecution,ver6.Iftheprecedingtargetpassedandnonewpracticewasselected,finishDRS.Otherwise,retuAcurriculumSTALLEDdecisioninsteadallowsoverdicts.Acompletedlifecycleneednothaveaforexecution,verification,explorathereferenceimplementation,notcompletesystemmessages;ellipsesmarkomittedpassages.Taskinstructions,currentmemory,projectoutcomes,andverifierreportsaresuppliedtheirwordingisretainedintheexceActorInstructions.Theacsuppliestheprojectinstructionandacomponents;itdoesnotincludethemodel-dependentvisual-actiondeclarationsorthetaskitself.screen:youronlywaytoactistosubmitoneCodeisyourcontrolchannel,notareinterpretationoftheproject:everyactiprogram,andthoseprogramsmayinspectandoperatethemachineorautomatearliteralrequirements,includinganynamed-application,native-editable-staactoragent'sself-assessment.PromptP2showsthetargetconfiguratioPromptPromptP2.IndependentcaTreatobservationsasevidence,notproofbyasseaffirmativelysupported.PublishFAILwhenconcreteevidenceestablishesamaterUNVERIFIEDwhenamaterAtthebeginningofthisinspectiontheharnesssavedacompleteQEMUchecitsfiles,processes,localhostservices,network,GUI,andIPCaninspectionends,theharnessrestoresthatcheckpoinRSIAgentplementaryexperiencearoundthetargetquery.Itsruntimeinputsincludetmemory,andcompletedwaveoutcomes.Theresultinghandoffspecifiesprojectsorastoppingdecision,withprojectfixturespreparedsepareconsideryourhypothesesandfreelychoosethenextwave.Whenfurthertarget-relevantexploratieepRecursiveSelf-exploration.TheDRSpromptmakesfurtherpractittheprecedingexperiencereveals.Inadditiontothetargetqueryandmemorreceivesthegroundedoutcomeandtheactoragent'sleaPromptP4.PromptP4.DeepRecursiveSamongplausiblecapabilitygaps.Youchoosethenextexperience,notmemorywordinvaluethanreturningcontroltotheunchangedtargetliB.3MemoryConsolidationandLearninthatperformedthetask,togetherwiththeterminaloutcomeandfullverifierreport.ThretainscontrolovermRSIAgentrevise,reorganize,delete,orleavememoryunchanged.Youownthecontent,representation,retrievalstrategy,scope,andstoppingdecision;thereisnorequiredschema,length,numberoffiles,ornumberofbankbeforeitsupdateispromoted.I