LongHorizon-Harness：面向真实任务的长时程智能体能力提升研究

1

arXiv:2608.01964v1[cs.CV]3Aug2026

LongHorizon-Harness:AdvancingLong-HorizonAgentsforReal-WorldTasks

ZiyuMa*,HailangHuang*,ShunZou*,YongWang*†,ShidongYang,YimingHu,FeiWei,XiangxiangChu

DreamXTeam,AlibabaGroup*Equalcontribution†Projectlead

Largelanguagemodel(LLM)agentsincreasinglyundertakelong-horizontasksthatrequiresustainedreasoning,tooluse,andrevisionacrossmanyinterdependentsteps.However,existingagentharnessesmaintaintaskexecution,taskstate,andcompletionassessmentwithinagrowingcontext,makingthestatedifficulttotrackandallowingincorrectself-assessmentstopropagateintolaterdecisions.Wereformulatelong-horizonexecutionasatask-statemanagementproblemandproposeLongHorizon-Harness,whichmaintainsthetaskstateexplicitlyoutsideexecutionandupdatesitonlywithfactsindependentlyverifiedfromtheenvironment.ItsManage-Execute-Audit(MEA)loopusesamanagertomaintainthetaskstateanddeterminethenextsubtask,afresh-contextexecutortoperformit,andaread-onlyauditortoverifytheresultingenvironmentstatebeforethenextround.AlightweightAgentAdaptersupportsinterchangeablemodelandharnessbackendswithoutmodifyingtheirnativeagentloops.LongHorizon-HarnessimprovesQwen3.7-Plusfrom51.8%to80.7%onWeaveBench,from69.7%to77.2%onTerminal-Bench2.1,andfrom2.8%to8.3%onOSWorld2.0.ItalsoraisesClaudeOpus4.7from20.0%to34.3%onanOSWorld2.0subset,demonstratingconsistentgainsacrossmodels,harnesses,andinteractiondomains.

Github:

/AMAP-ML/LongHorizon-Harness

Website:

https://lh-harness.pages.dev

Figure1Left:LongHorizon-Harnessimproveslong-horizonexecutionacrossbenchmarksandbackbones.Withthesamebackboneandexecutionbackend,itliftsWeaveBenchPassRatefrom51.8%to80.7%,Terminal-Bench2.1from69.7%to77.2%,andOSWorld2.0binarycompletionby3.0×,andthegainstransferfromQwen3.7-PlustoClaudeOpus4.7.Right:Auditedstatetransitions.Insteadofonecontinuouslygrowingsessionthatjudgesitsownprogress,amanagerre-plansthenextsubtaskfromauditedfacts,afresh-contextexecutorperformsit,andaread-onlyauditorcertifieswhatactuallychangedintheenvironment.Auditreportsaretheonlycross-roundmemory,andinterchangeablebackends(e.g.,ClaudeCode,Codex)serveeachrole.

2

1Introduction

Overthepastfewyears,largelanguagemodels(LLMs)haveevolvedfromconversationalmodelsintothedecision-makingcoreofautonomousagentsforsoftwareengineering(

Yangetal.

,

2024

;

Wangetal.

,

2025

;

Renetal.

,

2026

;

Dingetal.

,

2025

),general-purposeassistance(

Anthropic

,

2025

;

OpenAI

,

2025a

;

b

;

Anthropic

,

2026

),scientificdiscovery(

Sunetal.

,

2025

;

Weietal.

,

2025a

),computeruse(

Anthropic

,

2024

;

OpenAI

,

2025c

;

GoogleDeepMind

,

2025

;

Sageretal.

,

2026

;

Zhouetal.

,

2026

;

Zhengetal.

,

2026

),andmultimodalinteraction(

Zhangetal.

,

2023

;

Agasheetal.

,

2025

).Acrosstheseapplications,agentsincreasinglyfacelong-horizonexecution,whichrequiresrepeatedreasoning,tooluse,observation,andrevisionovermanyinterdependentsteps,sometimesacrossmultiplecontextwindowsorsessions.Thelengthoftasksthatagentscancompleteincreasinglydetermineshowmuchworkcanbedelegatedtothem.METR(

Kwaetal.

,

2026

)reportsthatthetask-completionhorizonofadvancedagentshasdoubledroughlyeverysevenmonths,withthetrendacceleratingtoaboutfourmonthsforrecentmodels.Frontiercodingagentscanalreadysustainhours-longworkonasingleproject(

OpenAI

,

2025a

;

Anthropic

,

2025

;

Yangetal.

,

2026a

).However,alongerhorizondoesnotbyitselfmakeexecutionreliable(

Dongetal.

,

2026

).

Thedifficultyoflong-horizonexecutionliesnotinanyindividualstep,butinsustainingcoherentprogressacrossalongsequenceofinterdependentactions.Acrosssystemsanddomains,threechallengesconsistentlyemerge:(i)Compoundingerrorsandgoaldrift.Errorsinearlieractionsordecisionsaccumulatealongthetrajectory,distortsubsequentchoices,andgraduallysteertheagentawayfromitsoriginalobjective(

Sun

etal.

,

2026

).(ii)Contextrot.Astheinteractionhistorygrows,relevantinformationbecomesincreasinglydifficulttoretrieveanduse,andagentperformancecandegradesharplyoncecontextutilizationcrossesacriticalthreshold(

Liuetal.

,

2024

;

Hongetal.

,

2025

).(iii)Task-stateloss.Long-horizontasksaredifficulttocompletewithoutanaccurateandup-to-datetaskstate(i.e.,therequirementstosatisfy,actionsalreadycompleted,artifactsproduced,andfactsdiscoveredfromtheenvironment),butagentsoftenfailtorecover,retain,andupdatethisstatethroughouttaskexecution.

Extensiveeffortshavestrengthenedagentsfromboththemodelandharnesssides.Frontiermodelscontinuetoscaleinsize,extendtheircontextwindows,andacquirestrongercodingandagenticcapabilitiesbytrainingonlarge-scale,high-qualitydata(

Weietal.

,

2025b

;

Chuetal.

,

2026

;

Lietal.

,

2025

;

Maetal.

,

2026

;

Jietal.

,

2025

;

Yangetal.

,

2026b

).Inparallel,agentharnessessuchasClaudeCode(

Anthropic

,

2025

),CodexCLI(

OpenAI

,

2025a

),andOpenClaw(

OpenClawContributors

,

2026

)havebecomethestandardsystemlayerfororganizingprompting,tooluse,contextmanagement,andmulti-stepexecutionaroundthemodel.Forlong-horizontasks,theseharnessesalreadysupportplanning,taskdecomposition,tooluse,andsubagentsthatexecuteorreviewworkinisolatedcontexts.However,existingharnessesstillfacetwostructurallimitations:(i)Taskexecutionandtask-statemanagementsharethesamegrowingcontext.Theagentusesthesamecontexttoexecutethetaskandmaintainitstaskstate,whilethegrowingexecutionhistorymakestaskstateincreasinglydifficulttotrack.(ii)Taskexecutionandcompletionassessmentremaincoupled.Theagentperformseachsubtaskandjudgeswhetherithasbeencompleted,withanincorrectjudgmentpotentiallybeingrecordedaspartofthetaskstateandusedasapremiseforsubsequentdecisions.

Toaddresstheselimitations,weproposeLongHorizon-Harness,aframeworkthatorganizeslong-horizonexecutionasasequenceofindependentlyauditedtask-statetransitions.Ourkeyideaistomaintainthetaskstateasanexplicitrecordoutsidetaskexecution,updateitonlywithfactsindependentlyverifiedfromtheenvironment,andderiveeachnextsubtaskfromthecurrentrecordandtheoriginalgoal.Specifically,LongHorizon-HarnessfollowsaManage-Execute-Audit(MEA)loopasshowninFig.

1

.Amanagerreadsthecurrenttaskstateanddefinesonesubtaskwithitsdependencies,constraints,andacceptancecriteria.Anexecutorperformsonlythissubtaskinafreshcontext,whilearead-onlyauditorindependentlyinspectstheenvironmenttodeterminewhatchanged,whatwascompleted,andwhatremainsunmet(

Zhugeetal.

,

2024

;

Zhengetal.

,

2023

).Themanagerupdatesthetaskstatefromtheauditresultandbeginsthenextround,whiletheexecutor’sinteractionhistoryisdiscardedaftereachroundsothatonlycompact,verifiedtaskstatepersistsacrossthetask.ThroughalightweightAgentAdapter,LongHorizon-Harnesspreservesthenativeagentloops(

Yaoetal.

,

2023

)ofexistingsystemsandsupportsinterchangeablebackendsforallthreeroles,spanningmodelssuchasClaudeOpus,GPT,andQwenandharnessessuchasCodexCLI(

OpenAI

,

2025a

),ClaudeCode(

Anthropic

,

2025

),OpenClaw(

OpenClawContributors

,

2026

),andHermesAgent(

Nous

Research

,

2026

).

3

Figure2OverviewofLongHorizon-Harness.LongHorizon-Harnessprocessesalong-horizontaskTthroughrepeatedManage-Execute-Auditrounds,shownbythedashedboxandnumberedflow❶–❸.ThemanagerreadstaskstateSiandconstructssubtaskcontractci,whichspecifiesthegoal,acceptancecriteria,boundaryconstraints,andrelevantpriorevidence(❶).Themanagermayinsteadrequestuserinformationorauthorizationthroughtheaskroute.TheselectedGUIorCLIexecutorperformsthesubtaskinafresh,budget-boundedcontextandmodifiestheenvironment(❷).Theauditorindependentlyinspectstheresultingenvironmentthroughread-onlytoolsandproducesauditreportvi(❸).Themanagerusesvitoupdatethetaskstatebeforethenextround.

WeevaluateLongHorizon-Harnessonthreerecentlong-horizonbenchmarks,WeaveBench(

Lietal.

,

2026

),OSWorld2.0(

Yuanetal.

,

2026

),andTerminal-Bench2.1(

Merrilletal.

,

2026

),whichtogethercovercross-interfacecoordination,desktopworkflows,andchallengingcommand-linetasks.Undermatchedmodelbackendsandevaluationprotocols,LongHorizon-HarnessraisesWeaveBenchPassRatefrom51.8%to80.7%usingQwen3.7-PlusandClaudeCode,nearlydoublingthestrongestofficiallyreportedresultof41.2%obtainedbyClaudeOpus4.7withClaudeCode.OnTerminal-Bench2.1,itimprovesperformancefrom69.7%to77.2%withQwen3.7-Plus.OnthefullOSWorld2.0benchmark,itraisesbinarycompletionfrom2.8%to8.3%withQwen3.7-Plus;ona34-tasksubset,itimprovestheresultfrom20.0%to34.3%withClaudeOpus4.7.Ourcontributionsaresummarizedasfollows:

•Wereformulatelong-horizonexecutionfromasinglegrowingtrajectoryintoatask-statemanagementproblem.Thetaskstateismaintainedexplicitlyoutsideexecution,updatedonlywithindependentlyverifiedfacts,andusedtodetermineeachnextsubtaskundertheoriginalgoal.

•WedesignLongHorizon-Harness,whichrealizesthisprinciplethroughaManage-Execute-Auditloop.Ineachround,themanagerdefinesonesubtaskfromthecurrenttaskstate,theexecutorperformsitinafreshcontext,andtheauditorindependentlyinspectstheenvironmentbeforethemanagerupdatesthestateandbeginsthenextround.

•WeevaluateLongHorizon-HarnessonWeaveBench,OSWorld2.0,andTerminal-Bench2.1.WithQwen3.7-Plus,itimprovesWeaveBenchPassRatefrom51.8%to80.7%,Terminal-Bench2.1from69.7%to77.2%,andOSWorld2.0binarycompletionfrom2.8%to8.3%,whilealsodeliveringconsistentgainswithClaudeOpus4.7.

4

2Method

2.1Overview

Givenalong-horizontaskTandacomputerenvironment,LongHorizon-Harnessexecutesthetaskthroughasequenceofdynamicallydeterminedroundsratherthanasinglecontinuouslygrowingsession.Theharnessmaintainsanexplicittaskstateoutsidetaskexecutionandadvancesitonlywithevidenceindependentlyverifiedfromtheenvironment.Acrossrounds,onlythetaskstateanditssupportingauditreportspersist;theexecutor’srawinteractiontrajectoryisdiscardedaftereachround.Fig.

2

providesanoverviewoftheframework.

EachroundfollowsaManage-Execute-Audit(MEA)loop.LetSidenotethetaskstateavailableatthebeginningofroundi,ei—1∈εthecurrentenvironmentstate,andVi—1=(v1,...,vi—1)theaccumulatedauditreports.Themanagerconstructsaboundedsubtaskcontractci.Afresh-contextexecutorperformsthecontract,transformstheenvironmentfromei—1toei,andreturnsanexecutionreportoi.Aseparateauditortheninspectseithroughread-onlytoolsandproducesauditreportvi.ThemanagerincorporatesviintothenexttaskstateSi+1beforedeterminingwhetheranotherroundisrequired.Theloopendswhentheauditedstatesatisfiestheoriginaltask,nopermittedsubtaskcanadvancetheremainingrequirements,userinputisrequired,ortheroundbudgetisexhausted.

2.2Manager

Themanagerownsthepersistenttaskstateanddetermineshowthetaskshouldproceed.IthasaccesstotheoriginaltaskT,thecurrenttaskstate,andallaccumulatedauditreports,buthasnodirectinterfacetothecomputerenvironment.Itcannotobserveapplicationstate,inspectworkspacecontents,invokeGUIorCLItools,ormodifytheenvironment.Itsdecisionsarebasedentirelyonthetaskstateandenvironmentevidencerecordedbytheauditors.

Afterroundi,themanagerupdatesthetaskstateandproducesthenextcontroldecision:

(Si+1,qi+1,ci+1)=Φmgr(T,Si,Vi),(1)

whereVi=(v1,...,vi)andqi+1isoneofExEcuTE,DoNE,BLockED,andAsk.Thecontractci+1isreturnedonlywhenfurtherexecutionisrequired.

Task-stateupdate.Thetaskstateisastructuredcollectionoftask-relevantrecords:arequirementrepresentsanobjectiveorconstraintderivedfromtheoriginaltask,anartifactrepresentsanoutputcreatedormodifiedduringexecution,andafactrecordsenvironmentinformationneededbysubsequentrounds.Eachrecordismarkedascompleted,pending,blocked,oruntrusted,andretainsreferencestotheauditevidencesupportingitscurrentstatus.TheinitialstateS1isconstructedfromTwithitsrequirementsmarkedaspending.Aftereachround,themanagerappliestheverifiedfindingsinvitoSi,addingorupdatingthecorrespondingrecordswhileleavingunresolvedonespending,blocked,oruntrustedasappropriate.Executorclaimsdonotdirectlychangethepersistentstate:arecordismarkedascompletedonlywhensupportedbycleanauditevidence.

Next-subtaskconstruction.ThemanagercomparesSi+1againsttheoriginaltask,selectsanunresolvedobjectivethatcanbeadvancedfromthecurrentstate,andchecksitsdependenciesandprerequisites.Itthenconstructsaboundedcontractci+1specifyingtheimmediategoal,acceptancecriteria,boundaryconstraints,andthetask-staterecordsandpriorauditreportsrelevanttoexecutionandverification.ThecontractisroutedtoaGUIorCLIexecutoraccordingtotheprimaryenvironmenttransitionitrequires.ThemanagerreturnsDoNEwhentheauditedstatesatisfiesTwithoutunresolvedintegrityviolations,BLockEDwhennopermittedactioncanadvancetheremainingrequirements,andAskwhenprogressrequiresuserinformationorauthorization;otherwise,itreturnsExEcuTEtogetherwithci+1.

2.3Executor

Theexecutorperformsthecontractselectedbythemanagerandistheonlyrolepermittedtointentionallymodifytheenvironment.Inroundi,itreceivestheoriginaltaskT,thecurrenttaskstateSi,thesubtask

5

contractci,andonlythepriorauditreportsreferencedbythecontract.Ittransformstheenvironmentfromei—1toei:

(ei,oi)=Φexec(T,Si,ci;ei—1),(2)

whereoisummarizestheactionsperformed,resultingstate,artifactsproducedormodified,andissuesencounteredduringexecution.Thereportoidescribestheexecutor’soutcomebutdoesnotestablishthatthecontracthasbeencompleted.

Fresh-contextexecution.Eachexecutorinvocationrunsasafresh,budget-boundedepisodecontainingonlytheinformationsuppliedforthecurrentround.Itdoesnotreceivetherawinteractiontrajectoriesofearlierrounds.Withintheepisode,theexecutormayperformmultiplecyclesofplanning,environmentinteraction,observation,andrevision.Whentheepisodeends,itsrawtrajectoryandinternalreasoningarediscarded;onlyoiisforwardedforauditing.

GUIandCLIcapabilityboundaries.GUIandCLIexecutorsoperatethroughdifferentenvironmentinterfaces.TheGUIexecutorreceivesscreen-orientedcapabilitiessuchastakingscreenshots,clicking,scrolling,andenteringtext,andisresponsiblefortransitionscenteredonapplicationandinterfacestate.TheCLIexecutorreceivescapabilitiessuchasshellexecution,fileediting,coding,andtesting,andisresponsiblefortransitionscenteredonworkspace,process,andprogramstate.

Theharnessexposesonlytheenvironmentinterfaceandtoolsetassignedtotheselectedexecutorrole.Capabilitiesoutsidethatroleareunavailableunlessexplicitlyprovidedbyitsconfiguration.Thisseparatesresponsibilityfordifferentclassesofstate-changingactionswhileallowingthemanagertoselecttheinterfaceappropriateforeachcontract.

Backendexecution.Executorsareinstantiatedthroughacommonagent-adapterinterface.Givenacontract,arole-specificenvironmentinterface,andanexecutionbudget,theadapterlaunchesanexistingbackendsuchasClaudeCode,CodexCLI,orOpenClawasoneboundedepisode.Thebackendretainsitsnativeplanningandtool-useloopandmaynormallyinvoketheshell,editfiles,writecode,runtests,orinteractwithapplicationswhenthosecapabilitiesareexposedbyitsassignedrole.Theharnessdoesnotreplacethebackend’sinternalexecutionprocess;itcontrolsthesuppliedcontext,availabletools,environmentpermissions,executionbudget,andreturnedreport.

2.4Auditor

Theauditorindependentlyverifiestheenvironmentstateproducedbytheexecutor.Afterexecution,itreceivestheoriginaltaskT,thetaskstateSi,thecontractci,thepriorauditreportsreferencedbythecontract,andtheexecutorreportoi.Itdoesnotreceivetheexecutor’srawinteractiontrajectoryorinternalreasoning.Theauditorinspectstheresultingenvironmenteiandproduces

vi=Φaud(T,Si,ci,oi;ei),(3)

whereviisappendedtothepersistentaudithistoryandreturnedtothemanagerbeforethenextround.

Independentenvironmentinspection.Theauditorstartsfromafreshcontextthatexcludestheexecutor’srawinteractiontrajectoryandinternalreasoning.Itmayuseoitolocaterelevantfiles,windows,logs,processes,orotheroutputs,butdeterminescompletionbyindependentlycomparingtheresultingenvironmentagainstthegoal,acceptancecriteria,andboundaryconstraintsinci.AGUIauditorexaminesapplicationandscreenstatethroughobservation-orientedinteractions,whereasaCLIauditorusesnon-mutatingcommandsandinspectiontoolstoexaminefiles,metadata,logs,processes,tests,andworkspacestate.Inbothcases,auditconclusionsmustbesupportedbyevidenceobtaineddirectlyfromtheenvironmentratherthanbytheexecutor’scompletionclaim.

6

Table1ResultsonWeaveBench.Grayrowsdenoteofficialresultsreportedby(

Lietal.

,

2026

),usingthebestreportedthinkingmodeforeachbackbone;modelsotherthanGPT-5.5andClaudeOpus4.7wereevaluatedonlywithOpENCLAw.BlackrowsdenoteourrunswithQwen3.7-Plus.Ourrunsuserootprivilegesinsidethetaskvirtualmachine,whereastheofficialresultsusearegularuseraccountandarethereforeincludedasreferencepointsratherthanmatchedcomparisons.PRdenotesfull-taskPassRate(%),andOveralldenotesthemeanscoreoverall114tasks.TheremainingcolumnsreportPassRateacrosstheeightbenchmarkdomains.

Model

Harness

PR↑

Overall↑

DSK

DOC

GAM

WEB

DAV

OPS

SPA

DES

ClaudeOpus4.7

CLAuDECoDE

41.2

0.532

55.6

47.1

23.5

53.3

23.1

50.0

33.3

40.0

OpENCLAw

35.1

0.482

55.6

29.4

23.5

66.7

15.4

41.7

16.7

20.0

HERmEsAGENT

28.1

0.516

33.3

47.1

11.8

26.7

30.8

50.0

8.3

10.0

CoDExCLI

13.2

0.378

16.7

11.8

11.8

6.7

7.7

25.0

16.7

10.0

GPT-5.5

CoDExCLI

35.1

0.499

38.9

29.4

23.5

53.3

15.4

50.0

58.3

10.0

OpENCLAw

33.3

0.466

38.9

35.3

35.3

21.4

23.1

38.5

33.3

40.0

HERmEsAGENT

31.6

0.466

55.6

29.4

35.3

40.0

7.7

25.0

25.0

20.0

CLAuDECoDE

14.9

0.299

33.3

11.8

11.8

0.0

15.4

16.7

25.0

0.0

GPT-5.4

OpENCLAw

22.8

0.465

55.6

35.3

5.9

0.0

23.1

23.1

8.3

20.0

GPT-5.3-codex

OpENCLAw

18.4

0.456

33.3

23.5

29.4

0.0

7.7

16.7

8.3

20.0

GPT-5.2-codex

OpENCLAw

6.1

0.321

5.6

11.8

0.0

0.0

15.4

16.7

0.0

0.0

GPT-5.1-codex

OpENCLAw

1.8

0.226

0.0

5.9

0.0

0.0

7.7

0.0

0.0

0.0

Gemini3.1pro

OpENCLAw

1.8

0.223

0.0

0.0

0.0

0.0

0.0

8.3

8.3

0.0

Qwen3.5-397B-A17B

OpENCLAw

0.9

0.318

0.0

0.0

0.0

0.0

0.0

8.3

0.0

0.0

Qwen3-VL-8B-Think

OpENCLAw

0.9

0.092

0.0

0.0

0.0

0.0

8.3

0.0

0.0

0.0

GUI-Owl-1.5-32B

OpENCLAw

0.0

0.065

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

Qwen3.7-Plus

ClaudeCode

51.8

0.702

83.3

76.5

29.4

46.7

53.8

66.7

16.7

20.0

LongHorizon-Harness(ClaudeCode)

80.7

0.835

88.9

100.0

58.8

73.3

84.6

91.7

66.7

80.0

Read-onlyauthority.Theauditormaychangeitsobservationviewwhennecessaryforinspectionbutcannotmodifytask-relevantenvironmentstate.Itcannotcreate,edit,overwrite,move,ordeleteprotectedartifacts,executestate-changingcommands,orperformGUIactionsthataltertheresultunderinspection.Theharnessmonitorstask-relevantworkspaceandartifactstatethroughoutauditing;anydetectedmutationisrecordedasanintegrityviolation,andtheresultingreportcannotsupportacompletedtask-staterecord.

Audit-resultconstruction.Theauditreportvirecordsthreeclassesoffindings.First,itassignsacompletionstatusofcomplete,incomplete,orblockedbyevaluatingthecontract’sacceptancecriteria.Second,itassignsanintegritystatusofclean,suspect,orviolationbycheckingworkspacemutations,artifactvalidityandprovenance,andrelevantdeletionconstraints.Third,itrecordsthetask-stateupdatessupportedbytheinspection,includingverifiedfacts,supportingevidence,andremaininggaps.

Theauditormayproposechangestorequirement,artifact,andfactrecords,butthemanagerdetermineshowthesefindingsareincorporatedintoSi+1.Consequently,oiremainsanunverifiedexecutionsummary,whereasviprovidestheenvironment-groundedevidencethatmayadvancethepersistenttaskstate.

3Experiments

3.1ExperimentalSetup

Comparedconfigurations.WeuseQwen-3.7-Plusastheprimarybackbonemodel,andadditionallyevaluateClaudeOpus-4.7toassessthegeneralityacrossbackbonemodels.Foreachbackbonemodel,wecomparetheoriginalClaudeCodeevaluationframework(

Anthropic

,

2025

)withLongHorizon-Harness,wherethelatterusesClaudeCodeasitsexecutionbackend.Unlessotherwisespecified,themanager,executor,andauditorareinstantiatedwiththesamebackbonemodel,ensuringthatthecomparisonreflectstheeffectoftask-statemanagementratherthandifferencesinmodelcapability.

7

Table2ResultsonOSWorld2.0.Grayrowsdenoteofficialresultsreportedby(

Yuanetal.

,

2026

)underbatched-andsingle-actionsettings,whiletheblackrowdenotesLongHorizon-HarnesswithQwen3.7-Plusinthehybridsetting.Binaryisthepercentageoftaskswhosefinalbenchmarkscoreequals1,andPartialisthemeanbenchmarkscoreoverall108tasks.Boldmarksthebestofficialresult.

Model

Harness/Mode

Binary↑

Partial↑

ClaudeOpus4.8

Batchedactions

20.6

54.8

ClaudeOpus4.7

Batchedactions

18.2

48.9

GPT-5.5

Batchedactions

13.0

49.5

ClaudeOpus4.8

Singleaction

18.5

49.3

ClaudeOpus4.7

Singleaction

13.9

49.1

ClaudeSonnet4.6

Singleaction

8.3

41.5

MiniMaxM3

Singleaction

4.6

22.3

Kimi2.6

Singleaction

4.6

22.1

Qwen3.7-Plus

Singleaction

2.8

21.5

Qwen3.7-Plus

LongHorizon-Harness(hybrid)

8.3

35.2

Implementationdetails.LongHorizon-HarnessintegratesexistingagentbackendsthroughaunifiedAgen-tAdapterinterface.Eachroleisassignedanindependentexecutionbudget:theexecutorislimitedto1800secondsperround,whileboththemanagerandtheauditorarelimitedto300seconds.WesetthemaximumnumberofMEAroundstoNmax=25.

Benchmarksandmetrics.WeevaluateLongHorizon-Harnessonthreerecentbenchmarksthatcovercomple-mentaryformsoflong-horizonexecution.

WeaveBench(

Lietal.

,

2026

)consistsof114tasksthatrequirecoordinatedGUIandCLIinteractionswithinthesameworkflow.WefollowthestandardWeaveBenchevaluationprotocol,usingthetaskdefinitions,workspaces,runtimeassets,andjudgetemplatesreleasedbytheofficialbenchmarkforalltasks.WereportPassRate,definedasthepercentageoffullypassedtasks,andOverall,definedastheaveragescoreacrossalltasks.Inaddition,wereportPassRateforeachoftheeightdomainscoveredbythebenchmark.

OSWorld2.0(

Yuanetal.

,

2026

)consistsof108desktopworkflowtasks,withamedianhumancompletiontimeofapproximately1.6hours.Weusetheofficialosworld-v2-2026.06.24releaseanditsstandardDocker-basedVMinfrastructure.Wereporttwometrics:BinaryandPartialAccuracy.BinaryAccuracycountsataskassuccessfulonlywhenitsfinalscoreis1,whilePartialAccuracyistheaveragescoreoveralltasks.

Terminal-Bench2.1(

Merrilletal.

,

2026

)isdesignedtoevaluatechallengingandrealisticcommand-linetasks.WefollowtheevaluationsetupusedfortheQwen-seriesmodels,andevaluateonTerminal-Bench2.1usingHarborwiththeDockerbackend.Finalresultsreporttheaveragescoreoverthreeindependentrunspertask,andarecomparedagainsttheofficialleaderboardresults.

3.2MainResults

Effectivenessonlong-horizoncomputeruse.Table

1

reportstheresultsonWeaveBench.TheClaudeCodeharnesswithQwen3.7-PlusachievesaPassRateof51.8%andameantaskscoreof0.702.UsingthesamemodelandretainingClaudeCodeastheexecutorbackend,LongHorizon-Harnessincreasestheseresultsto80.7%and0.835,respectively.Thismatchedcomparisonisolatesthecontributionoftheadditionaltask-statemanagementlayer.Performancealsoimprovesacrossalleighttaskdomains,showingthatthegainisnotconcentratedinaparticularapplicationcategory.Sincetheofficiallyreportedconfigurationsuseadifferentprivilegesetting,weincludethemasreferencepointsbutbaseourmainconclusiononthematchedClaudeCodecomparison.

8

Figure3ResultsonTerminal-Bench2.1.LongHorizon-HarnessretainsClaudeCodeasitsexecutorbackendandimprovessuccessratefrom69.7%to77.2%.Valuesmarkedwith∗areexternallyreportedmetrics.

Table3OSWorld2.0Opus4.7subset(34tasks).BothrowsuseClaudeOpus4.7asthebackbone.

Model

Harness/Mode

Binary↑

Partial↑

ClaudeOpus4.7

SINGLEAcTIoN

20.6

55.8

ClaudeOpus4.7

LongHorizon-Harness(hybrid)

35.3

66.9

Table

2

reportstheresultsonOSWorld2.0.WithQwen3.7-Plus,LongHorizon-Harnessincreasesbinarycompletionfrom2.8%to8.3%andpartialscorefrom21.5%to35.2%.Thehigherpartialscoreshowsthattheagentsatisfiesalargerfract