2025年思考模型Seed-Thinking-v1.5技术报告（英文版）-字节跳动

1Seed1.5-Thinking:AdvancingSuperbReasoningModelswithReinforcementLearningByteDanceSeedFullauthorlistinContributionsAbstractWeintroduceSeed1.5-Thinking,capableofreasoningthroughthinkingbeforeresponding,resultinginimprovedperformanceonawiderangeofbenchmarks.Seed1.5-Thinkingachieves86.7onindicatingitsbroaderapplicability.Comparedtootherstate-of-the-artreasoningmodels,Seed1.5-ThinkingisaMixture-of-Experts(MoE)modelwitharelativelysmallsize,featuring20Bactivatedand200Btotalparameters.Aspartofourefforttoainternalbenchmarks,BeyondAIMEandCodeforces,bothofwhichwillbepubliclyreleasedtosupportfutureresearch.Modeltriallink:/experience/ark.Date:April10,2025Figure1Benchmarkperformanceonreasoningtasks21Introductionmathematicalproblem-solving,andcodegeneration.Theseadvancementsunderscoreashifttowardmorechain-of-thought,andlarge-scalereinforcementlearning.Inthiswork,wepresentanewreasoningmodel,calledSeed1.5-Thinking.Thismodelhasachievedstrongperformanceinbothreasoningandnon-reasoningMathematicalReasoning:Formathcompetition,Seed1.5-Thinkingachieves86.7onAIME2024,matchingtheperformanceofo3-mini-highandsignificantlyoutperformingo1andDeepSeekR1,demonstratingcompetitivestrength.SinceAIME2024nolongerprovidessufficientdiscrimination,weconstructamorechallengingevaluationsetnamedBeyondAIME.AllproblemsinBeyondAIMEarenewlycuratedbyhumanexpertsanddesignedWhileSeed1.5-Thinkingsurpassesbotho1andR1,thereremainsaperformancegapcomparedtoo3andGeminipro2.5.Thisalsofurtherdemonstratesthediscriminativepowerofthenewevaluationset.CompetitiveProgramming:Fortheevaluationofcompetitiveprogramming,weadoptCodeforcesasourbenchmark.UnlikesomepriorworksthatrelyonEloScores,whichcontainsestimationandarenotdirectlycomparable,weadoptaconcreteevaluationprotocolbasedonthemostrecent12Codeforcessolvestheproblemwithinkattempts,i.e.,selectingthebestresultfromkgeneratedsubmissions.Wechoosetoreportpass@8sinceitprovidesmorestableresultsgapremainscomparedtoo3.Theevaluationsetwillbemadepubliclyavailableinafuturerelease.Science:Seed1.5-Thinkingreachesascoreof77.3onGPQA,closetoo3-levelperformance.Importantly,thisgainislargelyattributedtoimprovedgeneralizationfrommathematicaltraining,ratherthananreplicatereal-worlduserneeds.ThroughhumanevaluationsconductedagainstDeepSeekR1acrossdiversescenarios,Seed1.5-Thinkingdemonstratessignificantadvancements:itattainsan8.0%overallriseinusers’positivefeedback,therebyhighlightingitsaugmentedabilitytomanageintricateuserTherearethreekeypointsinthedevelopmentofhigh-qualityreasoningmodels:trainingdata,RLalgorithm,andRLinfrastructure.detail.DataForSFTtraining,unlikeconventionalpost-trainingdata,reasoningmodelsrelyonchain-of-thoughtdata,whichexplicitlyoutlinesthestep-by-stepreasoningprocess.Ourpreliminaryexperimentsshthattoomuchnon-CoTSFTdatacansignificantlyreducethemodel’sabilitytoexplore.ForRLtraining,weincorporatefourcategoriesofdata:non-reasoningdatalikecreativewritinganddialogue.Amongthese,thelogtoperformanceimprovementsontheARC-AGIbenchmarksignificantly.ThemathdataexhibitsstronggeneralizationcapabilitiesandcanleadtobroadperformanceimprovementsacrRLAlgorithmRLtrainingofreasoningmodelsishighlyunstableandoftencrashes,especiallyformodelstrainingofRLsystemsiscrucialforthesuccessofreasoningmodels.Toaddresstheselong-standingissues,wehavepioneeredVAPO[5]andDAPO[6]—twodistinctframeworkstailoredforactor-criticandpolicy-gradientRLparadigms,respectively.VAPOnowstandsasthestate-of-the-art(SOTA)solutioninactor-criticmethods,whileDAPOestablishesanewSOTAresultforpolicy-gradientapproaches3withoutcriticmodels.BytargetingthecoreinstabilityissuesinRLtraining,bothmethodsdeliverrobustandconsistenttrainingtrajectories,effectivelyenablingreliableoptimizationofreasoningmodels.RLInfrastructureThecomplexityofLargeLanguageModels(LLM)basedreinforcementlearningsystemsdemandsrobustinfrastructuretoensurescalability,reproducibility,andcomputationalefficiency.Tohandleheterogeneousworkloads,wedecouplestreamingrolloutarchitecturethatasynchronouslyprocessespartialtrajectorygenerationsthroughprioritizedsamplepools,achieving3×fasteriterationcyclesthansynchronousframeworks.Thesystemalsosupportsmixefaultrecovery,criticalformaintainingstabilityduringlarge-scaleRLruns.2Data2.1RLTrainingDataOurRLtrainingdataconsistsoftwomainparts:verifiableproblemswithdefinitiveanswersandnon-verifiableproblemswithoutdefinitiveanswers.Themodel’sreasoningabilityprimarilycomesfromthefirstpartand2.1.1VerifiableProblemsTheVerifiableproblemsprimarilycompriseSTEMquestionspairedwithanswers,codingproblemsequippedwithunittests,andlogicreasoningsthatareamenabletoautomatedverification.STEMDataOurdatasetconsistsofseveralhundredthousandhigh-quality,competition-gradeproblemsspanningmath-ematics,physics,andchemistry,withmathematicscomprisingthemajority(over80%).Theseproblemsaredrawnfromamixofopen-sourcedatasets,publiccompetitions(bothdomesticandinternational),andproprietarycollections.Fordatacleaning,wefirsteliminatequestionswithincompletestatements,inconsistentnotation,orunclearProblemsforwhichthemodelachievedawoNscore(worstofN)of1aredeemedtoosimpleandremoved.Finally,somequestionsmayhaveaninaccuratereferenceanswer.WeuseSOTAreasoningmodelstogeneratemultiplecandidateresponsesforeachquestion.Ifthemodel’sanswerswereinconsistentwiththereferenceanswer,butthemodel’soutputsshowedhighinternalconsistency,orinvolvedonlyaverysmallnumberofreasoningtokens,weconsiderthereferenceanswertobeincorrect.Humanexpertsthenconductmanualverificationonthesequestionstoensurethatthereferenceanswersarecorrect.Wealsoapplydataaugmentationtomakethedatamoresuitableforlearningandevaluation.Specifically,weconvertmultiple-choicequestionsintofill-in-the-blankorshort-answerformatstoeliminatethepossibilityofguessingandtobetterassessreasoningability.AndwemodifycertainmathproblemstoensurethattheanswAfterdatacleaningandaugmentation,wefinallyobtainatrainingsetof100kSTEMproblems.Duringtraining,weusemodel-basedSCodeDataForcodingproblems,weprioritizethesourceofhigh-qualityandchallengingalgorithmictasks,primarilydrawnfromesteemedcompetitiveprogrammingcontests.asetofunittests,andacheckerscript.Unittestsvalidatethefunctionalcorrectnessofsolutions,whilethecheckerscriptenforcesadditionalconstraintssuchasoutputformattingandedgecases.Wealsoperformdifficultyfiltering,ensuringthatproblemspossessanappropriatelevelofcomplexityandapplicabilitytoreal-worldalgorithmicreasoning.Forevaluation,themostaccurateformistosubmitthegeneratedcodetotheofficialplatforms.However,duringreinforcementlearning,real-timesubmissionisn’tfeasible.Thus,wedevelopedanoff-lineevaluation4setforefficientlocalvalidation.Ourobservationsindicateastrongcorrelationbetweenofflineevaluationresultsandofficialverdicts.Alltrainingandevaluationproblemsareintegratedintoanin-housecodesandbenvironment,enablingdirectexecutionandassessmentofmodel-generatedcode.Weensurethesandbox’sstabilityandhighthroughputtodeliverconsistentandaccuratefeedbackduringtheRLtrainingprocess.LogicalPuzzleDataForthelogicreasoningdata,wegather22commonlystudiedtasks,suchas24-point,mazes,Sudoku,etc.Foreachtask,weconstructadatageneratorandananswerverifier.Thedatageneratorcanautomaticallyproducealargeamountoftrainingandevaluationdata.Moreover,formanyofthetasks,wecanconfigurethedifficultyofthegeneratedproblems.Duringthetrainingprocess,wegraduallyadjustthedifficultyoftheabout10kpuzzleproblemsforRLtraining.2.1.2Non-verifiableProblemspreferences,involvingtaskslikecreativewriting,translation,knowledgeQA,role-playing,andsoon.ThepromptsareoriginatedfromRLtrainingdataforDoubao-1.5Pro[7].ThedatasethassufficientcoverageWediscarddatawithlowsamplescorevarianceandlowdifficulty.Tobespecific,weusetheSFTmodeltogeneratemultiplecandidatesforeachpromptandthenscorethemusingarewardmodel.PromptswithlowscorevariancesareremovedastheyexhibitlimitedsamplingdiversityandminimalpotentialforPromptsarealsoremovedwheretherewardscoreimprovementsurpassesacertainthresholdduringtheDoubao1.5ProRLtrainingprocess[8].Thisisbecausesuchdatamaybeoverlysimplisticoralreadyabundantlyrepresentedinthedataset.Offlineexperimentsshowthatoveroptimizingsuchsamplesleadstoprematurecollapseofthemodel’sexplorationspaceanddiminishtheperformance.Forthesenon-verifiabledata,weemployapairwiserewardingmethodforscoringandRLtraining.Bycomparingtherelativequalityoftwosamples,thisapproachaidsthemodelinbetterunderstandinguserpreferences,enhancingthequalityanddiversityofgeneratedresults.ThedetailoftherewardmodelisintroducedinAdvancedMathBenchmarkThecurrentreasoningmodelsusuallyuseAIMEasthego-tobenchmarktoevaluatabilities.However,withonly30problemsreleaseresults,makingitchallengingtoeffectivelydifferentiatebetweenstate-of-the-artreasoningmodels.TobetterSpecifically,wecollaboratewithmathematicsscompetitionformats.Wesystematicallyadaptexistingcompetitionquestionsthroughstructuralmodificationsarenevertrivialvalues—suchasnumbersexplicitlymentionedintheproblemstatement—toreducethechanceofmodelsguessingthecorrectanswThroughthisrigorousfilteringandcurationprocess,wecompileafinalsetof100problems,eachwithadifficultylevelequaltoorgreaterthanthatofthehardestquestionsinAIME.SimilartoAIME,allanswersareguaranteedtobeintegers(withoutbeingrestrictedtoaspecificnumericalrange),whichsimplifiesand3RewardModelingAsacrucialcomponentinRL,rewardmodelingdefinestheobjectiveorgoalthatthepolicyistryingto5Verifier-typeTrainingexamples(approximate)Humanlabeledtestset>98%Seed-Thinking-Verifier>99%Table1Accuracyoftwoverifier-types.Specifically,theaccuracyonthetrainingsetisderivedfromthetrainingstatistics.Additionally,wemanuallyannotated456samplestoformthetestset,whicharespecificallyselectedfromcasesthattheSeed-Verifiercannothandlestably.modelresponsesduringthetrainingstage.Forverifiableandnon-verifiableproblems,weemploydistinctrewardmodelingmethodologies.3.1RewardModelingforVerifiableProblemsWithproperprinciplesandthoughttrajectories,weutilizeLLMstojudgeawidearrayofverifiablequestionsacrossdiversescenarios.ThisapproachyieldsamoregeneralizedsolutionthatsurpassesthelimitationsofWehavedesignedtwoprogressiverewardmodelingsolutions,Seed-VerifierandSeed-Thinking-Verifier:•Seed-Verifierisbasedonasetofmeticulouslycraftedprincipleswrittenbyhumans.ItleveragesthepowerfulfoundationalcapabilitiesofLLMstoevaluateatripletconsistingofthequestion,referenceequivalent,itreturns“YES”;otherwise,itreturns“NO”.Theequivalencehereisnotaliteralexactmatchbutratheradeeperassessmentbasedoncomputationalrulesandmathematicalprinciplesthatproveaccuratelyreflectswhetherthemodel•Seed-Thinking-Verifierisinspiredbythehumanjudgmentprocess,whichgeneratesconclusivejudgmentsthroughmeticulousthinkingandin-depthanalysis.Toachievethis,wetrainedaverifierthatprovidesadetailedreasoningpathforitsevaluations.Specifically,wetreatedtTheSeed-Thinking-VerifiersignificantlyalleviatesthreemajorissuesassociatedwiththeSeed-Verifier:•RewardHacking:Non-thinkingmodelsmayexploitloopholestoreceiverewardswithouttrulyunder-standingtheproblem.ThedetailedreasoningprocessinSeed-Thinking-Verifiermakessuchhackingmoredifficult.•UncertaintyinPredictions:Incaseswherethereferenceandmodel-generatedanswersareessentiallyequivalent,whichmaydifferinformat,e.g.,219vs524288,theSeed-Verifiermightsometimesreturn“YES”andothertimes“NO”.TheSeed-Thinking-Verifierprovidesconsistentresultsbythoroughlyanalyzingthereasoningbehindtheanswers.TheabilityofSeed-Thinking-VerifiertoprovidedetailedreasoningallowsittobetteraddresstheseTable1presentstheperformanceoftheabovetwoverifiers.TheresultsindicatethattheSeed-Verifierstrugglestoeffectivelyhandlesomeparticularcases,whereastheSeed-Thinking-Verifierdemonstratesaremarkableabilitytoprovideaccuratejudgments.WhilethethinkingprocessofthelatterdoesconsumeasignificantamountofGPUresources,webelievethatthepreciseandrobustrewardresultsitgeneratesarecrucialforendowingthepolicywithstrongreasoningcapabilities.63.2RewardModelingforNon-verifiableProblemsFornon-verifiableproblems,wetrainarewardmodelforRLtraining.TherewardmodeltrainingdataisconsistentwiththehumanpreferencedatautilizedinDoubao1.5Pro[7],primarilyencompassingsuchascreativewritingandsummarization.whichevaluatesthesuperiorityoftworesponsesandusscore.Thisapproachenablesthemodeltodirectlycomparedifferencesbetweenresponsesduringscoring,therebyavoidingexcessivefocusonirrelevantdetails.ExperimentalresultsdemonstratethatthisrewardmodelingmethodimprovesthestabilityofRLtraining,particularlyinthemixedtrainingscenariosinvolvingbothnon-verifiableandverifiableproblems,byminimizingconflictsbetweenthetwodifferenttypesofrewardmodelingparadigms.Thisimprovementmaybeattributedtothepairwisegenerativerewardmodel’sinherentadvantageinmitigatingoutlierscoregenerationcomparedtoconventionalrewardmodels,thereforeavoidingsignificantdiscrepanciesinscoredistributionswiththeverifier.4Approach4.1SupervisedFine-TuningOurtrainingprocessstartswithsupervisedfine-tuning(SFT).TheSFTphasesetsasolidfoundationforthesubsequentreinforcementlearningstage.ComparedtoinitiatingRLfromabasemodel,theSFTmodelproducesmorereadableoutputs,exhibitsfewerinstancesofhallucination,anddemonstratesreducedharmfulness.WecurateanSFTdatacomprising400ktraininginstance,including3100knon-verifiableproblems.VerifiablepromptsarerandomlysampledfromRLtrainingset.Non-verifiabledataaresourcedfromtheSFTdatausedforDoubao-Pro1.5[7],coveringareassuchascreativewriting,knowledge-basedQA,safety,andfunctioncalling.Togeneratehigh-qualityresponseswithlongCoT,weemployaniterativeworkflowthatintegratesmodelsynthesis,humanannotation,andrejectionsampling.Initially,humanexpertsapplypromptengineeringtechniquesorengageininteractivedialogueswithaninternalmodeltoproduceresponseswithvariousreasoningpatterns.Afteraccumulatingtensofhigh-qualitycold-startsamples,wecantrainareasoningmodelSeed-Verifier.Whilethisworkflowisprimarilyappliedtomathematicaldata,weobserveitcangeneralizewelltootherdomains,suchascoding,logicpuzzleandevencreativewriting.Thus,forotherdomains,wealsoconductacoldstartprocessfollowedbyrejectionsamplingtoproducedetailedreasoningtrajectories.Duringtraining,eachinstanceistruncatedto32,000tokens.Wefine-tunethebasemodelfortwoepochsto2×10-6gradually.4.2ReinforcementLearningofdomains.Thisintegrationincorporatesthreedatacategories:•Verifiabledata,whichobtainsfeedbackfromaverifier.Thistypeofdataallowsfordirectvalidationofthemodel’soutputsagainstknowncriteria.•Aspecificclassofdatathatcombinesscoresfromboththeverifierandtherewardmodel.Thishybriddatatypeleveragesthestrengthsofbothverificationandreward-basedevaluation.7•Value-Pretraining:Wesampleresponsesfromafixedpolicy,suchasπsft,andupdatethevaluemodelusingtheMonte-Carloreturn.Thisprocessensuresthattheinitializedvaluemodelisfullyalignedwithourpolicyπsft.Maintainingthisalignmenthasbeenproventobecrucialforpreservingthemodel’sCoTpattern,enablingthemodeltogeneratecoherentandlogicalCoT.•Decoupled-GAE:ByemployingdifferentGeneralizedAdvantageEstimation(GAE)parameters,suchasλvalue=1.0andλpolicy=0.95,weallowthevaluemodeltoupdateinanunbiasedmanner.Meanwhile,thepolicycanindependentlybalanceitsownbiandstabletrainingofthemodel.•Length-adaptiveGAE:Wesetλpolicy=1-length.ThisapproachensuresamoreuniformdistributionofTemporalDifference(TD)errorsacrossbothshortandlongsequences.Asaresult,themodelcanhandlesequencesofvaryinglengthsmoreeffectivelyduringtraining.•DynamicSampling:Weemploydynamicsamplingandfilteroutpromptswithaccuracyscoresequalto1or0,retainingonlythoseinthebatchthatexhibiteffectivegradients.Thisprocesshelpspreventthedampeningofgradientsignalsduringmodeltraining.•Clip-Higher:IntheProximalPolicyOptimization(PPO)algorithm,wedecoupletheupperandlowerLCLIP(θ)=t[min(rt(θ)t,clip(rt(θ),1-ϵlow,1+ϵhigh)t)](1)Byincreasingthevalueofϵhigh,wecreatemoreroomfortheincreaseoflow-probabilitytokens.Thisencouragesthemodeltoexploreawiderrangeofpossibleresponses,enhancingitsabilitytodiscover•Token-levelLoss:Insteadofdefiningthepolicylossoverentireresponses,wedefineitoveralltokens.Thisapproachaddressestheimbalanceinthetoken-levelcontributiontothefinalloss,ensuringthateachtoken’simpactonthetrainingprocessisappropriatelyaccountedfor.•PositiveExampleLMLoss:ThislossfunctionisdesignedtoboosttheutilizationefficiencyofpositiveL(θ)=LPPO(θ)+µ*LNLL(θ)(2)Thisadditionallosstermhelpsthemodeltobetterlearnfrompositiveexamples,improvingitsoverallperformance.ofinterferencebetweendifferentdatadomains.Thisinterferencecanarisefromdisparitiesindifficultylevels,theriskofreward-hacking,andotherunderlyingfactors.TheseissuesmakeitextremelydifficulttoachieveuniformandsimultaneousimprovementsacrOnlineDataDistributionAdaptation.Thismethodtransformsthestationarypromptdistributionduringreinforcementlearningintoanadaptivedistributionthatbettercaterstothemodel’srequirementsduringtraining.Bydoingso,weminimizethenegativeimpactofdatainterferenceandensureamorebalancedimprovementacrossdifferentabilities.Asaresult,themodelcanenhanceitsperforman5Infrastructures5.1FrameworkThetrainingframeworkisbuiltusingHybridFlow[11]programmingabstraction.ThewholetrainingworkloadrunsontopofaRay[12]cluster.ThedataloaderandRLalgorithmisimplementedinasingleprocessRayActor(singlecontroller).Themodeltrainingandresponsegeneration(rollout)isimplementedinaRay8WorkerGroup.TheRayWorkerGroupexposesasetofAPIs(e.g.,generate_response/train_batch,etc,.),whichrunsheavytraining/generationworkloadviaSPMD(singleprogram,multipledata)insidetheWorkerGroup.ThesinglecontrollerinvokesvariousAPIsexposedbytheRayWorkerGrouptoconstructthetrainingflow.HybridFlowprogrammingabstractionenablesfastprototypingofRLalgorithmideaswithoutbotheringwithcomplexdistributedsystems.Seed1.5-Thinkingistrainedthroughhybridenginearchitecture[13],whereallthemodelsareco-located.ThispreventstheidletimeoftheGPUswhenswitchingbetweentrainingandgeneration.DuringLong-CoTgeneration,weobserveseverestragglerphenomenoncausedbythelargedifferenceoftherbetweenvariousprompts.ThiscausesmassiveGPUidletimeduringgeneration.Tomitigatethestraggleroflong-tailresponsegeneration,weproposeSRS(Streamiframeworkthatstrategicallydeploysstandalonestreaming-computeunitstotransformsystemconstraints5.2StreamingRolloutSystemTheSRSarchitectureintroducesstreamingrollouttodecouplemodelevolutionfromruntimeexecution,enablingdynamicadjustmentofon/off-policysampleratiosthroughparametricα:•Allocatetheremainingnon-completesegment(1-α)tooff-policyrolloutsfromversionedmodelsnapshots,seamlesslyintegratedthroughasynchronouscontinuationofpartialgenerationsonthestandaloneInaddition,wealsoimplementdynamicprecisionschedulingduringenvironmentinteractionphases,whichdeploysFP8policynetworksviapost-trainingquantizationwitherror-compenstokenimbalanceinMoEsystems,weimplementathree-tieredparallelarchitecturecombiningTP(tensorparallelism)forlayer-wisecomputation,EP(expertparallelism)withdynamicexpertassignment,andSP(sequenceparallelism)forcontextchunking.Ourkernelauto-tunerdynamicallyselectsoptimalCUDAkernelconfigurationsbasedonreal-timeloadmonitoring.5.3TrainingSystemToefficientlytraintheSeed1.5-Thinkingmodelatscale,wedesignahybriddistributedtrainingframeworkthatintegratesadvancedparallelismstrategies,dynamicworkloadbalancing,andmemoryoptimizations.Belowwedetailthecoretechnicalinnovationsdrivingthesystem’sefficiencyandscalability.•Parallelismmechanisms.WecomposeTP(tensorparallelism)/EP(expertparallelism)/CP(contextparallelism)withFullyShardedDataParallelism(FSDP)totrainSeed1.5-Thinking.Specifically,weappliedTP/CPforattentionlayers,andEPforMoElayers.•Sequencelengthbalancing.TheeffectivesequencelengthcanbeimbalancedacrossDPranks,leadingKARP[14]algorithmthatrearrangestheinputsequenceswithinonemini-batchtomakethembalance•Memoryoptimization.Weadoptlayer-wiserecomputation[15],activationoffloadandoptimizeroffloadtosupporttrainingoflargermicro-batchestooverlapthecommunicationoverheadcausedbyFSDP.•Autoparallelism.Toenableoptimalsystemperformance,wedevelopanautomatictuningsystem,referredtoasAutoTuner.Specifically,AutoTunermodelsthememoryusagefollowingaprofile-basedsolution[16].Then,itestimatestheperformanceandmemoryusagtheoptimalconfiguration.•Checkpoint.WeemployByteCheckpoint[17]tosupportcheckpointresumefromdifferentdistributedconfigurationswithminimaloverhead.Thisenablesuserstoelasticallytrainthetaskstoimprovecluster9BenchmarkSeed1.5-ThinkingDeepSeekR1OpenAIo3-miniGrok3BetaGemini2.5proMathematicsAIME2025AIME2024BeyondAIME74.0%65.0%86.5%77.3%86.7%79.8%87.3%83.9%48.0%42.4%63.6%-86.7%92.0%58.8%ScienceGPQAdiamondSuperGPQAMMLU-PRO77.3%71.5%79.7%80.2%62.1%60.5%52.2%62.8%87.0%85.6%82.4%84.6%84.0%65.3%86.3%CodeCodeforcesavg@8LiveCodeBenchv5AiderPolyglot36.3%32.0%50.9%-55.0%45.0%67.5%-64.9%64.3%74.1%70.6%54.2%56.9%68.6%-40.3%56.3%70.4%74.0%AgenticCodingSWE-benchverifiedSWE-benchverified*47.0%49.2%49.3%-47.0%46.2%44.5%-63.8%63.8%LogicreasoningARC-AGI39.9%18.3%25.8%31.9%27.6%FactualitySimpleQA12.9%30.1%13.8%43.6%52.9%InstructionCollieIFEval73.1%34.2%87.6%33.6%87.4%86.1%93.7%83.4%62.5%91.5%Table2ResultsofState-of-the-ArtReasoningModels*Resultsfromourinternalsandbox,whichmaydifferfromthereportedresultsduetoinconsistenciesinthetestingenvironment.*6ExperimentResults6.1AutoEvaluationResultsInmathematicalreasoning,Seed1.5-Thinkingachievestop-tierperformanceontheAIME2024benchmark,scoring86.7,matchingtheperformanceofOpenAI’so3-mini-highmodel.However,2.5Probutstilltrailsbehindo3-mini-high.Notably,Seed1.5-ThinkingdemonstrateslessimpressiveresultsonSimpleQA.Itisworthemphasizingthatthisbenchmarkprimarilyfunctionsasamemory-orientedmetric,whereperformanceismorestronglycorrelatedwithpre-trainedmodelscaleratherthangenuinereasoningcapabilities.6.2HumanEvaluationResultsToevaluatemodelperformanceonsubjectivetasks,whereautomatedmetricsareinsufficienttocaptureandadherencetohuman-centricpreferences,withapanelofdomain-expertevaluatorsratingmodeloutputsagainstDeepseekR1underpredefinedrubrics.Weusea5-pointordinalscale,rangingfrom0(verypoor)to4(excellent),andevaluatebothmodelsonsessionpromptswithmultiplerounds.Eachfullsessionisannotatedwithabinarywin/lossoutcometocapturetheoveralluserexperienceandasingle0-4scoreisSeed1.5-Thinkingachievesanoverallwinratioof8.0%ontheevaluatedsessions,indicatingsuperiorityincreativewritingtohumanitiesknowledgeelaboration.Figure2showstheper-roundlevelscoredistribution.Figure2RatingDistribution6.3Effectsofpre-trainmodelsRejectionSampling.Rejectionsamplinghasbeenidentifiedasavaluabletechniqueforimprovingmodelperformance[2].WeperformanablationtoexaminewhetherinitializingR