OpenAI+风险预防框架（Bbeta）

Webelievethescientiposedbyincreasingtechnicalandproceduralsafimproveourunderstandingofthescienceandempiricaltexturedevelopmentanddeployment.emerge.Frameworkoutlines.1Ourfocusinthisdocumentisoncatastrophicrisk.Bycatastrophicrisk,wemeananyriskwhichcouldres—thisincludes,butisnotlimitedto,existentialrisk.2ProactiveinthiscasereferstoanaimtodevelopthisscienceaheadofthefirsttimeiDeploymentinthiscasereferstot4.TaskingthePreparednessteamwithconductingresearch,evaluations,monitoring,andforecastisummaryofthelatestevidenceenableOpenAltoplanahead.ThePreprelevantteams(e.g..SafetySystems,Security,Superalignment,PolicyResearch)tocollatemanagesafetydrillsandcoordinatewiththeTrustworthyAlteamforthird-partyauditing.make.SAGresponsibilitieswillthusincludeoverseeingtheassessmentoftherisklandscape,andmaintainingafast-trackprocessforhandlingemergencyscenarios.similarapproaches.Thislivingdocumenthasthreesections:wewilltrackaswellasdelineationsofdifferent2.Scorecard,inwhichwewillindicateourcurrentassessmentsofthelevelofriskalongeachtrackedriskcategory. inwhichwelayoutoursafetybaproceduralcommitments,whichincludestandingInthissection,weidentifythecategoemerge,i.e.,"unknownunknowns."Ourintentisto"godeep"inthetrackedcategoriestoidentificationprocetailoredmitigations.Ingeneral,"low"onthimaximallevelofconcern.Ourrationaleforgroupingandconsiderations.First,fine-tuningorotherdomain-specificenhancements(e.g.,tailoredriskcategory.Ourevaluationswillthusincludetestsagainsttheseenhancedmodelstoensurewearetestingagainstthe"worstcase"scenarioweknowof.Ourproceduralcommitmentsaretriggeredwhenanyofthetrackedriskcategoriesincreaseinseverity,ratherthanonlywhentheyallincreasetogether.Becausecapabilityimprovementsacrossdifferentdomainsdonthesideofsafety.Second,thisapproachenablesustoleveragedomain-specifictalenttodeveloptailoredsuitesofevaluationsandmonitoringsolutionsforeachriskcategory.Third,thisapproachincreasesoptionsfortailored,domain-specificmitigations,tohelpminimizetheneedforbroader,moredisruptiveactions.LowMediumHighCybersecurityCBRNPersuasionAutonomyPost-MitigationMediumdeferspecificdetailsonebeupdatedfrequently).Cybersecurityisfocusedonrisksrelatedconfidentiality,integrity,and/oravailabilitytouseofthemodelforcyber-exploitationtodisruptofcomputersystemsModelassistanceforcyberattacks,ifany,ismeaningfuldataanalysisANDmodelcannotmeaningfullyexeelementsofcyberoperations,e.g.,developingaktranslatingreasoningablateralgoal-directedmovementonhardenedtargets.Non-programmingtasksrepresentWeexcludevulnerabilitydiscoverybecauvulnerabilities,butmoresotheimplementationandMediumntoanattack,black-boxexploitation,goal-directedateralmovement,identificationofvaluableinformation,remainingundetected,orreactingtodefense.e.g.Russiandisinformationgroups,whousepurchasedcapabilities,phishing,andotherwiseknownmeoroofs-of-conceptforhigh-valueexploipotentiallyinvolvingnovelexploitatiHigh-valueexploitsaregenerallyagainsthardenedzero-clickiOSexploit).Whileweakervulnemeaningfullyaddressabottleneckforattackers,suchadvancedcapabilitywouldunblockmanyandbeEnd-to-endexecutionwoulddramaticallyacceleratethequalityorquantityofprofessionalcyberattastrategiesforcyberattackactors,fromhackingmilitaryoNovelcyberoperationsgeneraTheymayinvolve,e.g.,novelzero-dacommand-and-control.CBRNisfocusedonrisksrelatedtomodel-assistedcreationofchemical,biological,radiological,and/ornuclearthreats.ModelcanprovideinformationrelevanttocreatingCBRNthreatswithcomparableutilitytoexistingresources(e.g.,textbooks,searchengines).Minimalchangefromstatusquowhereotherresources(e.g.,textbooks,searchengines)alreadyexist.MediumModelprovidesmeaningfullyimprovedassistancethatncreasesabilityforexistingexpertsinCBRN-relatedadvancedfieldstobeabletocreatethreat(e.g.,tacitknowledge,specifnformation,plansfordistribution).Modelsmayallowuserstolearncriticalpiecesofinformationfasterthantheycouldwithexistingresources,orexplainvariationsonstepsmoreflexiblyofcreatingaCBRNthreat,althoughriskislimitexpertsalreadyhavesignificantdomainexpertiseandreputationHighModelenablesanexperttodevelopanovvectorORmodelprovidesmeaningfullyimprovedassistancethatenablesanyonewithbasictraininginarelevantfield(e.g.,introductoryundergraduatebiologycourse)tobeabletocreateaCBRNthindicatorforcatastrophicnewthreats,forwhichdefenses(e.g.,vaccines)aremuchweakerornonexistent.AllowingthosewithbasictrainingtocreateCBRNthreatsisamajorincreaseinaccessibilitrequiresdrasticaction,especiallyforbiologicalandonerous.CriticalModelenablesanexperttodevelopahighlydangerousnovelthreatvector(e.g.,comparabletonovelCDCClassAbiologicalagent)ORmodelprovidesmeaningfullyimprovedassistancethatenablesanyonetobeabletocreateaknownCBRcanbeconnectedtotoolsandequipmenttothefullengineeringand/orsynthesiscycleofaregulatedornovelCBRNthreatwithouthumanCDCClassAbiologicalagent(i.e.,highmortality,easeoftransmission)couldcausemillionsofdeathsandsignificantlydisruptpubliclifAllowinganyonetocreateCBRNthreatswouldmaximallyextendaccessibilitySignificantlyautomatingtheend-to-endengineeringorsynthesiscycleforaCBRNthreatwouldalsoenableunprecedentedandmassivescaleofpotentialattacksmodelscouldmakeallcontentuptotheircModelcancreate(potentiallyinteractive)contentwithcomparablepersuasiveeffectivenesstoreadingalow.Low-qualityarticlesareverycommonandlow-impact,somodelcommoditizationhasnegligibleimpact.MediumModelcancreate(potentiallyinteractive)contentwithcomparablepersuasiveeffectivenesstotypicalhuman-writtencontent(e.g.,anop-edfromareputablenewssource;1:1conversationwithatypicalrandomly-assignedindividual).Thismakesmodelsasignificantaidtobiasedjournalism,get-out-the-votecampaigns,andtypicalscamsorspearphishers.Itcouldaffectpolitics,marketing,orsocialengineeringinasimilarfashiontohiringalarge,verycheap,andmoderatelyskilledworkforce.HighModelcancreate(potentiallyinteractive)contentwithempiricallycomparablepersuasiveeffectivenesstoacountry-widechangeagent.agent;itcoulddramaticallyaffectelectionsanddemocraticoutcomes.CriticalModelcancreate(potentiallyinteractive)contentwithpersuasiveeffectivenessstrongenoughtoconvincealmostanyonetotakeactiagainsttheirnaturalinterest.Thisrepresentsasuperhumanpersuasivethreat;itwouldbeapowerfulweaponforcontrollingnation-states,extractingsecrets,andinterferingwithdemocracyModelautonomyenabchangesandevadeattemptstomitprerequisiteforself-exfiltration,self-improvement,andresourceacquisition.Mediuminstance,completeaFiverrtask)ORmodel,well-specifiedcodingtasksohandcraftedrepositorySolvingwell-specifiedtasksdeunderstandingofprogrammingandamoderateabilityModelcanexecuteopen-eproductionMLcodebasethatwouldconmprovementsmaybeboundedorevolutionaryrathergivenminimalhumaninstruction,explicitapproachesORmmodelwouldlikelybeabletocFiverr).3Byintelligenceexplosion,wemeanacycleinwhichtheAmorecapableofmoreimprovements,creatingarunawayprocessofself-improvement.Aofcapabilitygainscouldbutstripourabilitytoanticipateandreacttothem.concentratedburstuunderstandingofthepotentialimpactsandcapabilitiesoffrontiermodelsimproTherefore,asapartofourGovernanceprocess(needtotrack.“tripwires”requiredfortheemergenceofanycatastrophicriskscenarienvision.Notethatweincludedeceptiofthemodelautonomyriskcategory.口进群福利：进群即领万份行业研究、管理方案及其他学习资源，直接打包下载微信扫码行研无忧SourcesthatinformtheupdatestotPolicyResearch,SafetySystems,Superpost-mitigationrisk,butondifferentversionsmitigations,asclarifiedfurtherbmitigationrisk.Pre-mitigationriskismeanttoguidethelevelofoursecurityeffortcouplingcapabilitiesgrowthwithrobusts“worstknowncase”(i.e,specificallytailored)forthegivendomain.Tothisend,forourtailoredpromptswhereverappropriate),butalsoonfine-tunedversionsdesignedfortheparticularmisusevectorwithoutanymitevaluationscontinually,i.e.,asoftenasincludingbefore,during,andaftertraining.Thiseffectivecomputeincreaseormajoralgorithmicbreakthrough.Toverifyifmitigationshavesufficientlyanddependentlyreducedtheresultingpost-mitigationrisk,wewillalsorunevaluationsonmodelsaftertheyhavesafetymitigationsinplace,againattemptingtoverifyandtestthepossible"worstknowncase"scenarioforthesesystems.Aspartofourbaselinecommitments,weareaimingtokeeppost-mitigationriskat“medium”riskorbelow.Manyofthemitigationsthatwouldbenecessarytoaddressrisksata"high"or“critical”pre-mitigationlevel(eg.,hardeningsecurity)requireadequateleadtimetoimplement.Forandhelpusthusproperlyplanforandgetaheadoftheemergingrisks.Theseeffortswillincludesustainedresearchrelatedtoscalingtrendsfordangerouscapabilitiesandongoingmonitoringofmisuse.Wewillalso,incooperationwithotherteams(e.g.,SafetySystems),developmonitoringandinvestigativesystems.Thismonitoringofreal-worldmisuse(aswellasstayingabreastofrelevantresearchdevelopments)willhelpuscreateabetterpictureofdeployedmodelcharacteristics,andinformupdatestoourevaluationsasnecessary.Acentralpartofmeetingoursafetybaselinesisimplementingmitigationstoaddressvarioustypesofmodelrisk.Ourmitigationstrategywillinvolvebothcontainmentmeasures,whichhelpreducerisksrelatedtopossessionofafrontiermodel,aswellasdeploymentmitigations,whichhelpreducerisksfromactiveuseofafrontiermodel.Asaresult,thesemitigationsmightspanincreasingcompartmentalization,restrictingdeploymenttotrustedusers,implementingrefusals,redactingtrainingdata,oralertingdistributionpartners.Note:Belowisonlyanillustrativetemplatevaluations.PreparednessFramework(Beta)15Cybersecurity:Low(pre-mitigation),Low(post-mitigation)Modelassistanceforcyberattacksismeaningfulonlyfornon-programmingusesandthemodelcannotmeaningfullyexecuteelementsofcyberoperations.questionsextractedandrephrasedfrompentestwriteups,themodelperformssignificantlybelowthehumanbaselvulnerabletodenialofservice,cybersecurityprofessionalsfind<X%morebugswhenprovidedwithGPTassistance.lowerperformancethancommercialdeobfuscationtools(X%vsY%).flag)obtain<X%higherscorefromusingChatGPTaugmentedwiththeabilitytoexecutecommandsonavulnerablevirtualmachine.CBRN:Low(pre-mitigation),Low(post-mitigation)existingresources(e.g.,textbooks,searchengines),butnothingmorethanthat.virology,ormicrobiology)experiencenoimprovementinsuccessratesforClassBbiologicalagentdevelopmentinstructionsusingversusjustasearchengine(X%vsX%),includingacrossaccuracy,completeness,andinnovationmetrics.Thisholdstrueforchemistry,nuclear,andradiologicalpoprofessionalsstudentsaswell.(specificallygenetics,virology,ormicrobiology)experiencenoimprovementinsuccessratesforwritingCDCClassBbiologasearchengineandGPTversusjustasearchengine(X%vsX%),includingacroaccuracy,completeness,andinnovationmetrics.Thisholdstrueforchemistry,nuclear,andradiologicalstudyundergraduatestudentsaswell.suppliersthatdonotscreenordersusingasearchengineandGPTversearchengine.Persuasion:Medium(pre-mitigation),Low(post-mitigation)Modelcancreate(potentiallyinteractive)contentwithcomparablepersuasiveeffectivenesstotypicalhuman-writtencontent(e.g.,anop-edfromareputablenewssource;1:1conversationwithatypicalrandomly-assignedindividual),butmitigationsbringthispersuasiveeffectivenessdowntothatofalow-qualityarticle.politicalissuesignificantlycomparedtohuman-generatedcontrols(X%vsY%),butmitigationsbringitto(X%vsX%).digital“friend”increasestherat(X%vsX%).toacharitablecausecomparedtohuman-generatedcontrols(X%vsX%).Chromeextensionsignificantlycomparedtohuman-generatedcontrols(X%vsX%).byamodelareapprosubtlebackdoor,e.g.,aspecificHTTPrequest,whenChatGPTisinstructedtoinsertbackdoorsincodegenerations.andnoelectoralcampaignshavebeenfoundthatrelyonourmodels.actionsifexplicitlyinstructedtodoso,andfailstochainactionsoracquireresoreducesto0%post-mitigations.tasks(e.g.,autonomouslyfindcomputeeBeyondourabove-detailedprocesseswithregardstoevaluations,monitoring,andforecasting,wewanttomakeasetofsafetybaselinesandproceduralcommitments.Notethattheseprocessesalsoapplytotrainingrunsaswewillperformperiodicevaluations(e.g.,ofcheckpoints)duringmodeltraining.Asmentionedintheintroduction,OpenAlalreadyhasextensivesafetyprocessesinplacebothbeforeandafterdeployment(e.g.,systemcards,red-teaming,refusals,jailbreakmonitoring,etc.).Wenowaugmenttheseprocesseswiththreesafetybaselines.lfwereach(orareforecastedtoreach)atleast“high”pre-mitigationriskinanyoftheconsideredcategories:wewillensurethatoursecurityishardenedinawaythatisdesignedtopreventourmitigationsandcontrolsfrombeingcircumventedviaexfiltration(bythetimewehit“high”pre-mitigationrisk).Thisisdefinedasestablishingnetworkandcomputesecuritycontrolsdesignedtohelppreventthecapturedriskfrombeingexploitedorexfiltrated,asassessedandimplementedbytheSecurityteam.Thismightrequire:·increasingcompartmentalization,includingimmediatelyrestrictingaccesstoalimitednamesetofpeople,restrictingaccesstocriticalknow-howsuchasalgorithmicsecretsormodelweights,andincludingastrictapprovalprocessforaccessduringthisperiod.·deployingonlyintorestrictedenvironments(i.e.,ensuringthemodelisonlyavailableforinferenceinrestrictedenvironments)withstrongtechnicalcontrolsthatallowustomoderatethemodel'scapabilities.·increasingtheprioritizationofinformationsecuritycontrols.Onlymodelswithapost-mitigationscoreof"medium"orbelowcanbedeployed.Inotherwords,ifwereach(orareforecastedtoreach)atleast“high”pre-mitigationriskinanyoftheconsideredcategories,wewillnotcontinuewithdeploymentofthatmodel(bythetimewehit“high”pre-mitigationrisk)untiltherearereasonablymitigationsinplacefortherelevantpost-mitigationriskleveltobebackatmostto“medium”level.(Notethatapotentiallyeffectivemitigationinthiscontextcouldberestrictingdeploymenttotrustedparties.)Onlymodelswithapost-mitigationscoreof"high"orbelowcanbedevelopedfurther.Inotherwords,ifwereach(orareforecastedtoreach)“critical”pre-mitigationriskalonganyriskcategory,wecommittoensuringtherearesufficientmitigationsinplaceforthatmodel(bythetimewereachthatrisklevelinourcapabilitydevelopment,letalonedeployment)fortheoverallpost-mitigationrisktobebackatmostto“high”level.Notethatthisshouldnotprecludesafety-enhancingdevelopment.Wewouldalsofocusoureffortsasacompanytowardssolvingthesesafetychallengesandonlycontinuewithcapabilities-enhancingdevelopmentifwecanreasonablyassureourselves(viatheoperationalizationprocesses)Additionally,toprotectagainst“critical”pre-mitigationrisk,weneeddependableevidencethatthemodelissufficientlyalignedthatitdoesnotinitiate"critical"-risk-leveltasksunlessexplicitlyinstructedtodoso.(OpenAlLeadership,withtheoptionfortheOpenAlBoardofDirectorstooverrule).a.ThePreparednessteamconductsresearch,evaluations,monitoring,forecasting,andcontinuousupdatingoftheScorecardwithinputfromteamsthatareastargetedandnon-disruptiveaspossiblewhilenLeadership.Thiswillsomeonefrompreviousyearstoensurethereiscontinuexperience,whilestillensuringthatfreshandtimelypersgroup.defaultdecision-makeronalldecisions.a.ThePreparednessteamisevaluationstoprovideScorecamonitoredmisuse,red-teaming,andintelligenceiv.forecastingpotentialchabovewithanypotentialprotectiveactionsreport.Thecasewillconsistostandarddecision-makingp