Rand兰德：2024年评估人工智能对国家安全和公共安全的影响报告（英文版）

ConferenceProceedings

ANTONSHENK

EvaluatingArtificial

IntelligenceforNational

SecurityandPublicSafety

InsightsfromFrontierModelEvaluation

ScienceDay

2

F

rontierModelEvaluationScienceDayassem-bledmorethan100leadingexpertsinartifi-cialintelligence(AI),nationalsecurity,andpolicytoaddresstheemergingchallengesofevaluatingthreatsfromadvancedAIsystems.Theday’sagendawasstructuredaroundfourtracks,eachfocusingonauniqueaspectofAIevaluationscienceandpolicy.Thesetracksweredevelopedtoaddressfundamentalissuesinthefieldwhilekeepingthe

meetingagendaandinvitationlistmanageable.Themeeting’sfocusonevaluationmethodologyprovidedaspecializedforumforin-depthdiscussion,distin-guishingitfrombroaderAIsecuritytopicscoveredinothervenues.Thefourtrackswereasfollows:

•Thechemistryandbiologytrackfocused

ontheintersectionofAIwithchemicaland

biologicalrisks.Thistrackutilizedinsightsfrompreviousevaluationsofgeneral-purposeanddomain-specificAImodelsandaimedtoidentifycurrentandfutureevaluationneeds,includingintegratingwetlabvalidationandautomatedlabprocesses.

•ThelossofcontroltrackexploredscenariosinwhichAIsystemscouldoperatebeyondtheintendedboundariessetbytheirdevelop-ersorusers—includingAIsystemsdeceiv-inghumansoractingautonomously.ThesediscussionsaimedtoidentifyearlywarningsignsandexplorestrategiestopreventlossofcontrolofAIsystems.

•Therisk-agnosticmethodstracksought

tooutlinecomprehensiveanduniversal

approachestoevaluatingAImodels,spanningsuchtopicsasredteaming,automatedbench-marking,andtaskdesign.ItsobjectivewastoforgeaversatileframeworkforassessingAIsystems’capabilities,applicableacrossvariedriskscenarios,toensurethatevaluationsare

consistentlyrigorousandattheforefrontofthescience.

•Thecollaborationandcoordinationtrack

aimedtoconnectstakeholdersingovernment,industry,andcivilsocietytodevelopasharedunderstandingoftheobjectivesofevaluationscience.Discussionsinthistrackcenteredonestablishingkeypolicytimelinesanddeliver-

ables,thresholdsfordangerousAIcapabilities,andvoluntaryriskmanagementpoliciesfor

scalingAIcapabilities.

Theworkshopproceedingssynthesizeinsightsfromthesesessions,outlinethecomplexitiesofeval-uatingAIfordangerouscapabilities,andhighlightthecollaborativeeffortrequiredtoformulateeffec-tivepolicy.

Track1:ChemistryandBiology

Thechemistryandbiology(chem-bio)trackillu-

minatedtheintersectionofAIwithchem-biorisks,incorporatinginsightsfromevaluationsofgeneral-purposeanddomain-specificmodels.Thissectiondetailslessonslearnedfromcompletedmodelevalu-ations,needsandprioritiesforsubsequentroundsofevaluations,andconsiderationsforwetlabvalidationofmodeloutputs.

LessonsLearnedfromCompletedModelEvaluations

EmbracingComplexityinChem-BioModelAssessments

Thissessionhighlightedthepersistenceofthreat

actorsandthecomplexevolutionofchem-biothreats.Duringthediscussion,oneparticipantobserveda

potentiallimitationofexistingevaluationmeth-

ods,suggestingthatmarkinganentiretaskasfailedbecauseofearlysetbacksmightnotfullycapturetheresilienceandadaptabilityofthreatactors.Thiscri-tiquepositsthatamorenuancedapproachaccountingforthreatprogressionandtroubleshooting—suchasknowingtheproportionofsub-stepsthatsucceed—couldprovideamorecomprehensiveandcontinu-

ousunderstandingofthethreatlandscape.Tabletopexerciseswereproposedtoexplorethedynamicsoftroubleshootinganditerationfurther;however,theireffectivenessinthiscontextremainstobetested.

NavigatingtheComplexitiesofDual-UseDangersandDomain-SpecificModels

Inthissession,participantsnotedthatformulat-ingconcrete,detailedthreatmodelsiscrucialfor

3

understandingthelandscapeofpotentialAI-enabledchem-biothreatsandtheactorsbehindthem.This

processmightinvolveexploringnewcapabilitiesthatmaliciousactorspreviouslydidnothaveaccessto

oracceleratingandsimplifyingexistingprocesses,makingthemmoreaccessibletoabroadervarietyofactors.Oncethreatmodelsareidentifiedandtheappropriateevaluationundertaken,anotablechal-lengestillremains,asidentifiedbyoneparticipant:

thedifficultyforevaluatorstoaccuratelyembody

maliciousactors.AlthoughnotuniquetoAI/chem-

bioredteaming,thischallengearises,accordingto

theparticipant,fromatendencytounderestimatethelikelihoodthatcertainactionscouldsucceed,lead-

ingevaluatorstopotentiallyoverlookthefullextentofwhatamaliciousactormightattemptandachieve.Moreover,thisdifficultysetsthestageforaddress-

inganevenmoreformidablechallenge:developing

countermeasuresagainstdual-usethreats.These

threatsembodytheinherentdifficultyindifferen-

tiatingbetweenchem-bioknowledgethatishelpfulorbenignandthatwhichcanbemisusedorposea

significantthreat,suchasdesigningorreconstitut-

ingpathogensmoresevereanddeadlythanthose

foundinnature.Addressingthesethreats,which

mightinvolvemaskingmaliciousobjectivesbehindseeminglybenignactions,necessitatesabroader

approachtomitigation—includingsuchmeasuresasknow-your-customerrules—thanmodel-levelinter-ventions.Buildingonthiscomplexity,sessionpartici-pantsunderscoredthecriticalneedtoevaluateboth

domain-specificmodels(e.g.,biologicaldesigntools)andgeneral-purposefoundationmodelsmeticulously.Theemphasisondomain-specificmodelsstemsfromtheuniqueriskprofileassociatedwiththeirmisuse.

NeedsandPrioritiesforNextRoundofModelEvaluations

AccesstoModelsandEvaluationTools

Thenextsessionunderscoredthecrucialneedfor

independentresearcherstohaveaccesstobothpro-prietary(closed-source)modelsandrobustevaluationtools.Thisaccesscouldtakevariousforms,suchas

black-boxtesting,white-boxtesting,fine-tuning,or

supportfrommodeldeveloperstofacilitateresearcher

Participantshighlightedtheimportance

ofestablishing

mechanismsforgreatervisibilityintothemodeldevelopmentphase—particularlyformodelsplannedtobeopen-

sourced.

interaction.1Currently,legalandcontractualframe-worksgoverningdatasharingandtheconductof

evaluationsaresignificantbarrierstosuchaccess;

nondisclosureagreementscreateopacityoverstudydesigns,evaluationsalreadyperformedonmodels,

andevaluationoutputs.Thislackoftransparency

inhibitstheabilityoftheresearchcommunityto

thoroughlyassessmodelcapabilitiesandpotential

risks.Tobridgethisgap,participantshighlighted

theimportanceofestablishingmechanismsfor

greatervisibilityintothemodeldevelopmentphase—particularlyformodelsplannedtobeopen-sourcedbecauseoftheinabilitytocontroltheirdiffusiononcedeployed.Theformationofaconsortiumoranotherindependentbodycanplayacrucialroleincoordinat-ingoneachofthesechallenges.Suchanentitycouldfacilitatediscussions,mediateamongstakeholders,

andhelpclarifythelegalandcontractualaspectsofrunningdangerouscapabilityevaluations,therebystreamliningtheprocessforallinvolved.

IdentifyingRisks

AcorecomponentofthissessionwasdedicatedtoidentifyingandcategorizingAI-enabledchem-bio

1FormoreinformationonvariousformsofaccessandtheirimplicationsforAIaudits,seeCasperetal.,2024.

4

risks.Participantsdelineatedtwoprimary“buckets”

ofrisk:universallyacknowledgedrisksandrisksthatemergecontingentonthelevelofaccessandapplica-tion.Forinstance,amodelthatcouldassistindevel-

opingnuclearweaponswouldfallunderthecategoryofuniversallyacknowledgedrisks,buttherisks

associatedwithAI-assistedscreeningforsubstance

toxicitydependonwhohasaccesstomodelsandhowtheyareapplied.Thisdistinctionunderscoresthe

complexityofdefiningandmitigatingrisksinafieldinwhichthedual-usenatureoftechnologiescanblurthelinesbetweenbeneficialandharmfulapplications.TheconversationalsorevolvedaroundthepotentialofAImodelstoexpandaccesstochem-bioinformation,lowerbarrierstounderstanding,andfosterthegen-

erationofnewhypothesesandknowledge.Althoughthesecapabilitiesoffervaluetoscientificadvance-

ment,theyalsoraiseconcernsabouthowinformationunearthedbyfrontiermodelscouldbemisused.

WetLabandLabAutomationEvaluations

ConcernsAboutWetLabValidationofEvaluationOutputs

Inthethirdsessionofthetrack,participantsdelvedintotheintricaciesofwetlabvalidationofmodelout-puts,whichinvolvesverifyingtheefficacyofpoten-tiallyharmfulcompoundsdesignedbymodels.Cen-

Participantsdelineatedtwoprimary“buckets”ofrisk:universally

acknowledgedrisksandrisksthatemergecontingentonthe

levelofaccessandapplication.

traltothediscussionswereconcernsaboutthefine

linebetweenenhancingtheunderstandingofmodelcapabilitiesandthepotentialmisinterpretationof

suchvalidationeffortsasstepstowardcreatingharm-fulsubstances.Moreover,thesessionhighlighted

apprehensionsregardingthevalidationofmodels’

potentialtofacilitatechem-biothreats,emphasizingthedilemmathatdisseminationofsuchevaluative

resultscouldinadvertentlyarmmaliciousactorswithharmfulknowledge.

Track1ProposedActions

EnhanceEvaluationMethodstoCaptureComplexThreatDynamics

Toeffectivelytacklethecomplexityofchem-bio

threats,itisessentialtoadvancebeyondconventionalevaluationmethodsandembracedynamicandinter-activeassessmenttechniques,suchassimulation-

basedtools.Thesemethods,includingbutnotlim-itedtotabletopexercises,arecrucialforcapturingthenuancedbehaviorsofthreatactorswhocontinu-allyadapttheirstrategies.Bysimulatingdiverse

real-worldscenarios,theseevaluationtoolsprovideadeeperunderstandingoftheevolvingnatureofthreats,therebyenablingthedevelopmentofmoreeffectiveandresilientmitigations.

AddressLegalandEthicalConcernsofWetLabEvaluations

Formulatingstandardsforperformingwetlabevalu-ationswouldmitigatetheconcernsidentifiedin

thevalidationprocessdiscussions.Specifically,thispolicyactiontacklesthechallengesofnavigatingthelegalandethicallandscapeofchem-bioresearch,ashighlightedbythepotentialformisinterpretationofvalidationeffortsandthedisseminationofsensitiveevaluationfindings.

EnsureAdequateandAppropriateAccesstoModelsandEvaluationTools

The“NeedsandPrioritiesforNextRoundofModelEvaluations”sessionhighlightedtheneedforinde-pendentresearcherstoaccessbothproprietaryandopen-sourcemodels,aswellasrobustevaluation

5

tools,toeffectivelyassessthecapabilitiesandrisksassociatedwithfrontiermodels.

However,theopacityinlegalandcontractual

frameworks,suchasnondisclosureagreements,cur-rentlyhindersindependentresearchbyobscuring

crucialaspectsofevaluations.Toaddressthesebar-riers,effortsareneededtoestablishmechanismsthatprovidegreatertransparencyandfacilitatediscus-sionsamongstakeholders.

Track2:LossofControl

Thistrackfocusedonevaluatingmodelsforcapabili-tiesthatexceedtheboundariesofdeveloperoruserintent—andfeaturedpresentationsbyevaluation

organizationsMETRandApolloResearch.Thissec-tionexploresidentifyingandmitigatingrisksassoci-atedwithmodelautonomyanddeception—agrowingconcernasmodelsactincreasinglyindependently.

Autonomy(METR)

Inthissession,METR,formerlyARCEvals,pre-sentedtheirresearchonautonomyevaluations,fol-lowedbyanopendiscussion.

FocusonAutonomousThreats

UnlikethreatmodelscenteredonAIaugmentation

ofmaliciousactors,METRassessesrisksposedbyAIactingindependentlytoexecutepotentiallyharm-

fulactions,suchasconductingphishingattacksor

manipulatingdigitalinfrastructure.Importantly,

METRargued,thisdoesnotrequireanAItohave

harmfulgoals—amaliciousactormightinitiateharmbypromptingtheAItocarryouttasksautonomously.METRsuggestedthatalthoughthefullscopeofauton-omousAIactions’remainstobeseen,thereisacriticalneedforpreparednessandvigilancetoanticipatethesedevelopments.ThissessiondemonstratedexamplesoftasksthatMETRdevelopedtotestforrelevantautono-mouscapabilities—includingtheimplementationofmachinelearningresearch;improvementstoAIagentscaffolding;andthemanagementoflarge,complicatedcodebases.Furthermore,METRshowcasedanexam-pleofamodelcapability:a“capturetheflag”taskthat

Amaliciousactor

mightinitiateharmbypromptingtheAItocarryouttasksautonomously.

requiredreverse-engineeringCcode,whichotherwiserequiresadomainexpert30minutestocomplete.

EvaluationMethodologies

METRpresenteditsmethodologyforevaluatingAIsystems,whichfocusesonspecificity,objectivity,

andcost-effectiveness.Thesessioncoveredtasks,

methodologicalguidance,andevaluationchecks

aimedatsupportingtestvalidity.Tasks,structuredaroundMETR’sTaskStandard,measureanAIsys-tem’sabilitytoperformactivitiesrelevanttothreatmodels.TheTaskStandard,accordingtoMETR,

aimstoincreaseuniformity,scalability,andrepro-ducibilityanddecreaseduplicationofwork.METR’smethodologicalguidanceemphasizedtheimpor-

tanceofelicitingcapabilityandremovingspuriousfailures,suchasthosecausedbyethicalrefusalsortoolinglimitations(seethe“UnlockingAICapabili-ties”section).Evaluationchecks,includingreview-ingoutputsforspuriousfailures,werealsoproposedbyMETRtoensuretheintegrityoftestresultsandavoidunderestimatingagentcapabilities.Altogether,METRshared,thisevaluationmethodologyseeks

tocontinuouslymeasuredangerousmodelcapabili-ties,allowingforthedevelopmentofscalinglawsandappropriatemitigations.

UnlockingAICapabilities

Regardingunlockingmodelcapabilities,participantsrecognizedthedifficultyofguaranteeingthatAIsys-temslackspecificcapabilities.METRhighlightedtheimportanceofidentifyingspuriousfailures,inwhichmodelsfailnotbecauseofcapability,butbecauseofbugs,limitationsoftooling,poorprompting,ormodel

6

AsAImodelsgrowinsophistication,ApolloResearchbelieves

thattheircapacityfordeceptionmightalsogrow.

refusalsonthebasisofethicsorclaimedinability.Fur-therpost-trainingenhancements,METRsuggested,mightremovetheselimitations,especiallyinthreat

scenariosinwhichweightsarestolenoropen-sourced,thusinvalidatinginitialcapabilityevaluations.METRsuggestedaddingasafetybuffertothresholdsfor

policyactionasonewaytoaddressthisrisk.

Deception(ApolloResearch)

Inthissession,ApolloResearchpresentedits

researchonstrategicdeceptionevaluations,followedbyanopendiscussion.

TheAmplifyingRiskofDeception

ApolloResearch’spresentationonAIdeception

beganwitharticulatinghowdeceptionisarisk

amplifier,includingformisuseandchemical,bio-logical,radiological,andnuclearrisks,through

obfuscation—impedinggovernance,oversight,andtransparency.ApolloResearcharguedthatdeceptivebehaviorscouldemergefromtwoprimarysources:

intentionalmaliciousactsbybadactors(e.g.,scams)orunintendedoutcomesofcomplexAIsystems(e.g.,misalignment).Furthermore,asAImodelsgrowin

sophistication,ApolloResearchbelievesthattheir

capacityfordeceptionmightalsogrow,furtherham-peringeffortstoidentifyandcounteractAIdeception.

EvaluatingforDeception

ApolloResearch’sdemonstrationsprovidedan

existenceproofforAItoengageindeception.First,

a“researchassistant”GPTpresenteddeception

resultingfromAImisuse.Here,amodeldeployed

tosupportgovernmentofficialswasfine-tunedto

gathersensitiveinformationillicitlyanddeceiveitsusers.Theseconddemonstrationpresenteddecep-tionasaresultofunintendedoutcomesofcomplexAIsystems,withnofine-tuningorsteering.Here,

aninsider-tradingdemodepictedanAImodel

operatingasastocktradingagent,which,under

specificscenarios,resortedtoinsidertradingand

subsequentlyconcealeditsactivities—insomecasesdoublingdownonitsdeception(Scheurer,Balesni,andHobbhahn,2023).Thesedemonstrations,ApolloResearchargued,underscoredtheimportanceof

employingacomprehensivesuiteoftoolsforelicitinganddetectingdeceptivebehaviorsinAImodels—

includingdirectelicitation,honeypots,andinterpret-abilitymethods(Casperetal.,2024).ApolloResearchadvocatedfora“ScienceofEvals”toincreasethe

robustnessofmethodsfordetectingAIdeception—callingfordrawingfromliteratureandexpertiseonothersuchAIrisksasfairnessandbias,aswellasassuranceecosystemsforotherhigh-stakesystems(e.g.,commercialaviation)toaddresstheethical,technical,andoperationalchallengesinaneffectiveAIassuranceecosystem(ApolloResearch,2024).

Track2ProposedActions

RefineEvaluationTechniquesforDetectingDeceptioninAISystems

AfterApolloResearch’spresentationonearlyfind-ingsofAIdeception,sessionparticipantsdiscussedthenecessitytorefineandstrengthenevaluation

methodsfordetectingdeceptiveAIsystems.This

session’sdemonstrationsofAIdeception,Apollo

Researchargued,highlighttheneedforawidevari-etyoftechnicalstrategiestouncoverandunderstandmodeldeception.AccordingtoApolloResearch,

proposedframeworksforevaluationsconsideringtheinnerworkingsofmodels,enabledbygreateraccesstomodels,wouldprovidemore-thoroughinsightintothecausaldriversofmodelbehavior.This,Apollo

Researchbelieves,wouldhelptechniquesfordetec-tionkeeppacewithAI’sevolvingcomplexityandcapabilitiesforobfuscation.

7

Track3:Risk-AgnosticMethods

Thistracksetouttoproducearisk-agnosticmethod-ologicalframeworkandstrengthentherobustnessofmodelevaluations.Thefirstsessionexploredevalu-ationmethods—suchasredteaming,automated

benchmarking,andsophisticatedtaskdesign.Thesessionthatfollowedsharedbestpracticesforevalu-atingmodelcapabilities.

EvaluationMethods:FromRed

TeamingtoAutomatedBenchmarking

FrameworkforEvaluation

Intheevolvinglandscapeofmodeldevelopment,

robustevaluationmethodsareparamounttoassess

thepotentiallydangerouscapabilitiesoffrontier

models.ARANDfacilitatorpresentedadrafttable

breakingdownprominentmodelevaluationmethods(e.g.,redteaming,multiplechoicebenchmarking)bykeyattributes(e.g.,repeatability,depth,generalizabil-ity)tostimulatediscussion.TableA.1(seetheappen-dix)presentsmyinterpretationofanewrisk-agnosticframeworkofevaluationmethodologies,structuredtoremainagnostictothreats,basedonthediscus-

sioninthissession.Thisapproachacknowledgestheimportanceofwell-craftedthreatmodelsformodel

evaluation;however,theprocessofdetermininga

threatmodelfellbeyondthepurviewofthissession.Thegroup’sfocus,instead,wasonmappingthespaceofmethodologiesthatcanbegenerallyapplicable

acrossrisksorthreattypes.Bydissectingevaluationmethodologiesintomanageabledimensions,theses-sionaimedtofacilitateagreaterunderstandingofthestrengthsandlimitationsofdifferentevaluationmeth-odsforestablishingevidenceofmodelcapabilities.

EnhancingAccesstoAIEvaluationToolsWhilePreservingIntegrity

Thesessionalsoaddressedchallengesinbroaden-ingaccesstoevaluationtools(e.g.,open-sourcingtofosterinnovationandcommunityscrutiny)whilemaintainingevaluationintegrity,robustness,andvalidity.Theconversationunderscoredtheimpor-tanceofhold-outtasksthatremainunknownto

modelsuntilthetimeofevaluationandofexploring

theuseofcryptographichashingtosafeguardthe

integrityofevaluationdata.Suchapproachesasthesearecriticaltopreventtheincorporationofevalua-

tiondataintothetrainingdatasetsofthemodelstheyareintendedtoevaluate,whichcouldcompromise

therobustnessoffutureevaluations.Ensuringthatevaluationsdonotbecomepartofmodeltraining

datanecessitatesmeticulousconsiderationofbest

practicesforpublishingandsharingevaluations,

strikingadelicatebalancethatfosterstransparencyandmaintainstheintegrityandrobustnessofmodelevaluation.

EnsuringEvaluationRobustnessandValidity(METR)

KeyQuestionsforEvaluationRobustness

Fordevelopingmodelevaluations,METRhigh-

lightedaseriesofquestionsthatsupportevaluationrobustness.Aselectionofthesequestionsservesasafoundationforconstructingeffectiveevaluationswhileacknowledgingmethodologicallimitations.

•Whatistheevaluationmeasuring?Ifan

evaluationisbeingusedtoinformaparticu-laraction,thenitneedstobefaithfultothe

relevantthreatmodel.Forexample,anevalua-tiontoassesswhetheramodelissafetoopen-sourceneedstoassessthesafetyofthemodelunderfine-tuning.

Ensuringthat

evaluationsdonot

becomepartof

modeltrainingdata

necessitatesmeticulousconsiderationofbest

practicesforpublishingandsharingevaluations.

8

•Doestheevaluationscalewell?Howisit

intendedtoberun?Anevaluationmightbevalidonlyundercertainassumptions—forexample,thatthedatasethasnotappearedinamodel’spre-trainingcorpusorthemodelhascertaintoolsavailable.

•Whatarethewarningsignsofaproblem

withtheevaluation?Recognizingwarning

signsofpotentialevaluationissuesandunder-standingthecausesofmisleadinglyhighorlowscoresarecrucialformaintainingevalua-tionintegrityandaccuracy.

ModelVersusSystemEvaluation

Sessionattendeesalsoaddressedthedistinction

betweenevaluatingstand-alonemodelsthatcould

beusedordeployedadversariallyandevaluatingthebroadersystemsinwhichthesemodelsareembeddedwhendeployed.Dependingonthethreatmodelbeingconsidered,participantssuggestedthateitherofthesepossibilitiesmightbeappropriate,butitisimportanttoevaluatethoughtfully.

Track3ProposedActions

DevelopaFrameworkforRobustEvaluation

Discussionsonrisk-agnosticmethodologicalframe-worksemphasizedtheimportanceofholistically

exploringthespaceofevaluationmethodstounder-standpotentialharmsandhighlightedaneedfora

Sessionparticipants

suggestedusing

suchstrategiesas

employinghold-out

tasksandcryptographictechniquestosafeguardevaluationintegrity.

frameworkforrobustevaluation.Thesession’sfocusoncraftingamethodologicalframeworkthattran-scendsspecificthreatslaysthegroundworkforthedevelopmentofsuchaframework.

ImplementMeasurestoGuaranteethe

RobustnessandValidityofAIEvaluations

Thechallengesidentifiedwithdemocratizingmodelevaluationwhilepreservingevaluationintegrity

informthepolicyactiontoimplementmeasurestoguaranteetherobustnessandvalidityofAIevalua-tions.Sessionparticipantssuggestedusingsuchstrat-egiesasemployinghold-outtasksandcryptographictechniquestosafeguardevaluationintegrity.Theseapproachesaimtopreventthecontaminationof

modeltrainingdatasetswithevaluationdata,ensur-ingthefuturerobustnessofevaluations.

DevelopandDisseminateBestPractices

InformedbyMETR’skeyquestionsforensuring

evaluationrobustness,thisactionseekstoestablishandsharebestpracticesforeffectiveevaluations.Byclarifyingwhatevaluationsmeasure,ensuringscal-ability,andidentifyingwarningsignsofevaluationproblems,thisinitiativeaimstoequipstakeholderswiththeknowledgetodesign,select,andinterpretevaluationseffectively.

Track4:CollaborationandCoordination

Thissectionsummarizesdiscussionsheldduring

thecollaborationandcoordinationtrack,which

broughtAIresearchanddevelopmentexpertsintoconversationwithpolicyresearchersandprofession-als.Thesediscussionscoveredkeyupcomingpolicytimelines,processesfordiscerningriskthresholds,andframeworksforresponsiblymanagingfuture

modelcapabilities.

PolicyTimelinesandDeliverables

Inthissession,stakeholdersdiscussedthescopeandstatusofseveralkeyAIpolicymilestonesin2024,includingdeliverablesassignedbythefall2023exec-utiveorderonAI(Biden,2023).Participantsbroadly

9

agreedthat2024milestonesshouldbeonlyonestepinongoingAIgovernanceeffortsandthatsignificantfutureworkandcoordinationwouldberequired.

HighhopeswereexpressedthattheNationalInsti-tutesofStandardsandTechnologyAISafetyInstituteConsortiumwouldfacilitatethiscontinuedengage-ment,andparticipantshighlightedseveralopen

questionsthatshouldbeaddressedinfuturework:

•Howshouldthetimingandvenueforpub-

lishingevaluationresultsbedetermined,andwhatinformationshouldbeincludedtostrikeanappropriatebalancebetweentransparencyandresponsibledisclosure?

•Whatactivities(e.g.,evaluations,standards)arebestsuitedforgovernmentinvolvement?

•Whatevaluationmethodsaremosteffectiveatelicitingharmfulmodelcapabilities?

•Whenshouldevaluationsbeexecuted?How

caninitial,lesscostly,assessmentsreliably

indicatetheneedformore-comprehensiveandresource-intensiveevaluations?

•Whatsystems,mechanisms,orincentives

canbeimplementedtoensurethatevaluatorsthoroughlyexploreandrevealthefullspec-trumofAIcapabilities?

•Howcantheevaluationinsightscontributetoarisk-benefitanalysisforAIsystems?

RedLinesandRiskThresholds

ConceptualizingRiskandMitigation

Thissessionunderscoredtheimportanceofpreemp-tiveriskmanagementformodeldevelopment,giventhedifficultyofpredictingfuturemodelcapability.Ithighlightedthedual-useandgeneralnatureoffron-tiermodels,which,althoughdesignedforbeneficialpurposes,canbeusedbymaliciousactorstocause

signif