评估未来人工智能的潜在风险、效益和政策（英文）

OECDpublishing

ASSESSINGPOTENTIALFUTUREARTIFICIAL

INTELLIGENCERISKS,BENEFITSANDPOLICYIMPERATIVES

OECDARTIFICIAL

INTELLIGENCEPAPERS

November2024No.27

ECD

BETTERPOLlcIESFORBETTERLIVES

2lASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVES

OECDARTIFICIALINTELLIGENCEPAPERS

Foreword

ThisreportreviewsresearchandexpertperspectivesonpotentialfutureAIbenefits,risksandpolicyactions.ItfeaturescontributionsfrommembersoftheOECDExpertGrouponAIFutures(“ExpertGroup”),whichisjointlysupportedbytheOECDAIandEmergingDigitalTechnologiesdivision(AIEDT)andStrategicForesightUnit(SFU),withregardtowhichitemsshouldbeconsideredhighprioritybypolicymakers.Italsoconsidersexistingpublicpolicyandgovernanceeffortsandremaininggaps.

TheExpertGroupisco-chairedbyStuartRussell(UniversityofCalifornia,Berkeley;CentreforHuman-CompatibleAI),FrancescaRossi(IBM)andMichaelSchönstein(FederalChancelleryofGermany).ThecompletelistofmembersandrelevantoutputsonAIfuturescanbefoundat

https://oecd.ai/site/ai-futures.

Becauseoftheprospectivenatureofpartofthisreportandthelackofrigorousstudyonsometopics,manyofthefuture-orientedaspectsofitscontentsarenecessarilyspeculative.

ThisreportwasdiscussedandreviewedbymembersoftheExpertGroupfromSeptember2023toJuly2024.ItwasalsodiscussedattheOECDWorkingPartyonArtificialIntelligenceGovernance(AIGO)atitsNovember2023meeting.ThispaperwasapprovedanddeclassifiedbywrittenprocedurebytheDigitalPolicyCommitteeon30October2024andpreparedforpublicationbytheOECDSecretariat.

ThisreportcontributestotheOECD’sAIinWork,Innovation,ProductivityandSkills(AI-WIPS)programme,whichprovidespolicymakerswithnewevidenceandanalysistokeepabreastofthefast-evolvingchangesinAIcapabilitiesanddiffusionandtheirimplicationsfortheworldofwork.AI-WIPSissupportedbytheGermanFederalMinistryofLabourandSocialAffairs(BMAS)andwillcomplementtheworkoftheGermanAIObservatoryintheMinistry’sPolicyLabDigital,Work&Society.Formoreinformation,visit

https://oecd.ai/wips

and

https://denkfabrik-bmas.de.

ItalsocontributestotheOECDHorizontalForesightInitiativeonAnticipatingandManagingEmergingGlobalTransformations,whichseekstodeveloppolicyframeworksandriskmanagementapproachestoincreasepreparednessforAI,syntheticbiologyandotherpotentialtransformativedevelopments.Formoreinformation,visit

https://oe.cd/global-transformations.

ThisreportwasdraftedbyJamieBerryhill(AIEDT),HamishHobbsandDexterDocherty(SFU)inclosecollaborationwithExpertGroupco-chairsandmembers.StrategicdirectionandeditingwereprovidedbyKarinePerset,HeadofAIEDT,andRafałKierzenkowski,OECDSeniorCounsellorforStrategicForesight.RiccardoRapparini,RobinStaes-Polet,MichaelaSullivan-Paul,PabloGomezAyerbeandMoritzvonKnebelmadeanalysisanddraftingcontributions.TheteamgratefullyacknowledgestheinputfromExpertGroupmembers,aswellasfromOECDcolleaguesJerrySheehan,AudreyPlonk,Hanna-MariKilpelainen,GalliaDaor,MollyLesher,JeremyWest,LuisAranda,AlistairNolan,SarahBérubéandRashadAbelsonoftheDirectorateforScience,TechnologyandInnovation(STI);UmaKalkarofSFU;RichardMayoftheDirectorateforFinancialandEnterpriseAffairs(DAF);StijnBroeckeoftheDirectorateforEmployment,LabourandSocialAffairs(ELS)andCharlesBaubion,GiuliaCibrario,JamesDrummond,AndrasHlacs,BeckyKing,CraigMatasick,MauricioMejiaandArturoRiveraPerezoftheDirectorateforPublicGovernance(GOV).TheteamalsothanksJohnTarverandAndreiaFurtadoforeditorialsupport.

ASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVESl3

OECDARTIFICIALINTELLIGENCEPAPERS

NotetoDelegations:

ThisdocumentisalsoavailableonO.N.EMembers&Partnersunderthereferencecode:

DSTI/CDEP/AIGO(2023)13/FINAL

Thisdocument,aswellasanydataandmapincludedherein,arewithoutprejudicetothestatusoforsovereigntyoveranyterritory,tothedelimitationofinternationalfrontiersandboundariesandtothenameofanyterritory,cityorarea.

Coverimage:©Kjpargeter/S.©OECD2024

Attribution4.0International(CCBY4.0)

ThisworkismadeavailableundertheCreativeCommonsAttribution4.0Internationallicence.Byusingthiswork,youaccepttobeboundbythetermsofthislicence

(/licenses/by/4.0/).

Attribution-youmustcitethework.

Translations-youmustcitetheoriginalwork,identifychangestotheoriginalandaddthefollowingtext:

Intheeventofanydiscrepancybetweentheoriginalworkandthetranslation,onlythetextoforiginalworkshouldbeconsideredvalid.

Adaptations-youmustcitetheoriginalworkandaddthefollowingtext:ThisisanadaptationofanoriginalworkbytheOECD.TheopinionsexpressedandargumentsemployedinthisadaptationshouldnotbereportedasrepresentingtheofficialviewsoftheOECDorofitsMembercountries.

Third-partymaterial-thelicencedoesnotapplytothird-partymaterialinthework.Ifusingsuchmaterial,youareresponsibleforobtainingpermissionfromthethirdpartyandforanyclaimsofinfringement.

YoumustnotusetheOECDlogo,visualidentityorcoverimagewithoutexpresspermissionorsuggesttheOECDendorsesyouruseofthework.

AnydisputearisingunderthislicenceshallbesettledbyarbitrationinaccordancewiththePermanentCourtofArbitration(PCA)ArbitrationRules2012.TheseatofarbitrationshallbeParis(France).Thenumberofarbitratorsshallbeone.

4lASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVES

OECDARTIFICIALINTELLIGENCEPAPERS

Tableofcontents

Foreword2

Executivesummary6

1IdentifyingdesirableAIfutures8

Governmentsshouldconsiderthemedium-andlong-termimplicationsofAI8

Policyactionstodaycanhelpachievedesirablefuturescenarios8

Governments’AIforesighteffortsareexpanding10

2AI’spotentialfuturebenefits11

TheExpertGroupidentifiedtenpriorityAIbenefitsforenhancedpolicyfocus11

BENEFIT1:Acceleratedscientificprogress11

BENEFIT2:Bettereconomicgrowth,productivitygainsandlivingstandards12

BENEFIT3:Reducedinequalityandpoverty13

BENEFIT4:Betterapproachestourgentandcomplexissues,includingmitigatingclimate

changeandadvancingotherSDGs13

BENEFIT5:Betterdecision-making,sense-makingandforecasting14

BENEFIT6:Improvedinformationproductionanddistribution15

BENEFIT7:Betterhealthcareandeducationservices15

BENEFIT8:Improvedjobquality16

BENEFIT9:Empoweredcitizens,civilsocietyandsocialpartners16

BENEFIT10:Improvedinstitutionaltransparencyandgovernance,instigatingmonitoringand

evaluation17

Policyeffortsrecognisepotentialfuturebenefits,butgapsmayexist17

3PotentialfutureAIrisks19

TheExpertGroupidentifiedtenpriorityAIrisksforenhancedpolicyfocus19

RISK1:Facilitationofincreasinglysophisticatedmaliciouscyberactivity20

RISK2:Manipulation,disinformation,fraudandresultingharmstodemocracyandsocial

cohesion20

RISK3:RacestodevelopanddeployAIsystemscauseharmsduetoalackofsufficient

investmentinAIsafetyandtrustworthiness21

RISK4:UnexpectedharmsresultfrominadequatemethodstoalignAIsystemobjectiveswith

humanstakeholders’preferencesandvalues22

RISK5:Powerisconcentratedinasmallnumberofcompaniesorcountries22

RISK6:MinortoseriousAIincidentsanddisastersoccurincriticalsystems23

RISK7:Invasivesurveillanceandprivacyinfringement24

RISK8:GovernancemechanismsandinstitutionsunabletokeepupwithrapidAIevolutions24

RISK9:AIsystemslackingsufficientexplainabilityandinterpretabilityerodeaccountability25

RISK10:Exacerbatedinequalityorpovertywithinorbetweencountries25

Policyeffortscouldhelpmanagefuturerisks,butsomegapsmayexist26

4Prioritypolicyactions29

TheExpertGroupidentifiedtenprioritypolicyactions29

POLICYACTION1:Establishclearerrules,includingonliability,forAIharms30

POLICYACTION2:Considerapproachestorestrictorpreventcertain“redline”AIuses30

ASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVESl5

OECDARTIFICIALINTELLIGENCEPAPERS

POLICYACTION3:Requireorpromotethedisclosureofkeyinformationaboutsometypesof

AIsystems31

POLICYACTION4:Ensureriskmanagementproceduresarefollowedthroughoutthelifecycle

ofAIsystemsthatmayposeahighrisk32

POLICYACTION5:MitigatecompetitiveracedynamicsinAIdevelopmentanddeploymentthat

couldlimitfaircompetitionandresultinharms33

POLICYACTION6:InvestinresearchonAIsafetyandtrustworthinessapproaches,including

AIalignment,capabilityevaluations,interpretability,explainabilityandtransparency34

POLICYACTION7:Facilitateeducational,retrainingandreskillingopportunitiestohelpaddress

labourmarketdisruptionsandthegrowingneedforAIskills35

POLICYACTION8:Empowerstakeholdersandsocietytohelpbuildtrustandreinforce

democracy36

POLICYACTION9:Mitigateexcessivepowerconcentration36

POLICYACTION10:TargetedactionstoadvancespecificfutureAIbenefits37

References45

Notes66

FIGURES

FigureB.1.Expertsidentifiedandranked21potentialfutureAIbenefits41

FigureB.2.Expertsidentifiedandranked38potentialfutureAIrisks42

FigureB.3.Expertsidentifiedandranked66potentialfutureAIpolicyactions43

6lASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVES

OECDARTIFICIALINTELLIGENCEPAPERS

Executivesummary

TheswiftevolutionofAItechnologiescallsforpolicymakerstoconsiderandproactivelymanageAI-drivenchange.TheOECD’s

ExpertGrouponAIFutures

wasestablishedtohelpmeetthisneedandanticipateAIdevelopmentsandtheirpotentialimpacts.Thisinitiativeaimstoequipgovernmentswithinsightstocraftforward-lookingAIpolicies.ThisreportdiscussesresearchandexpertinsightsonprospectiveAIbenefits,risks,andpolicyimperatives.Whileofferingguidanceforpolicymakers,decision-makersareencouragedtoremainawareofuncertainties,activelyseekdiverseperspectivesandvigilantlymonitorthesocietalimplicationsofAIinnovations.

GovernmentscanshapeAIpoliciestosteerdevelopmentstowarddesirablefutures

TheExpertGroupidentifiedcharacteristicsofdesirableAIfuturesthroughasurvey,discussionsandscenarioexploration.TheseincludewidelydistributedAIbenefits;respectforhumanrights,privacyandintellectualpropertyrights;moreandbetterjobs;resilientphysical,digitalandsocietalsystems;mechanismstomaximiseAIsecurityandpreventmisuse;stepstopreventexcessivepowerconcentration;strongriskmanagementpracticesfortraining,deploymentanduseofAIsystemsthatmaycarryhighrisksandinternationalandmulti-stakeholderco-operationfortrustworthyAI.ThesecharacteristicsembodytherealisationofAI’sbenefitsandmitigatingitsrisks.GovernmentscantakeactiontohelprealisepositiveAIfutures.TheOECDworkedwithExpertGroupmembersthroughthesurveyanddiscussionstoidentifypolicyandgovernancepriorities.AnnexAprovidesdetailsonthemethodologyfordoingso.

FuturebenefitsfromAIincludescientificbreakthroughsandbetterlives…

TheExpertGroupidentified21potentialfutureAIbenefits.Throughrankingandsynthesisofthese,asdiscussedinAnnexA,itputforthtenprioritybenefitsthatwarrantpolicyfocus:

1.acceleratedscientificprogress,suchasthroughdevisingnewmedicaltreatments;

2.bettereconomicgrowth,productivitygainsandlivingstandards;

3.reducedinequalityandpoverty,aidedthroughpovertyreductioneffortsandimprovedagriculture;

4.betterapproachestoaddressurgentandcomplexissues,includingmitigatingclimatechangeandadvancingotherSustainableDevelopmentGoals(SDGs);

5.betterdecision-making,sense-makingandforecastingthroughimprovedanalysisofpresenteventsandfuturepredictions;

6.improvedinformationproductionanddistribution,includingnewformsofdataaccessandsharing;

7.betterhealthcareandeducationservices,suchastailoredhealthinterventionsandtutoring;

8.improvedjobquality,includingbyassigningdangerousorunfulfillingtaskstoAI;

9.empoweredcitizens,civilsocietyandsocialpartners,includingthroughstrengthenedparticipation;

10.improvedinstitutionaltransparencyandgovernance,instigatingmonitoringandevaluation.

…butfuturerisksfromAIincludeharmstoindividualsandsocieties

TheExpertGroupidentified38potentialfutureAIrisks.Throughrankingandsynthesisofthese,itputforthtenpriorityriskswarrantingenhancedpolicyfocus:

ASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVESl7

OECDARTIFICIALINTELLIGENCEPAPERS

1.facilitationofincreasinglysophisticatedmaliciouscyberactivity,includingoncriticalsystems;

2.manipulation,disinformation,fraudandresultingharmstodemocracyandsocialcohesion;

3.racestodevelopanddeployAIsystemscauseharmsduetoalackofsufficientinvestmentinAIsafetyandtrustworthiness;

4.unexpectedharmsresultfrominadequatemethodstoalignAIsystemobjectiveswithhumanstakeholders’preferencesandvalues;

5.powerisconcentratedinasmallnumberofcompaniesorcountries;

6.minortoseriousAIincidentsanddisastersoccurincriticalsystems;

7.invasivesurveillanceandprivacyinfringementthatunderminehumanrightsandfreedoms;

8.governancemechanismsandinstitutionsunabletokeepupwithrapidAIevolutions;

9.AIsystemslackingsufficientexplainabilityandinterpretabilityerodeaccountability;

10.exacerbatedinequalityorpovertywithinorbetweencountries,includingthroughriskstojobs.

Someriskswerenotprioritisedbecausetheywereratedlessimportantoverall,thoughindividualExpertGroupmemberrankingsvariedsignificantly.Opinionsdivergedparticularlyaboutthepotentialriskofhumanslosingcontrolofartificialgeneralintelligence(AGI).Thisisahypotheticalconceptwherebymachinescouldhavehuman-levelorgreater“intelligence”acrossabroadspectrumofcontexts.

ProactivepoliciesandgovernancecanhelptocaptureAI’sbenefitsandmanagerisks

TheExpertGroupidentified66potentialpolicyapproachestoobtainAIbenefitsandmitigaterisks.Throughrankingandsynthesisofthese,itputforthtenpolicyprioritiestohelpachievedesirableAIfutures:

1.establishclearerrules,includingonliability,forAIharmstoremoveuncertaintiesandpromoteadoption;

2.considerapproachestorestrictorpreventcertain“redline”AIuses;

3.requireorpromotethedisclosureofkeyinformationaboutsometypesofAIsystems;

4.ensureriskmanagementproceduresarefollowedthroughoutthelifecycleofAIsystemsthatmayposeahighrisk;

5.mitigatecompetitiveracedynamicsinAIdevelopmentanddeploymentthatcouldlimitfaircompetitionandresultinharms,includingthroughinternationalco-operation;

6.investinresearchonAIsafetyandtrustworthinessapproaches,includingAIalignment,capabilityevaluations,interpretability,explainabilityandtransparency;

7.facilitateeducational,retrainingandreskillingopportunitiestohelpaddresslabourmarketdisruptionsandthegrowingneedforAIskills;

8.empowerstakeholdersandsocietytohelpbuildtrustandreinforcedemocracy;

9.mitigateexcessivepowerconcentration;

10.taketargetedactionstoadvancespecificfutureAIbenefits.

Governmentsrecognisetheimportanceoftheseissues,butmoreneedstobedone

Policyinitiativesrecognisetheimportanceoftheseissues.RecentdevelopmentsincludetherevisionoftheOECDAIPrinciples;finalisationoftheEuropeanUnionAIActandCouncilofEuropeFrameworkConventiononAIandHumanRights,DemocracyandtheRuleofLaw;executiveactionsincountriessuchastheUnitedStates,thelaunchofnationalAIsafetyandresearchinstitutes;commitmentsendorsedbyAIcompanies;effortstoincreaserelevanttalentingovernmentandapplyexistingregulationtothecontextofAI;publicinvestmentsinAIresearchanddevelopmentandinitiativesoftheUnitedNationsanditsagencies.Effortsonthehorizon,suchastheEULiabilityDirective,mayalsoadvancebeneficialAI.Yet,opportunitiesexisttotakemoreconcreteaction.GovernmentsshouldconsiderhowbesttoimplementprioritypolicyactionsandstrengthentheircapacitiestohelpanticipateandshapeAIfutures.

8lASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVES

OECDARTIFICIALINTELLIGENCEPAPERS

1

IdentifyingdesirableAIfutures

Governmentsshouldconsiderthemedium-andlong-termimplicationsofAI

Themediumtolong-termimplicationsofrapidlyadvancingAIsystemsremainlargelyunknownandfiercelydebated.ExpertsraisearangeofpotentialfuturerisksfromAI,someofwhicharealreadybecomingvisible.Atthesametime,expertsandothersexpectAItodeliversignificantorevenrevolutionarybenefits.Future-focusedactivitiescanhelpbetterunderstandAI’spossiblelonger-termimpactsandbeginshapingtheminthepresenttoseizeAI’sbenefitswhilemanagingitsrisks.

Tothisend,the

OECDExpertGrouponAIFutures

(“ExpertGroup”)isamulti-disciplinarygroupof70leadingAIexpertsthathelpsaddressfutureAIchallengesandopportunitiesbyprovidinginsightsintothepossibleAItrajectoriesandimpactsandbyequippinggovernmentswiththeknowledgeandtoolsnecessarytodevelopforward-lookingAIpolicies.1

Policyactionstodaycanhelpachievedesirablefuturescenarios

TheExpertGroup,throughasurvey,discussions,andscenarioexplorationexercises,presenteditsviewsonthecharacteristicsofdesirableAIfuturesinsocietyandgovernance(seemethodologyinAnnexA).ThesedesirablefuturesembodytherealisationofpotentialfutureAIbenefitsandthemitigationofkeyfuturerisks.Positivefutureswillnotoccurautomatically;theydemandconcreteactionbypolicymakers,companies,andotherAIactors.

BenefitsfromAIwouldbewidelydistributed

AIcanacceleratescientificresearchandgeneratesolutionsthatcontributetobreakthroughsinareassuchashealthcareandclimatechange.CertainpoliciescouldenableinnovationintrustworthyAI,ofwhichbenefitswouldbesharedwidelywithinandbetweencountriesandequitablydistributedacrossstakeholdergroups,sectorsandthepublic,whilepreventingsystemdeploymentsoruseswithsubstantialpotentialforharm.Allcountries,includingemerginganddevelopingeconomies,wouldbenefitfromAI’ssocio-economicpotentials.

AIwouldempowerpeople,civilsocietyorganisations(CSOs)andsocialpartners

PeoplewouldbeempoweredthroughAI,suchasthroughnewdata-driventoolstomakemoreinformeddecisions,includingafocusonwomenandmarginalisedcommunities.GovernmentswouldfacilitatethisbyleveragingAItoengagewithcitizensandincorporatetheirviewsintopolicymaking,thusreinforcingdemocracyandparticipationinpubliclife.ThecapabilitiesofCSOsandsocialpartnerssuchastradeunionswouldbestrengthenedbyAI,allowingthemtobetterconnectwithandgatherinsightsfromcitizensandworkers.Throughnewmeanstoanalyseopengovernmentdataandoutputs,AIwouldenableCSOsworldwidetoprovidestrongerindependentoversightofgovernment.ThisoversightrolewouldbefurtherfacilitatedbydisclosurerequirementsornormsforcertainAIsystemsthathelpunderstandtheirfunctioningandfosteranecosystemofindependentevaluators.Intheworkplace,theuseofAIwouldbetrustworthy,anditsbenefitswouldbedistributedfairly,withworkersandsocialpartnersalsoabletoleverageAItobolsterorganisingandinformcollectivebargaining.Thepublicwouldhaveaccesstoreliable,authenticinformation,enhancedandpersonalisededucationandreskillingopportunities.

ASSESSINGPOTENTIALFUTUREAIRISKS,BENEFITSANDPOLICYIMPERATIVESl9

OECDARTIFICIALINTELLIGENCEPAPERS

Humanrights,includingprivacy,wouldberespected

DevelopersanddeployersofAIsystemsandthirdpartiessuchasauditorswouldwidelyusebenchmarks,evaluations,andtechnicaltoolstodetect,mitigate,andcorrectharmfulbiasanddiscrimination.FrameworksandpracticestoensurethatAIsystemsaredesigned,developed,deployed,andusedinaccordancewithhumanrightswouldbeavailableandwidelyadopted.Policiesandsolutionstoprotectpersonaldatawouldbeinplace,especiallyforusecasesthatmaycarryhighriskandsystemsthatmayimpactvulnerablepopulations.

Intellectualpropertyrightswouldberespectedandclarifiedifneeded

Modeldeveloperswouldhaveclearguidanceonwhichdatacanbeusedtotrainmodelsandwhichdata

areprotectedbycopyright.Rightsholdersandothercontentgeneratorswouldbeempoweredtomakeeducateddecisionsabouthowtheirdataandcontentareused.

Robusttechnical,proceduralandeducationaltoolswouldhelpkeepAIsystemstransparent,explainableandalignedwithhumanstakeholders’values

AIactors—thoseactivelyparticipatingintheAIsystemlifecycle,includingorganisationsandindividualsthatdeployoroperateAI—couldleveragerobustprocedures,technicalapproaches,andothermethodstoprovidestrongassurancethatAIsystemsaresafeandtrustworthy.Thiswouldincludeensuringappropriatetransparencyandexplainabilityandaligningsystembehaviourswiththevaluesofhumanstakeholders.

Physical,digitalandsocietalsystemsandecosystemswouldberesilient

TechnicaltoolsandotherprotectivemeasuresagainstAI-facilitatedmaliciouscyberactivitywouldbedevelopedandavailabletoAIdevelopersanddeployers.Criticalinfrastructure,physicalandinformationsecurityrequirementswouldbeadaptedtoreflectrisksposedbytheuseofAI.Tohelpensuretheresilienceofsocietalsystems,governmentinitiativeswouldhelpthetransitionoflabourmarkets,includingreskillingeffortsandconsideringnewsocialsafetynets.Inaddition,aportfolioofeffortsatinternationalanddomesticlevelswouldreinforcedemocracyandinformationintegrity,includingviaeffectiveprocessesenablingfreeandfairelectionsandmitigatingmassdistributionofdisinformation.

EffectivemechanismsmaximiseAIsecurityandpreventmisusebybadactors

AIsystemswouldbedesigned,deployedandove