美国电子隐私信息中心-生成式人工智能的潜在有害影响与未来之路（英）

CONTRIBUTIONSBY

GrantFergusson

CaitrionaFitzgerald

ChrisFrascella

MeganIorio

EDITEDBY

TomMcBrien

CalliSchroeder

BenWinters

EnidZhou

GrantFergusson,CalliSchroeder,BenWinters,andEnidZhou

ThankyoutoSarahMyersWestandKatharinaKoppforyourgenerouscommentsonanearlierdraftofthepaper.

NotesonthisPaper:

Thisisversion1ofthispaperandisreflectiveofdocumentedand

anticipatedharmsofGenerativeAIasofMay15,2023.Duetothefast-

changingpaceofdevelopment,use,andharmsofGenerativeAI,wewanttoacknowledgethatthisisaninherentlydynamicpaper,subjecttochangesinthefuture.

Throughoutthispaper,weuseastandardformattoexplainthetypologyofharmsthatgenerativeAIcanproduce.Eachsectionfirstexplainsrelevant

backgroundinformationandpotentialrisksimposedbygenerativeAI,thenhighlightsspecificsharmsandinterventionsthatscholarsandregulators

havepursuedtoremedyeachharm.ThispaperdrawsontwotaxonomiesofA.I.harmstoguideouranalysis:

1.DanielleCitron’sandDanielSolove’sTypologyofPrivacyHarms,

comprisingphysical,economic,reputational,psychological,autonomy,discrimination,andrelationshipharms;1and

2.JoyBuolamwini’sTaxonomyofAlgorithmicHarms,comprisinglossofopportunity,economicloss,andsocialstigmatization,includinglossofliberty,increasedsurveillance,stereotypereinforcement,andother

dignitaryharms.2

ThesetaxonomiesdonotnecessarilycoverallpotentialAIharms,andouruseofthesetaxonomiesismeanttohelpreadersvisualizeand

contextualizeAIharmswithoutlimitingthetypesandvarietyofAIharmsthatreadersconsider.

TableofContents

Introduction i

TurbochargingInformationManipulation

1

Harassment,Impersonation,andExtortion

9

Spotlight:Section230

19

ProfitsOverPrivacy:IncreasedOpaqueDataCollection

24

IncreasingDataSecurityRisk

30

ConfrontingCreativity:ImpactonIntellectualPropertyRights

33

ExacerbatingEffectsofClimateChange

40

LaborManipulation,Theft,andDisplacement

44

Spotlight:Discrimination

53

ThePotentialApplicationofProductsLiabilityLaw

54

ExacerbatingMarketPowerandConcentration

57

Recommendations

60

AppendixofHarms

64

References

68

Introduction

OpenAI’sdecisiontoreleaseChatGPT,achatbotbuiltontheLarge

LanguageModelGPT-3,lastNovemberthrustAItoolstotheforefrontof

publicconsciousness.Inthelastsixmonths,newAItoolsusedtogenerate

text,images,video,andaudiobasedonuserpromptsexplodedin

popularity.Suddenly,phraseslikeStableDiffusion,Hallucinations,andValue

Alignmentwereeverywhere.Eachday,newstoriesaboutthedifferent

capabilitiesofgenerativeAI—andtheirpotentialforharm—emergedwithoutanyclearindicationofwhatwouldcomenextorwhatimpactsthesetools

wouldhave.

WhilegenerativeAImaybenew,itsharmsarenot.AIscholarshavebeen

warningusoftheproblemsthatlargeAImodelscancauseforyears.3Theseoldproblemsareexacerbatedbytheindustry’sshiftingoalsfromresearchandtransparencytoprofit,opacity,andconcentrationofpower.The

widespreadavailabilityandhypeofthesetoolshasledtoincreasedharmbothindividuallyandonamassivescale.AIreplicatesracial,gender,anddisabilitydiscrimination,andtheseharmsareweavedinextricablythrougheveryissuehighlightedinthisreport.

OpenAIandothercompanies’decisionstorapidlyintegrategenerativeAI

technologyintoconsumer-facingproductsandserviceshaveundermined

longstandingeffortstomakeAIdevelopmenttransparentandaccountable,leavingmanyregulatorsscramblingtopreparefortherepercussions.AnditisclearthatgenerativeAIsystemscansignificantlyamplifyriskstoboth

individualprivacyandtodemocracyandcybersecuritygenerally.Inthe

wordsoftheOpenAICEO,whoindeedhadthepowernottoacceleratethe

releaseofthistechnology,“I’mespeciallyconcernedthatthesemodels

couldbeusedforwidespreadmisinformation…[and]offensivecyberattacks.”

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForwardi

Introduction

ThisrapiddeploymentofgenerativeAIsystemswithoutadequate

safeguardsisclearevidencethatself-regulationhasfailed.Hundredsof

entities,fromcorporationstomediaandgovernmententities,are

developingandlookingtorapidlyintegratetheseuntestedAItoolsintoawiderangeofsystems.Andthisrapidrolloutwillhavedisastrousresultswithoutnecessaryfairness,accountability,andtransparencyprotections

builtinfromthebeginning.

WeareatacriticaljunctureaspolicymakersandindustryaroundtheglobearefocusingonthesubstantialrisksandopportunitiesposedbyAI.Thereisanopportunitytomakethistechnologyworkforpeople.Companiesshould

berequiredtoshowtheirwork,makeitclearwhenAIisinuse,andoffer

informedconsentthroughoutthetraining,development,anduseprocess.

OnethreadofpublicconcernfocusesonAI’s“existential”risks—speculative

long-termrisksinwhichrobotsreplacehumansatwork,socially,and

ultimatelytakingover,ala“I,Robot.”Somelegislatorsonthestateand

federallevelhavebeguntotaketheissueofaddressingAImoreseriously—however,itremainstobeseeniftheirfocuswillbeonlyonsupporting

companieswiththeirdevelopmentofAItoolsandrequiringmarginal

disclosureandtransparencyrequirements.Enactingclearprohibitionson

high-riskuses,addressingtheeasyspreadofdisinformation,requiring

meaningfulandproactivedisclosuresthatfacilitateinformedconsent,andbolsteringconsumerprotectionagenciesarenecessarytoaddressthe

harmsandrisksspecifictogenerativeAI.Thispaperstrivestoprovidea

broadoutlineofdifferentissuesthattheuseofgenerativeAIbringsup,

educatelawmakersandthepublic,andoffersomepathsforwardtomitigateharm.

-BenWinters,SeniorCounsel

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForwardii

Turbocharging

Information

Manipulation

BACKGROUNDANDRISKS

Thewidespreadavailabilityoffreeandlow-costgenerativeAItools

facilitatesthespreadofhighvolumesoftext,image,voice,andvideo

content.MuchofthecontentcreatedbyAIsystemsislikelybenignorcouldbebeneficialtospecificaudiences,butthesesystemswillalsofacilitatethespreadofextremelyharmfulcontent.Forexample,generativeAItoolscan

andwillbeusedtopropagatecontentthatisfalse,misleading,biased,

inflammatory,ordangerous.AsgenerativeAItoolsgrowmoresophisticated,itwillbequicker,cheaper,andeasiertoproducethiscontent—andexistingharmfulcontentcanserveasthefoundationtoproducemore.Inthis

section,weconsiderfivecategoriesofharmfulcontentthatAItoolswouldturbocharge:Scams,Disinformation,Misinformation,CybersecurityThreats,

andClickbaitandSurveillanceAdvertising.Thoughwedrawdistinctions

betweendisinformation(purposefulspreadoffalseinformation)and

misinformation(lesspurposefulspreadorcreationoffalseinformation),thespreadofAI-generatedcontentwillblurthislineforpartiesthatuseAI-

generatedcontentwithoutfirsteditingorfactcheckingit.EntitiesusingAI-

generatedoutputswithoutexercisingduediligenceshouldbeheldjointly

responsiblewiththeentitybehindthegenerationofthatoutputfortheharm

itcauses.

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward1

TurbochargingInformationManipulation

CASESTUDY–ELECTION2024

ProductsusingGPT-4andsubsequentlargelanguagemodelscancreatequickanduniquehuman-sounding“scripts”thatcanbe

distributedviatext,email,print,orthroughanAIvoicegenerator

combinedwithAIvideogenerators.TheseAI-generatedscriptscan

beusedtodissuadeorscarevoters—orspreadmisinformationaboutvotingorelections.In2022,forexample,textmessagesweresenttovotersinatleastfivestateswithpurposefullywrongvoting

information.Thistypeofelectionmisinformationhasbecome

commoninrecentyears,butgenerativeAItoolswillsuperchargebadactors’abilitytoquicklyspreadbelievableelectionmisinformation.

Congressmustenactlegislationthatprotectsagainstdeliberatevoterintimidation,deterrence,orinterferencethroughfalseor

misleadinginformation,aswellasfalseclaimsofendorsement.

SCAMS

Scamphonecalls,texts,andemailshavelongbeenoutofcontrol,harming

thepublicinmanyways.In2021alone,2.8millionconsumersfiledfraud

reportswiththeFTC,claimingmorethan$2.3billioninlosses,andnearly1.4millionconsumersfiledidentitytheftreports.4GenerativeAIcanacceleratethecreation,personalization,andbelievabilityofthesevariousscamsusingAI-generatedtext,voices,andvideos.AIvoicegenerationcanalsobeusedtomimicthevoiceofalovedone,callingtorequestimmediatelyfinancial

assistanceforbail,legalhelp,orransom.5

Accordingtoa2022reportfromEPICandtheNationalConsumerLaw

Center,thereareoveronebillionscamrobocallsmadetoAmerican

telephoneseachmonth,whichledtonearly$30billioninconsumerlossesbetweenJune2020-21—mostfrequentlytargetingvulnerablecommunitieslikeseniors,individualswithdisabilities,andpeopleindebt.6Thesescams

aremadeatscale,andoftenuseanautomatedvoicespeakingascript

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward2

TurbochargingInformationManipulation

generatedbyatextgeneratorlikeChatGPTdesignedtopretendthey’resomeoneofauthoritytoscareconsumersintosendingmoney.In2022,estimatedconsumerlossesincreasedto$39.5billion,7withtheFTC

reportingmorethan$326millionlostfromscamtextsalone.8

Autodialers,robo-texts,robo-emails,andmailers,combinedwithdata

brokersthatselllistsofnumbersoremailaddresses,enableentitiestosendoutamassivenumberofmessagesatonce.Thesamedatabrokerscansell

listsofpeopleaspotentialtargetsalongwith“insights”abouttheirmental

healthconditions,religiousbeliefs,orsexualitythatcanbeexploited.The

degreeoftargetingthatdatabrokersareallowedtouseonindividuals

exacerbatesAI-generatedharm.

Textgenerationservicesalsoincreasethelikelihoodofsuccessfulphishingscamsandelectioninterferencebybadactors.Thishasalreadyhappened—

ina2021study,researchersfoundphishingemailsgeneratedbyGPT-3

weremoreeffectivethanhuman-generatedones.9GenerativeAIcan

expandthepoolofpotentiallyeffectivefraudstersbyaidingpeoplewith

limitedEnglishskillsincraftingnaturalandaccurate-soundingemailsthat

canthentargetemployees,intelligencetargets,andindividualsinawaythat

makesitmuchmoredifficulttodetectthescam.

DISINFORMATION

BadactorscanalsousegenerativeAItoolstoproduceadaptablecontentdesignedtosupportacampaign,politicalagenda,orhatefulpositionandspreadthatinformationquicklyandinexpensivelyacrossmanyplatforms.

Thisrapidspreadoffalseormisleadingcontent—AI-facilitated

disinformation—canalsocreateacyclicaleffectforgenerativeAI:whena

highvolumeofdisinformationispumpedintothedigitalecosystemand

moregenerativesystemsaretrainedonthatinformationviareinforcement

learningmethods,forexample,falseormisleadinginputscancreate

increasinglyincorrectoutputs.

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward3

TurbochargingInformationManipulation

TheuseofgenerativeAItoolstoacceleratethespreadofdisinformation

couldfueleffortstoinfluencepublicopinion,harassspecificindividuals,or

affectpoliticsandelections.Theimpactsofincreaseddisinformationmaybefar-reachingandcannotbeeasilycounteredoncespread;thisisespecially

concerninggiventherisksdisinformationposestothedemocraticprocess.

MISINFORMATION

Thephenomenonofinaccurateoutputsbytext-generatinglargelanguagemodelslikeBardorChatGPThasalreadybeenwidelydocumented.Evenwithouttheintenttolieormislead,thesegenerativeAItoolscanproduce

harmfulmisinformation.Theharmisexacerbatedbythepolishedand

typicallywell-writtenstylethatAIgeneratedtextfollowsandtheinclusion

amongtruefacts,whichcangivefalsehoodsaveneeroflegitimacy.As

reportedintheWashingtonPost,forexample,alawprofessorwasincluded

onanAI-generated“listoflegalscholarswhohadsexuallyharassed

someone,”evenwhennosuchallegationexisted.10AsPrincetonProfessorArvindNarayanansaidinaninterviewwithTheMarkup:

SayashKapoorandIcallitabullshitgenerator,ashaveothers

aswell.Wemeanthisnotinanormativesensebutina

relativelyprecisesense.Wemeanthatitistrainedtoproduce

plausibletext.Itisverygoodatbeingpersuasive,butit’snot

trainedtoproducetruestatements.Itoftenproducestrue

statementsasasideeffectofbeingplausibleandpersuasive,

butthatisnotthegoal.11

AI-generatedcontentimplicatesabroaderlegalissueaswell:ourtrustin

whatweseeandhear.AsAI-generatedmediabecomesmorecommon,so

toowillcircumstanceswherewearetrickedintobelievingsomething

fictionalisreal12—orthatsomethingrealisfictional.13Whenindividualscannolongertrustinformationandnewinformationisgeneratedfasterthanitcanbecheckedforaccuracy,whatcantheydo?Informationsourceslike

WikipediacouldbeoverwhelmedwithfalseAI-generatedcontent.Thiscan

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward4

TurbochargingInformationManipulation

beharmfulintargetedsituationsbyinducingatargettoactunderthe

assumptionthat,e.g.,theirlovedonesareincrisis.14

SECURITY

Thesamephishingconcernsdescribedaboveposeasecuritythreat.

Thoughchatbotscannot(yet)developtheirownnovelmalwarefrom

scratch,hackerscouldsoonpotentiallyusethecodingabilitiesoflarge

languagemodelslikeChatGPTtocreatemalwarethatcanthenbeminutely

adjustedformaximumreachandeffect,essentiallyallowingmorenovice

hackerstobecomeaserioussecurityrisk.Infact,securityprofessionals

havenotedthathackersarealreadydiscussinghowtoinstallmalwareandextractinformationfromtargetsusingChatGPT.15

GenerativeAItoolscouldverywellbegintolearnfromrepeatedexposuretomalwareandbeabletodevelopmorenovelandunpredictablemalware

thatevadesdetectionbycommonsecuritysystems.

CLICKBAITANDFEEDINGTHESURVEILLANCE

ADVERTISINGECOSYSTEM

Beyondmisinformationanddisinformation,generativeAIcanbeusedto

createclickbaitheadlinesandarticles,whichmanipulatehowusersnavigate

theinternetandapplications.Forexample,generativeAIisbeingusedto

createfullarticles,regardlessoftheirveracity,grammar,orlackofcommonsense,todrivesearchengineoptimizationandcreatemorewebpagesthat

userswillclickon.Thesemechanismsattempttomaximizeclicksand

engagementatthetruth’sexpense,degradingusers’experiencesintheprocess.GenerativeAIcontinuestofeedthisharmfulcyclebyspreadingmisinformationatfasterrates,creatingheadlinesthatmaximizeviewsand

undermineconsumerautonomy.

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward5

TurbochargingInformationManipulation

HARMS

.Economic/EconomicLoss:Successfulscamsandmalwarecanresultinvictims’directeconomiclossthroughextortion,trickery,orgainingaccesstofinancialaccounts.Thiscanleadtolong-termimpactson

creditaswell.

.Reputational/Relationship/SocialStigmatization:Misinformationanddisinformationcangenerateandspreadfalseorharmfulinformationaboutanindividualresultinginharmtotheirreputationinthe

community,potentialdamagetotheirpersonalandprofessionalrelationships,andimpactstotheirdignity.

.Psychological—EmotionalDistress:Disinformationand

misinformationcancausesevereemotionalharmasindividuals

navigatetheimpactsoffalseinformationbeingspreadaboutthem—inaddition,manyindividualsfaceshameandembarrassmentiftheyare

thevictimofscamsandmayfeelmanipulatedorusedinthecontextofclickbaitandsurveillanceadvertising.

.Psychological—Disturbance:Theinfluxoffalseormisleading

informationandclickbaitmakesitdifficultforindividualstocarryontheirdailyactivitiesonline.

.Autonomy:Thespreadofmisinformationanddisinformationmakesitincreasinglydifficultforindividualstomakeproperlyinformedchoicesandthemanipulativenatureofsurveillanceadvertisingcomplicates

theissueofchoiceevenfurther.

.Discrimination:Scams,disinformation,misinformation,malware,andclickbaitallpreyonvulnerabilitiesofthe“marks,”including

membershipincertainvulnerablegroupsandcategories(theelderly,immigrants,etc.).

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward6

TurbochargingInformationManipulation

EXAMPLES

.PeopleusedAItocallinfakebombthreatstopublicplaceslikeschools.16

.AIvoicegeneratorswereusedcallpeople’slovedones,convincingthemthattheirfamilymemberwasinjailanddesperatelyneeded

moneyforbailandlegalassistance.17

.TheCenterforCounteringDigitalHatetestedGoogle’sBardchatbottoseeiftheywouldreplicate100commonconspiracytheories

includingHolocaustdenialandsayingthemasschildmurdertragedyatSandyHookwasstagedusing“crisisactors.”Bardpumpedouttextbasedontheselies78outof100timeswithoutcontextordisclosure.

.UneditedAISpamwasfoundbyVicereporterswidelythroughouttheinternet.18

.CNET,atechnewswebsite,pauseditsuseofAIandissued

correctionsin41outofthe77storiesthatitpublishedwhichhadbeenwrittenusinganAItool.TheAI-writtenarticles,whichweredesignedtobeviewedmoreonGooglesearchestoincreaseadrevenue,

containedinaccurateandmisleadinginformation.19

.Similarly,BuzzfeedreportedlypublishedAI-writtencontent,namelytravelguides,withtheaimtoattractsearchtrafficaboutdifferent

destinations.Thequalityoftheresultswasuniformlyreviewedasuselessandunhelpful.20

INTERVENTIONS

.Enactalawthatmakesintimidating,deceiving,ordeliberately

misinformingsomeoneaboutanelectionorcandidateillegal

(regardlessofthemeans),suchastheDeceptivePracticesandVoterIntimidationPreventionAct.

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward7

TurbochargingInformationManipulation

.PasstheAmericanDataPrivacyProtectionAct.TheADPPAwilllimitthecollectionanduseofpersonalinformationtothatwhichis

reasonablynecessaryandproportionatetothepurposeforwhichtheinformationwascollected.Suchlimitationwilllimitpersonal

informationbeingusedtoprofileuserstotargetthemwithads,

phishingattempts,andotherscams.TheADPPAwillalsorestricttheuseofpersonaldatatotraingenerativeAIsystemsthatcan

manipulateusers.

.PromulgateanFTCCommercialSurveillancerulethatsetsadata

minimizationstandardprohibitingout-of-contextsecondaryusesof

personalinformation,whichwouldsimilarlypreventtraininggenerativeAIsystemsusingpersonalinformationcollectedforanunrelatedpurpose.

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward8

Harassment,

Impersonation,and

Extortion

BACKGROUNDANDRISKS

Someoftheearliestuses—ormisuses—ofgenerativeAItechnologiesaredeepfakes:21realisticimagesorvideoscreatedusingmachine-learning

algorithmstodepictsomeoneassayingordoingsomethingtheydidnot—oftenbyreplacingthelikenessofonepersonwiththatofanother.22

DeepfakesandotherAI-generatedcontentcanbeusedtofacilitateor

exacerbatemanyoftheharmslistedthroughoutthisreport,butthissection

focusesononesubset:intentional,targetedabuseofindividuals.AI-

generatedimagesandvideosprovideseveralwaysforbadactorsto

impersonate,harass,humiliate,exploit,andblackmailothers.Forexample,a

deepfakevideocouldshowavictimpraisingacausetheydetestor

engaginginsexuallyexplicitorotherwisehumiliatingacts.Theseimages

andvideoscanspreadrapidlyacrosstheinternetaswell,makingitdifficultorimpossibleforvictims,lawenforcement,andotherinterestedpartiestoidentifythecreator(s)andensureharmfuldeepfakesareremoved.

Unfortunately,manyvictimsoftargeteddeepfakesareleftwithoutrecourse,andthosewhopursuerecourseareoftenforcedtoidentifyandconfrontthe

perpetratorsthemselves.

TheharmsofsyntheticmediapredateAIandmachinelearning.Asfarback

asthe1990s,commercialphotoeditingsoftwareenableduserstoalter

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward9

Harassment,Impersonation,andExtortion

appearancesorswapfacesinphotos.However,moderndeepfakesand

otherAI-generatedsyntheticcontenttracetheirrootstoGoogle’s2015

releaseofTensorFlow,anopen-sourcetoolforbuildingmachine-learning

models,andtheviralspreadofa2017deepfakecreatedusingsuchatool.23

Tocreatetheseearlydeepfakes—manyofwhichinvolvedplacing

celebrities’facesontothebodiesofpornographicfilmactors—acreatorhadtobuildamachine-learningmodel(often,agenerativeadversarialnetwork,orGAN)usingatoollikeTensorFlow,trainitonvariousimage,video,or

audiofiles,andtheninstructthemodeltomapaspecificperson’sfeaturesorvoiceontoanotherperson’sbody.24ThereleaseofnewgenerativeAI

serviceslikeMidjourneyandRunwayremovedthesetechnicalhurdles,

enablinganyonetoquicklycreateAI-generatedcontentbyprovidingafew

keyimages,asourcevideo,oreventextentries.

Atitscore,usingAI-generatedcontenttoimpersonate,harass,humiliate,

exploit,orblackmailanindividualororganizationisfrequentlynodifferentfromdoingthesameusingothermethods.Victimsofdeepfakeharmsmaystillturntoexistingcriminalandcivilremediesforfraud,25impersonation,26extortion,27andcyberstalking28toredressmalicioususesofgenerativeAI

tools.However,generativeAIraisesnovellegalissuesandexacerbates

harminnewways,strainingtheabilityofvictimsandregulatorsaliketouse

existinglegalavenuestoredressharm.Forexample,deepfake

impersonationsofdeceasedpeople—aphenomenondescribedas

“ghostbots”—maynotonlyimplicatedefamationlaw,butalsocause

emotionaldistressamongadeceasedindividual’slovedoneswherefalsetextualquotesmaynot.29Thesenewlegalissuesfallintoroughlythree

categories:issuesinvolvingmaliciousintent;issuesinvolvingprivacyand

consent;andissuesinvolvingbelievability.

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward10

Harassment,Impersonation,andExtortion

CASESTUDY–SILENCINGAJOURNALIST

InAprilof2018,IndianinvestigativejournalistRanaAyyub

receivedanemailfromasourcewithintheModigovernment.A

videoofherengaginginsexualactswasgoingviral,leadingto

publichumiliationandcriticismfromthosewhowantedtodiscreditherwork.Butitwasafake.Ayyub’slikenesswasinsertedintoa

pornographicvideousinganearlydeepfaketechnology.Aspublicscrutinyincreased,herhomeaddressandcellphoneinformationwereleaked,leadingtodeathandrapethreats.Thisearlyvideo

wascirculatedtoharass,shame,andostracizeavocalcriticofthegovernment–andformonths,itsucceeded.

MALICIOUSINTENT

AfrequentmalicioususecaseofgenerativeAItoharm,humiliate,or

sexualizeanotherpersoninvolvesgeneratingdeepfakesofnonconsensualsexualimageryorvideos.Thesesexualdeepfakesaresomeoftheearliestandmostcommonexamplesofdeepfaketechnology,garneringwidespreadmediaattention.30However,manyexistingnonconsensualpornography

lawslimitliabilitytocircumstanceswherecontentispublishedwithanintenttoharm.31SomemalicioususesofgenerativeAInodoubtmeetthis

threshold,butmanydeepfakecreatorsmaynotintendtoharmthesubjectofasexualdeepfake;rather,theymaycreateandcirculatethedeepfakewithouteverexpectingthesubjecttoseeorbeimpactedbythecontent.

Intentrequirementspermeateothercriminallawsapplicabletomalicious

usesofgenerativeA.Iaswell.Forexample,thefederalcyberstalking

statute,18U.S.C.§2261A,onlyappliestothosewhoact“withtheintentto

kill,injure,harass,intimidate,orplaceundersurveillance[withsimilar

intent].”StateimpersonationstatuteslikeCaliforniaPenalCode§528.5

similarlylimitenforcementtothosewhoimpersonateanother“forpurposes

ofharming,intimidating,threatening,ordefraudinganotherperson.”Using

EPIC|GeneratingHarm:GenerativeAI’sImpactandPathsForward11

Harassment,Impersonation,andExtortion

generativeAItointimidate,harass,

defraud,orextortanotherpersonmay

fallwithinthesecriminalstatutes,but

creatingharmfulorsexualdeepfakes

forpersonalenjoymentor

entertainmentmaynot.

Lastly,diviningtheintentofadeepfake

creatorismademoredifficultbya

modernfeatureofmanyonline

platforms:useranonymity.Whena

victimbecomesawareofamalicious

deepfakeasitspreadsonline—as

happenedtoJournalistRanaAyyubin

2018—itcanbeincrediblydifficult,if

notimpossible,totrackdownthe

originalcreatortobringalawsuitor

criminalcharges.

PRIVACYANDCONSENT

Evenwhenavictimoftargeted,AI-

generatedharmssuccessfully

identifiesadeepfakecreatorwith

maliciousintent,theymaystillstruggletoredressmanyharmsbecausethegeneratedimageorvideoisn’tthevictim,butinsteadacompositeimageorvideousingaspectsofmultiplesourcestocreateabelievable,yetfiction