人工智能在游戏中的伦理问题 The Ethics of AI in Games

1

arXiv:2305.07392v1[cs.HC]12May2023

TheEthicsofAIinGames

DavidMelhart,JulianTogelius,BenedikteMikkelsen,ChristofferHolmgrd,GeorgiosN.Yannakakis

modl.ai

Copenhagen,Denmark

david@modl.ai,julian@modl.ai,benedikte@modl.ai,christoffer@modl.ai,georgios@modl.ai

Abstract—Videogamesareoneoftherichestandmostpopularformsofhuman-computerinteractionand,hence,theirroleiscriticalforourunderstandingofhumanbehaviourandaffectatalargescale.Asartificialintelligence(AI)toolsaregraduallyadoptedbythegameindustryaseriesofethicalconcernsarise.Suchconcerns,however,havesofarnotbeenextensivelydiscussedinavideogamecontext.MotivatedbythelackofacomprehensivereviewontheethicsofAIasappliedtogames,wesurveythecurrentstateoftheartinthisareaanddiscussethicalconsiderationsofthesesystemsfromtheholisticperspectiveoftheaffectiveloop.Throughthecomponentsofthisloop,westudytheethicalchallengesthatAIfacesinvideogamedevelopment.Elicitationhighlightstheethicalboundariesofartificiallyinducedemotions;sensingshowcasesthetrade-offbetweenprivacyandsafegamingspaces;anddetection,asutilisedduringin-gameadaptation,poseschallengestotransparencyandownership.Thispapercallsforanopendialogueandactionforthegamesoftodayandthevirtualspacesofthefuture.Bysettinganappropriateframeworkweaimtoprotectusersandtoguidedeveloperstowardssaferandbetterexperiencesfortheircustomers.

IndexTerms—artificialintelligence,ethics,videogames,affec-tivecomputing

I.INTRODUCTION

Videogamesarekeytoourunderstandingofhumanbe-haviourduetotheirvastpopularity,themulti-modalwaysplayerscaninteractwiththem,andthevariouswaysgamescanexpressemotionandadapttoaplayer’sstyle.Eventhoughval-uessuchastransparency,trustworthinessandresponsibilityarecoreaspectsofethicalsystemsinotherdomains,videogamespresentuniquechallengesintermsofethics.Darkpatternsin

gamedesign[1],predatorymonetisationstrategies[2],andthe

black-boxnatureofgameshindertransparency[3]andraise

severalethicalconcerns.Theseissuesarefar-reachingfrom

gamedesignanddevelopment[4],[5]tosocietalimpactand

researchethics[6]

.

Inthissurveypaper,weaimtoaddresstheethicalcon-siderationsofgameAItoolsandmethodsthroughthelensofaffectivecomputing.Inparticular,wefocusprimarilyonplayer

modelling[7]asafieldofgameresearchthatconsiderstheag

-

gregation,simulation[8],andunderstandingofgameplayand

userexperienceingames.We,thus,structurethediscussion

ofAIethicsingamesaroundtheaffectivegameloop[9](see

Fig.

1).Theaffectivegameloopdescribestherelationships

betweenemotionexpression,elicitation,detection,prediction,andsubsequentreaction.Itpresentsacomplexgamesystemwhichfacilitatestheseprocessesandadaptstotheuser’semotionalresponse.ThisloopcanassistAIsystemstogen-eratepersonalisedaspectsofgamessuchasagentbehaviour,

Figure1.TheAffectiveGameLoop

[9].Theloopreliesonthegame’s

parameterspacetoelicitanemotionalresponse.ThisresponseissensedbyanAImodelthatdetectschange(s)intheplayer’semotionalstate.Theoutputoftheaffectmodelcanbeusedtoadaptthegamecontentandgenerateanewsetofstimulifortheplayer.

levelsandimages[10]orguideanorchestrationprocess[11],

[12]acrosscreativefacetssuchastext,levelsandvisuals

.Theconceptoftheaffectivegameloophasbeenexplored

thoroughlyinacademia[13]–[17].Meanwhile,theadoption

ofaffect-drivenadaptationsystemsingameshasbeengradualoverthelasttwentyyears;indicativeyetrepresentativeexam-plesincludeFac¸ade(ProceduralArts,2005)—seeFig.

2—and

Nevermind(FlyingMollusk,2016)—seeFig.

3.

Thepaperisstructuredasfollows.Afteranoverviewofrelatedliterature(Section

II),wediscusstheethicaldimen

-sionsofgameAIthroughthephasesoftheaffectivegameloop—foradetailedstructureofoursurveyseeTable

I.

Inparticular,Section

III

coversaspectsofelicitationandhowdarkpatternsareusedtomanipulateandreducetheplayers’emotionalagencyinharmfulorexploitativeways;Section

IV

takesathoroughlookatsensingandissuesrelatedtothetradeoffbetweenprivacyandcontrol,andmaliciousactioningames;Section

V

discussesaffectdetectionandthecomplexitiesoftransparencyinlimitedinformationsystemssuchasgames;andfinallySection

VI

reflectsonquestionsofdataandmodelownershipduringtheaffect-drivenadaptationphase.ThepaperendswithadiscussiononseveralotherissuesrelatedtogameAIethicsincludingAIalgorithmicbiases,computefairness,andin-gametoxicityandviolence.

2

TableI

ASPECTSOFTHEAFFECTIVEGAMELOOPWITHTHEIRASSOCIATEDAIVIRTUES(

INTRODUCEDBYBOSTROMANDYUDKOWSKY[18]);

MAJOR

PITFALLS;ANDPOSITIVEINITIATIVESUPHOLDINGAIVIRTUESANDBENEFITINGEND-USERS.

AffectiveLoop

Section

AIVirtues

[18]

MajorPitfalls

PositiveInitiatives

Elicitation

III

Responsibility,

Auditability

Predatorymonetisation,darkdesignpatterns

[1],

andgeneratingharmfulcontent

Governmentstighteningregulationaroundpredatorymonetisationpractices

[19];

Prolificdesignerstakingacriticalstanceagainstdarkpatterns(e.g.SixtoStartCEOAdrianHon

[20])

Sensing

IV

Transparency,

Incorruptibility

LackoftransparencyindatainferredbyAIsystems

Ubisoft’sMLmodelssensingfortoxicbehaviourinForHonor(2017)

[21]

Detection

V

Transparency,

Auditability,

Predictability

AIsystemsoverfittingtoskewedpopulationsandperpetuatingharmfulhistoricalbiases

Gamestudiossharingdatasetswiththeacademiccommunity(e.g.EAand

Nintendo

[22],Ubisoft

[23]andRiot

Games

[24])

.

Adaptation

VI

Responsibility,

Transparency,

Incorruptibility

Unclearchainofresponisibiliyandownershipofdataandoutputofhuman-AIco-creation

Microsoft’sXboxTransparencyReportsshowingactionstakenincontent

moderation1.

Figure2.InFac¸ade(ProceduralArts,2005),theplayercaninteractwiththegame’sagentsthroughfree-formtext.TheunderlyingAIrespondstotheplayerinputbasedonitssemanticandemotionalcontent.

Figure3.InNevermind(FlyingMollusk,2016),theplayerexploresdream-likehorrorenvironments.Thegamecontentisadjustedbasedontheplayer’semotionalstatebyintroducingmoredangersastheplayer’sstresslevelincreases.

II.RELATEDWORK

ThissectionreviewstheliteratureonethicsresearchandethicalframeworksinAIandgamesresearch.

A.EthicsinArtificialIntelligence

EthicshasbeenaconstantchallengeinthefieldofAIfuelledbyacademicandpracticalinterestintothegovernanceofautonomoussystemsandpublicanxietytowardsdata-driven

black-boxinfrastructures[25],[26].Ethicalframeworkshave

beendevelopedtoaddresstheseanxieties,generallyaimingtoprovideguidelinesforcreatingbeneficial,transparent,andtrustworthyapplications.

OneofthemostpopularethicsframeworksappliedinAIconsistsofthevirtuesofResponsibility,Transparency,Auditability,Incorruptibility,andPredictabilityintroduced

byBostromandYudkowsky[18].The

responsibilityofAIalgorithmsreferstotheirclearoversightonthechainofresponsibilityastheoutputofthealgorithmcanbeattributed

toeitherindividualsororganisations[27].Aswewilldiscuss

inSection

III

and

VI,aclearchainofresponsibilityisoften

lostbetweentheoriginaldata,theinferredmodels,andlarge-scaleensemblearchitecturesusingthird-partyAI.

Transparencyisoneofthemorecomplexethicalvirtues

andcornerstonesofAITrustworthiness[28],[29].Ontheone

hand,itcanrefertoakindofalgorithmictransparencythatpromotesAIdecision-makingprocessesthatareexplainable

[30]andclearlyunderstoodbytheirusers[27].Ontheother

hand,itcanrefertoasystemictransparencyandopennessofAI-poweredapplications;thatislegalaccesstoAIinfrastruc-

turesthemselves[28].AswewillshowinSection

IV,Sec

-tion

V,andSection

VI,theindustryhasatroubledrelationship

withtransparency,withmanycompaniesnotdisclosingtheiruseofplayermodelstogainfurtherinsightsfromuserdata.Whiletransparencyinrelationtodirectdata-collectionisclear,“inferred”informationsuchascomputationalmodelsandtheir

outputismuchlessprotectedbylegalframeworks[31]

.

AuditabilityimpliesthatthecorrectnessoftheoutputofAIsystemsshouldbeverifiablebyathirdparty.AswediscussinSection

III

andSection

V,auditabilityandgeneral

transparencyisaseriousblindspotofthevideogames

industry.Althoughsomeofthisblind-spotcanbeattributed

totheinherentopacityofAIsystems[32],thereisadefinite

limitationraisedbylegalopacityrestrictingaccesstoAI

architectureandtrainingdatabyexternalauditors[33]

.

Incorruptibilitymeansthatthesystemisrobustagainst

3

manipulation.Eventhoughtheobfuscationofdatasets,al-gorithms,andtheiroutputdefinitelyprovidessomelevelofprotection,obfuscationisfundamentallyclashingwiththeprincipleoftransparency.Duetotheirinteractivitygamesareunderconstantsiegebymalicioususers,however,theircorruptionisnotnecessarilyanoutsideforce.AsGebrupointsout,AIbiastendstoexacerbatethesociopoliticalandsocioeconomicdisparitiesinoursocietyastheyperpetuateinherentbiasesofthecreatorsofAImodelsandoursocial

reality[34].AswediscussinSection

IV,whileoneofthe

primarygoalsofappliedAIinthegameindustryistoincreasetherobustnessofsystemsagainstexternalattacks,thereismuchlessdiscussionandtransparencyabouttheinherentbiasintheemployedAIsystems.

Finally,predictabilityreferstoself-consistentAIoutputsandalgorithmicbehaviour.PredictabilityisalessprominentyetimportantaspectofAIethics,whichaimstopushappli-

cationstowardsamorereliableandfairimplementation[27]

.PredictabilitygoesalongwaytowardseliminatingAIbias,whichwedetailinSection

V.

TheaforementionedvirtuesarebeingunderstoodasthecornerstonesofAIethicsand

solidified[3],[35]—insomeshapeorform—inthe

IEEE

EthicallyAlignedDesignGuidelines[36],the

HumaneAI

EthicalFramework[27]andthenewlyemergingconceptof

TrustworthyAI[29],[37],whichalsoplaysafundamentalrole

inthenewEthicalGuidelinesforArtificialIntelligenceofthe

EuropeanUnion[38]

.

Arecentmeta-reviewbyYuetal.

[26]oftheAAAI,

AAMAS,ECAIandIJCAIconferencesmappedoutthefieldofEthicsinAI(EAI)andidentifiedfourmajorareasunderthisdomain.ThefirstcategoryisresearchfocusingonleveragingAItechniquestoexplorequestionsofethicsfacedbyhumans.Thesecondandthirdcategoriesfocusoninternaldecision-makingframeworksforAIagentsactingeitherasindividualunitsorcollectives.Finally,thelastcategoryfocusesonethicsinhuman-computerinteractions.Foracompletereviewofall

theseavenuesofresearchwerefertoYuetal.[26];herewe

focusonlyonthelattercategoryasitisthemostrelevanttothedomainandpurposesinvestigatedinthispaper.Aspositioned

byYuetal.[26],[39]andechoedbythelargerresearch

[3],[35],[40]andpolicymaking[36],[37]communities,

ethicalHCIsystemsshouldconservetheautonomyofhumans,bebeneficialtotheuser,andminimiseunderlyingrisks.Summarisingthissentimentinrelationtoaffectivecomputing,theIEEEEthicallyAlignedDesignGuidelinesexplicitlystate:

“Toensurethatintelligenttechnicalsystemswillbeusedtohelphumanitytothegreatestextentpossibleinallcontexts,autonomousandintelligentsystemsthatparticipateinorfacilitatehumansocietyshouldnotcauseharmbyeitheramplifyingordampening

humanemotionalexperience”[36,page6]

.

Beyondthescopeofemotionalautonomy,however,thereisalsothequestionoftransparencyandautonomyinhuman-AIinteractioningeneral.Rovatsosraisestheissueofthegeneraldistrusttowardsmachinesandwhetheritisethicalfor

anAIsystemtoconcealitself[41].Althoughitcanbeeasy

toconsidertotaltransparencyasthemostethical,theissueis

morecomplex.Anewethicalconundrumemergeswhenweconsiderthatinsomehuman-computerinteractions,alackof

transparencycanimprovetheefficacyofthesystem[42].If

thisistrue,wouldn’ttheperformancedrop—thatwasinducedbyincreasedtransparency—hurttheuserinthelongrun?Wouldinthissituationtotaltransparencytakeawayfromtheuser’sautonomy?Ontheotherhand,couldanopaquesystemevenpresentfairchoicestotheuser?Thesequestions—raised

byRovatsos[41]—presupposeabenevolentsystem.However,

AIisnotalwaysdesignedtobebenevolent.Perhapsthemoststrikingexampleofthisislethalautonomousweaponsystems,whicharedesignedtokillhumanswithoutconsider-

ableoversight[43].Eventhoughreal-lifekillingrobotsmight

seemtoberemovedfromthedomainofgames,pushingamilitaryagendaandaidingbothrecruitmentandresearchhas

neverbeenfarfromvideogames[6].Andaswediscussat

manypointsinthispaperneitherisemotionalexploitationnorpsychologicalmanipulation.

AIresearchersfromthefieldsofcomputerscience,engi-neering,robotics,medicine,games,andmorearecallingforstrongerregulationsonexploitativeandharmfulAIandapush

forbenevolentAIapplications[6],[43],[44].Theirfearsare

notunfoundedascurrentresearchandindustrialapplicationofAIaremorethancapableofexploitingandharminghumans

enmasse,fromsocialengineering[45],throughpsychological

manipulation[2]andexacerbatingexistingsocioeconomic

disparitiestophysicalharm[6],[43]

.

B.AIEthicsinGameResearch

AIethicsingameresearchisafairlyunder-researchedarea.Thehandfulofpapersexistentintheliteraturefocusmainly

onplayermodelling[3],[46],ethicaldevelopmentpractices

[4],[5]andethicalpracticesinresearch[6].Incontrast,more

workhasbeencarriedoutongamesasethicalsystems[47]

andtheoutcomeofresponsibilityofgamedesign[1]

.

Inareviewofthefieldofplayermodelling,Mikkelsen

videdanoverviewofemergingethicalissues[3]

.

Intheiranalysisrelyingontheframeworklaidoutby[18],

Mikkelsenetal.identifiedanumberofareasofconcernfrommonetisationthroughcontentmanagementtodynamicadaptationandprivacy.Mostissuesemergingintheseareasareconnectedtothelackoftransparencyandauditabilityofcomputermodels,especiallyinindustrialsettings.Onesolutiontothelackoftransparencyandinterpretabilityis

offeredbythefieldofexplainableAI[48],[49].Apossible

avenueforadaptingexplainableAIisthroughopenplayer

models[46],whicharebasedonOpenLearnerModelsapplied

togames[50].Openplayermodelsincorporateanexplanatory

moduleintoagivenAIapplicationwhichgivesclearfeedbacktotheuseronthebehaviourandpredictionsofthemodel.Asthemoduleprovidingtransparencyisremovedfromthemainpipelineofthealgorithm,inprincipleopenplayermodelscanbecost-effectivetoimplementinexistingsystemsaswell.Nevertheless,althoughexplainableAIprinciplescanhelpbuildmoretransparentsystemsintheory,thepracticalapplicationofsuchframeworksappearstobechallenging.Black-boxalgorithmssuchasdeeplearningneuralnetworks

4

areverypopularindatascienceduetotheirperformance,andtheyarenotoriouslyhardtoexplainandinterpret—despite

advances[49].Ontheotherhand,effectivewhite-boxsystems

arestillanopenchallengetothefield[48]andmightnot

evenbesustainableordesirablefromabusinessperspectiveasthegamesindustryisknowntotreatdatasets,data-processingpipelines,andAImodelsasstrictly-kepttradesecrets.

BeyondtheconcernsoftransparencythereisanalarmingissueofintentionallyharmfulusageofAImodelsthatexploit

addictionandirresponsiblespendinghabits[3],[5].Despitea

growingconcernagainstaggressiveanddeceptivemonetisa-tiontechniques—oftenaimedatchildren—thereisstillalackoflegalandpracticalframeworksthatarecapableofaddress-

ingsuchissues[2].Kingetal.[2],forinstance,examined13

differentpatentsconnectedtovideogamemonetisationandfoundthatalmostallofthemreliedontheexploitationoftheplayers’datatooptimisethedeliveryandtimingofadsandpurchaseoffers.TheynotethatwiththeexpansionofAImethodsitisexpectedthatsuchsystemswillbecomemoresophisticatedandubiquitousinthefuture,makingtheissueofethicsinplayermodellingmorepressingthanever.

C.StateofAIEthicsinPractice

Althoughethicalframeworkshavebeendevelopedtopro-videguidance,thelackofspecificityoftenleadstoasmallscaleofadoption.Ifwelookatthecoreissueoftrans-parency,whichisalsooftenrequiredfortheassessmentofothercomponentsofAItrustworthiness,wefindthatbothaffectivecomputingandgamesapplicationsarelaggingbehind

[51].Thisistruedespitetheissueoftransparencybeing

proppedupbymorerobustlegalframeworksthanmanyothercomponentsofethicalAI.IntheEuropeanUnion,theGeneral

DataProtectionRegulation(GDPR)[52]ismeanttogive

alegalframeworkandtransparencytodatahandling(alsoinanAIcontext).However,areviewofseriousgames—gamesdevelopedforhealthcare,educational,hiring,orothernon-entertainmentpurposes—foundthattwoyearsaftertheadoptionofGDPR,ithashadlittletonoeffectontheresearch

community[53].Similarly,inarecentexplorationofaffective

computingthroughthelensofGDPRlawsHauselmannfoundthatthefieldfacesseriousissuesintermsoftransparency,

responsibility,andpredictability[54].Hauselmannhighlights

thedelicatenatureofemotionaldataassomethingthatisnotnecessarilyprotectedundercurrentlegalframeworksbutextremelypersonaltotheusers.However,thequestionofemotionaldataisfurthercomplicatedbythefactthatwhileuserbehaviourisrelativelyeasytoobserveandrecord,emo-tionaldataisoftenextractedthroughmeansofperipheralsignalsandmachinelearning.Inthissenseaffectivedata

isinferredandnotobserved[31];asaresult,themajority

ofaffectivecomputingapplicationsappeartobeinherentlyopaque.Asthereshouldbearighttoanaccurateportrayalofpersonaldata,inaccuratepredictorsmightinfringeonthepersonalrightsofusers.Thisishardtoprove,however,asthesemodelsareoftendifficulttoaudit.Thisphenomenonisamplifiedbecausetherearefewerpracticalconcernsforinaccuratemodelsuptoacertaindegree.Oftenevenifauser

isprofiledinaccurately,animperfectpredictioncanstillbe

usedtogreateffectinanadaptivesystem[55].Moreover,

commercialapplicationsoftensafeguardtheirmodelsastradesecretsorcannothandletheconstraintsandoverheadofimplementingethicalsafeguardsonafundamentallevel.

Theaboveexamplesfocuspredominantlyontheresearchcommunity;inthegamesindustry,theproblemoftransparencycanbeevenmoreprevalent.Moreoftenthannot,usersareunawareofthedatacollectedandinferredbyalgorithms.AsKrgerpointsout,datacollectioningamesisgenerallymadeinvisibletotheplayersasitis“wovenintoagame’s

environment”[51].Giventhisopaquenessandablas

atti-tudeofuserstowards—whattheyperceiveas—anonymousplay,itisquestionabletowhichextentregulationssuchastheaforementionedGDPRcouldreasonablybeupheld.A

thoroughreviewoffivecompaniesbyVakkurietal.[35]

revealedthateventhoughdevelopersmightconsiderethicsasanimportantquestion,theyhavelittletonotoolstoaddressitinasystematicmanner.MitigationofethicalrisksinAIsystemsthusbecomeslow-priorityandgenerallyaddressedinapost-andad-hocmanner,ifatall.Reviewsofgame

industryapplicationsthatspanfromplayermodelling[3],

[46],todata-drivengamedevelopment[5],andtoprocedural

contentgeneration[4]revealasimilarpattern.Becausethe

gamesindustryisafast-movingfieldwithgrowingpressureonproducingmoreandmorecontentwiththeadventoflive-servicegames,theapplicationofethicalframeworkstoAIingames—includingbutnotlimitedtoplayermodelling—remainsanafter-thoughtwithoutclearwaystointegratethemitigationofethicalproblemsintoexistingindustrypipelines.

III.ELICITATION–BOUNDARIESOFARTIFICIALLY

INDUCEDEMOTIONS

WestartexaminingtheaffectivegameloopfromtheElicita-tionphase.Doingsowearefacedwiththeethicalboundariesofartificiallyinducedemotions.Althoughinherentlypersonalandsubjective,emotionsdonotenjoylegalprotectiontothe

sameextentasotherpersonaldata[54].Thecoreissueswe

encounterinthisareaareownershipandautonomyoverone’s

ownemotions.Theso-calleddarkdesignpatterns[1]have

beenusedingamestocompelplayers’behaviourthroughaffectivemanipulationandwiththeadventofbigdataanalysisandmachinelearning,thereisapotentialforanewwaveof

darkdesignpatterns[2].Asgamesareoftenmarketedtowards

children,theethicalsideoftheemotionelicitationingames,theiruseandtheirgoalhavetobeconsidered.Importantly,thechallengeofdarkdesignpatternsiscoretogamedesignprinciplesbutnotnecessarilytotheAIalgorithmassociatedwithagame.OneshouldthustakeadiveintotheproblematicethicalaspectsofthegamedesignpriortoexaminingtheroleofAIwithinaparticulargame.

Whileafewyearsagolootboxes—i.e.virtualitemsthatcanberedeemedforotherrandomitemsthatprovidesome

valuetotheplayers[56]—madewaves[57]–[59],thenew

monetisationtechniquesweepingacrosstheindustryisthebattlepassorseasonpasssystem.Unlikepreviousiterationsofpremiumsubscriptions,in-gamecurrencies,downloadable

5

contentpacks,lootboxes,andgatedprogression,thebattlepasssystemdoesnotpromiseanyimmediatetangiblerewardtoplayers.Instead,playersbuyintoaccesstotime-limitedcontentupdates,whichtheystillhavetounlockin-game

withinagiventimeframe[60].Thistypeofmonetisation

reformulatesthevaluepropositionofonlinegamesandshifts

thefocusfromcommoditiestoservices[61].Whiletheloot

boxesofyesteryearweredesignedtooperateonthesame

psychologicalbuttonsasgambling[57],[58],emergingbattle

passsystemsbuildmoreonafeelingofmissingout[60]

andsocietalpressure[62].Inmanymodernonlinegames—

suchasFortnite(EpicGames,2017),Ap