OWASP LLM人工智能网络安全与治理清单（英文版）

LLMAICybersecurity&GovernanceChecklist

FromtheOWASPTop10forLLMApplicationsTeam

Version:1.0

Published:February19,2024

RevisionHistory

Revision

Date

Author(s)

Description

0.1

2023-11-01

SandyDunn

initialdraft

0.5

2023-12-06

SD,Team

publicdraft

0.9

2023-02-15

SD,Team

pre-releasedraft

1.0

2024-02-19

SD,Team

publicreleasev1.0

Theinformationprovidedinthisdocumentdoesnot,andisnotintendedto,constitutelegaladvice.Allinformationisforgeneralinformationalpurposesonly.

Thisdocumentcontainslinkstootherthird-partywebsites.SuchlinksareonlyforconvenienceandOWASPdoesnotrecommendorendorsethecontentsofthethird-partysites.

1

Overview

5

1.1

ResponsibleandTrustworthyArtificialIntelligence

6

1.2

WhoisThisFor?

7

1.3

WhyaChecklist?

7

1.4

NotComprehensive

7

1.5

LargeLanguageModelChallenges

7

1.6

LLMThreatCategories

8

1.7

ArtificialIntelligenceSecurityandPrivacyTraining

9

1.8

IncorporateLLMSecurityandgovernancewithExisting,EstablishedPracticesandControls9

1.9

FundamentalSecurityPrinciples

9

1.10

Risk

10

1.11

VulnerabilityandMitigationTaxonomy

10

2

DeterminingLLMStrategy

11

2.1

DeploymentStrategy

13

3

Checklist

14

3.1

AdversarialRisk

14

3.2

ThreatModeling

14

3.3

AIAssetInventory

14

3.4

AISecurityandPrivacyTraining

15

3.5

EstablishBusinessCases

15

3.6

Governance

16

3.7

Legal

17

3.8

Regulatory

18

3.9

UsingorImplementingLargeLanguageModelSolutions

19

3.10

Testing,Evaluation,Verification,andValidation(TEVV)

19

3.11

ModelCardsandRiskCards

20

3.12

RAG:LargeLanguageModelOptimization

21

3.13

AIRedTeaming

21

4

Resources

22

A

Team

32

Overview

Everyinternetuserandcompanyshouldpreparefortheupcomingwaveofpowerfulgenerativeartificialintelligence(GenAI)applications.GenAIhasenormouspromiseforinnovation,efficiency,andcommercialsuccessacrossavarietyofindustries.Still,likeanypowerfulearlystagetechnology,itbringsitsownsetofobviousandunexpectedchallenges.

Artificialintelligencehasadvancedgreatlyoverthelast50years,inconspicuouslysupportingavarietyofcorporateprocessesuntilChatGPT’spublicappearancedrovethedevelopmentanduseofLargeLanguageModels(LLMs)amongbothindividualsandenterprises.Initially,thesetechnologieswerelimitedtoacademicstudyortheexecutionofcertain,butvital,activitieswithincorporations,visibleonlytoaselectfew.However,recentadvancesindataavailability,computerpower,GenAIcapabilities,andthereleaseoftoolssuchasLlama2,ElevenLabs,andMidjourneyhaveraisedAIfromanichetogeneralwidespreadacceptance.TheseimprovementshavenotonlymadeGenAItechnologiesmoreaccessible,buttheyhavealsohighlightedthecriticalneedforenterprisestodevelopsolidstrategiesforintegratingandexploitingAIintheiroperations,representingahugestepforwardinhowweusetechnology.

•Artificialintelligence(AI)isabroadtermthatencompassesallfieldsofcomputersciencethatenablemachinestoaccomplishtasksthatwouldnormallyrequirehumanintelligence.MachinelearningandgenerativeAIaretwosubcategoriesofAI.

•MachinelearningisasubsetofAIthatfocusesoncreatingalgorithmsthatcanlearnfromdata.Machinelearningalgorithmsaretrainedonasetofdata,andthentheycanusethatdatatomakepredictionsordecisionsaboutnewdata.

•GenerativeAIisatypeofmachinelearningthatfocusesoncreatingnewdata.

•Alargelanguagemodel(LLM)isatypeofAImodelthatprocessesandgenerateshuman-liketext.Inthecontextofartificialintelligencea"model"referstoasystemthatistrainedtomakepredictionsbasedoninputdata.LLMsarespecificallytrainedonlargedatasetsofnaturallanguageandthenamelargelanguagemodels.

OrganizationsareenteringunchartedterritoryinsecuringandoverseeingGenAIsolutions.TherapidadvancementofGenAIalsoopensdoorsforadversariestoenhancetheirattackstrategies,introducingadualchallengeofdefenseandthreatescalation.

Businessesuseartificialintelligenceinmanyareas,includingHRforrecruiting,emailspamscreening,SIEMforbehavioralanalytics,andmanageddetectionandresponseapplications.However,thisdocument’sprimaryfocusisonLargeLanguageModelapplicationsandtheirfunctionincreatinggeneratedcontent.

ResponsibleandTrustworthyArtificialIntelligence

AschallengesandbenefitsofArtificialIntelligenceemerge-andregulationsandlawsarepassed-theprinciplesandpillarsofresponsibleandtrustworthyAIusageareevolvingfromidealisticobjectsandconcernstoestablishedstandards.The

OWASPAIExchangeWorkingGroup

ismonitoringthesechangesandaddressingthebroaderandmorechallengingconsiderationsforallaspectsofartificialintelligence.

Figure1.1:Imagedepictingthepillarsoftrustworthyartificialintelligence

WhoisThisFor?

TheOWASPTop10forLLMApplicationsCybersecurityandGovernanceChecklistisforleadersacrossexecutive,tech,cybersecurity,privacy,compliance,andlegalareas,DevSecOps,MLSecOps,

andCybersecurityteamsanddefenders.Itisintendedforpeoplewhoarestrivingtostayaheadin

thefast-movingAIworld,aimingnotjusttoleverageAIforcorporatesuccessbutalsotoprotectagainsttherisksofhastyorinsecureAIimplementations.Theseleadersandteamsmustcreatetacticstograbopportunities,combatchallenges,andmitigaterisks.

ThischecklistisintendedtohelpthesetechnologyandbusinessleadersquicklyunderstandtherisksandbenefitsofusingLLM,allowingthemtofocusondevelopingacomprehensivelistofcriticalareasandtasksneededtodefendandprotecttheorganizationastheydevelopaLargeLanguageModelstrategy.

ItisthehopeoftheOWASPTop10fortheLLMApplicationsteamthatthislistwillhelporganizationsimprovetheirexistingdefensivetechniquesanddeveloptechniquestoaddressthenewthreatsthatcomefromusingthisexcitingtechnology.

WhyaChecklist?

Checklistsusedtoformulatestrategiesimproveaccuracy,defineobjectives,preserveuniformity,andpromotefocuseddeliberatework,reducingoversightsandmisseddetails.Followingachecklistnotonlyincreasestrustinasafeadoptionjourney,butalsoencouragesfutureorganizationsinnovationsbyprovidingasimpleandeffectivestrategyforcontinuousimprovement.

NotComprehensive

AlthoughthisdocumentintendstosupportorganizationsindevelopinganinitialLLMstrategyinarapidlychangingtechnical,legal,andregulatoryenvironment,itisnotexhaustiveanddoesnotcovereveryusecaseorobligation.WhileusingthisdocumentisOrganizationsshouldextendassessmentsandpracticesbeyondthescopeoftheprovidedchecklistasrequiredfortheirusecaseorjurisdiction.

LargeLanguageModelChallenges

LargeLanguagemodelsfaceseveralseriousanduniqueissues.OneofthemostimportantisthatwhileworkingwithLLMs,thecontrolanddataplanescannotbestrictlyisolatedorseparable.AnothersignificantchallengeisthatLLMsarenondeterministicbydesign,yieldingadifferentoutcomewhenpromptedorrequested.LLMsemploysemanticsearchratherthankeywordsearch.Thekeydistinctionbetweenthetwoisthatthemodel’salgorithmprioritizesthetermsinitsresponse.Thisisasignificantdeparturefromhowconsumershavepreviouslyusedtechnology,andithasanimpactontheconsistencyandreliabilityofthefindings.Hallucinations,emergingfromthegapsandtrainingflawsinthedatathemodelistrainedon,aretheresultofthismethod.

Therearemethodstoimprovereliabilityandreducetheattacksurfaceforjailbreaking,modeltricking,andhallucinations,butthereisatrade-offbetweenrestrictionsandutilityinbothcostandfunctionality.

LLMuseandLLMapplicationsincreaseanorganization’sattacksurface.Somerisksassociated

withLLMsareunique,butmanyarefamiliarissues,suchastheknownsoftwarebillofmaterials(SBoM),supplychain,datalossprotection(DLP),andauthorizedaccess.TherearealsoincreasedrisksnotdirectlyrelatedtoGenAI,butGenAIincreasestheefficiency,capability,andeffectivenessofattackerswhoattackandthreatenorganizations.

AdversariesareincreasinglyharnessingLLMandGenerativeAItoolstorefineandexpeditetraditional

methodsofattackingorganizations,individuals,andgovernmentsystems.LLMfacilitatestheirabilitytoenhancetechniquesallowingthemtoeffortlesslycraftnewmalware,potentiallyembeddedwithnovelzero-dayvulnerabilitiesordesignedtoevadedetection.Theycanalsogeneratesophisticated,unique,ortailoredphishingschemes.Thecreationofconvincingdeepfakes,whethervideooraudio,furtherpromotestheirsocialengineeringploys.Additionally,thesetoolsenablethemtoexecuteintrusionsanddevelopinnovativehackingcapabilities.Inthefuture,more“tailored”andcompounduseofAItechnologybycriminalactorswilldemandspecificresponsesanddedicatedsolutionsfor

anorganization’sappropriatedefenseandresiliencecapabilities.

OrganizationsalsofacethethreatofNOTutilizingthecapabilitiesofLLMssuchasacompetitivedisadvantage,marketperceptionbycustomersandpartnersofbeingoutdated,inabilitytoscalepersonalizedcommunications,innovationstagnation,operationalinefficiencies,thehigherriskofhumanerrorinprocesses,andinefficientallocationofhumanresources.

UnderstandingthedifferentkindsofthreatsandintegratingthemwiththebusinessstrategywillhelpweighboththeprosandconsofusingLargeLanguageModels(LLMs)againstnotusingthem,makingsuretheyaccelerateratherthanhinderthebusiness’smeetingbusinessobjectives.

LLMThreatCategories

Figure1.2:ImagedepictingthetypesofAIthreats

ArtificialIntelligenceSecurityandPrivacyTraining

Employeesthroughoutorganizationsbenefitfromtrainingtounderstandartificialintelligence,generativeartificialintelligence,andthefuturepotentialconsequencesofbuilding,buying,orutilizingLLMs.Trainingforpermissibleuseandsecurityawarenessshouldtargetallemployeesaswellasbemorespecializedforcertainpositionssuchashumanresources,legal,developers,datateams,andsecurityteams.

Fairusepoliciesandhealthyinteractionarekeyaspectsthat,ifincorporatedfromtheverystart,willbeacornerstonetothesuccessoffutureAIcybersecurityawarenesscampaigns.Thiswillnecessarilyprovideuserswithknowledgeofthebasicrulesforinteractionaswellastheabilitytoseparategoodbehaviorfrombadorunethicalbehavior.

IncorporateLLMSecurityandgovernancewithExisting,EstablishedPracticesandControls

WhileAIandgeneratedAIaddanewdimensiontocybersecurity,resilience,privacy,andmeetinglegalandregulatoryrequirements,thebestpracticesthathavebeenaroundforalongtimearestillthebestwaytoidentifyissues,findvulnerabilities,fixthem,andmitigatepotentialsecurityissues.

•Confirmthemanagementofartificialintelligencesystemsisintegratedwithexistingorganizationalpractices.

•ConfirmAIMLsystemsfollowexistingprivacy,governance,andsecuritypractices,withAIspecificprivacy,governance,andsecuritypracticesimplementedwhenrequired.

FundamentalSecurityPrinciples

LLMcapabilitiesintroduceadifferenttypeofattackandattacksurface.LLMsarevulnerabletocomplexbusinesslogicbugs,suchaspromptinjection,insecureplugindesign,andremotecodeexecution.Existingbestpracticesarethebestwaytosolvetheseissues.Aninternalproductsecurityteamthatunderstandssecuresoftwarereview,architecture,datagovernance,andthird-partyassessmentsThecybersecurityteamshouldalsocheckhowstrongthecurrentcontrolsaretofindproblemsthatcouldbemadeworsebyLLM,suchasvoicecloning,impersonation,orbypassingcaptchas.Givenrecentadvancementsinmachinelearning,NLP(NaturalLanguageProcessing),NLU(NaturalLanguageUnderstanding),DeepLearning,andmorerecently,LLMs(LargeLanguageModels)andGenerativeAI,itisrecommendedtoincludeprofessionalsproficientintheseareasalongsidecybersecurityanddevopsteams.Theirexpertisewillnotonlyaidinadoptingthesetechnologiesbutalsoindevelopinginnovativeanalysesandresponsestoemergingchallenges.

Risk

ReferencetoriskusestheISO31000definition:Risk="effectofuncertaintyonobjectives."LLMrisksincludedinthechecklistincludesatargetedlistofLLMrisksthataddressadversarial,safety,legal,regulatory,reputation,financial,andcompetitiverisks.

VulnerabilityandMitigationTaxonomy

Currentsystemsforclassifyingvulnerabilitiesandsharingthreatinformation,likeOVAL,STIX,CVE,andCWE,arestilldevelopingtheabilitytomonitorandalertdefendersaboutvulnerabilitiesandthreatsspecifictoLargeLanguageModels(LLMs)andPredictiveModels.Itisexpectedthatorganizationswillleanontheseestablishedandrecognizedstandards,suchasCVEforvulnerabilityclassificationandSTIXfortheexchangeofcyberthreatintelligence(CTI),whenvulnerabilitiesorthreatstoAI/MLsystemsandtheirsupplychainsareidentified.

DeterminingLLMStrategy

TherapidexpansionofLargeLanguageModel(LLM)applicationshasheightenedtheattentionandexaminationofallAI/MLsystemsusedinbusinessoperations,encompassingbothGenerativeAIandlong-establishedPredictiveAI/MLsystems.Thisincreasedfocusexposespotentialrisks,suchasattackerstargetingsystemsthatwerepreviouslyoverlookedandgovernanceorlegalchallengesthatmayhavebeendisregardedintermsoflegal,privacy,liability,orwarrantyissues.ForanyorganizationleveragingAI/MLsystemsinitsoperations,it’scriticaltoassessandestablishcomprehensivepolicies,governance,securityprotocols,privacymeasures,andaccountabilitystandardstoensurethesetechnologiesalignwithbusinessprocessessecurelyandethically.

Attackers,oradversaries,providethemostimmediateandharmfulthreattoenterprises,people,andgovernmentagencies.Theirgoals,whichrangefromfinancialgaintoespionage,pushthemtostealcriticalinformation,disruptoperations,anddamageconfidence.Furthermore,theirabilitytoharnessnewtechnologiessuchasAIandmachinelearningincreasesthespeedandsophisticationofattacks,makingitdifficultfordefensestostayaheadofattacks.

Themostpressingnon-adversaryLLMthreatformanyorganizationsstemfrom"ShadowAI":

employeesusingunapprovedonlineAItools,unsafebrowserplugins,andthird-partyapplicationsthatintroduceLLMfeaturesviaupdatesorupgrades,circumventingstandardsoftwareapprovalprocesses.

Figure2.1:Imageofoptionsfordeploymentstrategy

DeploymentStrategy

Thescopesrangefromleveragingpublicconsumerapplicationstotrainingproprietarymodelsonprivatedata.Factorslikeusecasesensitivity,capabilitiesneeded,andresourcesavailablehelpdeterminetherightbalanceofconveniencevs.control.However,understandingthesefivemodeltypesprovidesaframeworkforevaluatingoptions.

Figure2.2:Imageofoptionsfordeploymenttypes

Checklist

AdversarialRisk

AdversarialRiskincludescompetitorsandattackers.

□Scrutinizehowcompetitorsareinvestinginartificialintelligence.AlthoughtherearerisksinAIadoption,therearealsobusinessbenefitsthatmayimpactfuturemarketpositions.

□Investigatetheimpactofcurrentcontrols,suchaspasswordresets,whichusevoicerecognitionwhichmaynolongerprovidetheappropriatedefensivesecurityfromnewGenAIenhancedattacks.

□UpdatetheIncidentResponsePlanandplaybooksforGenAIenhancedattacksandAIMLspecificincidents.

ThreatModeling

Threatmodelingishighlyrecommendedtoidentifythreatsandexamineprocessesandsecuritydefenses.Threatmodelingisasetofsystematic,repeatableprocessesthatenablemakingreasonablesecuritydecisionsforapplications,software,andsystems.ThreatmodelingforGenAIacceleratedattacksandbeforedeployingLLMsisthemostcosteffectivewaytoIdentifyandmitigaterisks,protectdata,protectprivacy,andensureasecure,compliantintegrationwithinthebusiness.

□Howwillattackersaccelerateexploitattacksagainsttheorganization,employees,executives,orusers?Organizationsshouldanticipate"hyper-personalized"attacksatscaleusingGenerativeAI.LLM-assistedSpearPhishingattacksarenowexponentiallymoreeffective,targeted,andweaponizedforanattack.

□HowcouldGenAIbeusedforattacksonthebusiness’scustomersorclientsthroughspoofingorGenAIgeneratedcontent?

□CanthebusinessdetectandneutralizeharmfulormaliciousinputsorqueriestoLLMsolutions?

□CanthebusinesssafeguardconnectionswithexistingsystemsanddatabaseswithsecureintegrationsatallLLMtrustboundaries?

□Doesthebusinesshaveinsiderthreatmitigationtopreventmisusebyauthorizedusers?

□CanthebusinesspreventunauthorizedaccesstoproprietarymodelsordatatoprotectIntellectualProperty?

□Canthebusinesspreventthegenerationofharmfulorinappropriatecontentwithautomatedcontentfiltering?

AIAssetInventory

AnAIassetinventoryshouldapplytobothinternallydevelopedandexternalorthird-partysolutions.

□CatalogexistingAIservices,tools,andowners.Designateataginassetmanagementforspecificinventory.

□IncludeAIcomponentsintheSoftwareBillofMaterial(SBOM),acomprehensivelistofallthesoftwarecomponents,dependencies,andmetadataassociatedwithapplications.

□CatalogAIdatasourcesandthesensitivityofthedata(protected,confidential,public)

□EstablishifpentestingorredteamingofdeployedAIsolutionsisrequiredtodeterminethecurrentattacksurfacerisk.

□CreateanAIsolutiononboardingprocess.

□EnsureskilledITadminstaffisavailableeitherinternallyorexternally,followingSBoMrequirements.

AISecurityandPrivacyTraining

□ActivelyengagewithemployeestounderstandandaddressconcernswithplannedLLMinitiatives.

□Establishacultureofopen,andtransparentcommunicationontheorganization’suseofpredictiveorgenerativeAIwithintheorganizationprocess,systems,employeemanagementandsupport,andcustomerengagementsandhowitsuseisgoverned,managed,andrisksaddressed.

□Trainallusersonethics,responsibility,andlegalissuessuchaswarranty,license,andcopyright.

□UpdatesecurityawarenesstrainingtoincludeGenAIrelatedthreats.Voicecloningandimage

cloning,aswellasinanticipationofincreasedspearphishingattacks

□AnyadoptedGenAIsolutionsshouldincludetrainingforbothDevOpsandcybersecurityforthedeploymentpipelinetoensureAIsafetyandsecurityassurances.

EstablishBusinessCases

SolidbusinesscasesareessentialtodeterminingthebusinessvalueofanyproposedAIsolution,balancingriskandbenefits,andevaluatingandtestingreturnoninvestment.Thereareanenormousnumberofpotentialusecases;afewexamplesareprovided.

□Enhancecustomerexperience

□Betteroperationalefficiency

□Betterknowledgemanagement

□Enhancedinnovation

□MarketResearchandCompetitorAnalysis

□Documentcreation,translation,summarization,andanalysis

Governance

CorporategovernanceinLLMisneededtoprovideorganizationswithtransparencyandaccountability.IdentifyingAIplatformorprocessownerswhoarepotentiallyfamiliarwiththetechnologyorthe

selectedusecasesforthebusinessisnotonlyadvisedbutalsonecessarytoensureadequate

reactionspeedthatpreventscollateraldamagestowellestablishedenterprisedigitalprocesses.

□Establishtheorganization’sAIRACIchart(whoisresponsible,whoisaccountable,whoshouldbeconsulted,andwhoshouldbeinformed)

□DocumentandassignAIrisk,riskassessments,andgovernanceresponsibilitywithintheorganization.

□Establishdatamanagementpolicies,includingtechnicalenforcement,regardingdataclassificationandusagelimitations.Modelsshouldonlyleveragedataclassifiedfortheminimumaccesslevelofanyuserofthesystem.Forexample,updatethedataprotectionpolicytoemphasizenottoinputprotectedorconfidentialdataintononbusiness-managedtools.

□CreateanAIPolicysupportedbyestablishedpolicy(e.g.,standardofgoodconduct,dataprotection,softwareuse)

□PublishanacceptableusematrixforvariousgenerativeAItoolsforemployeestouse.

□DocumentthesourcesandmanagementofanydatathattheorganizationusesfromthegenerativeLLMmodels.

Legal

ManyofthelegalimplicationsofAIareundefinedandpotentiallyverycostly.AnIT,security,andlegalpartnershipiscriticaltoidentifyinggapsandaddressingobscuredecisions.

□ConfirmproductwarrantiesareclearintheproductdevelopmentstreamtoassignwhoisresponsibleforproductwarrantieswithAI.

□ReviewandupdateexistingtermsandconditionsforanyGenAIconsiderations.

□ReviewAIEULAagreements.End-userlicenseagreementsforGenAIplatformsareverydifferentinhowtheyhandleuserprompts,outputrightsandownership,dataprivacy,compliance,liability,privacy,andlimitsonhowoutputcanbeused.

□OrganizationsEULAforcustomers,Modifyend-useragreementstopreventtheorganizationfromincurringliabilitiesrelatedtoplagiarism,biaspropagation,orintellectualpropertyinfringementthroughAI-generatedcontent.

□ReviewexistingAI-assistedtoolsusedforcodedevelopment.Achatbot’sabilitytowritecodecanthreatenacompany’sownershiprightstoitsproductifachatbotisusedtogeneratecodefortheproduct.Forexample,itcouldcallintoquestionthestatusandprotectionofthegeneratedcontentandwhoholdstherighttousethegeneratedcontent.

□Reviewanyriskstointellectualproperty.Intellectualpropertygeneratedbyachatbotcouldbeinjeopardyifimproperlyobtaineddatawasusedduringthegenerativeprocess,whichissubjecttocopyright,trademark,orpatentprotection.IfAIproductsuseinfringingmaterial,itcreatesariskfortheoutputsoftheAI,whichmayresultinintellectualpropertyinfringement.

□Reviewanycontractswithindemnificationprovisions.Indemnificationclausestrytoputtheresponsibilityforaneventthatleadstoliabilityonthepersonwhowasmoreatfaultforitorwhohadthebestchanceofstoppingit.EstablishguardrailstodeterminewhethertheprovideroftheAIoritsusercausedtheevent,givingrisetoliability.

□ReviewliabilityforpotentialinjuryandpropertydamagecausedbyAIsystems.

□Reviewinsurancecoverage.Traditional(D&O)liabilityandcommercialgeneralliabilityinsurancepoliciesarelikelyinsufficienttofullyprotectAIuse.

□Identifyanycopyrightissues.Humanauthorshipisrequiredforcopyright.Anorganizationmayalsobeliableforplagiarism,propagationofbias,orintellectualpropertyinfringementifLLMtoolsaremisused.

□EnsureagreementsareinplaceforcontractorsandappropriateuseofAIforanydevelopmentorprovidedservices.

□RestrictorprohibittheuseofgenerativeAItoolsforemployeesorcontractorswhereenforceablerightsmaybeanissueorwherethereareIPinfringementconcerns.

□AssessandAIsolutionsusedforemployeemanagementorhiringcouldresultindisparatetreatmentclaimsordisparateimpactclaims.

□MakesuretheAIsolutionsdonotcollectorsharesensitiveinformationwithoutproperconsentorauthorization.

Regulatory

TheEUAIActisanticipatedtobethefirstcomprehensiveAIlawbutwillapplyin2025attheearliest.TheEUśGeneralDataProtectionRegulation(GDPR)doesnotspecificallyaddressAIbutincludesrulesfordatacollection,datasecurity,fairnessandtransparency,accuracyandreliability,andaccountability,whichcanimpactGenAIuse.IntheUnitedStates,AIregulationisincludedwithinbroaderconsumerprivacylaws.TenUSstateshavepassedlawsorhavelawsthatwillgointoeffectbytheendof2023.

FederalorganizationssuchastheUSEqualEmploymentOpportunityCommission(EEOC),theConsumerFinancialProtectionBureau(CFPB),theFederalTradeCommission(FTC),andtheUSDepartmentofJusticeśCivilRightsDivision(DOJ)arecloselymonitoringhiringfairness.

□DetermineCountry,State,orotherGovernmentspecificAIcompliancerequirements.

□Determinecompliancerequirementsforrestrictingelectronicmonitoringofemployeesandemployment-relatedautomateddecisionsystems(Vermont,California,Maryland,NewYork,NewJersey)

□DeterminecompliancerequirementsforconsentforfacialrecognitionandtheAIvideoanalysisrequired(Illinois,Maryland,Washington,Vermont)

□ReviewanyAItoolsinuseorbeingconsideredforemployeehiringormanagement.

□ConfirmthevendorścompliancewithapplicableAIlawsandbestpractices.

□AskanddocumentanyproductsusingAIduringthehiringprocess.Askhowthemodelwastrained,andhowitismonitored,andtrackanycorrectionsmadetoavoiddiscriminationandbias.

□Askanddocumentwhataccommodationoptionsareincluded.

□Askanddocumentwhetherthevendorcollectsconfidentialdata.

□Askhowthevendorortoolstoresanddeletesdataandregulatestheuseoffacialrecognitionandvideoanalysistoolsduringpre-employment.

□Reviewotherorganization-specificregulatoryrequirementswithAIthatmayraisecomplianceissues.TheEmployeeRetirementIncomeSecurityActof1974,forinstance,hasfiduciarydutyrequirementsforretirementplansthatachatbotmightnotbeabletomeet.

UsingorImplementingLargeLanguageModelSolutions

□ThreatModelLLMcomponentsandarchitecturetrustboundaries.

□DataSecurity,verifyhowdataisclassifiedandprotectedbasedonsensitivity,includingpersonalandproprietarybusinessdata.(Howareuserpermissionsmanaged,andwhatsafeguardsareinplace?)

□AccessControl,implementleastprivilegeaccesscontrolsandimplementdefense-in-depthmeasures

□TrainingPi