AI在网络防御中的机遇

OpportunitiesforAIincyberdefenceUseofAIbycybersecurityteams

AustralianGovernment

AustraliansignalsDirectorate

AUSTRALIAN

DIRECTRATE

ASDGNA

cybrrty

centre

CscAutrs"·

communicationssecurityEstablishmentcanada

canadiancentre

forcybersecurity

centredelascuritdes

tcommunicationscanadacentrecanadien

pourlacyberscurit

Nationalcybersecuritycentre

NEWZEALAND

Nationalcybersecuritycentre

apartofGCHQ

Tableofcontents

Introduction· 4

Audience· 4

Thecybersecuritylandscapeisevolving· 5

IntegratingAIforcybersecurity· 7

SecurelyadoptingAI· 13

Conclusion· 17

Furtherinformation· 17

AppendixA:CybersecurityquestionsforAIvendors· 18

AppendixB:AIcapabilityquestionsforcybersecurityvendors· 21

.au

Introduction

Artificialintelligence(AI)israpidlyreshapingthecybersecuritylandscape.AshighlycapableAI

becomesmorewidelyavailable,maliciousactorsareusingittodelivercyberthreatsatgreaterscaleandspeed.Organisationsthatdon’tre-evaluateandimprovetheirdefenceswillremainvulnerabletotheseAI-enabledcyberthreats.

Cybersecurityhastraditionallyreliedonspecialisedteamsandreactiveworkflowstomanagerisk.Theseapproachesremainimportant,butthescaleandcomplexityofthemoderncybersecurity

landscapeincreasinglystrainthem.Heavydependenceonmanualprocessescanmakeitdifficulttoprioritiserisks,investigatepotentialthreatsandmaintainconsistentdefensivecoverage.

AIpresentsasignificantopportunityforcyberdefenders.Whenusedsafely,securelyandresponsiblyAIcan:

•strengthenprioritisationofcyberrisks

•improvedetectionofthreatsandvulnerabilities

•supportfasterresponseandrecovery

•reducerelianceonrepetitivemanualtasks.

Thisguidancewasco-sealbytheCanadianCentreforCyberSecurity(CyberCentre),theNew

ZealandNationalCyberSecurityCentre(NCSC-NZ)andtheUnitedKingdomNationalCyberSecurityCentre(NCSC-UK),andaddresseshoworganisationscanuseAItostrengthenorganisationalcybersecuritywhilemanagingtherisksofusingAI.ItoutlineshowthecybersecuritylandscapeisevolvinganddescribeshoworganisationscanuseAIalignedwiththe

Informationsecuritymanual

(ISM)

cybersecurityfunctionsofGovern,Identify,Protect,Detect,RespondandRecover.Italsosetsout

principlesforsecurelyadoptingAI,alongwithkeyquestionsforcyberdefenderstoaskAIvendorstosupportsecureuse.

Humanoversight,governanceand

SecurebyDesign

practicesremainessential.AIcansignificantlyenhancecybersecurity,butitisnotareplacementforstrongcybersecurityfundamentals.Poorly

designedorpoorlygovernedAIsystemscanintroducenewattackpaths.Thiscanoccurthrough

excessivesystemaccess,relianceonuntrustedinputs,orautomatedactionswithoutadequate

safeguards.Organisationsthatlackrobustidentityandaccessmanagement,secureconfiguration,patchmanagement,networksegmentation,monitoringandincidentresponseprocessesare

unlikelytoachievebettersecurityoutcomessimplybydeployingAI.

Audience

Thisguidancesupportscyberdefenders,suchasChiefInformationSecurityOfficersandseniorsecurityleaderswhoareresponsibleforcybersecuritystrategy,operationsandriskoutcomes.

Thepublicationassumesafoundationalunderstandingofcybersecurityconceptsandsupportsinformeddecision-makingaboutsafe,secureandresponsibleuseofAIwithincybersecurity.

4OpportunitiesforAIincyberdefence

OpportunitiesforAIincyberdefence5

Thecybersecurity

landscapeisevolving

ThecybersecuritylandscapeisrapidlyevolvingasmaliciousactorsincreasinglyuseAItoimprovethespeed,scaleandsophisticationofattacks.Toadapt,cyberdefenderscanuseAItostrengthenanalysis,prioritisationanddefensivedecisionmaking.

SpectrumofAIuseincybersecurity

OrganisationsareincreasinglyexploringhowAIcanbeusedtoenhancecybersecurityoutcomes.

Thefocusshouldremainonimprovingsecuritywithintheorganisation,notingthatsimplyidentifyingvulnerabilitiesdoesnot,onitsown,strengthensecurity.Withoutappropriatecontext,prioritisation

andremediation,poorlyimplementedAIusecouldintroduceadditionalriskratherthanreduceit.

OrganisationsshoulddrawonaspectrumofAIcapabilitiesdependingontheirobjectives,risk

appetiteandavailabletechnology.ThisspectrumrangesfromadvancedfrontierAImodels,throughtogeneralpurposelargelanguagemodels(LLM),toembeddedAIfeatureswithinexistingsecurity

tools.Foralistofquestionstosupportvendorassessment,refertoAppendicesAandB.

Attheleadingedge,frontierAImodelscanperformawidevarietyoftasksandreflectthe

capabilitiespresentintoday’smostadvancedmodels.ComparedtomorecommongenerativeAI

systems,frontierAIenablemorecomplexreasoning,broadertaskcoverage,andtighterintegrationwithtools,dataandoperationalworkflows.FrontierAIusealsocarriescostandsustainability

considerations,includingtokenconsumption,computedemandsandongoingoperational

expenditure.OrganisationsshouldconsiderthesefactorswhenselectingAImodels,ensuringthe

levelofmodelcapabilityiscommensuratewiththerequirementsandvalueoftheintendedusecase.

Inacybersecuritycontext,frontierAIshouldbeintegratedtoaugmentexistingtoolsandprocesses,ratherthandeployingitasastandalonesolution.Withincybersecurityoperations,organisations

achievesaferandmoreeffectiveAIusebydeployingmodernandfit-for-purposesecuritysoftware

thatconstrains,testsandgovernsAIcapabilitiestosupportspecificISMfunctions.

ThejointFiveEyes

statementurgesorganisationsleaderstouseAItostrengthendefence

.

Theapproachanorganisationtakeswilldependonwhatitistryingtoachieveandwhatithas

accessto.Alayeredstrategyallowsorganisationstocombinethesecapabilities,usingmore

advancedAIwhereappropriatewhileleveragingexistingtoolstoenhanceoperationalefficiency.

ThisensuresAIisusedtostrengthenexistingcybersecuritypracticesratherthanreplacethem,supportingsafe,secureandresponsibleadoptionacrosstheorganisation.

NoteveryorganisationcanadoptfrontierAIatthesamepace,especiallywhereresourcesand

skillsarelimited,creatinggapsbetweencyberdefenders.The

WorldEconomicForum

highlightsthatsmallerandpublic-sectororganisationsfacelowercyberresilience,withcriticalinfrastructureshowingparticularlylowconfidenceinrespondingtomajorincidents.

Thesechallengespointtotheneedforstrongerlocalreadiness,trustedpartnerships,andpracticalsupport.

6OpportunitiesforAIincyberdefence

AgenticAI

AgenticAIreferstoAIsystemsthatcanindependentlyplan,decideandtakeactionstoachievea

goal,ratherthansimplyrespondingtoindividualprompts.ThesesystemsuseadvancedAImodels,suchasLLMs,tounderstandtheirenvironmentandreasonaboutoptions.Theyarecombinedwithaccesstotools,data,memoryandworkflows,enablingthemtoactandoperatewithadegreeofautonomy.

UnliketraditionalAIorchatsystems,agenticAIactivelyworkstowardoutcomesevenwhen

objectivesarelooselydefined.Itcanoperatewithlimitedongoinghumanoversight,adaptits

behaviourbasedonresults,andinsomecasescreatesubtasksorsubagentstocompletecomplexwork.Whilehumanssetgoals,constraintsandpermissions,agenticAIsystemsexecutegoal

directedactionsovertimeratherthansimplyprovidingadviceoroutputs.

Formoreinformation,refertoourco-authoredpublicationbyCybersecurityandInfrastructure

SecurityAgency(CISA),NationalSecurityAgency(NSA),CanadianCentreforCyberSecurity(CCCS),NationalCyberSecurityCentreNewZealand(NCSC-NZ),NationalCyberSecurityCentre(NCSC-UK)on

CarefuladoptionofagenticAIservices

.

MaliciousactorsareleveragingAI

MaliciousactorsareincreasinglyleveragingAItoacceleratedevelopmentofcyberattacksand

deploythematscale.TheyembedAImodelsdirectlyintomaliciousworkflowstoautomate

reconnaissance,developattacktooling,analysecompromiseddataandgeneratetailoredmaliciousoutputs.Thisacceleratesvulnerabilitydiscovery,andshortensthetimebetweenvulnerability

discoveryandexploitation,leavingdefenderswithlesswarningandresponsetime.

AIalsolowersthetechnicalbarrierstoentry.LessskilledmaliciousactorscanuseAItoperformactivitiesthatpreviouslyrequiredspecialistexpertise,suchasproducingevasivemalware,

conductinglarge-scaledataanalysisorexecutingconvincingsocialengineeringcampaigns.

AsAIcapabilitiesbecomemoreaccessible,maliciousactorswillcontinuetoincreasethespeed,

scaleandimpactofmaliciouscyberoperations.Tostayahead,organisationsshouldstrengthendefensivecapabilitybyreinforcingcybersecurityfundamentals.Theseincludeminimisingattacksurface,promptlypatchingsystems,implementinglayereddefence-in-deptharchitectures,

increasingautomationandimprovingthreatdetection.Withoutcorrespondingadvancesin

defence,AI-enabledthreatsarelikelytoerodetheeffectivenessoftraditionalsecurityapproaches.

Forfurtherinformation,refertotheUnitedKingdom’sNationalCyberSecurityCentre’s(UK’sNCSC)

ImpactofAIoncyberthreatfromnowto2027

assessment.

OpportunitiesforAIincyberdefence7

IntegratingAIforcybersecurity

TheuseofAIforcybersecurityshouldalignwiththeISManditsassociatedcontrols.TheISMgroupscybersecurityprinciplesinto6functions:Govern,Identify,Protect,Detect,RespondandRecover.ThissectionoutlineshoworganisationscanapplyAIwithineachfunctiontosupportcybersecurity,as

illustratedinFigure1.

ManyAIusecasesinvolveprocessingsensitiveoperationalinformation.OrganisationsshouldapplyAIwhilemaintainingappropriatecontrolsandprotectinginformation,inlinewiththeirsecurityanddatahandlingobligations.Thisincludesenforcingleastprivilegeaccess,restrictingdataexposuretoAIsystems,validatingoutputsbeforeuseandmaintainingauditabilityofAI-assistedactions.

Organisationsshouldalsoundertakeongoingassessmentofcosts,benefitsandrisks,recognisingthatbothAIcapabilitiesandmaliciousactoruseofAIwillcontinuetoevolve.

GovernIdentifyProtectDetectRespondRecover

FrontierAImodels

Createlearningenvironments

Generatedynamicattackscenarios

Support activitiesSummarise&interpretinformation

Analyse&informrestorationactivities

AdvancevisibilityIdentifypatterns

SupportteamsIdentify issues

Analyse

data

Identify

gaps

Automate&enrichdiscovery

Figure1.ApplyingAIacrossthesixcybersecurityfunctions

8OpportunitiesforAIincyberdefence

Govern

Developandmaintainastrongandresilientcybersecurityculture

TheGovernfunctionfocusesonhowcybersecurityandcyberriskaredirected,understoodand

managedacrosstheorganisation.Itestablishesclearaccountability,decisionmakingstructureandoversightmechanismsthatconnectexecutiveleadershipwithtechnicalteams.Asthreatsevolve,

effectivegovernancesupportstimelyreviewandadaptationofpolicies,controlsandassurancearrangementstomaintainorganisationalresilience.

AI-specificthreatintelligencesharingshouldalsobestrengthened,leveragingASD’sACSC(includingitsPartnershipProgram:

.au/partnershipprogram

),theNationalCyberThreatNotificationSystem(NCTNS),andCISA’sAICybersecurityCollaborationPlaybooktosupporttimely,trustedcollaborationacrosspartners.

AsorganisationsadoptAIincybersecurity,governanceshoulddefineclearexpectationsforitsuse,oversight,andhumanaccountability.ThisincludesapplyingboundedautonomytokeepAIwithindefinedlimits,especiallyincriticalsystemsandsecurityoperations.ItalsorequiresrecoverabilitysoAIfunctionscanbesafelyrestrictedorrestored,ensuringoperationscontinueifsystemsfailorarecompromised.

AImaysupportorganisationsto:

•identifyinconsistenciesinriskevaluationacrossbusinessunits,systemsorprojects

•analysesupplychainrisks,includingsoftwaredependencies,vulnerabilityexposureandvendorsecuritypractices

•supportthecreationanduseofinventories,suchassoftwarebillofmaterials(SBOM)andcryptographicbillofmaterials(CBOM)

•strengthenpolicyinterpretationandcompliance,suchasusinganinternalAIassistantto

providecontextawareguidancebasedonorganisationalpoliciesandregulatoryrequirements

•prioritisecybersecuritydecisionsbasedonriskassessment.

OpportunitiesforAIincyberdefence9

Identify

Identifyassetsandassociatedsecurityrisks

TheIdentifyfunctionfocusesonunderstandingwhatassetsexistwithintheenvironmentandthesecurityrisksassociatedwiththem.Itestablishesvisibilityofsystems,software,dataandconfigurationstoenableinformedrisk-baseddecisions.

AImaysupportorganisationsto:

•enhanceassetdiscoverybyusingnetworktelemetrytoidentifyunmonitoredorhiddenassetsacrosstheenvironment

•prioritisepatchingdecisionsbyusingmultiplefactorsincludingseverityratings,exploitavailability,threatintelligenceandoperationalimpactofexploitation

•identifyandassessvulnerabilitychainingbylinkingmultiplelowerseverityvulnerabilitiesintoattackpaths

•identifyinsecureconfigurationsandrecommendremediationactions

•reviewlogsamplestoidentifyactionsanadversarycouldtakewithouttriggeringalerts,andassistwithrefiningdetectionlogic

•analyseSBOMsandCBOMstomapallsoftwarecomponentstoidentifyoutdatedlibraries,hiddenrisks,andsupplychainexposures

•mapAItrainingandfine-tuningdatasourcestoassessprovenance,retentionandpoisoningexposure.

Redteamscenario:AI-drivenattackpathanalysis

Anorganisation’sredteamusesasuitableAImodeltoanalyseitsenterpriseenvironment,

ingestingsystemarchitecture,identityrelationships,andvulnerabilitydataatscale.The

modelisconfiguredwithrelativelybroadpermissions,allowingittocorrelatefindingsacrossnetwork,applicationandaccesscontrollayers.

Overtime,theAIidentifieshowmultiplelow-andmedium-severityweaknessescancombineintomeaningfulattackpathsthatwouldbedifficultandtime-consumingforhumansto

detectmanually.Forexample,itlinksabenignmisconfiguration,excessivepermissionsandaminorsoftwareflawintoasequencethatcouldenableprivilegeescalationandlateral

movement.

Bycontinuouslygeneratingandrefiningthesescenarios,theredteamusesAItouncover

high-impactchainsthatarenotobviouswhenissuesareconsideredinisolation.This

improvestestingcoverageandhelpsorganisationprioritiseremediationbasedonreal-worldattackfeasibilityratherthanindividualvulnerabilityseverity.

FormoreinformationonSBOMs,refertotheUnitedStates’CybersecurityandInfrastructureSecurityAgency’s

ASharedVisionofSoftwareBillofMaterials(SBOM)forCybersecurity

.

10OpportunitiesforAIincyberdefence

Protect

Implementandmaintaincontrolstomanagesecurityrisks

TheProtectfunctionfocusesonimplementingandmaintainingsafeguardsthatreducethelikelihoodandimpactofcybersecurityincidents.Thisincludesmaintainingsecureconfigurations,managingidentitiesandaccess,reducingattacksurface,andensuringcontrolsremaineffectiveassystems

andthreatschange.

AImaysupportorganisationsto:

•prioritisehardeningactionsbasedonexploitabilityandenvironmentalcontexttoreducetheattacksurface

•enhancesecurityarchitecturebyevaluatingcomplexenvironments,trustboundariesanddataflowstosimulaterealisticattackpathsanddetectweaknesses

•analyseidentities,rolesandbehavioursofusersandagentstoidentifybreachesofleastprivilegeaccess,detectexcessiveorunintendedpermissions,andidentifyprivilegecreepororphanedaccounts

•analysereal-timetrafficpatternstoidentifypotentialsecuritythreats,includinganomalousorautonomousAIdrivenactivity,andrecommendimprovementstonetworksegmentationrules

•scansourcecode,infrastructureascode,andpipelinedefinitionstodiscovervulnerabilitiesandbusinesslogicflawsthatpattern-basedscannersmiss

•zero-trustandmoderndefensiblearchitecturesshouldextendtoAIagentsaswellashumanusers.Thisinvolvesreplacingstaticcredentialswithdynamictokensandenforcingclear

privilegeboundariesonwhatagentscanaccessandexecute.

Codereviewscenario:AIidentifyinghiddenvulnerabilities

AnorganisationdeploysanAItooltoreviewsourcecodethatautomaticallyanalyses

sourcecodewheneveradevelopersubmitschanges.Duringaroutineupdate,adeveloperintroducesafeaturethatprocessesuserinputandstoresitinadatabase.Whilethecode

functionsasintended,theAIidentifiesthattheinputisnotproperlyvalidatedbeforestorage.

Althoughthisissueappearsminorandwouldlikelypasstraditionalchecks,sincethesyntaxiscorrectandnoobviousruleisviolated,theAIrecognisesthebroadersecurityimplication.Ithighlightsthatamaliciousactorcouldexploitthisgaptoinsertmaliciousdata,potentiallyleadingtocompromise.

Byflaggingtheissueearly,theorganisationisabletocorrectthevalidationlogicbeforerelease.Thisreducestheriskofintroducingexploitablevulnerabilitiesintoproductionandstrengthensoverallsecuritywithoutslowingdevelopmenttime.

ForfurtherinformationonsecurityevaluationsofAI,refertotheFrontierModelForum’s

Technical

Report:ManagingAdvancedCyberRisksinFrontierAIFrameworks

,whichoutlinesemergingindustryviewsonhowfrontierAImodelscansupportcyberriskmanagement,includingtheidentificationofvulnerabilitiesandinsecuredefaults.

OpportunitiesforAIincyberdefence11

Detect

Detectandanalysecybersecurityeventstoidentifycybersecurityincidents

TheDetectfunctionfocussesonidentificationandanalysisofcybersecurityeventstoenablethe

timelydetectionofpotentialincidents.Itsupportscontinuousmonitoringacrosssystems,networksandidentitiestosurfaceanomalousactivityandemergingthreats.

AImaysupportorganisationsto:

•detectcybersecurityeventsandincidents,withanalystjudgementtovalidatefindings

•leverageframeworkssuchas

MITREATLAS

™,OWASPTop10forLLMApplications,OWASPTop10forAgenticApplication,NISTAIRMFGenerativeAIProfileandMAESTROthreatmodelto

informAIsupporteddetectionofmaliciousbehaviourstargetingAIenabledsystems

•analysenetworktelemetry(forexample,flows,logs,DomainNameSystem,andapplicationprogramminginterfacecalls)todetectanomalousbehaviour,suchasunexpected

communicationbetweenservicesorconnectionstoknownmaliciousendpoints

•detectAImisusethroughAI-specifictelemetry,suchasmodelinputs(includingpromptmanipulation),decisiontraces,policychecksandconfidencesignals,whileprotectingtheintegrityofdetectionlogsfromtampering

•analysebehaviourandconfigurationcontexttodistinguishlegitimateactivityfromsuspiciousactivitytominimisefalsepositivesandfalsenegatives

•baselinehigh-riskactivitiessuchasidentity,privilegedaccessandremoteaccesstoassistinanomalydetection.

Securityoperationscentreplatformscenario:AI-driventhreatdetectionandtriage

Anorganisationhasasecurityoperationscentre(SOC)platformthatusesAI-assisted

detectiontotriagealertsacrossidentity,endpoint,networkandcloudtelemetry.Tohelp

analystsmanagehighalertvolumes,theplatformcorrelatesdataacrosstheDomainNameSystem,applicationprogramminginterface(API)transactions,andnetworkflows.Italso

trackspatternsinprivilegedaccessandsurfacesprioritisedincidentsforanalystreview.

Overtime,theorganisationbuildsconfidenceintheplatform’sabilitytosuppressfalse

positivesandhighlightthealertsthatmattermost.TheAIcontinuouslyrefinesits

understandingofnormalandabnormalbehaviour,improvingdetectionaccuracyacrosstheenvironment.

Inthisscenario,theSOCplatformenablesanalyststoidentifyandrespondtothemostcriticalthreatsmorequickly,reducingresponsetimesandalleviatingoperationaloverloadwhile

maintainingstrongsecurityvisibility.

12OpportunitiesforAIincyberdefence

Respond

Respondtocybersecurityincidents

TheRespondfunctionfocusesoneffective,timelyandcoordinatedactionduringcybersecurityincidents.Itensuresorganisationscancontainandmitigateincidentswhilemaintainingcriticaloperations.

AImaysupportorganisationsto:

•assistanalystsbycorrelatingalerts,logsandforensicartefactsintoacoherentexplanationoftheincident

•interpretalertsandobservedsystemoruserbehavioursinthecontextoftheorganisation’sspecificsystems,architectureandriskenvironment

•reducerelianceonmanualsearchesduringinvestigations

•sequenceresponseactionsacrossidentity,endpointandnetworkcontrols

•draftincidentupdatesthatbridgerespondersandexecutivesbyaligningwithdefinedcommunicationexpectations,withrespondersvalidatingaccuracybeforedistribution

•augmentsurgecapacitybyautomatingtriageandpreparingorinitiatingpre-approvedandreversibleresponseactionsinparallelacrossconcurrentincidents,withhumanapproval

requiredforhigh-impactcontainment,disruption,recoveryorcommunicationdecisions,whichisespeciallycriticalinAI-drivenattackscenarios

•proposecontainmentandremediationactionsforuncertaincases,alignedwithorganisationalincidentresponseplaybooks,forreviewbyhumanincidentresponders.

Recover

Resumenormalbusinessoperationsfollowingcybersecurityincidents

TheRecoverfunctionfocusesonrestoringsystemsandservicesfollowingacybersecurityincidentsonormalbusinessoperationscansafelyresume.Itprioritisesandcontrolsrecoveryactivitiesbasedonverifiedsystemintegrityandacceptedresidualrisk.

AImaysupportorganisationsto:

•analyserebuildandrestorationpathwaystosupportrecoveryplanning,sequencingandassuranceactivities

•triggerautomatedrecoveryactionsandplaybookstoremediateimpactedassetsandreturnthemtoasecure,trustedstate,withhumanapprovalfordestructiveorirreversiblechangessuchassystemrestorationordatarollback

•validatesystemandservicerestorationsagainstknownbaselinestoconfirmintegritybeforeresumingoperations

•rollbackAImodelstopreviousversionswherecompromise,poisoningorunintendedmodeldriftissuspected

•verifytheintegrityandexpectedbehaviourofAImodelsandassociateddatathroughAI-specificvalidationchecksbeforere-enablingAI-augmentedorautonomousfunctions

•enablerapidandsaferesumptionofcriticalbusinessservicesbyidentifyingpotentialcascadingfailuresacrossinterconnectedsystemsandreducingtheriskofoccurrence

•generateandevaluaterecoverysequencesfornovelorcomplexincidenttypesthatfalloutsidepredefinedautomationorsecurityorchestration,automationandresponseplaybooks

•identifyweaknessesinrecoveryarrangements,dependenciesorassumptionsproactively,beforetheyareencounteredduringarealincident.

OpportunitiesforAIincyberdefence13

SecurelyadoptingAI

WhenadoptingAI,organisationsshouldprioritisestrongsecuritypractices.Whilenomitigation

strategycanprovidecompleteprotection,organisationsshouldimplementastrongcybersecuritybaselinealignedwithASD’s

ISM

andthe

EssentialEight

,tomateriallyreducecybersecurityrisk.

AsAIcapabilitiesandthecyberthreatenvironmentevolves,organisationsshouldregularlyreviewandupdatetheircontrolstoensureAIcontinuestobeusedsafely,securely,responsiblyandin

accordancewithorganisationalrisktolerance.Criticalinfrastructure(CI)shouldplaceparticularfocusonmaintainingresilient,continuouslyassuredcontrolsgivenitsheightenedriskandimpact.

DatareadinessisacoreprerequisiteforAIadoption.AIsupportedcyberdefencewillonlybeas

reliableastheunderlyingdataitdrawson,includingassetinventories,loggingcoverage,identity

andprivilegedata,vulnerabilityrecordsandconfigurationdata.Organisationsshouldassessand

upliftthequality,coverageandcurrencyofthisdatabeforeandduringAIadoption,recognisingthatgapsorinconsistencieswilldirectlylimittheaccuracyandvalueofAIgeneratedinsights.

AdoptingAIincyberdefencedependsonworkforcecapabilityasmuchastechnology.CyberdefendersshouldunderstandhowtouseAItoolseffectively,includinghowtovalidateoutputs,recogniselimitationsandavoidoverreliance.Organisationsshouldincorporatetheseskillsintoongoingworkforcedevelopmenttosupporteffectivehuman-AIteaming.

OrganisationsshouldalsoensureAIsystemsusedforcybersecurity,andAIsystemsused

operationally,areprotectedfromadversarialtechniquessuchaspromptinjection,modelevasionandmodelextraction,inlinewith

GuidelinesforsecureAIsystemdevelopment,

NationalInstituteofStandardsandTechnology’s(NIST)

AdversarialMachineLearning:ATaxonomyandTerminologyof

AttacksandMitigations(NISTAI100-2E2025)

and

MITREATLAS™

.

Humanoversight

AIshouldsupportcyberdefenders,notreplacehumanjudgment,particularlywheredecisionscouldaffecthighconsequencecyberorsafetyenvironments.AsAIsystemsbecomemorecapableand

beginrecommendingortakingactions,stronghumanoversightisessential.

AIuseintroducesrisksthatcandegradereliabilityifnotproperlymanaged.Theserisksinclude

hallucinatedormisleadingoutputs,adversarialmanipulation(suchaspromptinjectionormodel

evasion),overrelianceonAI,oversightfatigueandoperationalpressuresrelatedtoscale,availabilityorcost.

WhileAIcansignificantlyincreasespeedandscale,organisationsshouldverifyitsoutputsagainstevidenceandcontext.OngoingmonitoringandhumanoversighthelpensurethatAI-enabled

accelerationimprovesaccuracyandreliability,ratherthanintroducingnewerrors.

Organisationsshould:

•ensureAIsupports,rathe