版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领
文档简介
Universaladversarialperturbations
Seyed-MohsenMoosavi-Dezfooli∗†AlhusseinFawzi∗†
seyed.moosavi@epfl.chalhussein.fawzi@epfl.ch
OmarFawzi‡PascalFrossard†
omar.fawzi@ens-lyon.frpascal.frossard@epfl.ch
Abstract
Chihuahua
Givenastate-of-the-artdeepneuralnetworkclassifier,
weshowtheexistenceofauniversal(image-agnostic)and
verysmallperturbationvectorthatcausesnaturalimages
tobemisclassifiedwithhighprobability.Weproposeasys-
tematicalgorithmforcomputinguniversalperturbations,
andshowthatstate-of-the-artdeepneuralnetworksare
highlyvulnerabletosuchperturbations,albeitbeingquasi-
imperceptibletothehumaneye.Wefurtherempiricallyan-
alyzetheseuniversalperturbationsandshow,inparticular,
thattheygeneralizeverywellacrossneuralnetworks.The
surprisingexistenceofuniversalperturbationsrevealsim-
portantgeometriccorrelationsamongthehigh-dimensional
decisionboundaryofclassifiers.Itfurtheroutlinespoten-
tialsecuritybreacheswiththeexistenceofsingledirections
intheinputspacethatadversariescanpossiblyexploitto
breakaclassifieronmostnaturalimages.1
Joystick
Thresher
Tibetanmasti
Chihuahua
Jay
Labrador
Labrador
Tibetanmasti
1.Introduction
LycaenidBrabancongrion
Canwefindasinglesmallimageperturbationthatfools
astate-of-the-artdeepneuralnetworkclassifieronallnat-
uralimages?Weshowinthispapertheexistenceofsuch
quasi-imperceptibleuniversalperturbationvectorsthatlead
tomisclassifynaturalimageswithhighprobability.Specif-
ically,byaddingsuchaquasi-imperceptibleperturbation
tonaturalimages,thelabelestimatedbythedeepneu-
ralnetworkischangedwithhighprobability(seeFig.1).
Suchperturbationsaredubbeduniversal,astheyareimage-
agnostic.Theexistenceoftheseperturbationsisproblem-
aticwhentheclassifierisdeployedinreal-world(andpos-
siblyhostile)environments,astheycanbeexploitedbyad-
Balloon
Labrador
Borderterrier
Figure1:Whenaddedtoanaturalimage,auniversalper-
turbationimagecausestheimagetobemisclassifiedbythe
deepneuralnetworkwithhighprobability.Leftimages:
Originalnaturalimages.Thelabelsareshownontopof
eacharrow.Centralimage:Universalperturbation.Right
images:Perturbedimages.Theestimatedlabelsoftheper-
turbedimagesareshownontopofeacharrow.
∗Thefirsttwoauthorscontributedequallytothiswork.
†E´colePolytechniqueFe´de´raledeLausanne,Switzerland
‡ENSdeLyon,LIP,UMR5668ENSLyon-CNRS-UCBL-INRIA,
Universite´deLyon,France
1Toencouragereproducibleresearch,thecodeisavailableatgitHub.
Furthermore,avideodemonstratingtheeffectofuniversalperturbations
onasmartphonecanbefoundhere.
1
versariestobreaktheclassifier.Indeed,theperturbation
processinvolvesthemereadditionofoneverysmallpertur-
bationtoallnaturalimages,andcanberelativelystraight-
forwardtoimplementbyadversariesinreal-worldenviron-
ments,whilebeingrelativelydifficulttodetectassuchper-
turbationsareverysmallandthusdonotsignificantlyaffect
datadistributions.Thesurprisingexistenceofuniversalper-
turbationsfurtherrevealsnewinsightsonthetopologyof
thedecisionboundariesofdeepneuralnetworks.Wesum-
marizethemaincontributionsofthispaperasfollows:
•Weshowtheexistenceofuniversalimage-agnostic
perturbationsforstate-of-the-artdeepneuralnetworks.
•Weproposeanalgorithmforfindingsuchperturba-
tions.Thealgorithmseeksauniversalperturbationfor
asetoftrainingpoints,andproceedsbyaggregating
atomicperturbationvectorsthatsendsuccessivedata-
pointstothedecisionboundaryoftheclassifier.
•Weshowthatuniversalperturbationshavearemark-
ablegeneralizationproperty,asperturbationscom-
putedforarathersmallsetoftrainingpointsfoolnew
imageswithhighprobability.
•Weshowthatsuchperturbationsarenotonlyuniver-
salacrossimages,butalsogeneralizewellacrossdeep
neuralnetworks.Suchperturbationsarethereforedou-
blyuniversal,bothwithrespecttothedataandthenet-
workarchitectures.
•Weexplainandanalyzethehighvulnerabilityofdeep
neuralnetworkstouniversalperturbationsbyexamin-
ingthegeometriccorrelationbetweendifferentparts
ofthedecisionboundary.
singleperturbationvectorthatfoolsthenetworkonmost
naturalimages.Perturbinganewdatapointthenonlyin-
volvesthemereadditionoftheuniversalperturbationtothe
image(anddoesnotrequiresolvinganoptimizationprob-
lem/gradientcomputation).Finally,weemphasizethatour
notionofuniversalperturbationdiffersfromthegeneral-
izationofadversarialperturbationsstudiedin[19],where
perturbationscomputedontheMNISTtaskwereshownto
generalizewellacrossdifferentmodels.Instead,weexam-
inetheexistenceofuniversalperturbationsthatarecommon
tomostdatapointsbelongingtothedatadistribution.
2.Universalperturbations
Weformalizeinthissectionthenotionofuniversalper-
turbations,andproposeamethodforestimatingsuchper-
turbations.LetµdenoteadistributionofimagesinRd,and
kˆdefineaclassificationfunctionthatoutputsforeachim-
agex∈Rdanestimatedlabelkˆ(x).Themainfocusofthis
paperistoseekperturbationvectorsv∈Rdthatfoolthe
classifierkˆonalmostalldatapointssampledfromµ.That
is,weseekavectorvsuchthat
kˆ(x+v)=kˆ(x)for“most”x∼µ.
Wecoinsuchaperturbationuniversal,asitrepresentsa
fixedimage-agnosticperturbationthatcauseslabelchange
formostimagessampledfromthedatadistributionµ.We
focushereonthecasewherethedistributionµrepresents
thesetofnaturalimages,hencecontainingahugeamount
ofvariability.Inthatcontext,weexaminetheexistenceof
smalluniversalperturbations(intermsofthe`pnormwith
p∈[1,∞))thatmisclassifymostimages.Thegoalisthere-
foretofindvthatsatisfiesthefollowingtwoconstraints:
Therobustnessofimageclassifierstostructuredandun-
structuredperturbationshaverecentlyattractedalotofat-
tention[19,16,20,3,4,12,13,14].Despitetheimpressive
performanceofdeepneuralnetworkarchitecturesonchal-
lengingvisualclassificationbenchmarks[6,9,21,10],these
classifierswereshowntobehighlyvulnerabletoperturba-
tions.In[19],suchnetworksareshowntobeunstableto
verysmallandoftenimperceptibleadditiveadversarialper-
turbations.Suchcarefullycraftedperturbationsareeither
estimatedbysolvinganoptimizationproblem[19,11,1]
orthroughonestepofgradientascent[5],andresultina
perturbationthatfoolsaspecificdatapoint.Afundamental
propertyoftheseadversarialperturbationsistheirintrin-
sicdependenceondatapoints:theperturbationsarespecif-
icallycraftedforeachdatapointindependently.Asare-
sult,thecomputationofanadversarialperturbationfora
newdatapointrequiressolvingadata-dependentoptimiza-
tionproblemfromscratch,whichusesthefullknowledge
oftheclassificationmodel.Thisisdifferentfromtheuni-
versalperturbationconsideredinthispaper,asweseeka
1.kvkp≤ξ,
x∼µk(x+v)=kˆ(x)
2.Pˆ
≥1−δ.
Theparameterξcontrolsthemagnitudeoftheperturbation
vectorv,andδquantifiesthedesiredfoolingrateforall
imagessampledfromthedistributionµ.
Algorithm.LetX={x1,...,xm}beasetofimages
sampledfromthedistributionµ.Ourproposedalgorithm
seeksauniversalperturbationv,suchthatkvkp≤ξ,while
foolingmostdatapointsinX.Thealgorithmproceedsit-
erativelyoverthedatapointsinXandgraduallybuildsthe
universalperturbation,asillustratedinFig.2.Ateachiter-
ation,theminimalperturbation∆vithatsendsthecurrent
perturbedpoint,xi+v,tothedecisionboundaryoftheclas-
sifieriscomputed,andaggregatedtothecurrentinstance
oftheuniversalperturbation.Inmoredetails,providedthe
currentuniversalperturbationvdoesnotfooldatapointxi,
weseektheextraperturbation∆viwithminimalnormthat
allowstofooldatapointxibysolvingthefollowingopti-
Algorithm1Computationofuniversalperturbations.
∆v2
v
∆v1
x1,2,3
R3
R2
1:input:DatapointsX,classifierkˆ,desired`pnormof
theperturbationξ,desiredaccuracyonperturbedsam-
plesδ.
2:output:Universalperturbationvectorv.
3:Initializev←0.
4:whileErr(Xv)≤1−δdo
5:foreachdatapointxi∈Xdo
6:ifkˆ(xi+v)=kˆ(xi)then
7:Computetheminimalperturbationthat
sendsxi+vtothedecisionboundary:
R1
∆vi←argmrinkrk2s.t.kˆ(xi+v+r)=kˆ(xi).
2s.t.kˆ(xi+v+r)=kˆ(xi).
Figure2:Schematicrepresentationoftheproposedalgo-
rithmusedtocomputeuniversalperturbations.Inthisil-
lustration,datapointsx1,x2andx3aresuper-imposed,and
theclassificationregionsRi(i.e.,regionsofconstantesti-
matedlabel)areshownindifferentcolors.Ouralgorithm
proceedsbyaggregatingsequentiallytheminimalperturba-
tionssendingthecurrentperturbedpointsxi+voutsideof
thecorrespondingclassificationregionRi.
8:Updatetheperturbation:
v←Pp,ξ(v+∆vi).
9:endif
10:endfor
11:endwhile
mizationproblem:
∆vi←argmrinkrk2s.t.kˆ(xi+v+r)=kˆ(xi).(1)
2s.t.kˆ(xi+v+r)=kˆ(xi).(1)
Toensurethattheconstraintkvkp≤ξissatisfied,theup-
dateduniversalperturbationisfurtherprojectedonthe`p
ballofradiusξandcenteredat0.Thatis,letP
p,ξbethe
projectionoperatordefinedasfollows:
v0kv−v0k2subjecttokv0k
Pp≤ξ.
p,ξ(v)=argmin
Then,ourupdateruleisgivenbyv←P
p,ξ(v+∆vi).Sev-
eralpassesonthedatasetXareperformedtoimprovethe
qualityoftheuniversalperturbation.Thealgorithmister-
minatedwhentheempirical“foolingrate”ontheperturbed
datasetXv:={x1+v,...,xm+v}exceedsthetarget
threshold1−δ.Thatis,westopthealgorithmwhenever
Xm
1
Err(Xv):=
1kˆ(xi+v)=kˆ(xi)≥1−δ.
m
i=1
ThedetailedalgorithmisprovidedinAlgorithm1.Interest-
ingly,inpractice,thenumberofdatapointsminXneed
notbelargetocomputeauniversalperturbationthatisvalid
forthewholedistributionµ.Inparticular,wecansetm
tobemuchsmallerthanthenumberoftrainingpoints(see
Section3).
Theproposedalgorithminvolvessolvingatmostmin-
stancesoftheoptimizationprobleminEq.(1)foreachpass.
Whilethisoptimizationproblemisnotconvexwhenkˆisa
standardclassifier(e.g.,adeepneuralnetwork),severalef-
ficientapproximatemethodshavebeendevisedforsolving
thisproblem[19,11,7].Weuseinthefollowingtheap-
proachin[11]foritsefficency.Itshouldfurtherbenoticed
thattheobjectiveofAlgorithm1isnottofindthesmallest
universalperturbationthatfoolsmostdatapointssampled
fromthedistribution,butrathertofindonesuchperturba-
tionwithsufficientlysmallnorm.Inparticular,different
randomshufflingsofthesetXnaturallyleadtoadiverse
setofuniversalperturbationsvsatisfyingtherequiredcon-
straints.Theproposedalgorithmcanthereforebeleveraged
togeneratemultipleuniversalperturbationsforadeepneu-
ralnetwork(seenextsectionforvisualexamples).
3.Universalperturbationsfordeepnets
Wenowanalyzetherobustnessofstate-of-the-artdeep
neuralnetworkclassifierstouniversalperturbationsusing
Algorithm1.
Inafirstexperiment,weassesstheestimateduniversal
perturbationsfordifferentrecentdeepneuralnetworkson
theILSVRC2012[15]validationset(50,000images),and
reportthefoolingratio,thatistheproportionofimagesthat
changelabelswhenperturbedbyouruniversalperturbation.
Resultsarereportedforp=2andp=∞,wherewe
respectivelysetξ=2000andξ=10.Thesenumerical
valueswerechoseninordertoobtainaperturbationwhose
normissignificantlysmallerthantheimagenorms,such
thattheperturbationisquasi-imperceptiblewhenaddedto
`2
`∞
CaffeNet[8]VGG-F[2]VGG-16[17]VGG-19[17]GoogLeNet[18]ResNet-152[6]
X85.4%85.9%90.7%86.9%82.9%89.7%
Val.85.687.0%90.3%84.5%82.0%88.5%
X93.1%93.8%78.5%77.8%80.8%85.4%
Val.93.3%93.7%78.3%77.8%78.9%84.0%
Table1:FoolingratiosonthesetX,andthevalidationset.
naturalimages2.ResultsarelistedinTable1.Eachresult
isreportedonthesetX,whichisusedtocomputetheper-
turbation,aswellasonthevalidationset(thatisnotused
intheprocessofthecomputationoftheuniversalpertur-
bation).Observethatforallnetworks,theuniversalper-
turbationachievesveryhighfoolingratesonthevalidation
set.Specifically,theuniversalperturbationscomputedfor
CaffeNetandVGG-Ffoolmorethan90%ofthevalidation
set(forp=∞).Inotherwords,foranynaturalimagein
thevalidationset,themereadditionofouruniversalper-
turbationfoolstheclassifiermorethan9timesoutof10.
Thisresultismoreovernotspecifictosucharchitectures,
aswecanalsofinduniversalperturbationsthatcauseVGG,
GoogLeNetandResNetclassifierstobefooledonnatural
imageswithprobabilityedging80%.Theseresultshavean
elementofsurprise,astheyshowtheexistenceofsingle
universalperturbationvectorsthatcausenaturalimagesto
bemisclassifiedwithhighprobability,albeitbeingquasi-
imperceptibletohumans.Toverifythislatterclaim,we
showvisualexamplesofperturbedimagesinFig.3,where
theGoogLeNetarchitectureisused.Theseimagesareei-
thertakenfromtheILSVRC2012validationset,orcap-
turedusingamobilephonecamera.Observethatinmost
cases,theuniversalperturbationisquasi-imperceptible,yet
thispowerfulimage-agnosticperturbationisabletomis-
classifyanyimagewithhighprobabilityforstate-of-the-art
classifiers.Werefertothesupp.materialfortheoriginal
(unperturbed)images,aswellastheirgroundtruthlabels.
Wealsorefertothevideointhesupplementarymaterialfor
real-worldexamplesonasmartphone.Wevisualizetheuni-
versalperturbationscorrespondingtodifferentnetworksin
Fig.4.Itshouldbenotedthatsuchuniversalperturbations
arenotunique,asmanydifferentuniversalperturbations(all
satisfyingthetworequiredconstraints)canbegeneratedfor
thesamenetwork.InFig.5,wevisualizefivedifferent
universalperturbationsobtainedbyusingdifferentrandom
shufflingsinX.Observethatsuchuniversalperturbations
aredifferent,althoughtheyexhibitasimilarpattern.This
ismoreoverconfirmedbycomputingthenormalizedinner
productsbetweentwopairsofperturbationimages,asthe
normalizedinnerproductsdonotexceed0.1,whichshows
thatonecanfinddiverseuniversalperturbations.
2Forcomparison,theaverage`2and`∞normofanimageinthevali-
dationsetisrespectively≈5×104and≈250.
Whiletheaboveuniversalperturbationsarecomputed
forasetXof10,000imagesfromthetrainingset(i.e.,in
average10imagesperclass),wenowexaminetheinfluence
ofthesizeofXonthequalityoftheuniversalperturbation.
WeshowinFig.6thefoolingratesobtainedontheval-
idationsetfordifferentsizesofXforGoogLeNet.Note
forexamplethatwithasetXcontainingonly500images,
wecanfoolmorethan30%oftheimagesonthevalidation
set.Thisresultissignificantwhencomparedtothenum-
berofclassesinImageNet(1000),asitshowsthatwecan
foolalargesetofunseenimages,evenwhenusingaset
Xcontaininglessthanoneimageperclass!Theuniversal
perturbationscomputedusingAlgorithm1havethereforea
remarkablegeneralizationpoweroverunseendatapoints,
andcanbecomputedonaverysmallsetoftrainingimages.
Cross-modeluniversality.Whilethecomputedpertur-
bationsareuniversalacrossunseendatapoints,wenowex-
aminetheircross-modeluniversality.Thatis,westudyto
whichextentuniversalperturbationscomputedforaspe-
cificarchitecture(e.g.,VGG-19)arealsovalidforanother
architecture(e.g.,GoogLeNet).Table2displaysamatrix
summarizingtheuniversalityofsuchperturbationsacross
sixdifferentarchitectures.Foreacharchitecture,wecom-
puteauniversalperturbationandreportthefoolingratioson
allotherarchitectures;wereporttheseintherowsofTable
2.Observethat,forsomearchitectures,theuniversalpertur-
bationsgeneralizeverywellacrossotherarchitectures.For
example,universalperturbationscomputedfortheVGG-19
networkhaveafoolingratioabove53%forallothertested
architectures.Thisresultshowsthatouruniversalperturba-
tionsare,tosomeextent,doubly-universalastheygeneral-
izewellacrossdatapointsandverydifferentarchitectures.
Itshouldbenotedthat,in[19],adversarialperturbations
werepreviouslyshowntogeneralizewell,tosomeextent,
acrossdifferentneuralnetworksontheMNISTproblem.
Ourresultsarehoweverdifferent,asweshowthegeneral-
izabilityofuniversalperturbationsacrossdifferentarchitec-
turesontheImageNetdataset.Thisresultshowsthatsuch
perturbationsareofpracticalrelevance,astheygeneralize
wellacrossdatapointsandarchitectures.Inparticular,in
ordertofoolanewimageonanunknownneuralnetwork,a
simpleadditionofauniversalperturbationcomputedonthe
VGG-19architectureislikelytomisclassifythedatapoint.
woolIndianelephantIndianelephantAfricangreytabbyAfricangrey
commonnewtcarouselgreyfoxmacawthree-toedslothmacaw
Figure3:Examplesofperturbedimagesandtheircorrespondinglabels.Thefirst8imagesbelongtotheILSVRC2012
validationset,andthelast4areimagestakenbyamobilephonecamera.Seesupp.materialfortheoriginalimages.
(a)CaffeNet(b)VGG-F(c)VGG-16
(d)VGG-19(e)GoogLeNet(f)ResNet-152
Figure4:Universalperturbationscomputedfordifferentdeepneuralnetworkarchitectures.Imagesgeneratedwithp=∞,
ξ=10.Thepixelvaluesarescaledforvisibility.
Visualizationoftheeffectofuniversalperturbations.
Togaininsightsontheeffectofuniversalperturbationson
naturalimages,wenowvisualizethedistributionoflabels
ontheImageNetvalidationset.Specifically,webuildadi-
rectedgraphG=(V,E),whoseverticesdenotethelabels,
anddirectededgese=(i→j)indicatethatthemajority
ofimagesofclassiarefooledintolabeljwhenapplying
theuniversalperturbation.Theexistenceofedgesi→j
thereforesuggeststhatthepreferredfoolinglabelforim-
agesofclassiisj.WeconstructthisgraphforGoogLeNet,
andvisualizethefullgraphinthesupp.materialforspace
constraints.Thevisualizationofthisgraphshowsaverype-
culiartopology.Inparticular,thegraphisaunionofdisjoint
components,wherealledgesinonecomponentmostlycon-
necttoonetargetlabel.SeeFig.7foranillustrationoftwo
connectedcomponents.Thisvisualizationclearlyshowsthe
existenceofseveraldominantlabels,andthatuniversalper-
turbationsmostlymakenaturalimagesclassifiedwithsuch
Figure5:DiversityofuniversalperturbationsfortheGoogLeNetarchitecture.Thefiveperturbationsaregeneratedusing
differentrandomshufflingsofthesetX.Notethatthenormalizedinnerproductsforanypairofuniversalperturbationsdoes
notexceed0.1,whichhighlightsthediversityofsuchperturbations.
VGG-FCaffeNetGoogLeNetVGG-16VGG-19ResNet-152
VGG-F93.7%71.8%48.4%42.1%42.1%47.4%
CaffeNet74.0%93.3%47.7%39.9%39.9%48.0%
GoogLeNet46.2%43.8%78.9%39.2%39.8%45.5%
VGG-1663.4%55.8%56.5%78.3%73.1%63.4%
VGG-1964.0%57.2%53.6%73.5%77.8%58.0%
ResNet-15246.3%46.3%50.5%47.0%45.5%84.0%
Table2:Generalizabilityoftheuniversalperturbationsacrossdifferentnetworks.Thepercentagesindicatethefoolingrates.
Therowsindicatethearchitectureforwhichtheuniversalperturbationsiscomputed,andthecolumnsindicatethearchitecture
forwhichthefoolingrateisreported.
90
80
70
60
50
40
30
20
10
0
500100020004000
NumberofimagesinX
Figure6:Foolingratioonthevalidationsetversusthesize
ofX.Notethatevenwhentheuniversalperturbationis
computedonaverysmallsetX(comparedtotrainingand
validationsets),thefoolingratioonvalidationsetislarge.
images.WeusetheVGG-Farchitecture,andfine-tunethe
networkbasedonamodifiedtrainingsetwhereuniversal
perturbationsareaddedtoafractionof(clean)trainingsam-
ples:foreachtrainingpoint,auniversalperturbationis
addedwithprobability0.5,andtheoriginalsampleispre-
servedwithprobability0.5.3Toaccountforthediversity
ofuniversalperturbations,wepre-computeapoolof10dif-
ferentuniversalperturbationsandaddperturbationstothe
trainingsamplesrandomlyfromthispool.Thenetworkis
fine-tunedbyperforming5extraepochsoftrainingonthe
modifiedtrainingset.Toassesstheeffectoffine-tuningon
therobustnessofthenetwork,wecomputeanewuniversal
perturbationforthefine-tunednetwork(usingAlgorithm1,
withp=∞andξ=10),andreportthefoolingrateofthe
network.After5extraepochs,thefoolingrateonthevali-
dationsetis76.2%,whichshowsanimprovementwithre-
specttotheoriginalnetwork(93.7%,seeTable1).4Despite
thisimprovement,thefine-tunednetworkremainslargely
vulnerabletosmalluniversalperturbations.Wetherefore
labels.Wehypothesizethatthesedominantlabelsoccupy
largeregionsintheimagespace,andthereforerepresent
goodcandidatelabelsforfoolingmostnaturalimages.Note
thatthesedominantlabelsareautomaticallyfoundbyAlgo-
rithm1,andarenotimposedaprioriinthecomputationof
perturbations.
Fine-tuningwithuniversalperturbations.Wenowex-
aminetheeffectoffine-tuningthenetworkswithperturbed
3Inthisfine-tuningexperiment,weuseaslightlymodifiednotionof
universalperturbations,wherethedirectionoftheuniversalvectorvis
fixedforalldatapoints,whileitsmagnitudeisadaptive.Thatis,foreach
datapointx,weconsidertheperturbedpointx+αv,whereαisthesmall-
estcoefficientthatfoolstheclassifier.Weobservedthatthisfeedbacking
strategyislesspronetooverfittingthanthestrategywheretheuniversal
perturbationissimplyaddedtoalltrainingpoints.
4Thisfine-tuningproceduremoreoverledtoaminorincreaseinthe
errorrateonthevalidationset,whichmightbeduetoaslightoverfitting
oftheperturbeddata.
windowshade
leopardnematode
microwave
diningtable
cashmachine
television
sliderule
dowitcher
refrigerator
tray
mosquitonet
pillow
spaceshuttle
greatgreyowl
computerkeyboardplatypus
pencilbox
quilt
fountain
wardrobe
platerack
digitalclock
Arcticfox
envelope
medicinechest
Figure7:TwoconnectedcomponentsofthegraphG=(V,E),wheretheverticesarethesetoflabels,anddirectededges
i→jindicatethatmostimagesofclassiarefooledintoclassj.
repeatedtheaboveprocedure(i.e.,computationofapoolof
10universalperturbationsforthefine-tunednetwork,fine-
tuningofthenewnetworkbasedonthemodifiedtraining
setfor5extraepochs),andweobtainedanewfoolingra-
tioof80.0%.Ingeneral,therepetitionofthisprocedure
forafixednumberoftimesdidnotyieldanyimprovement
overthe76.2%foolingratioobtainedafteronestepoffine-
tuning.Hence,whilefine-tuningthenetworkleadstoamild
improvementintherobustness,weobservedthatthissim-
plesolutiondoesnotfullyimmuneagainstsmalluniversal
perturbations.
4.Explainingthevulnerabilitytouniversal
perturbations
Thegoalofthissectionistoanalyzeandexplainthehigh
vulnerabilityofdeepneuralnetworkclassifierstouniver-
salperturbations.Tounderstandtheuniquecharacteristics
ofuniversalperturbations,wefirstcomparesuchperturba-
tionswithothertypesofperturbations,namelyi)random
perturbation,ii)adversarialperturbationcomputedfora
randomlypickedsample(computedusingtheDFandFGS
methodsrespectivelyin[11]and[5]),iii)sumofadversar-
ialperturbationsoverX,andiv)meanoftheimages(or
ImageNetbias).Foreachperturbation,wedepictaphase
transitiongraphinFig.8showingthefoolingrateonthe
validationsetwithrespecttothe`2normoftheperturba-
tion.Differentperturbationnormsareachievedbyscaling
accordinglyeachperturbationwithamultiplicativefactorto
havethetargetnorm.Notethattheuniversalperturbationis
computedforξ=2000,andalsoscaledaccordingly.
Observethattheproposeduniversalperturbationquickly
reachesveryhighfoolingrates,evenwhentheperturbation
isconstrainedtobeofsmallnorm.Forexample,theuni-
versalperturbationcomputedusingAlgorithm1achieves
afoolingrateof85%whenthe`2normisconstrainedto
ξ=2000,whileotherperturbationsachievemuchsmaller
ratiosforcomparablenorms.Inparticular,randomvec-
torssampleduniformlyfromthesphereofradiusof2000
onlyfool10%ofthevalidationset.Thelargedifference
betweenuniversalandrandomperturbationssuggeststhat
theuniversalperturbationexploitssomegeometriccorrela-
tionsbetweendifferentpartsofthedecisionboundaryofthe
classifier.Infact,iftheorientationsofthedecisionbound-
aryintheneighborhoodofdifferentdatapointswerecom-
pletelyuncorrelated(andindependentofthedistancetothe
decisionboundary),thenormofthebestuniversalperturba-
tionwouldbecomparabletothatofarandomperturbation.
Notethatthelatterquantityiswellunderstood(see[4]),
asthenormoftherandomperturbationrequiredtofoola
specificdatapointpreciselybehavesasΘ(√dk
rk
2),where
disthedimensionoftheinputspace,andkrk
2isthedis-
tancebetweenthedatapointandthedecisionboundary(or
equivalently,thenormofthesmallestadversarialperturba-
tion).FortheconsideredImageNetclassificationtask,this
quantityisequalto√dk
rk2≈2×10
4,formostdatapoints,
whichisatleastoneorderofmagnitudelargerthantheuni-
versalperturbation(ξ=2000).Thissubstantialdifference
betweenrandomanduniversalperturbationstherebysug-
gestsredundanciesinthegeometryofthedecisionbound-
ariesthatwenowexplore.
Foreachimagexinthevalidationset,wecom-
putetheadversarialperturbationvectorr(x)=
argminrkrk
2s.t.kˆ(x+r)=kˆ(x).Itiseasytosee
thatr(x)isnormaltothedecisionboundaryoftheclas-
sifier(atx+r(x)).Thevectorr(x)hencecapturesthe
localgeometryofthedecisionboundaryintheregion
surroundingthedatapointx.Toquantify
温馨提示
- 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
- 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
- 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
- 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
- 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
- 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
- 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
最新文档
- 2027届黑龙江省牡丹江市高三第四次模拟考试高考物理试卷(含答案解析)
- 十五五智能人体感应传感器:赋能智慧农业非接触式监控实践
- 前沿融合:区块链赋能无人机编队表演版权确权与NFT交易
- 中东市场:智能CO2监测器助力石油炼化满足高标合规新需求
- 双碳约束下:智能护肩枕绿色制造如何成为企业ESG新名片
- 中国医药在股市中的发展潜力与风险剖析
- 一点就到家观后感心得
- 2026年儿童主题公园行业报告:多孩家庭游玩需求与市场前景
- 2026年医疗健康行业患者信用评估与反欺诈研究报告
- 2026年钢铁行业市场供需分析报告
- 北京科技大学辅导员招录考试真题与答案2026
- 2026年环保中级工程师模拟试题(含答案)
- 2026-2030年中国出租车行业发展趋势及发展前景研究报告
- 2023年自考03709马克思主义基本原理概论重点笔记
- 2027届高三语文10月名校模拟试题汇编之修改病句题(学生版+教师版)
- 糖尿病的中医护理知识培训汇报
- 2026中国新能源汽车保险产品创新及理赔风险控制与大数据定价模型报告
- 耳鼻喉科临床诊疗指南(2025版)
- 2026年中国电信笔试题库
- 小学四年级英语沪教牛津版上册期末综合复习教案
- (2026年)米格列醇与阿卡波糖比较分析课件
评论
0/150
提交评论