人工智能机器学习Universal adversarial perturbations_第1页
人工智能机器学习Universal adversarial perturbations_第2页
人工智能机器学习Universal adversarial perturbations_第3页
人工智能机器学习Universal adversarial perturbations_第4页
人工智能机器学习Universal adversarial perturbations_第5页
已阅读5页,还剩22页未读, 继续免费阅读

付费下载

下载本文档

版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领

文档简介

Universaladversarialperturbations

Seyed-MohsenMoosavi-Dezfooli∗†AlhusseinFawzi∗†

seyed.moosavi@epfl.chalhussein.fawzi@epfl.ch

OmarFawzi‡PascalFrossard†

omar.fawzi@ens-lyon.frpascal.frossard@epfl.ch

Abstract

Chihuahua

Givenastate-of-the-artdeepneuralnetworkclassifier,

weshowtheexistenceofauniversal(image-agnostic)and

verysmallperturbationvectorthatcausesnaturalimages

tobemisclassifiedwithhighprobability.Weproposeasys-

tematicalgorithmforcomputinguniversalperturbations,

andshowthatstate-of-the-artdeepneuralnetworksare

highlyvulnerabletosuchperturbations,albeitbeingquasi-

imperceptibletothehumaneye.Wefurtherempiricallyan-

alyzetheseuniversalperturbationsandshow,inparticular,

thattheygeneralizeverywellacrossneuralnetworks.The

surprisingexistenceofuniversalperturbationsrevealsim-

portantgeometriccorrelationsamongthehigh-dimensional

decisionboundaryofclassifiers.Itfurtheroutlinespoten-

tialsecuritybreacheswiththeexistenceofsingledirections

intheinputspacethatadversariescanpossiblyexploitto

breakaclassifieronmostnaturalimages.1

Joystick

Thresher

Tibetanmasti

Chihuahua

Jay

Labrador

Labrador

Tibetanmasti

1.Introduction

LycaenidBrabancongrion

Canwefindasinglesmallimageperturbationthatfools

astate-of-the-artdeepneuralnetworkclassifieronallnat-

uralimages?Weshowinthispapertheexistenceofsuch

quasi-imperceptibleuniversalperturbationvectorsthatlead

tomisclassifynaturalimageswithhighprobability.Specif-

ically,byaddingsuchaquasi-imperceptibleperturbation

tonaturalimages,thelabelestimatedbythedeepneu-

ralnetworkischangedwithhighprobability(seeFig.1).

Suchperturbationsaredubbeduniversal,astheyareimage-

agnostic.Theexistenceoftheseperturbationsisproblem-

aticwhentheclassifierisdeployedinreal-world(andpos-

siblyhostile)environments,astheycanbeexploitedbyad-

Balloon

Labrador

Borderterrier

Figure1:Whenaddedtoanaturalimage,auniversalper-

turbationimagecausestheimagetobemisclassifiedbythe

deepneuralnetworkwithhighprobability.Leftimages:

Originalnaturalimages.Thelabelsareshownontopof

eacharrow.Centralimage:Universalperturbation.Right

images:Perturbedimages.Theestimatedlabelsoftheper-

turbedimagesareshownontopofeacharrow.

∗Thefirsttwoauthorscontributedequallytothiswork.

†E´colePolytechniqueFe´de´raledeLausanne,Switzerland

‡ENSdeLyon,LIP,UMR5668ENSLyon-CNRS-UCBL-INRIA,

Universite´deLyon,France

1Toencouragereproducibleresearch,thecodeisavailableatgitHub.

Furthermore,avideodemonstratingtheeffectofuniversalperturbations

onasmartphonecanbefoundhere.

1

versariestobreaktheclassifier.Indeed,theperturbation

processinvolvesthemereadditionofoneverysmallpertur-

bationtoallnaturalimages,andcanberelativelystraight-

forwardtoimplementbyadversariesinreal-worldenviron-

ments,whilebeingrelativelydifficulttodetectassuchper-

turbationsareverysmallandthusdonotsignificantlyaffect

datadistributions.Thesurprisingexistenceofuniversalper-

turbationsfurtherrevealsnewinsightsonthetopologyof

thedecisionboundariesofdeepneuralnetworks.Wesum-

marizethemaincontributionsofthispaperasfollows:

•Weshowtheexistenceofuniversalimage-agnostic

perturbationsforstate-of-the-artdeepneuralnetworks.

•Weproposeanalgorithmforfindingsuchperturba-

tions.Thealgorithmseeksauniversalperturbationfor

asetoftrainingpoints,andproceedsbyaggregating

atomicperturbationvectorsthatsendsuccessivedata-

pointstothedecisionboundaryoftheclassifier.

•Weshowthatuniversalperturbationshavearemark-

ablegeneralizationproperty,asperturbationscom-

putedforarathersmallsetoftrainingpointsfoolnew

imageswithhighprobability.

•Weshowthatsuchperturbationsarenotonlyuniver-

salacrossimages,butalsogeneralizewellacrossdeep

neuralnetworks.Suchperturbationsarethereforedou-

blyuniversal,bothwithrespecttothedataandthenet-

workarchitectures.

•Weexplainandanalyzethehighvulnerabilityofdeep

neuralnetworkstouniversalperturbationsbyexamin-

ingthegeometriccorrelationbetweendifferentparts

ofthedecisionboundary.

singleperturbationvectorthatfoolsthenetworkonmost

naturalimages.Perturbinganewdatapointthenonlyin-

volvesthemereadditionoftheuniversalperturbationtothe

image(anddoesnotrequiresolvinganoptimizationprob-

lem/gradientcomputation).Finally,weemphasizethatour

notionofuniversalperturbationdiffersfromthegeneral-

izationofadversarialperturbationsstudiedin[19],where

perturbationscomputedontheMNISTtaskwereshownto

generalizewellacrossdifferentmodels.Instead,weexam-

inetheexistenceofuniversalperturbationsthatarecommon

tomostdatapointsbelongingtothedatadistribution.

2.Universalperturbations

Weformalizeinthissectionthenotionofuniversalper-

turbations,andproposeamethodforestimatingsuchper-

turbations.LetµdenoteadistributionofimagesinRd,and

kˆdefineaclassificationfunctionthatoutputsforeachim-

agex∈Rdanestimatedlabelkˆ(x).Themainfocusofthis

paperistoseekperturbationvectorsv∈Rdthatfoolthe

classifierkˆonalmostalldatapointssampledfromµ.That

is,weseekavectorvsuchthat

kˆ(x+v)=kˆ(x)for“most”x∼µ.

Wecoinsuchaperturbationuniversal,asitrepresentsa

fixedimage-agnosticperturbationthatcauseslabelchange

formostimagessampledfromthedatadistributionµ.We

focushereonthecasewherethedistributionµrepresents

thesetofnaturalimages,hencecontainingahugeamount

ofvariability.Inthatcontext,weexaminetheexistenceof

smalluniversalperturbations(intermsofthe`pnormwith

p∈[1,∞))thatmisclassifymostimages.Thegoalisthere-

foretofindvthatsatisfiesthefollowingtwoconstraints:

Therobustnessofimageclassifierstostructuredandun-

structuredperturbationshaverecentlyattractedalotofat-

tention[19,16,20,3,4,12,13,14].Despitetheimpressive

performanceofdeepneuralnetworkarchitecturesonchal-

lengingvisualclassificationbenchmarks[6,9,21,10],these

classifierswereshowntobehighlyvulnerabletoperturba-

tions.In[19],suchnetworksareshowntobeunstableto

verysmallandoftenimperceptibleadditiveadversarialper-

turbations.Suchcarefullycraftedperturbationsareeither

estimatedbysolvinganoptimizationproblem[19,11,1]

orthroughonestepofgradientascent[5],andresultina

perturbationthatfoolsaspecificdatapoint.Afundamental

propertyoftheseadversarialperturbationsistheirintrin-

sicdependenceondatapoints:theperturbationsarespecif-

icallycraftedforeachdatapointindependently.Asare-

sult,thecomputationofanadversarialperturbationfora

newdatapointrequiressolvingadata-dependentoptimiza-

tionproblemfromscratch,whichusesthefullknowledge

oftheclassificationmodel.Thisisdifferentfromtheuni-

versalperturbationconsideredinthispaper,asweseeka

1.kvkp≤ξ,

x∼µk(x+v)=kˆ(x)

2.Pˆ

≥1−δ.

Theparameterξcontrolsthemagnitudeoftheperturbation

vectorv,andδquantifiesthedesiredfoolingrateforall

imagessampledfromthedistributionµ.

Algorithm.LetX={x1,...,xm}beasetofimages

sampledfromthedistributionµ.Ourproposedalgorithm

seeksauniversalperturbationv,suchthatkvkp≤ξ,while

foolingmostdatapointsinX.Thealgorithmproceedsit-

erativelyoverthedatapointsinXandgraduallybuildsthe

universalperturbation,asillustratedinFig.2.Ateachiter-

ation,theminimalperturbation∆vithatsendsthecurrent

perturbedpoint,xi+v,tothedecisionboundaryoftheclas-

sifieriscomputed,andaggregatedtothecurrentinstance

oftheuniversalperturbation.Inmoredetails,providedthe

currentuniversalperturbationvdoesnotfooldatapointxi,

weseektheextraperturbation∆viwithminimalnormthat

allowstofooldatapointxibysolvingthefollowingopti-

Algorithm1Computationofuniversalperturbations.

∆v2

v

∆v1

x1,2,3

R3

R2

1:input:DatapointsX,classifierkˆ,desired`pnormof

theperturbationξ,desiredaccuracyonperturbedsam-

plesδ.

2:output:Universalperturbationvectorv.

3:Initializev←0.

4:whileErr(Xv)≤1−δdo

5:foreachdatapointxi∈Xdo

6:ifkˆ(xi+v)=kˆ(xi)then

7:Computetheminimalperturbationthat

sendsxi+vtothedecisionboundary:

R1

∆vi←argmrinkrk2s.t.kˆ(xi+v+r)=kˆ(xi).

2s.t.kˆ(xi+v+r)=kˆ(xi).

Figure2:Schematicrepresentationoftheproposedalgo-

rithmusedtocomputeuniversalperturbations.Inthisil-

lustration,datapointsx1,x2andx3aresuper-imposed,and

theclassificationregionsRi(i.e.,regionsofconstantesti-

matedlabel)areshownindifferentcolors.Ouralgorithm

proceedsbyaggregatingsequentiallytheminimalperturba-

tionssendingthecurrentperturbedpointsxi+voutsideof

thecorrespondingclassificationregionRi.

8:Updatetheperturbation:

v←Pp,ξ(v+∆vi).

9:endif

10:endfor

11:endwhile

mizationproblem:

∆vi←argmrinkrk2s.t.kˆ(xi+v+r)=kˆ(xi).(1)

2s.t.kˆ(xi+v+r)=kˆ(xi).(1)

Toensurethattheconstraintkvkp≤ξissatisfied,theup-

dateduniversalperturbationisfurtherprojectedonthe`p

ballofradiusξandcenteredat0.Thatis,letP

p,ξbethe

projectionoperatordefinedasfollows:

v0kv−v0k2subjecttokv0k

Pp≤ξ.

p,ξ(v)=argmin

Then,ourupdateruleisgivenbyv←P

p,ξ(v+∆vi).Sev-

eralpassesonthedatasetXareperformedtoimprovethe

qualityoftheuniversalperturbation.Thealgorithmister-

minatedwhentheempirical“foolingrate”ontheperturbed

datasetXv:={x1+v,...,xm+v}exceedsthetarget

threshold1−δ.Thatis,westopthealgorithmwhenever

Xm

1

Err(Xv):=

1kˆ(xi+v)=kˆ(xi)≥1−δ.

m

i=1

ThedetailedalgorithmisprovidedinAlgorithm1.Interest-

ingly,inpractice,thenumberofdatapointsminXneed

notbelargetocomputeauniversalperturbationthatisvalid

forthewholedistributionµ.Inparticular,wecansetm

tobemuchsmallerthanthenumberoftrainingpoints(see

Section3).

Theproposedalgorithminvolvessolvingatmostmin-

stancesoftheoptimizationprobleminEq.(1)foreachpass.

Whilethisoptimizationproblemisnotconvexwhenkˆisa

standardclassifier(e.g.,adeepneuralnetwork),severalef-

ficientapproximatemethodshavebeendevisedforsolving

thisproblem[19,11,7].Weuseinthefollowingtheap-

proachin[11]foritsefficency.Itshouldfurtherbenoticed

thattheobjectiveofAlgorithm1isnottofindthesmallest

universalperturbationthatfoolsmostdatapointssampled

fromthedistribution,butrathertofindonesuchperturba-

tionwithsufficientlysmallnorm.Inparticular,different

randomshufflingsofthesetXnaturallyleadtoadiverse

setofuniversalperturbationsvsatisfyingtherequiredcon-

straints.Theproposedalgorithmcanthereforebeleveraged

togeneratemultipleuniversalperturbationsforadeepneu-

ralnetwork(seenextsectionforvisualexamples).

3.Universalperturbationsfordeepnets

Wenowanalyzetherobustnessofstate-of-the-artdeep

neuralnetworkclassifierstouniversalperturbationsusing

Algorithm1.

Inafirstexperiment,weassesstheestimateduniversal

perturbationsfordifferentrecentdeepneuralnetworkson

theILSVRC2012[15]validationset(50,000images),and

reportthefoolingratio,thatistheproportionofimagesthat

changelabelswhenperturbedbyouruniversalperturbation.

Resultsarereportedforp=2andp=∞,wherewe

respectivelysetξ=2000andξ=10.Thesenumerical

valueswerechoseninordertoobtainaperturbationwhose

normissignificantlysmallerthantheimagenorms,such

thattheperturbationisquasi-imperceptiblewhenaddedto

`2

`∞

CaffeNet[8]VGG-F[2]VGG-16[17]VGG-19[17]GoogLeNet[18]ResNet-152[6]

X85.4%85.9%90.7%86.9%82.9%89.7%

Val.85.687.0%90.3%84.5%82.0%88.5%

X93.1%93.8%78.5%77.8%80.8%85.4%

Val.93.3%93.7%78.3%77.8%78.9%84.0%

Table1:FoolingratiosonthesetX,andthevalidationset.

naturalimages2.ResultsarelistedinTable1.Eachresult

isreportedonthesetX,whichisusedtocomputetheper-

turbation,aswellasonthevalidationset(thatisnotused

intheprocessofthecomputationoftheuniversalpertur-

bation).Observethatforallnetworks,theuniversalper-

turbationachievesveryhighfoolingratesonthevalidation

set.Specifically,theuniversalperturbationscomputedfor

CaffeNetandVGG-Ffoolmorethan90%ofthevalidation

set(forp=∞).Inotherwords,foranynaturalimagein

thevalidationset,themereadditionofouruniversalper-

turbationfoolstheclassifiermorethan9timesoutof10.

Thisresultismoreovernotspecifictosucharchitectures,

aswecanalsofinduniversalperturbationsthatcauseVGG,

GoogLeNetandResNetclassifierstobefooledonnatural

imageswithprobabilityedging80%.Theseresultshavean

elementofsurprise,astheyshowtheexistenceofsingle

universalperturbationvectorsthatcausenaturalimagesto

bemisclassifiedwithhighprobability,albeitbeingquasi-

imperceptibletohumans.Toverifythislatterclaim,we

showvisualexamplesofperturbedimagesinFig.3,where

theGoogLeNetarchitectureisused.Theseimagesareei-

thertakenfromtheILSVRC2012validationset,orcap-

turedusingamobilephonecamera.Observethatinmost

cases,theuniversalperturbationisquasi-imperceptible,yet

thispowerfulimage-agnosticperturbationisabletomis-

classifyanyimagewithhighprobabilityforstate-of-the-art

classifiers.Werefertothesupp.materialfortheoriginal

(unperturbed)images,aswellastheirgroundtruthlabels.

Wealsorefertothevideointhesupplementarymaterialfor

real-worldexamplesonasmartphone.Wevisualizetheuni-

versalperturbationscorrespondingtodifferentnetworksin

Fig.4.Itshouldbenotedthatsuchuniversalperturbations

arenotunique,asmanydifferentuniversalperturbations(all

satisfyingthetworequiredconstraints)canbegeneratedfor

thesamenetwork.InFig.5,wevisualizefivedifferent

universalperturbationsobtainedbyusingdifferentrandom

shufflingsinX.Observethatsuchuniversalperturbations

aredifferent,althoughtheyexhibitasimilarpattern.This

ismoreoverconfirmedbycomputingthenormalizedinner

productsbetweentwopairsofperturbationimages,asthe

normalizedinnerproductsdonotexceed0.1,whichshows

thatonecanfinddiverseuniversalperturbations.

2Forcomparison,theaverage`2and`∞normofanimageinthevali-

dationsetisrespectively≈5×104and≈250.

Whiletheaboveuniversalperturbationsarecomputed

forasetXof10,000imagesfromthetrainingset(i.e.,in

average10imagesperclass),wenowexaminetheinfluence

ofthesizeofXonthequalityoftheuniversalperturbation.

WeshowinFig.6thefoolingratesobtainedontheval-

idationsetfordifferentsizesofXforGoogLeNet.Note

forexamplethatwithasetXcontainingonly500images,

wecanfoolmorethan30%oftheimagesonthevalidation

set.Thisresultissignificantwhencomparedtothenum-

berofclassesinImageNet(1000),asitshowsthatwecan

foolalargesetofunseenimages,evenwhenusingaset

Xcontaininglessthanoneimageperclass!Theuniversal

perturbationscomputedusingAlgorithm1havethereforea

remarkablegeneralizationpoweroverunseendatapoints,

andcanbecomputedonaverysmallsetoftrainingimages.

Cross-modeluniversality.Whilethecomputedpertur-

bationsareuniversalacrossunseendatapoints,wenowex-

aminetheircross-modeluniversality.Thatis,westudyto

whichextentuniversalperturbationscomputedforaspe-

cificarchitecture(e.g.,VGG-19)arealsovalidforanother

architecture(e.g.,GoogLeNet).Table2displaysamatrix

summarizingtheuniversalityofsuchperturbationsacross

sixdifferentarchitectures.Foreacharchitecture,wecom-

puteauniversalperturbationandreportthefoolingratioson

allotherarchitectures;wereporttheseintherowsofTable

2.Observethat,forsomearchitectures,theuniversalpertur-

bationsgeneralizeverywellacrossotherarchitectures.For

example,universalperturbationscomputedfortheVGG-19

networkhaveafoolingratioabove53%forallothertested

architectures.Thisresultshowsthatouruniversalperturba-

tionsare,tosomeextent,doubly-universalastheygeneral-

izewellacrossdatapointsandverydifferentarchitectures.

Itshouldbenotedthat,in[19],adversarialperturbations

werepreviouslyshowntogeneralizewell,tosomeextent,

acrossdifferentneuralnetworksontheMNISTproblem.

Ourresultsarehoweverdifferent,asweshowthegeneral-

izabilityofuniversalperturbationsacrossdifferentarchitec-

turesontheImageNetdataset.Thisresultshowsthatsuch

perturbationsareofpracticalrelevance,astheygeneralize

wellacrossdatapointsandarchitectures.Inparticular,in

ordertofoolanewimageonanunknownneuralnetwork,a

simpleadditionofauniversalperturbationcomputedonthe

VGG-19architectureislikelytomisclassifythedatapoint.

woolIndianelephantIndianelephantAfricangreytabbyAfricangrey

commonnewtcarouselgreyfoxmacawthree-toedslothmacaw

Figure3:Examplesofperturbedimagesandtheircorrespondinglabels.Thefirst8imagesbelongtotheILSVRC2012

validationset,andthelast4areimagestakenbyamobilephonecamera.Seesupp.materialfortheoriginalimages.

(a)CaffeNet(b)VGG-F(c)VGG-16

(d)VGG-19(e)GoogLeNet(f)ResNet-152

Figure4:Universalperturbationscomputedfordifferentdeepneuralnetworkarchitectures.Imagesgeneratedwithp=∞,

ξ=10.Thepixelvaluesarescaledforvisibility.

Visualizationoftheeffectofuniversalperturbations.

Togaininsightsontheeffectofuniversalperturbationson

naturalimages,wenowvisualizethedistributionoflabels

ontheImageNetvalidationset.Specifically,webuildadi-

rectedgraphG=(V,E),whoseverticesdenotethelabels,

anddirectededgese=(i→j)indicatethatthemajority

ofimagesofclassiarefooledintolabeljwhenapplying

theuniversalperturbation.Theexistenceofedgesi→j

thereforesuggeststhatthepreferredfoolinglabelforim-

agesofclassiisj.WeconstructthisgraphforGoogLeNet,

andvisualizethefullgraphinthesupp.materialforspace

constraints.Thevisualizationofthisgraphshowsaverype-

culiartopology.Inparticular,thegraphisaunionofdisjoint

components,wherealledgesinonecomponentmostlycon-

necttoonetargetlabel.SeeFig.7foranillustrationoftwo

connectedcomponents.Thisvisualizationclearlyshowsthe

existenceofseveraldominantlabels,andthatuniversalper-

turbationsmostlymakenaturalimagesclassifiedwithsuch

Figure5:DiversityofuniversalperturbationsfortheGoogLeNetarchitecture.Thefiveperturbationsaregeneratedusing

differentrandomshufflingsofthesetX.Notethatthenormalizedinnerproductsforanypairofuniversalperturbationsdoes

notexceed0.1,whichhighlightsthediversityofsuchperturbations.

VGG-FCaffeNetGoogLeNetVGG-16VGG-19ResNet-152

VGG-F93.7%71.8%48.4%42.1%42.1%47.4%

CaffeNet74.0%93.3%47.7%39.9%39.9%48.0%

GoogLeNet46.2%43.8%78.9%39.2%39.8%45.5%

VGG-1663.4%55.8%56.5%78.3%73.1%63.4%

VGG-1964.0%57.2%53.6%73.5%77.8%58.0%

ResNet-15246.3%46.3%50.5%47.0%45.5%84.0%

Table2:Generalizabilityoftheuniversalperturbationsacrossdifferentnetworks.Thepercentagesindicatethefoolingrates.

Therowsindicatethearchitectureforwhichtheuniversalperturbationsiscomputed,andthecolumnsindicatethearchitecture

forwhichthefoolingrateisreported.

90

80

70

60

50

40

30

20

10

0

500100020004000

NumberofimagesinX

Figure6:Foolingratioonthevalidationsetversusthesize

ofX.Notethatevenwhentheuniversalperturbationis

computedonaverysmallsetX(comparedtotrainingand

validationsets),thefoolingratioonvalidationsetislarge.

images.WeusetheVGG-Farchitecture,andfine-tunethe

networkbasedonamodifiedtrainingsetwhereuniversal

perturbationsareaddedtoafractionof(clean)trainingsam-

ples:foreachtrainingpoint,auniversalperturbationis

addedwithprobability0.5,andtheoriginalsampleispre-

servedwithprobability0.5.3Toaccountforthediversity

ofuniversalperturbations,wepre-computeapoolof10dif-

ferentuniversalperturbationsandaddperturbationstothe

trainingsamplesrandomlyfromthispool.Thenetworkis

fine-tunedbyperforming5extraepochsoftrainingonthe

modifiedtrainingset.Toassesstheeffectoffine-tuningon

therobustnessofthenetwork,wecomputeanewuniversal

perturbationforthefine-tunednetwork(usingAlgorithm1,

withp=∞andξ=10),andreportthefoolingrateofthe

network.After5extraepochs,thefoolingrateonthevali-

dationsetis76.2%,whichshowsanimprovementwithre-

specttotheoriginalnetwork(93.7%,seeTable1).4Despite

thisimprovement,thefine-tunednetworkremainslargely

vulnerabletosmalluniversalperturbations.Wetherefore

labels.Wehypothesizethatthesedominantlabelsoccupy

largeregionsintheimagespace,andthereforerepresent

goodcandidatelabelsforfoolingmostnaturalimages.Note

thatthesedominantlabelsareautomaticallyfoundbyAlgo-

rithm1,andarenotimposedaprioriinthecomputationof

perturbations.

Fine-tuningwithuniversalperturbations.Wenowex-

aminetheeffectoffine-tuningthenetworkswithperturbed

3Inthisfine-tuningexperiment,weuseaslightlymodifiednotionof

universalperturbations,wherethedirectionoftheuniversalvectorvis

fixedforalldatapoints,whileitsmagnitudeisadaptive.Thatis,foreach

datapointx,weconsidertheperturbedpointx+αv,whereαisthesmall-

estcoefficientthatfoolstheclassifier.Weobservedthatthisfeedbacking

strategyislesspronetooverfittingthanthestrategywheretheuniversal

perturbationissimplyaddedtoalltrainingpoints.

4Thisfine-tuningproceduremoreoverledtoaminorincreaseinthe

errorrateonthevalidationset,whichmightbeduetoaslightoverfitting

oftheperturbeddata.

windowshade

leopardnematode

microwave

diningtable

cashmachine

television

sliderule

dowitcher

refrigerator

tray

mosquitonet

pillow

spaceshuttle

greatgreyowl

computerkeyboardplatypus

pencilbox

quilt

fountain

wardrobe

platerack

digitalclock

Arcticfox

envelope

medicinechest

Figure7:TwoconnectedcomponentsofthegraphG=(V,E),wheretheverticesarethesetoflabels,anddirectededges

i→jindicatethatmostimagesofclassiarefooledintoclassj.

repeatedtheaboveprocedure(i.e.,computationofapoolof

10universalperturbationsforthefine-tunednetwork,fine-

tuningofthenewnetworkbasedonthemodifiedtraining

setfor5extraepochs),andweobtainedanewfoolingra-

tioof80.0%.Ingeneral,therepetitionofthisprocedure

forafixednumberoftimesdidnotyieldanyimprovement

overthe76.2%foolingratioobtainedafteronestepoffine-

tuning.Hence,whilefine-tuningthenetworkleadstoamild

improvementintherobustness,weobservedthatthissim-

plesolutiondoesnotfullyimmuneagainstsmalluniversal

perturbations.

4.Explainingthevulnerabilitytouniversal

perturbations

Thegoalofthissectionistoanalyzeandexplainthehigh

vulnerabilityofdeepneuralnetworkclassifierstouniver-

salperturbations.Tounderstandtheuniquecharacteristics

ofuniversalperturbations,wefirstcomparesuchperturba-

tionswithothertypesofperturbations,namelyi)random

perturbation,ii)adversarialperturbationcomputedfora

randomlypickedsample(computedusingtheDFandFGS

methodsrespectivelyin[11]and[5]),iii)sumofadversar-

ialperturbationsoverX,andiv)meanoftheimages(or

ImageNetbias).Foreachperturbation,wedepictaphase

transitiongraphinFig.8showingthefoolingrateonthe

validationsetwithrespecttothe`2normoftheperturba-

tion.Differentperturbationnormsareachievedbyscaling

accordinglyeachperturbationwithamultiplicativefactorto

havethetargetnorm.Notethattheuniversalperturbationis

computedforξ=2000,andalsoscaledaccordingly.

Observethattheproposeduniversalperturbationquickly

reachesveryhighfoolingrates,evenwhentheperturbation

isconstrainedtobeofsmallnorm.Forexample,theuni-

versalperturbationcomputedusingAlgorithm1achieves

afoolingrateof85%whenthe`2normisconstrainedto

ξ=2000,whileotherperturbationsachievemuchsmaller

ratiosforcomparablenorms.Inparticular,randomvec-

torssampleduniformlyfromthesphereofradiusof2000

onlyfool10%ofthevalidationset.Thelargedifference

betweenuniversalandrandomperturbationssuggeststhat

theuniversalperturbationexploitssomegeometriccorrela-

tionsbetweendifferentpartsofthedecisionboundaryofthe

classifier.Infact,iftheorientationsofthedecisionbound-

aryintheneighborhoodofdifferentdatapointswerecom-

pletelyuncorrelated(andindependentofthedistancetothe

decisionboundary),thenormofthebestuniversalperturba-

tionwouldbecomparabletothatofarandomperturbation.

Notethatthelatterquantityiswellunderstood(see[4]),

asthenormoftherandomperturbationrequiredtofoola

specificdatapointpreciselybehavesasΘ(√dk

rk

2),where

disthedimensionoftheinputspace,andkrk

2isthedis-

tancebetweenthedatapointandthedecisionboundary(or

equivalently,thenormofthesmallestadversarialperturba-

tion).FortheconsideredImageNetclassificationtask,this

quantityisequalto√dk

rk2≈2×10

4,formostdatapoints,

whichisatleastoneorderofmagnitudelargerthantheuni-

versalperturbation(ξ=2000).Thissubstantialdifference

betweenrandomanduniversalperturbationstherebysug-

gestsredundanciesinthegeometryofthedecisionbound-

ariesthatwenowexplore.

Foreachimagexinthevalidationset,wecom-

putetheadversarialperturbationvectorr(x)=

argminrkrk

2s.t.kˆ(x+r)=kˆ(x).Itiseasytosee

thatr(x)isnormaltothedecisionboundaryoftheclas-

sifier(atx+r(x)).Thevectorr(x)hencecapturesthe

localgeometryofthedecisionboundaryintheregion

surroundingthedatapointx.Toquantify

温馨提示

  • 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
  • 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
  • 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
  • 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
  • 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
  • 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
  • 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。

评论

0/150

提交评论