深度学习及其应用- 课件 0405GNN

GraphNeuralNetworkDeepLearninganditsApplicationHowtorepresentagraph?2SJTUDeepLearningLecture.

UndirectedGraphDirectedGraphAdditionalelementsofagraph3SJTUDeepLearningLecture.

Distinctionofgraph-structureddata4SJTUDeepLearningLecture.Differentfromimagedata:Itingeneraldoesnothavea2Dgridstructureandinsteadrepresentsrelationshipsbetweenobjects.Inaddition,graphdataisnotlimitedtovisualinformationandcanrepresentrelationshipsbetweenabstractentities.ImagedataDistinctionofgraph-structureddata5SJTUDeepLearningLecture.Differentfromtextdata:Itrepresentsrelationshipsbetweenobjectsratherthanjustsequencesofwords.Graphdatacanalsobeusedtorepresentrelationshipsbetweenentitiesintextdata,suchasco-occurrencerelationshipsbetweenwordsinadocument.“苟利国家生死以，岂因祸福避趋之”TextdataDistinctionofGraph-structuredData6SJTUDeepLearningLecture.Differentfrom3-Dpointclouds:Itrepresentsrelationshipsbetweenobjectsratherthanjustacollectionofpointsandtheirpositioncoordinates.Graphdatacanalsobeusedtorepresentrelationshipsbetweenpointsinapointcloud,suchasnearestneighborrelationships.PointcloudsGraph-structureddataareubiquitous!7SJTUDeepLearningLecture.SocialnetworksMoleculesGraph-structureddataareubiquitous!8SJTUDeepLearningLecture.KnowledgegraphsProgressinGraphGenerationHoogeboom,E.,Satorras,V.G.,Vignac,C.,&Welling,M.(2022,June).Equivariantdiffusionformoleculegenerationin3D.InInternationalConferenceonMachineLearning(pp.8867-8887)9SJTUDeepLearningLecture.AI-aidedDrugDiscoveryRelatedTasks10SJTUDeepLearningLecture.Nodeclassification:Inthistask,thegoalistopredictacategoricallabelforeachnodeinagraph.Forexample,inasocialnetwork,nodesmayrepresentpeople,andthetaskcouldbetopredicttheiroccupationbasedontheconnectionsbetweenthem.2.

Graphclassification:Thistaskinvolvespredictingacategoricallabelforanentiregraph.Inamoleculegraph,thetaskcouldbetopredictwhetheramoleculeistoxicornot.3.

Linkprediction:Thistaskinvolvespredictingtheexistenceofedgesbetweennodesinagraph.Forexample,inasocialnetwork,thetaskcouldbetopredictwhethertwopeoplearelikelytobecomefriendsbasedontheirotherexistingconnections.Messagepassing11SJTUDeepLearningLecture.Goal:Integratetheinformationfromneighboringnodestoencodecontextualgraphinformation.Theideabehindmessagepassingistoallowinformationtoflowbetweennodes,allowingthenetworktolearntherelationshipsandpatternsinthegraphstructure.Bypassingmessagesbetweennodes,thenetworkcancapturethedependenciesandinteractionsbetweennodes,leadingtoimprovedrepresentationandbetterperformanceongraph-relatedtaskse.g.nodeclassification,graphclassification,andlinkprediction.

Messagepassinggeneralformulation12SJTUDeepLearningLecture.Messagepassinggeneralformulation13SJTUDeepLearningLecture.Ineachiterationofmessagepassing,Theaggregateoperationcollectsinformationfromneighboringnodesandsummarizesitintoacompactrepresentation,whichisthenpassedontothetargetnode.Thisoperationallowsthenetworktogatherinformationfromthesurroundingnodesandmakeuseofittoupdatetherepresentationofthetargetnode.Theupdateoperation,takestheinformationfromtheaggregateoperationandupdatestherepresentationofthetargetnode.Thisupdatedrepresentationincludestheinformationfromtheoriginalnodefeaturesaswellastheinformationfromitsneighboringnodes.Theupdatedrepresentationthenservesasinputforthenextiterationofmessagepassing.Overview14SJTUDeepLearningLecture.Some

typicalGNNs

15SJTUDeepLearningLecture.1.GraphConvolutionalNetworks(GCN)[1]:ProposedbyKipfandWellingin2016,GCNisoneofthepioneeringGNNmodels.ItadaptstheconvolutionoperationfromtraditionalCNNstoworkongraph-structureddata,enablingfeaturelearningonnodesinthegraph.2.GraphSAGE

[2]:DevelopedbyHamilton,Ying,andLeskovecin2017,GraphSAGEisaninductiveGNNmodelthatlearnstogenerateembeddingsfornodesbyaggregatinginformationfromtheirlocalneighborhood.Thismodelisparticularlyusefulforgraphswithunseennodesorevolvingovertime.3.GraphAttentionNetworks(GAT)[3]:IntroducedbyVelickovicetal.in2017,GATincorporatesattentionmechanismsintographneuralnetworks.GATallowsnodestoweightheimportanceoftheirneighborsdynamically,enablingthemodeltofocusonmorerelevantinformationduringtheaggregationprocess.IssuesinGNNs16SJTUDeepLearningLecture.Scalability:GNNscanstrugglewithlarge-scalegraphs,asthecomputationalcomplexityandmemoryrequirementsincreasewiththesizeofthegraph.DesigningscalableGNNsthatcanefficientlyhandlelargegraphswhilemaintaininghighperformanceisacriticalchallenge.

e.g.[NodeFormer,Wuetal.],[DIFFormer,Wuetal.]Heterogeneousgraphs:Graphswithdifferenttypesofnodesandedges(heterogeneousgraphs)arecommoninreal-worldapplications.DevelopingGNNsthatcaneffectivelyhandleandexploittherichinformationinheterogeneousgraphsisstillanongoingresearcharea.IssuesinGNNs17SJTUDeepLearningLecture.Dynamicgraphs:Manyreal-worldgraphsaredynamic,withnodesandedgesbeingaddedorremovedovertime.DevelopingGNNscapableofadaptingtoandlearningfromdynamicgraphsremainsanopenchallenge.e.g.[EasyDGL,Chaoetal.]Oversmoothing:OversmoothingcanoccurinGNNswhenthenumberoflayersislarge.Oversmoothingmeansthatthefeaturesofdifferentnodesbecomeincreasinglysimilarduringthemessage-passingandinformationaggregationprocesses,leadingtoreduceddiscriminativepower.Summary18SJTUDeepLearningLecture.Inconclusion,GraphNeuralNetworks(GNNs)haveemergedasapowerfultoolforlearningfromgraph-structureddata.Byleveraginglocalandglobalinformationthroughmessage-passingandaggregationmechanisms,GNNshavedemonstratedremarkableperformanceinvariousdomains,includingsocialnetworkanalysis,recommendationsystems,anddrugdiscovery.Despitetheirimpressiveachievements,GNNsstillfacechallengessuchasscalability,oversmoothing,generalization,andsoon.OngoingresearchisfocusedonaddressingtheseissuesandfurtherimprovingGNNsfordiverseapplications.AswecontinuetoadvanceourunderstandingofGNNs,wecanexpectthesemodelstoplayanincreasinglysignificantroleinaddressingcomplexproblemsacrossawiderangeofdomains,unlockingnewpossibilitiesfordata-drivendecision-makingandinsights.References[Ref1]

Kipf,ThomasN.,andMaxWelling."Semi-SupervisedClassificationwithGraphConvolutionalNetworks."

InternationalConferenceonLearningRepresentations.[Ref2]Hamilton,Will,ZhitaoYing,andJureLeskovec."Inductiverepresentationlearningonlargegraphs."

Advancesinneuralinformationprocessingsystems

30(2017).[Ref3]Veličković,Petar,etal."GraphAttentionNetworks."

InternationalConferenceonLearningRepresentations.[Ref4]Wu,Qitian,etal."Nodeformer:Ascalablegraphstructurelearningtransformerfornodeclassification."

AdvancesinNeuralInformationProcessingSystems

35(2022):27387-27401.[Ref5]Wu,Qitian,etal."DIFFormer:Scalable(Graph)TransformersInducedbyEnergyConstrainedDiffusion."

TheEleventhInternationalConferenceonLearningRepresentations.[Ref6]Chen,Chao,etal."E