利用大语言模型促进综合图学习能力

#利用大语言模型

促进综合图学习能力02

大语言模型进行图学习的现状概述01

为什么应用大语言模型进行图学习

03

大语言模型促进跨领域跨任务的统一图学习04

潜在研究方向目录#01为什么应用大语言模型进行图学习为什么应用大语言模型进行图学习大语言模型的能力图数据的特征为什么应用大语言模型进行图学习大语言模型的能力LLMs

have

demonstrated

their

strong

text

encoding/decoding

ability.为什么应用大语言模型进行图学习大语言模型的能力LLMs

have

shown

newly

found

emergent

ability

(e.g.,

reasoning).为什么应用大语言模型进行图学习图数据的特征In

real

world,

text

and

graph

usually

appears

simultaneously.Text

data

are

associated

with

rich

structure

information

in

the

form

of

graphs.Graph

data

are

captioned

with

rich

textual

information.#02大语言模型进行图学习的现状概述大语言模型进行图学习的现状概述不同的图数据应用场景图任务中大语言模型的不同角色不同的图数据应用场景大语言模型进行图学习的现状概述大语言模型进行图学习的现状概述不同的图数据应用场景：Pure

GraphDefinition:

Graph

with

no

text

information

or

no

semantically

rich

text

information.

eg.

traffic

graphsor

power

transmission

graph.Problems

on

Pure

Graphs:graph

reasoning

tasks

likeconnectivityshortestpathsubgraph

matchinglogical

rule

induction…大语言模型进行图学习的现状概述不同的图数据应用场景：Pure

GraphGraph

with

no

text

information

or

no

semantically

rich

text

information.

eg.

traffic

graphs

or

powertransmission

graph.大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Paired

Graph大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Paired

Graph大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Attributed

Graph大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Attributed

Graph大语言模型进行图学习的现状概述图任务中大语言模型的不同角色LLM

as

Enhancer/EncoderLLM

as

PredictorLLM

as

Aligner大语言模型进行图学习的现状概述Eembedding-based图任务中大语言模型的不同角色:LLM

as

Enhancer/EncoderExplanation-based大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Explanation-basedICLR’24Basically,

using

T

andA,to

generate

P

and

E,then

use

T,

A,

P,

Easenriched

text

feature.大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Explanation-based大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Embedding-basedObservation:Fine-tune-based

LLMsmay

fail

at

low

labelingrate

settings.Chen,

Z.,

Mao,

H.,

Li,

H.,

Jin,

W.,

Wen,

H.,

Wei,

X.,

Wang,

S.,

Yin,

D.,

Fan,

W.,

Liu,

H.,

&

Tang,

J.

(2024).

Exploring

thePotential

of

Large

Language

Models

(LLMs)

in

Learning

on

Graphs

(arXiv:2307.03393).

arXiv.Low

label

ratioHigh

label

ratio大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Embedding-basedObservation:Under

embedding-basedstructure,

thebination

ofdeep

sentence

embedding

withGNNs

makes

a

strong

baseline.Low

label

ratioChen,

Z.,

Mao,

H.,

Li,

H.,

Jin,

W.,

Wen,

H.,

Wei,

X.,

Wang,

S.,

Yin,

D.,

Fan,

W.,

Liu,

H.,

&

Tang,

J.

(2024).

Exploring

thePotential

of

Large

Language

Models

(LLMs)

in

Learning

on

Graphs

(arXiv:2307.03393).

arXiv.High

label

ratio大语言模型进行图学习的现状概述图任务中大语言模型的不同角色:LLM

as

PredictorFlatten-basedGNN-based大语言模型进行图学习的现状概述LLM

asPredictor:

Flatten-based大语言模型进行图学习的现状概述LLM

asPredictor:

Flatten-based大语言模型进行图学习的现状概述LLM

as

Predictor:

GNN-based大语言模型进行图学习的现状概述LLM

as

Predictor:

GNN-based大语言模型进行图学习的现状概述图任务中大语言模型的不同角色:LLM

as

Aligner大语言模型进行图学习的现状概述LLM

as

Aligner:Contrastive大语言模型进行图学习的现状概述LLM

as

Aligner:DistillationMavromatis,

Costas,

et

al.

"Train

your

own

gnn

teacher:

Graph-aware

distillation

on

textualgraphs."

Joint

European

Conference

on

Machine

Learning

and

Knowledge

Discovery

in

Databases.Cham:

Springer

Nature

Switzerland,

2023.#03大语言模型促进跨领域跨任务的统一图学习大语言模型促进跨领域跨任务的统一图学习“Cross

Domain”

before

LLMsCross

Domain

Graph

Learning

with

LLM大语言模型促进跨领域跨任务的统一图学习“Cross

Domain”

before

LLMsKDD’20:

“We

design

Graph

Contrastive

Coding(GCC)—a

self-supervised

graph

neural

network

pre-training

framework—to

capture

the

universal

networktopological

properties

across

multiple

networks.”Limitation:

the

node

features

are

notthe

same,

among

graphs

fromdifferent

domain.大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMOne

for

all:Towards

trainingone

graphmodel

for

allclassificationtasks大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLM大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMOFA

successfully

enableda

single

graph

model

to

be

effective

onall

graph

datasets

across

differentdomains

as

OFA-joint

performs

well

onall

datasets.Further,

we

can

see

that

OFA-jointachieves

better

results

on

most

of

thedatasetspared

toOFA-ind.

Thismay

indicate

that

by

leveraging

thetext

feature,

the

knowledge

learnedfrom

one

domain

can

be

useful

for

thelearning

of

other

domains.大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMOverview

of

UniGraph

framework.

In

pre-training,

we

employ

a

self-supervised

approach,

leveragingTAGs

to

unify

diverse

graph

data.

This

phase

involves

a

cascaded

architecturebining

LMs

andGNNs.大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMWe

can

observe

thatpre-training

ongraphs

from

the

same

domain

enhancesthe

performance

of

downstream

tasks.This

suggests

that

in-domain

transferremains

simpler

than

cross-domaintransfer.Experiment

results

in

few-shot

transfer.大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLM大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLM#04潜在研究方向What

LLMs

truly

learned

from

GraphsObservation

1:LLMs

interpret

inputs

more

as

contextual

paragraphs

than

as

graphs

with

topologicalstructures.

Neither

linearizing

nor

rewiring

ego-graph

has

significant

impact

on

theclassification

performance

of

LLMs.Linearize

ego-graph:

We

create

a

linearized

version

of

the

graph-structured

prompts

by

onlykeeping

all

neighbors’

text

attributes

in

the

prompts.What

LLMs

truly

learned

from

GraphsObservation

1:LLMs

interpret

inputs

more

as

contextual

paragraphs

than

as

graphs

with

topologicalstructures.

Neither

linearizing

nor

rewiring

ego-graph

has

significant

impact

on

theclassification

performance

of

LLMs.Rewire

ego-graph:We

randomly

rewire

theego-graph

by

differentstrategies.

Then

wepare

theperformance

of

MPNNsand

LLMs

under

eachstrategy.What

LLMs

truly

learned

from

GraphsObservation

2:LLMs

benefit

from

structural

information

only

when

the

neighborhood

is

homophilous,which

means

the

neighbors

contain

phrases

related

to

the

groundtruth

label

of

the

targetnode.What

LLMs

truly

learned

from

GraphsObservation

3:LLMs

benefit

from

structural

information

when

the

target

node

does

not

contain

enoughphrases

for

the

model

to

make

reasonable

prediction.Truly

“Generative”

Cross

Domain

LLM-based

Graph

LearningIs

there

universal

structure

features

that

benefit

for

graph

learningof

graph

from

different

domain?How

can

theseplex

topological

features,

instead

of

the

textcontext,

be

really

captured

by

LLMs?Large

Language

Models

on

Graphs:

Aprehensive

Survey言鹏韦浙江大学信息资源管理系20研究生某著名企业巴巴通义实验室实习生ReferencesZhao

W

X,

Zhou

K,

Li

J,

et

al.

A

survey

of

large

language

models[J].

arXiv

preprint

arXiv:2303.18223,

2023.Huang,

Jin,

et

al.

"Can

llms

effectively

leverage

graph

structural

information:

when

and

why."

arXiv

preprintarXiv:2309.16595

(2023).Tan,

Yanchao,

et

al.

"MuseGraph:

Graph-oriented

Instruction

Tuning

of

Large

Language

Models

for

Generic

GraphMining."

arXiv

preprint

arXiv:2403.04780(2024).He,

Yufei,

and

Bryan

Hooi.

"UniGraph:

Learning

a

Cross-Domain

Graph

Foundation

Model

From

Natural

Language."

arXivpreprint

arXiv:2402.13630

(2024).Liu,

Hao,

et

al.

"One

for

all:

Towards

training

one

graph

model

for

all

classification

tasks."

arXiv

preprintarXiv:2310.00149

(2023).Qiu,

Jiezhong,

et

al.

"Gcc:

Graph

contrastive

coding

for

graph

neural

network

pre-training."

Proceedings

of

the

26th

ACMSIGKDD

international

conference

on

knowledge

discovery

&

data

mining.

2020.Mavromatis,

Costas,

et

al.

"Train

your

own

gnn

teacher:

Graph-aware

distillation

on

textual

graphs."

Joint

European

Conferenceon

Machine

Learning

and

Knowledge

Discovery

in

Databases.

Cham:

Springer

Nature

Switzerland,

2023.Wen,

Z.,

&

Fang,

Y.

(2023).

Prompt

tuning

on

graph-augmented

low-resource

text

classification.

arXiv

preprintarXiv:2307.10230.Tang,

Jiabin,

et

al.

"Graphgpt:

Graph

instruction

tuning

for

large

language

models."

arXiv

preprint

arXiv:2310.13023

(2023).Guo,

J.,

Du,

L.,

&

Liu,

H.

(2023).

Gpt4graph:

Can

large

language

models

understand

graph

structured

data?

an

empiricalevaluation

and

benchmarking.

arXiv

preprint

arXiv:2305.15066.ReferencesXie,

Han,

et

al.

"Graph-aware

language

model

pre-training

on

a

large

graph

corpus

can

help

multiple

graphapplications."

Proceedings

of

the

29th

ACM

SIGKDD

Conference

on

Knowledge

Discovery

and

Data

Mining.

2023.Zhikai

Chen,

Haitao

Mao,

Hang

Li,

Wei

Jin,

Hongzhi

Wen,

Xiaochi

Wei,

Shuaiqiang

Wang,

Dawei

Yin,

Wenqi

Fan,

Hui

Liu,

et

al.Exploring

the

potential

of

large

language

models

(llms)

in

learning

on

graphs.

arXiv

preprint

arXiv:2307.03393,

2023Ruosong

Ye,

Caiqi

Zhang,

Runhui

Wang,

Shuyuan

Xu,

and

Yongfeng

Zhang.

2024.

Language

is

All