2025AICon全球人工智能开发与应用大会-上海站:基于MLLM的视频语义理解与时空感知_第1页
2025AICon全球人工智能开发与应用大会-上海站:基于MLLM的视频语义理解与时空感知_第2页
2025AICon全球人工智能开发与应用大会-上海站:基于MLLM的视频语义理解与时空感知_第3页
2025AICon全球人工智能开发与应用大会-上海站:基于MLLM的视频语义理解与时空感知_第4页
2025AICon全球人工智能开发与应用大会-上海站:基于MLLM的视频语义理解与时空感知_第5页
已阅读5页,还剩42页未读, 继续免费阅读

下载本文档

版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领

文档简介

Info

Q

极客传媒基于MLLM的视频语义理解与时空感知演讲人:赵波上海交通大学人工智能学院AiCon全球人工智能开发与应用大会1AiCon全球人工智能开发与应用大会长视频理解评测集

MLVU02长视频理解大模型

Video-XL03

空间理解大模型

SpatialBot时空理解评测集

STI-Bench2小时级长视频理解4

InfoQ

极客传媒全球人工智能开发与应用大会5InfoQ

极客传媒全球人工智能开发与应用大会·

长视频的挑战·

信

息

量

大·

信息冗余度高·

模型上下文有限·

视频内容理解·

情

节·

动

作·

推

理长视频理解任务存在的问题:·

视频短且简单·

分类任务为主·

缺乏细节任务·

视频主题单

一·

缺乏复杂问题(a

)Relationship

(b)Way

of

Speaking

(c)Scene/Place(d)'Like'ratio

(e)View

Count6InfoQ

极客传媒全球人工智能开发与应用大会传统视频理解评测集MLVU:

更丰富的视频内容和长度7全球人工智能开发与应用大会MLVU:Benchmarking

Multi-task

Long

Video

Understanding.Zhou

et

al.CVPR

2025.DailyFoodPetTravelBadminton(120.30分0VideoDuration

(min)FantasyAdventureGeography200300NumberofVideosComedyActionFamilyBasketbal

sportInfoQ

极客传媒e小centriicLife

Record400

500PingpongHandcrafFootball5oftware

omacmacnTutorialCookingnceTV

SeriesScienceActionFictionDocumentaryMovieGame3Simulationuspense100D3FActionOrder329ActionCount266TopicReasoning355AnomalyRecognition

239Plot

QA589EgoReasoning

405Needle

QA415Open-ended

TaskVideoSummarization257Sub-SceneCaptioning247Holistic

Task

Single-Detail

Task

Multi-Detail

Task8MLVU:更丰富的任务类型Close-ended

TaskInfoQ

极客传媒全球人工智能开发与应用大会MLVU:更丰富的任务类型Holistic

LVU9

InfoQ

极客传媒全球人工智能开发与应用大会Prompt:

Please

summarize

the

main

content

of

this

video.StandardAnswer:Thevideostartswithsomeone

in

blue

pantsenteringabright

room,talkingtoanother

in

a

black

shirt,and

then

…Q:What

is

the

person

in

the

game

doing?(A)Fighting

with

a

game

boss

(B)Building

an

automatic

farm(C)Exploring

a

haunted

house

(D)Designing

a

character's

outfitQ:What

type

of

abnormality

in

this

surveillance

video?(A)Fighting

Vandalism(D)Assault(c)VideoSummarization(b)Anomaly

Recognition(a)TopicReasoningQ:What

isthe

man

inthevideodoingonthe

lake

shoreduringthesunnysummer?(A)Swimming

(B)Catching

the

drone(C)Sunbathing(D)Launching

the

drone(g)Sub-ScenePrompt:

Please

describe

how

the

man

in

the

white

suit

saved

the

woman

wearing

red

high

heels

when

she

was

about

to

fall

due

to

a

twisted

ankle...Standard

Answer:Theman

inthewhitesuit

hookedatreewithonefoot

and

used

his

handto

grab

her,preventing

herfrom

falling.MLVU:更丰富的任务类型Q:Whatdoesthecartoonmouse

useto

hit

the

cartoon

cat?(A)Stick(B)Stone

(C)Vase

(D)HammerCaptioningQ:Where

was

the

baking

glove

before

I

hung

it

on

the

hook?(A)On

the

kitchen

count

(B)By

the

window(C)On

the

oven

(D)In

the

dishwasher(f)Plot

Question-AnsweringSingle-Detail

LVU10

Info

Q

极客传媒(d)Needle

QuestionAnswering(e)Ego

ReasoningMLVU:更丰富的任务类型Multi-Detail

LVU11

InfoQ极客传媒全球人工智能开发与应用大会Q:Ordertheseactionsfromthevideo:(1)waterskiing,

(2)playingtrombone,(3)makingjewelry.(D)2->3->1Q:

How

many

times

does

the

action

of

"carving

apumpkin"occurin

this

video?(A)0

(B)2

(C)4(D)6(h)ActionOrder(i)Action

CountBenchmarks#Videos#QAPairsLen.

(s)Close-EndedOpen-EndedVarious

GenresMulti-LevelMulti-DimensionReferring

QANExT-QA

[50]1,0008,56439.5√√√XXXTVQA[21]15,25315,25311.2√XXXXXMSRVTT-QA

[52]2,90072,82115.2√XXXXXMVBench

[26]3,6414,00016.0√X√XXXMovie101

[58]1016144X√XXXXEgoSchema

[34]5,0635,063180√XXXXXMovieChat-1K[41]1301,950500√√XX√XVideo-MME*[13]9002,7001024√X√√XXLongVideoBench*[49]3,7636,678473√X√√X√MLVU1,7303,102930√√√√√√MLVU:

更丰富的问答形式InfoQ

极客传媒全球人工智能开发与应用大会12MethodsDateInputHolisticSingle

DetailMultiAODetailACM-AvgG-AvgTRARVS*NQAERPQASSCFullmarkRandom-一一10016.710016.710-10016.710016.710016.710-10016.710016.710016.710-Image

MLLMsOtter-I[23]LLaVA-1.6[29]IntemVL-2[8]Claude-3-Opust

[2]Qwen-VL-Max

[4]2023-052024-012024-072024-032024-0116

frm16

frm16

frm16

frm16

frm17.663.785.753.875.817.917.951.330.853.82.03

2.00

2.55

2.833.0016.713.348.314.015.017.026.447.217.026.418.030.052.020.04.843.904.205.253.6720.015.721.432.910.020.716.716.715.06.711.717.127.147.521.832.22.97

3.103.903.253.92Short

Video

MLLMsOter-V

[23]mPLUG-Owl-V

[54]VideoChat

[25]Video-LLaMA-2[59]

VideoChat2-HD[26]Video-LLaVA

[28]ShareGPT4Video

[7]VideoLLaMA2[9]2023-052023-042023-052024-082024-062023-112024-052024-0616

frm16

frm16

frm16

frm16

frm8

frm16

frm16

frm16.525.326.452.774.770.373.680.212.815.412.812.843.638.525.653.82.18

2.20

2.15

2.23

2.83

20.9

2.532.8016.76.718.313.335.02.3031.736.722.613.217.017.034.026.445.354.722.022.022.012.030.026.038.054.04.205.014.90

4.875.145.064.725.0912.914.315.715.721.420.017.142.913.320.011.78.323.321.78.316.716.716.717.718.837.429.334.248.43.193.613.533.553.993.683.633.95Long

Video

MLLMsMovieChat

[41]Movie-LLM[42]LLaMA-VID

[27]MA-LMM[16]MiniGPT4-Video

[3]LongVA[60]Video-CCAM[11]Video-XL[40]LLaVA-Onevision

[24]GPT40⁺[37]MLVU:

评测结果2048

frm1

fps1

fps1000

frm90

frm256

frm96

frm256

frm32

frm0.5fps2023-072024-032023-112024-042024-042024-062024-082024-092024-082024-0518.7

27.5

20.9

44.0

64.9

81.3

79.1

78.0

83.5

83.715.021.715.013.315.023.3

26.7

31.7

23.3

35.0

1310.325.623.1

23.146.241.038.5

28.256.4

68.82.30

2.10

2.70

3.04

2.50

2.90

2.65

3.40

3.754.9423.310.021.713.320.0

46.745.050.046.742.93.24

4.934.154.614.274.924.495.02

5.096.8015.111.3

11.330.2

30.239.6

52.841.558.447.817.120.018.6

18.615.7

17.124.3

48.635.7

46.216.5

18.918.122.431.7

42.146.146.351.7

54.52.77

3.52

3.43

3.83

3.39

3.91

3.57

4.214.425.8716.0

16.016.0

14.030.0

46.056.0

46.058.0

57.1InfoQ

极客传媒全球人工智能开发与应用大会7B

72B14Info

Q

极客传媒全球人工智能开发与应用大会180s

360s180s360s180s

360s360s

600s180s

360s180s

360sFigure3:Experimental

performances

on

MLVU

under

various

video

lengths:GPT4-turbo(orange),

MiniGPT4-Video(red),VideoChat2(green),Video-LLaMA-2(

blue),and

MA-LMM(cyan).MLVU:视频时长带来的挑战PlotQuestionAnsweringMultiple-choiceAverageAction

Count基于MLLM

的长视频理解AnswerLLMProjectorVisual

EncoderSingle

imageMulti

imageVideoInfoQ

极客传媒全球人工智能开发与应用大会15there

was

light口O

OOOOOOLet

there

<b>1

be

light

<b)2

〈b〉1

<b>2

and

there

<b)

was

light

(b)Chunk1

Chunk

2Video-XL:

视觉

Token

压缩·

推理前的Token

压缩/挑选会带来严重的信息损失·利用LLM固有的

Next-token

Prediction

能力实现视觉语义

Token压缩[1]Long

Context

Compression

with

Activation

Beacon.A

plug-in

module

for

transformer-basedLLMs

to

enable

effective,efficient,and

flexible

compression

of

long

contexts.Video-XL:TowardsVision

Language

Models

ForExtra-LongVideo

Understanding.Shuetal.CVPR2025OralInfo

Q

极客传媒thereOlight

Oand

O全球人工智能开发与应用大会be

O16Video-XL:视觉

Token

压缩Answer

1

Encoding

current

Interval

by

interleaving

VSTs2.Off-load

visual

tokens个LLM

with

VST

CompressionIntervaliVisualEncoder→

SimilarityCalculationDynamicIntervalPartitionProjector17Info

Q

极客传媒ThresholdδVisualtokens

VSTInterval

i+1S1S2

→S3Interval

i3.Encoding

next

Interval

by

previous

VSTsVideo

Text

tokensInterval

i4.VSTsproxyvisualtokensSingle

image

Multi

imageDepth

ScoreCalculation全球人工智能开发与应用大会Interval

i

Interval

i+1→→1.EncodingcurrentIntervalbyinterleavingVSTs2.Off-loadvisualtokensIntervaliIntervali3.EncodingnextIntervalbypreviousVSTs4.VSTsproxyvisualtokensIntervaliIntervali+1Intervali

Interval

i+1●

Interleaving

visual

summarization

tokens(VSTs)X₇Interleave

Vⁱ→Xⁱ'=[x₁…x₆,〈vs⟩₁

…

,x₉,w₇,〈vs⟩₉]●

Encoding(i+1)Intervals

by

the

KVs

of

previous

VSTsVideo-XL:

视觉

Token

压缩18InfoQ极客传媒●Partition

tokens

into

intervals19InfoQ

极客传媒全球人工智能开发与应用大会◎LongVideoQ:Whatsthecorrectorderofthe

following

actions?

VideoXLGPTPrompt:Pleaseextractactionsorevents

in

orderA:Standing

in

the

back->Running->Counting

dollars14sThe

characterisrunningin

ahairy

arm..Thegreencharacteriscountingdollars...Video-XL:

训练数据集构建OsClip

TwoCaptioncartooncharactersare

standingin

the

back

of

a

man..人工校验后:20k

QAsDLongVideoTime…2

iwl

临

的

0

0

9Lnlewtnapod

ia.&u0

J,hi

rl几

r

a

r

d

576

Tokens

per

Grid

144TokensperGridpl

5

地OriginalImagesImageasMulti-Grid

SplitEncode

2x2

FlattenAveragePoolVideo-XL:

数据扩充LLMOriginal

VideoVideoasImagewithMulti-Grid576TokensperFrame144

Tokens

per

Frame20Info

Q

极客传媒&lby:“92-Ⅱfp

d

t

p全球人工智能开发与应用大会1TL(

明00anpau画

sindke

i

tood

lan

i执idobodudiuaipabjclMovie10.3%8.6%14.3%24.1%11.5%Surveillance9.3%13.2%Game

DocumentaryTVSeries分)500

1500

2500

3500

4500

Number

of

Videos5500

650021InfoQ极客传媒全球人工智能开发与应用大会Video-XL:

训练数据集

VICO

开源Cartoon

Sports

Video

Length

DistributionEgocentricVideo

Duration8.6%Video-XL:

实验结果Model

Size

M-avg

G-avg

M-avg

G-avg

W/o

sub

W

sub

VNBenchVideoVistaLongVideo.VideoChat.

MVBenchProprietary

ModelsGPT-4V

[34]49.25.3543.3

4.6759.563.348.9

59.1

4.06

43.5GPT-4o

[35]64.6

5.80

54.95.87

71.9

71.264.478.3

66.7

-Gemini-1.5-Pro

[38]

-

-

-

-

75.0

81.3

66.764.0-

-35.117.230.733.841.142.945.53.99

3.433.683.63-3.913.57-4.2139.5-39.937.939.952.0-52.647.950.351.955.5VideoChat2[17]LLaMA-VID

[18]

VideoLLaVA

[20]

ST-LLM

[26]Shargpt4Video

[3]LLaVA-Next-Video

PLLaVA

[46]LongVA÷[50]VideoLLaMA2÷[4]Video-CCAM÷[6]

Long-LLaVA

[41]Video-XL7B7B7B7B7B[51]

34B

7B

7B8x7B9B13B7BMLLMs43.8-41.642.343.654.954.349.752.4-61.03.99

4.22

3.843.77-4.333.98-4.5012.4

10.8

12.4

22.7-20.1-41.5

24.935.6

52.161.647.9

33.247.346.456.358.564.961.656.956.6

49.3

53.6

56.7

60.4

67.4

60.5

69.070.662.3

41.443.0

54.951.2-46.653.964.6-55.339.339.1-39.7

50.540.2

47.836.0

43.1-50.72.98

2.89

2.843.15-3.26

3.123.263.17MLVU

DevMLVU

Test

VideoMMEInfoQ

极客传媒Open-source全球人工智能开发与应用大会221le4·一Video-XL=

!

LongVA……

LLaVA-NeXT-Video-

·-LongVILA1285121024Frames8070605040302010128

512一●

一

Video-XLLongVALLaVA-NeXT-VideoLongVILA10242048Video-XL:

效率对比全球人工智能开发与应用大会MemoryUsage(GB)InfoQ

极客传媒86420Frames204823TFLOPsRetentionRateCompression

Ratio在MLVU上不同固定帧压缩率下模型性能Video-XL:

压缩率分析24InfoQ

极客传媒What

color

is

the

cars?Black.What

shape

of

musical

instrument

is

the

building

in

the

video?Guitar.Video-XL:

效果可视化与对比What

is

thewoman

doing

at

the

end

ofthe

video?25Video-XL全球人工智能开发与应用大会InfoQ

极客传媒Takingphotos.USerUSerVideo-XLVideo-XLUSerPleasesummarizethisvideo,including

its

main

content.USerThe

video

begins

with

SpongeBob

SquarePants

and

Patrick

Star

in

a

whimsical

setting,transitioning

through

various

scenes

that

include

a

pirate

ship,a

stage

performance,and

a

series

of

comedic

and

dramatic

events.Initially,SpongeBobisseenin

apirate-themedenvironment,followed

by

a

scene

where

he

and

Patrick

are

on

a

stage,engaging

in

a

performance

or

pre-

sentation.The

narrative

progresses

to

show

them

in

different

settings,includingaroomwithagreen

curtain

backdrop,

where

they

interact

with

objects

like

a

large

blue

bucket

and

a

microphone.\n\nAs

the

video

continues,the

focus

shifts

to

a

moreelaboratestagesetup.Thecharactersengageinvariousactivities,such

ascleaningthestage,interactingwith

props,

and

reacting

to

each

other's

actions.The

scene

evolves

from

a

simple

cleaning

task

to

a

more

complex

scenario

involving

a

largeblue

curtain,whichismanipulatedbythe

characters,

leading

to

a

chaotic

situation

with

the

curtain.falling

and

reveal-

ing

a

red

substance

on

the

floor.n\nThroughout

the

video,the

characters'interactions

and

the

unfolding

events

suggest

astoryline

centered

around

cleaning,performance,and

unexpected

outcomes.The

video

concludes

with

SpongeBob

and

Squidward

engaged

in

a

cleaning

activity

amidst

a

chaotic

environment,highlighting

a

comedic

and

dynamic

conclusion

to

their

adventures

on

stage.Video-XL:

效果可视化与对比26

Info

Q

极客传媒AiCon全球人工智能开发与应用大会Video-XLDoes

this

video

contain

any

advertisement?If

yes,which

is

the

content

of

the

ad?The

video

you've

provided

is

from

a

movie

trailer,indicating

that

there

isn't

actual

content

to

summa-

rize

here

for

an

insert.Yes,the

video

contains

an

advertisement

that

displays

a

black

Casio

watch

with

alarge

digital

dis-

play.

It

includes

various

details

about

the

product

and

is

presentedwith

colorful

backgrounds.Video-XL:

效果可视化与对比27Info

Q

极客传媒AiCon全球人工智能开发与应用大会LLaMA-VIDVideo-XLUSerPlease

describe

the

movements

ofthe

heroine

in

the

movie.USerIn

one

key

scene,she

is

shown

kneeling

down

on

the

deck

of

the

ship

with

her

hands

to

her

face,ap-pearing

overwhelmed

or

distressed.The

woman

first

appears

to

be

in

a

state

of

surprise

or

shock,

as

indicated

by

her

wide

eyes

and

mouthslightly

agape.Outdoors,she's

shown

standing

with

arms

stretched

out,

lookingupwardsasifinawe

or

contemplationofhersurroundings,perhapsofthe

sunset

or

the

water

beneath

their

feet.This

is

followedby

a

momentary

hug

between

the

two

characters,

where

they

share

a

close

embrace.Info

Q

极客传媒Video-XL:

效果可视化与对比Video-XLAiCon全球人工智能开发与应用大会LLaMA-VID2829

InfoQ

极客传媒全球人工智能开发与应用大会空间理解30

InfoQ

极客传媒全球人工智能开发与应用大会仅用RGB

图片无法精确感知距离三维空间感知Human|RGB

+DepthBCantherobotclose

the

gripper

nowto

pick

up

the

rag?

,

ince

the

grippe

already

incontactwiththe

rag,

the

robot

can

proceedtoclosethegripperto

pick

up

the

rag.-r

isXs40sTYeGP■传统

MLLM

未见过深度图■传统

MLLM

未在深度相关任务上训练过■传统多模态数据集缺乏深度相关数据MLLM

深度感知与空间推理能力GPT-40XYes,the

gripper

has

touched

the

rag.

The

gripper

is

visibly

in

contact

with

the

rag

in

the

image.HumanRGB

+Depth

BHas

the

gripper

touched

the

rag?InfoQ

极客传媒全球人工智能开发与应用大会3132InfoQ

极客传媒全球人工智能开发与应用大会SpatialBot:MLLM

深度感知与空间推理SpatialBotThe

gripper's

depth

value

is

584,and

the

rag's

is

875.Since

584<875,it

can

be

inferred

that

the

gripperhas

not

touched

the

rag.SpatialBot:PreciseSpatial

UnderstandingwithVision

Language

Models.Caietal.ICRA2025Human

RGB

+Depth

BCanthe

robotclose

the

gripper

now

to

pick

up

the

rag?No,because

the

gripper

has

not

reachedrag.Therobot

needs

tolower

andstretchforward

thegripper.HumanRGB+Depth

BHas

thegrippertouched

the

rag?SpatialBotSpatialBot

APIDepth(point)=depth_value_of_the_point

Depth(G

)=depth_map[

□x,国

.y]Input

from

HumanPrevious

thoughts

ofSpatialBotReturned

values

from

APIImage

Encoder&ProjectorText

EncoderLarge

Language

ModelInstructions

for

API

Output

for

Human

回SpatialBot

模型架构The

depthvalue

ofthe

woman

on

the

left

is

1357.The

depthvalue

of

the

woman

in

the

middle

is

1300.The

depthvalue

ofthe

woman

on

the

right

is

1400.Sothewomaninthemiddle

istheclosest

to

thecamera.@APIThe

depthvalue

ofthe

woman

on

the

left

is:Depth(

①)The

depth

value

ofthe

woman

in

the

middle

is:

Depth(②)The

depthvalue

ofthe

woman

on

therightis:Depth(

③).@SpatialBotDepth(

①)=1357Depth(

②)=1300Depth(

3)=1400Whostandsclosesttothe

camera?Conversation

with

SpatialBotInfo

Q

极客传媒全球人工智能开发与应用大会33Low-LevelIPoint

Depth:DepthofaSingle

Point,1

IDirectly

RetrievedfromaDepth

Map

lQ:What

is

the

depth

value

ofthe

point(0.60,0.58)

?A:The

point(0.60,0.58)corresponds

to

a1depth

value

of

…

.Q:What

is

the

depth

value

of

the

point(0.31,0.82)?A:The

point(0.31,0.82)

corresponds

to

adepth

value

of

…Middle-Level;RGB-DepthAlignment:DepthofObject,Using

Bounding

Boxor

MaskQ:What

is

the

metric

depth

of

the

object:

ROLEXclock?4300,and

mean

depth

4512.Q:Howfaristhepickuptruckfromthecamera?High-LevelCounting

and

EnumerationQ:How

many

orange

bowls

are

there

onthe

table?A:There

are

3

orange

bowls.Q:How

many

bowls

that

are

upside-downA:

.there

on

the

table?OneareLow-LevelI

Depth

DescriptionandAnalysis

IIQ:Describe

the

depth

map.A:The

depth

map

appears

to

depict

anoutdoor

scene

with

a

ground

plane

thatextends

into

the

distance.There

areingupwardsfromthe

middleofthe

image.Middle-LevelProximity

RelationshipsQ:Which

is

closer

to

the

camera,theROLEX

clock,or

the

pickup

truck?A:The

clockhas

a

depth

value

of4909,whileQ:Which

appears

to

be

more

distant,thepoint(0.62,0.72)

or

the

point(0.39,0.74)?iA:Thepoint(0.62,0.72)corresponds

to

adepth

of9312,and

the

point(0.39,0.74)

…High-LevelSpatial

RelationshipUnderstandingQ:Which

bowl

is

closest

to

the

gripper?A:The

orange

bowl.Q:If

the

red

cup

is

to

the

right

side

of

thegripper

,what

is

the

spatial

relationshipgripper,

andcloserto

thecamera.34Info

Q极

客

传

媒SpatialQA

训练数据the

pickuptruckcorrespondstoametricdepth

of17802,so

theROLEXclock

is

closer

to

the

camera.IA:The

pickuptruck

is

17802

mm

away

fromithecamera.iA:The

clock

translatesto

a

depthvalue

ofbetween

the

purple

cupand

the

gripper?A:The

purple

cup

is

to

the

left

side

of

thestructuresorobjects

in

the

middle

to

upperpartsoftheimagethat

arefurtheraway.4909,with

max

depth

5051,min

depthAverticalstructure,canbeseen

extend-全球人工智能开发与应用大会SpatialQA-E

训练数据PrintedTeacup23435Info

Q极客传媒全球人工智能开发与应用大会CentralClosestThinnestTallestis

the

depth

value

of

the

sign?3B

SpatialQA:4137(GT:4137)is

the

depth

value

of

the

white

car?3B

SpatialQA:28482(GF:28479)is

closerto

the

camera,sign

or

white

ar?

3B

SpatialQA:Thesigncorrespondstodepthof4137,andthe

white

car

correspondsdepth

measurement

of28482Since4137<28482,be

inferred

that:the

sign

is

closer

to

thecamera3B:The

sign5

4B:The

sign

V34B:The

signsignAiCon1.What

is

the

depth

value

of

the

woman?Phi2-3B-SpatialQA:1241(GT:1240)2.What

is

the

depth

value

of

the

cat?Phi2-3B-SpatialQA:990(GT:987)3.Which

is

closer

to

the

camera,cat

or

woman?Phi2-3B-SpatialQA:cat

corresponds

to

a

depth

level

of990,and

woman

correspondsto

a

depth

level

of1241.Since990<1241,it

can

be

inferred

that:cat

is

closer

tothe

camera,so

the

answer

is:cat.Phi2-3B:The

cat.QWen1.5-4B:They

are

equally

close.

×LLaVA-34B:The

woman.XGPt-40:The

woman.

X全球人工智能开发与应用大会4.Whatis

the

arrow

of

signpointing

to?Answer

with

the

option's

letter

from

the

given

choices

directly.(A)The

car.(B)Thegrass.(C)The

sky.(D)Thebuilding.Phi2-3B-SpatialQA:(B)Phi2-3B:(A)

×QWen1.5-4B:(D)

×LLaVA-34B:(C)×GPT-40:(A)

×Info

Q

极客传媒4Whereisthe

cat?Answerwith

thefrom

the

given

choices

directly(A)The

cat

is

in

the

woman's

bag(B)The

cat

is

in

the

woman's

pocket(C)The

cat

is

out

of

the

woman's

bag(D)The

cat

is

not

with

the

womanPhi2

3B

SpatialQA:(A)Phi2

3B:(C)QWen15

4B:(C)LaVA

34B:(C)GPT

4o:(A)1.WhatPhi2-

-2.WhatPhi2-

-3.WhichPhi2-

-measurementto

ait

canPhi2-

.QWen1.-LLaVA-.GPt-40:The

.SpatialBench-

×-

×.-

×-.-

-......caoption's

letter36..GPT-4o-RGBGPT-4o-RGBD-

-70.661.885.090.084.585.251.751.743.340.0Bunny-Phi2-3B-RGB70.650.075.089.451.726.7Bunny-Phi2-3B-RGBD85.850.075.090.443.328.3SpatialBot-Phi2-3B-RGB84.164.780.088.061.728.3SpatialBot-Phi2-3B-RGBD>9961.880.091.755.026.7Bunny-Phi3-4B-RGB32.358.875.091.031.716.7Bunny-Phi3-4B-RGBD63.352.960.085.431.718.3SpatialBot-Phi3-4B-RGB83.264.775.091.04023.3SpatialBot-Phi3-4B-RGBD>9967.770.091.735.021.7QWen-1.5-4B-RGB42.250.075.091.626.715.0QWen-1.5-4B-RGBD74.644.170.090.725.015.0SpatialBot-QWen1.5-4B-RGB89.952.975.088.646.818.3SpatialBot-QWen1.5-4B-RGBD>9952.960

温馨提示

  • 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
  • 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
  • 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
  • 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
  • 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
  • 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
  • 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。

评论

0/150

提交评论