【全球软件开发大会】AIGC浪潮下WebNN的演进与实践

演讲人：付俊伟胡宁馨,

英特尔首席工程师,

W3C

Web

Neural

Network

(WebNN)

标准的起草和主

要编辑者,

Chromium

committer

and

Chromium

WebNN

组件的主要拥有者张敏,

IntelWebNN

团队的技术经理,

Chromium

and

ONNX

Runtime

WebNN

EP

的开发者,WebNN

developer

preview的作者付俊伟,

英特尔高级软件工程师,

Chromium

committer

and

Chromium

WebNN的

基础架构设计和Chromium

Shape

Detection

API

主要开发者

03

如何使用WebNN

02

WebNN的架构设计

04

WebNN的性能对比

01

WebNN出现的背景目

录WebNN

ExecutionProviderofONNX

RuntimeWebwithGPUaccelerationfrom

DirectML.

Runningon

Intel®CoreUltra7

processor

155Hwith

integratedArcGPU.https://microsoft.github.io/webnn-developer-preview/WebNNOperationmatMulgathersigmoidsoftmaxDirectMLGEMMGATHERLOGISTICSOFTMAXTFLiteBATH_MATMULGATHERACTIVATION_SIGMOIDACTIVATION_SOFTMAXCoreMLmatmulgather_along_axissigmoidsoftmaxStableDiffusioni

Image

DecoderImageGenerationText

EncoderAcat

under

the

snowUnetStep1UnetStep4UnetStep2UnetStep3框架WebAPIAPIextensionsWebGPUWebNNWeb引擎Web

Browser(e.g.,Chrome/Edge)JavaScriptRuntime(e.g.,

Electron/Node.js)运用场景系统MLAPIs硬件Transformers.jsTensorFlow.jsMediaPipeWebONNXRuntime

WebWebAssemblyBackground

SegmentationGPUNPUImageClassificationCPUNoiseSuppressionOtherML

OSAPIsNatural

LanguageObject

DetectionWindowsStudio

EffectsTFLiteCoreMLDirectMLComputational

Graph(Web)

inputoutput

WebNN

为Web带来了神经网络的统一抽象CompiledGraph

(Native)device

type:

cpu/gpu/npupowerpreference:high-perf/low-powerOutputBuffers

(CPU/GPU)InputBuffers

(CPU/GPU)MLGraphMLContextMLGraphBuildertmptmpWebNN

API

二

Other

Web

APIFused

conv2doutputbiasfilterinput↓

createbuild

↓compute

Dataflow

Call

flowcompileconv2daddreluRendererProcessGPU/Utility

ProcessApps/FrameworksNativeMLAPIsOS

DriversHardwareWeb

ApplicationWebNNMojo

ClientWebNNMojo

ServerJS

ML

FrameworksMLGraphBuilderMLGraphMLContext

↓

BNNS/MPSDirectMLMCDMDirectMLBackendCoreMLCoreMLBackendTFLiteBackendTFLiteXNNPACK/

DelegateNPUCPUGPUmacOS

WindowsAndroid/ChromeOS/LinuxChromiumIPCWasmKernelsinputWebNN

GraphIntermediateWasmKernelsBrowserswithWebNN

supportWeb

ApplicationTensorFlowLiteWebONNXRuntime

WebNPUGPUCPUNative

CPUKernelsNativeNPUKernelsNative

GPUKernels

母

ONNX

1.18

release

WasmKernelsWebGL

KernelsWebGPU

KernelsWebNN

GraphPost-ProcessingConv2dMatMulTensorF2Pre-processingIntegrationStatusPrototype

Availableintermediateintermediateweightsweightsbiasweb

Assemblybackend

web

NN

backendWebNN

ExecutionProviderofONNX

RuntimeWebwithGPUaccelerationfrom

DirectML.

Runningon

Intel®CoreUltra7

processor

155Hwith

integratedArcGPU.https://microsoft.github.io/webnn-developer-preview/VanillaJS(plainJavaScript)useofWebNNAPI,with

NPUaccelerationfrom

DirectML.Runningon

Intel®CoreUltra7

processor

155Hwith

integrated

Intel®AI

Boost

NPU.4.4.53.03.1

3.31.2

3.03.23.03.23.03.12.

.03.12.

.03.12.8

2.93.12.92.52.52.72.32.21.81.01.01.01.01.01.01.01.01.01.01.01.01.01.01.0The

average

performanceof

listed

15

models

onwebNN

Oncpu

isabout

93%of

nativeXNN

packMediaPipeModelsInferencePerformance(Normalized

/HigherisBetter)100.0%90.0%

80.0%

70.0%

60.0%

50.0%

40.0%

30.0%

20.0%

10.0%

0.0%•Browser:

Chrome

Canary

118.0.5943.0•DUT:Dell/Linux/i7-1260P,

singlep-core•Workloads:

MediaPipe

solutionmodels

(FP32,batch=1)

WebNNXNNPackNative

XNNPackWebNNvsNative5.04.5

4.03.53.02.5

2.0

1.5

1.0

0.5

0.0WebNNvs.

Nat

ive

RatioInferenceSpeedup3.2

3.3

WasmSIMD•CPU:Intel(R)Core(TM)Ultra7

155H

3.80

GHz•

GPU:

Intel(R)

Arc(TM)

Graphics

WebNN

GPU

Native

DirectML

WebNN

GPU

vs.

Native

DirectML•GPUDriver:3512weoNN

Direct

ML

is

about83%

of

native

DML

On87.9

87.3

87.4

88.882.6

82.471.4100001000100101120.0100.080.060.040.020.00.095.089.5

87.676.5•

Browser:Chrome

Canary126.0.6459.0•

OS:Windows11Pro23H2•

DUT:

Asus

Zenbook88.178.672.0

71.591.4

93.286.779.0WebNN

DirectMLvs.

Native

DirectMLInferenceT

ime(ms)(Logscale)95.673.0Percentage(%)91.585.381.575.881.586.1%73.4%62.7%•

Browser:Chrome

Canary126.0.6459.0•

OS:Windows11Pro23H2•

DUT:

Asus

Zenbook•CPU:Intel(R)Core(TM)Ultra7

155H

3.80

GHz•

NPU:Intel(R)AI

Boost•

NPU

Driver:381The

averageperformance

oflisted

4models

onWebNNDirectMLis

about

80%

of

native

DML

onMTL

NPUMobileNetV2SqueezeNet

1.0ResNet50v1EffiecientNet

Lite4 WebNN

DirectML

NPUNative

NPUWebNN

NPUvs

Native100.0%90.0%80.0%

70.0%

60.0%

50.0%

40.0%

30.0%

20.0%

10.0%

0.0%8.007.006.005.004.003.002.001.000.00WebNNDirectMLvs

Nativeon

MTLNPUWebNNvs

Nat

ive

(%)InferenceT

ime

(ms)95.8%SpeechtoText

PoC

Demofor

KhanAcademy

Khanmigo.WebNN

ExecutionProviderofONNX

RuntimeWebwith

NPUaccelerationfrom

DirectML.

Runningon

Intel®CoreUltra7

processor

155Hwith

integrated

Intel®AI

Boost

NPU.•

在Microsoft

Edge

和

Google

chrome

开发版本上试用

weoNN•

打开about://flags

网址

和

搜索Enables

web

NN

APPI"

、

weoNN

•

web

NN

标