800G光模块白皮书

ENABLINGENABLINGTHETHENEXTNEXTGENERATIONGENERATION

OFOFCLOUDCLOUD&&AIAIUSINGUSING800GB/S800GB/S

OPTICALOPTICALMODULESMODULES

EnablingTheNextGenerationOf

Cloud&AiUsing800Gb/sOpticalModules

1.CloudExpansionSetsPacefor

OpticalModules

Cloudcomputingandstoragehavetakenoverasthetechnologicalbackbonetoamajorityofourmodernbusinessapplications

providinginfrastructure,platform,softwareorvirtuallyanythingasaservice,andtopersonalappliancescoveringphones,

laptopsandvarioussmartdevices.UnlikewirelessinfrastructureandstandardslikeLTEand5G,wherethestandardizationand

technologyareaheadoftheactualapplications,providinga“builditandtheywillcome”businessmodel,therapidandall-

encompassingexpansionofcloudapplicationsandservicesvigorouslypushesthedevelopmentofhigh-techelectronicsand

optics,whichoftenseemtorunbehindthepacesetbytheendusers.Theexponentialresourcegrowthofartificialintelligence

applicationsandtheinherentneedforhighbandwidthforthetransportofbigdataputsafurtherstrainondatacenter

architecturesandtheunderlyinginterconnects.Thus,thedeploymentsoftheAIcloud,aregainingmomentum.

Cloudapplications,AR/VR,AI,and5Gapplicationgeneratemoreandmoretraffic.Theexplosivegrowthoftrafficrequireshigher

bandwidth.AsshowninFigure1,globalinterconnectionbandwidthcapacitywillgrowata48%CAGRin2017-2021.

WORLDWIDEGROWTH

10,000(Tbps)

8,000CAGR:+48%

6,000

4,000

2,000

0

20172018201920202021

US.EUAPLATAM

Figure1–GlobalInterconnectionIndex(Source:Equinix)

AsshowninFigure2,marketanalystsareprojectingafirstadoptionof400GDatacommodulesin2020withalargeradoption

of2x400G/800Gmodulesin2022-23.

$7,000

$6,000

LIGHTCOUNTING

$5,000

$4,000

$3,000

Sales($M)

$2,000

$1,000

$-

20202021202220232024

100G200G400G2x400G

Figure2–Projectionofthemarketrevenuefordatacommodules(Source:LightCounting)

01www.800G

EnablingTheNextGenerationOf

Cloud&AiUsing800Gb/sOpticalModules

“OurLightCountingForecastmodelindicatesthatoperatorsofClouddatacenterswillneedtodeploy800Gopticsby2023-2024

tokeepupwiththegrowthofdatatraffic,”statedfounderandCEOofLightCountingMarketResearch,VladimirKozlov,PhD.

“Mostof800Gwillbestillpluggabletransceivers,butweexpecttoseesomeimplementationofco-packagedopticsaswell.”

DatacentercloudarchitecturesarebeingpacedbythecapacityscalingofswitchingASICs,whichisdoublingapproximately

everytwoyears,unfazedbythetalkabouttheendofMoore’sLaw.Today,12.8Tb/sEthernetswitchingchipsarebeing

commerciallydeployedwithfirstchipdesignfirmsalreadyprototyping25.6Tb/ssiliconfordeploymentnextyear,asshownin

Figure3.Thisputsfurtherpressureontothedensificationofopticalinterconnects,whichdonotscaleatthespeedofCMOSdue

tothelackofacommondesignmethodologyacrossthevariouscomponentsandacommonlargescaleprocess.

Inthepastfewyears,therapidexpansionofcloudserviceswasfueledbytherapidadoptionandpriceerosionof100Gshort

reachopticalmodulesbasedondirectdetectiontechnologyandnon-returntozero

(NRZ)modules.Afterthebeginningofthe400GbEBandwidthAssessmentactivity

inIEEEinMarch2011,initialdeploymentof400Gopticalmodulesisfinally

startingin2020withastrongerrampprojectedfor2021,asshowninFigure2.

Infact,intheinitialusecases,400Gmoduleswillbemainlyusedtotransport

4x100Gover500minDR4applicationand2x200GFR4opticsover2km,not

makinguseofthe400GbEMAC.Atthesametime,itseemsunlikelythat

IEEEwouldsoonstandardizethenextgenerationofoptics,suchas800GbE,

meaningthatthestandardizationofhigherdensityopticsforthetransportof

8x100GbEor2x400GbEforthe25.6Tb/sand51.2Tb/sswitchinggenerations

wouldbewellbehindactualmarkettimelinerequirementsof2021-22.This

raisestheneedfor800Gindustryinteroperabilityoutsideoftheestablished

standardbodies.

800G

QSFP112-DD&OSFP

400G32x@1U/64x@2U

QSFP56-DD&OSFP

32x@1U25.6T/51.2T

100GQSFP28

64x@2U12.8T256/512Lanes

100GQSFP28

32x@1U6.4T256Lanes

10GSerdes

40GQSFP+

32x@1U3.2T256Lanes25GSerdes

50GSedes

128Lanes

EthernetswitchingchipcapacityswitchingEthernet1.28T

100GSerdes

128Lanes

20132015201720192021-22

Figure3–Historicalevolutionofdatacenterswitchingchipcapacity

www.800G02

EnablingTheNextGenerationOf

Cloud&AiUsing800Gb/sOpticalModules

2.DataCenterArchitectures

Thehyperscaledatacentermarketisquitefragmentedwithrespecttotheuseddatacenterarchitecturesorthedemandfor

pluggableoptics.DatacentersforoperatorswithalargerexternalcustomerbaseofferingXaaSaremorelikelydominatedby

north-southclient-to-servertrafficandcouldhavemoresmallergeographicalclusters.Ontheotherhand,operatorswitha

largeinternaldemandforcloudcomputingandstorageseemoreeast-westtrafficbetweenserversandcouldoperatetheir

datacentersashugeclusterswithahigherradix.Evenincaseofsimilarusecases,theoperatorscandeployindividualflavorsof

networkarchitectureorhaveasubjectivepreferencetoacertaininterconnectssolutionsuchasPSM4orCWDM4orothercost-

downvariantsofthereof,suchas100GCWDM4-OCP.

Onecanderiveatleasttwomaintypesoftypicaldatacentersarchitectures.Figure4showsthecommonabstractionofahyper-

scaledatacenteranditsopticalinterconnectroadmap.Ingeneral,thesearchitecturesarelarger,haveacertainconvergence

fromlayertolayer,e.g.3:1,andrelyoncoherentZRinterconnectsattheSpinelayer.Animportantboundaryconstraintfor800G

networkinginthiscaseisthat200Ginterconnects,albeitnotserial,areusedattheservertoTORlayer,whereastheTOR-leaf/

spinelayerwouldtypicallyrelyonPSM44x200Ginafan-outconfiguration.

DC

TypicalOpticalmoduleevolution

Scenario4Scenario4100GQSFP28400GQ-DD800G

(DCI)DWDMZRZR

Spine.....

Scenario340GQSFP+100GQSFP28400GQ-DD800G

Scenario3(Spine-Leaf)eSR4/LR4CWDM4/PSM4DR4/FR4PSM4/FR4

.....

Leaf

Scenario240GQSFP+100GQSFP28400GQ-DD800G

Scenario2

(Leaf-TOR)SR4SR4/PSM4SR8/DR4PSM8/4

TOR.....

Scenario1Scenario110GSFP+25GSFP28100G200G

(TOR-Server)AOC/DACAOC/DACAOC/DACAOC

Server

2012201620192022

Figure4–Typicalhyperscaledatacenterinterconnectroadmap

Forthetypicalhyperscaledatacenternetwork(DCN),deploying200Gserverswillrequirean800Gfabric.It’satraffic

convergencenetwork,whichdependsonthebalancebetweenservicerequirementsandCapexoptimization.Table1showsthe

detailedreachrequirementsdependingontheDCNlayer.

03www.800G

EnablingTheNextGenerationOf

Cloud&AiUsing800Gb/sOpticalModules

Table1–DetailedrequirementsofthetypicalhyperscaleDCN

ScenarioServertoTORTORtoLeafLeaftoSpineDCI

Bandwidth200G800G800G800G

4mwithinrack;≥70m

Distance500m/2km80km-120km

20mcross-rack100mispreferred

ModulesizeQSFP-DD/OSFPQSFP-DD/OSFPQSFP-DD/OSFPQSFP-DD/OSFP

Figure5showsthedatacenternetworkarchitectureofanAIcluster,withlesslayersthanthehyperscalenetworkduetothe

factthatitlacksanyconvergencebetweenthelayers.ThedesignofanAIcloudimpliesdifferenttrafficflowswithmuchlarger

bigdataflowsandlessfrequentswitching.

.....

Spine

Opticalmodulerateevolution(AI/HPCCluster)

Scenario2

Scenario2

400GPSM4800GPSM8

(Spine-Leaf)

.....

Leaf

Scenario1

Scenario1

2*200GE2*400GE

(Leaf-Server)

Server

20192021

Figure5–AI/HPCopticalinterconnectroadmap

FortheAI/HPCclusterDCN,deploying400Gserverswillrequirean800Gfabric.ThisDCNdoesn’thaveanytrafficconvergence,

withfasterdeploymentthaninthecaseofFigure4.Table2showsthedetailedrequirements.

Table2–DetailedrequirementsoftheAI/HPCclusterDCN

ScenarioServertoLeafLeaftoSpine

Bandwidth400G800G

4mwithinrack;

Distance500m

20mcross-rack

ModulesizeQSFP-DD/OSFPQSFP-DD/OSFP

Latency92ns(IEEEPMAlayer)92ns(IEEEPMAlayer)

Notexplicitlyshown,butalsorelevant,areDCnetworksforsmallercloudorenterprises,wherethedownstreamtotheserveris

decoupledfromthefan-outratesoftheLeaf-Spinelayerandtypicallyhasslowerserverinterconnectspeeds.

www.800G04

EnablingTheNextGenerationOf

Cloud&AiUsing800Gb/sOpticalModules

3.8x100GSolutionforSRScenario

3.1800GSRscenariorequirementanalysis

Fortheclassof100m,theindustryisfacingthebasiclimitationsofVCSELsignalingatspeedsof100G/lane.Here,multi-mode

technologywilllikelyallowforreachesof30-50m,thusonlypartiallycoveringtheSRclass,whichisprimarilyemployedby

Chinesehyperscaledatacenteroperators.TheMSAtargetsthedevelopmentofalow-cost8x100GmoduleforSRapplications,

coveringthesweetspotof60-100m,asshowninFigure6.Particularly,theMSAisintendedtospecifyalowercosttransmitter

technologywiththepotentialtoleveragesub-linearcostscalingwithahighdegreeofintegration.Suchamodulewouldallow

foranearlytime-to-marketdense800Gsolution.Alowcost800GSR8couldalsosupportthepotentialtrendsofanincreasing

switchradixanddecreasingservercount-per-rack,whichmaycombinetofavormiddle-of-the-rack(MoR)andend-of-the-

rack(EoR)ortop-of-the-rack(ToR)architectures,byprovidingalowcostserial100Gserverinterconnect.AsshowninFigure

6,theMSAwilldefinealowercostPMDforsinglemodefiberinterconnectsbasedon100GPAM4.Duetothelowlatency

requirementsofSRapplications,KP4forwarderrorcorrection(FEC)willbeusedend-to-endwithasimpleclockrecoveryand

dataequalizationunitinthemodule.Finally,theMSAwillspecifyaconnectorforthePSM8moduleswhichenablesafan-out

to8x100G.

MACANDHIGHERLAYERS

RECONCILIATION

8x100G

400GMII400GMII

400GBASE-R400GBASE-R

PCSPCS

DSP

PMAPMA

400GAUI-4C2M400GAUI-4C2M

PMAPMA

800GMSA

PMDPMD

MEDIUMMEDIUM8x100G

PSM8

Figure6–800GSR8blockdiagrams

3.2TechnicalFeasibilityof8x100Gsolutions

Asmentionedabove,signalingrateupto100Gperlanemaylimittheevolvementofmulti-modefiber(MMF)basedsolution

from400G-SR8to800G-SR8.BasedonthetheoreticalmodelusedinIEEE,wecanreckonthatthetransmissiondistancethat

MMFcansupportisnomorethan50masthebaudrateupto50G(SeeTable3).Thelimitationfactorsarefromthelimited

bandwidthofVCSELandthemodaldispersionofMMF.Withtheoptimizationindevices,fibermediumaswellasenhanced

DSPalgorithms,100mMMFtransmissionmayberealizedatthecostofhigherexpense,higherlatency,andlargerpower

consumption.Hence,in800GPluggableMSA,werecommendthatthe800G-SR8scenarioistakenoverbySMFbasedsolution.

05www.800G

EnablingTheNextGenerationOf

Cloud&AiUsing800Gb/sOpticalModules

Table3–FiberchannelbandwidthandtransmissiondistanceofMMFreckonedbythetheoreticalmodelusedinIEEE

Fiberchannel

Transmission

BitrateSignalTypeFiberTypebandwidthIEEEstandards

Distance(m)

(GHz•km)

50GbpsPAM4OM42.301100m50G-SR,100G-SR2

200G-SR4,400G-

50GbpsPAM4OM31.54170mSR8

100GbpsPAM4OM4/OM52.301/2.37750mDefinednow

100GbpsPAM4OM31.54135m-

InordertoguaranteetheadvantagesonthecostandpowerconsumptionoftheSMFbasedsolution,reasonablePMD

standardrequirementsareindispensablein800G-SR8.ThePMDrequirementstobedefinedshouldensurethat1)diverse

transmittertechniques,suchasDML,EML,andsiliconphotonics(SiPh),canbeappliedinsuchscenario;2)allthepotentialof

thecomponentscanbereleasedadequatelytoachievethetargetinglinkperformance;3)keyparametersinPMDlayersshould

berelaxedasmuchaspossible,inthecontextofmaintainingareliablelinkperformance.Accordingtothesethreeprinciples,we

willconductsomebriefinvestigationsanddiscussionsasfollows.

ThepowerbudgetoftheSMFbased800G-SR8solutionwouldbequitesimilarwiththatdefinedinIEEE400G-SR8.Theonly

issuetobedeterminedistheinsertionlossofnewdefinedPSM8SMFconnectors.ItmeansthatthepowerbudgetinSR

scenariocanbeachievedwithoutahitchbasedoncurrentlymatureopticalandelectroniccomponentsandDSPASICsused

in400GEopticalinterconnection.Therefore,apartfromspecifyingtheconnectorforthePSM8modules,thekeyissueforthe

definitionofPMDparametersin800SR8scenarioistofindoutthesuitableopticalmodulationamplitude(OMA),extinction

ratio(ER),transmitterdispersioneyeclosurequaternary(TDECQ)ofthetransmitterandsensitivityofreceiver.Inordertoset

theseparametersintothesuitableposition,thebiterrorration(BER)performanceofthediversetransmittersisinvestigatedand

assessed.

EMLBERvs.OMASiPh.BERvs.OMADMLBERvs.OMA

1.00E-021.00E-021.00E-02

1.00E-031.00E-031.00E-03

1.00E-041.00E-041.00E-04

1.00E-051.00E-051.00E-05

BER1.00E-06BER1.00E-06BER1.00E-06

1.00E-071.00E-071.00E-07

FEC:KP4FEC:KP4FEC:KP4

1.00E-081.00E-081.00E-08

EMLonlinetestresultSiPh.onlinetestresultDMLonlinetestresult

1.00E-091.00E-091.00E-09

-10-8-6-4-202-10-8-6-4-202-10-8-6-4-202

OMA(dBm)OMA(dBm)OMA(dBm)

(a)(b)(c)

Figure7–(a)EMLBERvs.OMAresultsbasedoncommercialavailable400GDSPASICs;(b)SiliconPhotonicsBERvs.OMAresults

basedoncommercialavailable400GDSPASICs,(c)DMLBERvs.OMAresultsbasedoncommercialavailable400GDSPASICs

Figure7showsthreeBERvsOMAcurvesof100GbpsPAM4signal,whichcorrespondtodifferenttransmittertechnologies

respectively,asonlineresultsandobtainedusingcommercial400GDSPASICs.Actually,theBERperformancesofEMLandSiPh

for100GperlaneillustratedinFigure7(a)and(b)arewell-knownresultssincethesetwosolutionshavebeenextensively

discussedinthepastfewyears.ConsideringrelativelylowlaunchingopticalpowerofSiPhtransmitterandgoodenough

sensitivityofallthreesolutions,theminimumOMArequirementin800G-SR8isrecommendedtoberelaxedappropriately.

www.800G06

EnablingTheNextGenerationOf

Cloud&AiUsing800Gb/sOpticalModules

TheBERperformanceoftheDMLinFigure7(c)showsthattheOMAsensitivityinthiscaseiscomparablewiththatinthecase

ofEMLorSiPh,eventhoughthecommercialDMLusedinherehasrelativelylowerbandwidththanEMLandSiPh.Thisresult

impliesthatthecommercialDSPASICsusedinpracticehavemuchstrongerequalizationabilitythanthereferencereceiverIEEE

definedin400GE,andthusitcansupportthetransmitterwithcomparativelylowbandwidthtoachievethetargetingpower

budget800G-SRrequired.InordertoreleasethepotentialoftheDSPunitadequatelyfor800GSR8PMD,referencereceiverfor

compliancetest(i.e.TDECQ)requirestobere-definedtomatchthepracticalequalizationabilityofcommercialDSPs,i.e.more

tapsnumbersthancurrentlydefined5tapsaredesired.Meanwhile,consideringtherelativelylowsensitivityrequirementinSR

scenarioandrestrictionofthepowerconsumptionofthe800Gmodule,alowcomplexityDSPmodeisrecommendedinfuture

modules.AnotherkeyparameterisERthatisrelatedtothepowerconsumptiondirectly.AlowerERisfavoredaslongasitdoes

notimpactthereliabilityofthelink.Basedontheaboveanalysis,webelievethatalowcostandpowerconsumptionSMF-

basedsolutionisfeasibleandpromisingin800G-SR8scenario.

4.4x200GSolutionforFRScenario

4.1800GFRscenariorequirementanalysis

200GperlanePAM4technologyisthenextmajortechnologicalstepforopticalintensitymodulated,directdetection

interconnectsandwillbethefoundationfora4-lane800Gconnectivity,aswellasanessentialbuildingblockforfuture1.6Tb/

sinterconnects.AsshowninFigure8,theMSAwilldefinethefullPMDandpartialPMAlayersincludinganewlowpower,low

latencyFECasawrapperontopoftheKP4FECofthe112Gelectricalinputsignals,inordertoimprovethenetcodinggain

(NCG)ofthemodem.Oneofthekeygoalsofthisindustryalliancewillbethedevelopmentofnewwidebandwidthelectrical

andopticalanalogcomponentsforthetransmitterandreceiverassembliesincludingdigital-to-analogandanalog-to-digital(AD/

DA)converters.Inordertoachievetheaggressivepowerenveloptargetsofpluggablemodules,theDSPchipswillbedesigned

inCMOSprocesswithlowernmnodeandemploylowpowersignalprocessingalgorithmstoachieveequalizationofthechannel.

MACANDHIGHERLAYERS

RECONCILIATION

8x112G8x112G

400GMII400GMII

400GBASE-R400GBASE-R

PCSPCS

DSPDSP

PMAPMA

400GAUI-4C2M400GAUI-4C2M1234

PMAPMAMux

800GMSA

PMD

MEDIUM

NewPMD4x224G4x224G

PSM4CWDM4

Figure8–800GFR4/PSM4blockdiagrams

07