V4ACIH_Multi-Tier_mihiguch

MinakoHiguchi,LionelHercot,TechnicalMarketingEngineer,INSBU,Lastupdate:03/25/2019,H-releaseMulti-Tiertopologysupport,PriortoH-release,ACIfabricmustbe2-tieredtopologywithSpineLeaf.InH-release,ACIfabriccannowbeMulti-tieredtopologywithSpineLeaf-Tier-2-Leaf.Usecase:Customerswhoprefer3tiersforcablingconsiderationandbettervisibilityandenforcementofpolicydomainoveraFEXbasedtopology,Summary,2Tier(Spine,Leaf),3Tier(Spine,Leaf,Tier-2-leaf),Spine,Leaf,Spine,Leaf(Tier-1-Leaf),Tier-2-leaf,Multi-PodoftenusedtosupportMultipleBlocksconnectedwithinthesameDC(betweenhalls,buildings,withinthesameCampus)Multi-TierprovidesanotheroptiontoaddressesCablinglimitationsEaseMigrationsfromNexus7/5/2Kdesigns,Multi-TierFabricTopologiesCablePlanSimplification,ACI:Multi-TierArchitecture,SeamlessMigrationFromLegacy3-TierArchitectures,ThreeTierACIFabric,VerticalExpansionOfACIPolicyDomain,InvestmentProtection:AddressCablingLimitationorReuseExistingCablePlan,ReplaceFEXArchitectureWith2ndTierLeaf:BetterVisibility&PolicyEnforcement,1,2,3,SimplifyN2/N5/N7kMigrationtoACI,4,Spine:AnyEX/FX/Cspines(9332C,9364C)Tier-1Leaf:AnyEX/FX/FX2exceptN9K-C93180LC-EXTier-2Leaf:AnyEX/FX/FX21stgenisnotsupportedMaxnumberofTier-1-leaf+Tier-2-leafisequaltothemaxnumberofLeafinthefabric(200perpod.400perMulti-Pod)MaxnumberofTier-2-leafperLeafis48.,SupportedPlatformsinACI4.1,MultipleTiersofswitcheswithinasingle“Pod”InitialSupportforLeafLeafSpine,LeafSpineSpinetofollowTier-1LeafthatsconnectedtoTier-2LeafcanhaveEP/L3out/APICconnectivitysameasusual.APICcontrollercanbeconnectedtoTier-1LeaforTier-2Leaf,Multi-TierFabricTopologiesconsiderations,2ndTierLeaffabricportconnectsto1stTierLeafsfabricport.Allportsof1stTierLeafcanbeconvertedtofabricportusingportprofilefeature2ndTierLeafcanconnecttomultiple1stTierLeaf.ItcouldbeanadvantageforACIdesignwherecustomercanconnecttomorethan2upstreamswitchesincomparisontotraditionaldoublesidedvPCdesignwithonly2upstreamswitches.,Connectivityrequirementto2ndTierLeaf,Configuration,FabricInventoryFabricMembershipNodesPendingRegistrationCreateFabricNodeMember,Tier-2-leafconfigurationcanbedoneaspartofregistration.IfthenodeisalreadyrunningasTier-1leaf,weneedto“Removefromcontroller”first,thenregisteritasTier-2leaf.,Spine,Tier-1-leaf,Tier-2-leaf,Usefabriclink:SpinetoTier-1leafconnectionTier-1leaftoTier-2leafconnection,Usefabriclink:SpinetoTier-1leafconnectionTier-1leaftoTier-2leafconnection,Multi-Tierplatformsupportmatrix,Multi-tierarchitecturedoesntchangetheLeafscaleforaPod.Both1sttierand2ndtierwillbecountedasleafwhilecalculatingtotalleafscaleTier-1orTier-2optionconfigurationcanbedoneaspartofregistration.Inordertoconvertaleaftoa2ndtierleafortheotherwayaround,aswitchhastobedecommissioned(Removefromcontroller),configuredandthenregister.Multi-tierarchitecturewillsupportMulti-Pod,RemoteLeafandMulti-SiteinACI4.1Tier-2leafscannotbeconnectedtoaRemoteleaf(actingas1stTier).”SupportforvPodandGolfwillcomeinlaterrelease.GXLeaf(400G)willbeabletouseas1sttierand2ndtierleaf.(roadmap)WhenBUMtrafficisoriginatedbyaTier-1leaf,BUMtrafficissenttoTier-2leafnodesconnectedtothatTier-1leafinadditiontospinesevenifthatVRFandBDarenotdeployedontheTier-2leaf.MaxnumberofECMPlinkssupportedbyTier-2-leaftoTier-1-leafis18Recommendation:keepatleastoneofthedefaultuplinksusedasfabriclinktoensurerecoveryincaseoffactoryreset,Considerations,Leafswitchesoninitialbootstillperformcablingchecktodisableanyleaftoleaflink.ThusthelinksbetweenTier-1LeaftoTier-2Leafbecomesout-of-service,butinterfaceisupandDHCPpacketsstillgothroughforswitchregistration.Duringregistration,theswitchtypeissetforaleaf.Afterregistration,thecablingcheckisrelaxedtoallowTier-1toTier-2link.Tier2leafwillstoreitsswitchtypeinNVRAMsoonsubsequentrebootsitdoesnotneedtogothroughthecommissioningprocessandcanjoinfabricandenforcetopologychecksimmediately,SwitchBootBehaviorandcablingcheck,Spine,Leaf(Tier-1-Leaf),Tier-2-leaf,SpinetoTier-2leaflinkwillbeout-of-servicebasedonswitchtypeconfiguration.,Tier-2toTier-2leaflinkwillbeout-of-service,Tier-1toTier-1leaflinkwillbeout-of-service,Onceregistrationisdone,Tier-1toTier-2linkisUP,Spine:N9332C,N9364Cor9500Leaf:N9336C-FX2Tier-2-leaf(sLeaf):N93180/93108EX/FXor9336C-FX2Usecase:LeafisEoR.Tier-2-leafisToR.40/100GfabriclinkforSpineandLeaf40/100GfabriclinkforLeafandTier-2-leaf10/25G/40G/100GserverlinkWantstoavoidfull-meshcablingsacrossrows.,Topologyexample(10G/25GAccess),Spine:N9332C,N9364Cor9500Leaf:N93180EX/FX*or9336C-FX2Tier-2-leaf(sLeaf):N9348FXPUsecase:LeafisEoR.Tier-2-leafisToR.40/100GfabriclinkforSpineandLeaf10/40G/100GfabriclinkforLeafandTier-2-leaf1GserverlinkWantstoavoidfull-meshcablingsacrossrows.*N9K-C93180LC-EXcantbeLeaf,Topologyexample(1GAccess),MigrationConsiderations,2Tier(Spine,Leaf)or2Tier+FEX,3Tier(Spine,Leaf,Tier-2-leaf),Spine,Leaf,Spine,Leaf,Tier-2-leaf,Tier-2-leafmustbe4.1orlater.IfleafsthatwillbeaddedasTier-2-leaf,youneedtoupgradetheswitchesto4.1first.,FEX,MigrationConsiderations,2Tier(Spine-Leaf),3Tier(Spine-Leaf-Tier-2-leaf),Spine,Leaf,Spine,Leaf,Tier-2-leaf,1stgenleafscantbeusedforMulti-Tier,but1stgenleafsasdedicatedleaf(withoutaTier-2-leafconnection)canco-exit.Spinemustbe2ndgen.,3TiercapableleafswithTier-2-leafconnection,1stgenleafswithoutTier-2-leafconnection,1stgenleafs,Useca