V4ACIG_Disable_Dataplane_Learning_mihiguch

MinakoHiguchi,TechnicalMarketingEngineer,INSBU,Lastupdate:Sep26th2018,ACIversion4.0(Ganga),DisableDataplaneIPLearning,OverviewUsecaseexamplesConsiderations,Agenda,Overview,Bydefault,ACIfabriclearnsendpointIPandMACthroughARP/GARP/NDandalsothroughdataplane.Intraditionalnetwork,IPandMACmappingishappenedthroughARP/GARP/ND(notthroughdataplane).SothereisadifferencebetweenACIfabricandtraditionalnetwork.Forsomeusecases,weneedtodisabledataplaneendpointIPlearning.(Forexample,DSRetc)Thisdocumentexplainsthebehaviorandconfigurationexample,usingoneusecase.Relateddocuments,ACIEndpointlearning,Wehavetwooptions.(Priorto4.0release)ConfigureL4-L7VIPunderEPGNeedtospecify/32hostIPtodisabledatapathlearningGoodforthecasewhenweknowwhichIPweshoulddisabledataplanelearning.TheonlytestedandsupportedusecaseforthisoptioniswithLayer2DSR.Thereisdesignconsideration.(Pleaseseeendpointlearningwhitepaperfordetail)DisabledataplanelearningonBDGoodforthecasewhenwehavemanyIPsweshoulddisabledataplanelearningorwearenot100%surewhichIPsweshoulddisabledataplanelearning.(forexample,serversaremanagedbydifferentteam)ItsforPBRusecaseonly.Thereisaconsiderationforgeneralusecase.DisablingdataplanelearningattheBDlevelisnotconsideredaGeneralAvailabilityfeature.PleaseseeEDCS-1545591fordetail.,TodisabledataplaneIPlearningPriorto4.0release,DisabledataplanelearningonBDIfingressToRisSugarbowl,remoteToRdoesdataplanelearningifitsnotserviceEPGsinceDL(DisableLearning)bitisNotsetiniVXLANheaderpercurrentimplementation.Soitdoesntworkwithnonservicegraphconfiguration.NorthStarbasedToRisfinesinceNorthStarbasedToRdoesntcheckwhetheritsserviceEPGornot.ButitsnotvalidatedbyQAandtherearesomeconsideration.ForgeneralusecasebesidesPBR,wearegoingtosupportDisableDataplaneIPlearningperVRFin4.0release.,Knownlimitation:CSCvc20477,DisabledataplaneIPlearningonVRFItsavailablefornon-PBRcaseaswell.DataplaneIPlearningisdisabledonVRFL3multicastissupported.LocalMACsandremoteMACsstillgetlearnedfromdataplane.LocalIPsarenotlearnedfromdataplane.RemoteIPsarenotlearnedfromunicastpacketsbutfrommulticastpacketsfromdataplane.LocalIPsstillgetlearnedfromARP/GARP/ND.WhenDataplaneIPlearningisdisabledonVRFExistingremoteIPsareflushedimmediately.ExistinglearntlocalIPsareretained,butwillbeagedunlesscontrolplanepacketskeepitaliveBounceentrieswillbedeletedwhenbouncetimerexpires,TodisabledataplaneIPlearningAfter4.0release(NEW),TenantNetworkingVRFsVRF,Configuration,IPData-planeLearningEnabledbydefault,APICCLIconfiguration,F2-APIC1(config)#tenantABCF2-APIC1(config-tenant)#vrfcontextvrfF2-APIC1(config-tenant-vrf)#ipdataplanelearning?disabledDisableipDataPlaneLearningVrfKnobF2-APIC1(config-tenant-vrf)#ipdataplanelearningdisabledF2-APIC1(config-tenant-vrf)#noipdataplanelearningdisabled,SwitchCLIcheckstatus,F2-P1-Leaf-301#showsysteminternalepmvrfABC:vrfdetailVRFABC:vrfvrftype:Tenant:vrfvalid:yescontextid:7:vnid:2392068Scope:2392068:Sclass:16386EPretentionpolicyvalid:YesLocalEPtimeout:900:RemoteEPtimeout:300EPbouncetimeout:630:EPholdtimeout:300EPmovefrequency:256Valid:Yes:LearnEnable:Yes:IPLearnEnable:NoEndpointcount:5:,Usecaseexample,Usecaseexample(whenweneeddisabledataplaneIPlearning?),Usecaseexample1,IP:MAC:SDefaultGW:54,Server2(Standby)VIP:00DefaultGW:54MAC:B,Server1(Active)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,1:TrafficfromsourceSrcIP:SrcMAC:SDestIP:00DestMAC:BDsubnetMAC,E1/1,E1/1,E1/2,Leaf1,Leaf2,Spine,Ifitsroutedtraffic,remoteMACisnotlearnt.,Usecaseexample1(cont)Server2takesoverActiverole,IP:MAC:SDefaultGW:54,Server2(Active)VIP:00DefaultGW:54MAC:B,Server1(Standby)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,2:GARPVIP:00MAC:B,E1/1,E1/1,E1/2,Leaf1,Spine,Leaf2,Inthisexample,remoteIPinformationonLeaf1isnotchangedasbothserver1andserver2areundersameleaf.,Usecaseexample1(cont)Server1(standby)sendRST,IP:MAC:SDefaultGW:54,Server2(Active)VIP:00DefaultGW:54MAC:B,Server1(Standby)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,3:SendRSTforexistingconnectionSource:IP:00SourceMAC:A,E1/1,E1/1,E1/2,Leaf1,*Becauseofdataplanelearning,EPtableisupdated*,Spine,Leaf2,Inthisexample,remoteIPinformationonLeaf1isnotchangedasbothserver1andserver2areundersameleaf.,Usecaseexample1(cont)Clienttriestore-connect,IP:MAC:SDefaultGW:54,Server2(Active)VIP:00DefaultGW:54MAC:B,Server1(Standby)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,E1/1,E1/1,E1/2,Leaf1,*TrafficstillgoestoServer1!,5:TrafficfromsourceSrcIP:SrcMAC:SDestIP:00DestMAC:BDsubnetMAC,Spine,Leaf2,UsecaseexampleIfdataplaneIPlearningisdisabled,LocalMACsandremoteMACsstillgetlearnedfromdataplane.LocalIPsarenotlearnedfromdataplane.RemoteIPsarenotlearnedfromunicastpacketsbutfrommulticastpacketsfromdataplane.LocalIPsstillgetlearnedfromARP/GARP/ND.,Usecaseexample1,IP:MAC:SDefaultGW:54,Server2(Standby)VIP:00DefaultGW:54MAC:B,Server1(Active)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,1:TrafficfromsourceSrcIP:SrcMAC:SDestIP:00DestMAC:BDsubnetMAC,E1/1,E1/1,E1/2,Leaf1GotoSpineproxy,Leaf2,Spine,RemoteIPsarenotlearnedfromunicastpackets,*PolicyisappliedonegressLeaf,Usecaseexample1(cont)Server2takesoverActiverole,IP:MAC:SDefaultGW:54,Server2(Active)VIP:00DefaultGW:54MAC:B,Server1(Standby)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,2:GARPVIP:00MAC:B,E1/1,E1/1,E1/2,Leaf1,Spine,Leaf2,RemoteIPsarenotlearnedfromunicastpackets,LocalIPsstillgetlearnedfromARP/GARP/ND,Usecaseexample1(cont)Server1(standby)sendRST,IP:MAC:SDefaultGW:54,Server2(Active)VIP:00DefaultGW:54MAC:B,Server1(Standby)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,3:SendRSTforexistingconnectionSource:IP:00SourceMAC:A,E1/1,E1/1,E1/2,Leaf1,Spine,Leaf2,LocalMACsstillgetlearnedfromdataplane.LocalIPsarenotlearnedfromdataplane,RemoteIPsarenotlearnedfromunicastpackets,Usecaseexample1(cont)Clienttriestore-connect,IP:MAC:SDefaultGW:54,Server2(Active)VIP:00DefaultGW:54MAC:B,Server1(Standby)VIP:00DefaultGW:54MAC:A,VRF1,AllBDsareinsameVRF,BD154,BD254,E1/1,E1/1,E1/2,Leaf1,5:TrafficfromsourceSrcIP:SrcMAC:SDestIP:00DestMAC:BDsubnetMAC,Spine,Leaf2,IfsourceanddestinationendpointsareunderdifferentLeafs.EPGEPG(L3traffic)Policyisappliedonegressleafsinceingressleafcantresolvedestinationclass.EPGEPG(L2traffic)PolicycanbeappliedoningressleafsinceremoteleaflearnsremoteMAC.(butnotremoteIP)EPGL3outPolicycanbeappliedoningressleafsincethedestinationisL3out.,DisabledataplaneIPlearningconsiderationwherecontractpolicyisapplied?,Considerations,DataplaneLearningDisablefeaturescomparison,2ndgenleafs(EX/FX/FX2)1stgen,*InnerMACofVXLANpacketfromfrontpanelportcannotbelearned.Pleaseseelateroftheslides.,AnycastserviceWhenanycastIP/MACisenabled:Localendpointlearningisenabledforpacketsfromanycastsource(MAC/IP)Remoteendpointlearningdisabledforpacketswithanycastsource(MAC/IP)WithdataplaneIPlearningfortheVRFenabled,localanycastIPandMACarelearntfrombothdataandcontrolplane.RemoteanycastIP/MACisnotlearnt.WithdataplaneIPlearningfortheVRFdisabled,localanycastIPisnotlearntfromdataplane.LocalanycastIPmaybeagedoutunlessitsrefreshedviahosttracking(ARP/ND).,FeatureInteraction,RogueEndpointDetectionWithdataplaneIPlearningfortheVRFenabled,rogueIPswillbedetectedviadataandcontrolplanemoves.WithdataplaneIPlearningfortheVRFdisabled,rogueIPswillonlybedetectedviacontrolplanemoves,notdataplanemoves.,FeatureInteraction,DirectServerReturn(DSR)ForDSRusecase,useofL4-L7VIPknobisstillrecommended.WithL4-L7VIPconfigurationandIPdataplaneIPlearningenabledDSRwillworkasexpectedandlearningfortheL4-7VIPconfiguredisdoneonlythroughcontrolplane.WithL4-L7VIPconfigurationandIPdataplaneIPlearningdisabledClienttoLoadBalancer:Noimpacthere.NoremoteIPlearntforLBVIP(DSREP).Ingressleafwillusespine-proxy.IfDSREPisinspineCOOPdatabase,proxylookupwillbesuccessful,otherwisewewillgenerategleanforLBandlearnitviacontrolpath.LoadBalancertoserver:Noimpacthere.WesupportL2DSRonly.ItmeanstrafficbetweenLB/ServerisL2.NoIPlearninghappens.ServertoClient:NoRemoteIPforclient.Itwillusespine-proxy.Ifcliententryisdeletedinspine,itshouldbere-learntviaglean.ForclientbehindL3out,noRemoteIPislearntinthiscase.,FeatureInteraction,L4-L7VIPanddisableDataplaneIPlearningexampleClientLB,EPGClient,VRF1(endpointdataplaneIPlearningisdisabled),EPGLB,EPGWeb,1:TrafficfromclientSrcIP:SrcMAC:SDestIP:00(VIP)DestMAC:BDsubnetMAC,Leaf12:GotoSpineproxy,Spine3:GotoLeaf2.,IP:MAC:SDefaultGW:54,BD154,BD254,E1/1,Leaf1,Leaf2,Leaf3,VIP:00MAC:LBDefaultGW:54,RealIP:01Loopback:00(VIP)MAC:DDefaultGW:54,E1/1,E1/1,L4-L7VIPanddisableDataplaneIPlearningexampleLBserver,EPGClient,VRF1(endpointdataplaneIPlearningisdisabled),EPGLB,EPGWeb,4:TrafficfromLBSrcIP:SrcMAC:LBDestIP:00(VIP)DestMAC:D,Leaf25:GotoSpineproxyorGotoLeaf3,Leaf37:Trafficgoestodestination.,Spine,IP:MAC:SDefaultGW:54,BD154,BD254,E1/1,Leaf1,Leaf2,Leaf3,VIP:00MAC:LBDefaultGW:54,RealIP:01Loopback:00(VIP)MAC:DDefaultGW:54,E1/1,E1/1,*RemoteMACisleant,L4-L7VIPanddisableDataplaneIPlearningexampleserverclient,EPGClient,VRF1(endpointdataplaneIPlearningisdisabled),EPGLB,EPGWeb,Leaf39:Trafficgoestospineproxy,Spine10:GotoLeaf1,IP:MAC:SDefaultGW:54,BD154,BD254,E1/1,Leaf1,Leaf2,Leaf3,VIP:00MAC:LBDefaultGW:54,RealIP:01Loopback:00(VIP)MAC:DDefaultGW:54,E1/1,E1/1,8:TrafficfromWebSrcIP:00(VIP)SrcMAC:DDestIP:DestMAC:BDsubnetMAC,Leaf111:GotoClient,*RemoteIPisnotlearnedfromdataplane,*LocalIPisnotlearnedfromdataplane(WithL4-L7VIPunderEPGLB,VIPisnotlearntfromWebEPGevenviacontrolplane.),Northstar/DonnerbasedToRConsiderations,Gen1leafhasconsiderations.WecanpreventremoteIP/MAClearn,butlocalMACislearntviaCPUthroughBroadcom.EvenwithdisabledataplaneIPlearningperVRF,remoteMACaddressesarenotlearned.Thus,HardwareProxymodeonthecorrespondingBDsmustbeconfigured.LocalinnerMACaddressesofVXLANpacketsarenotlearned.,Northstar/DonnerbasedToRconsiderations,Northstar/DonnerbasedToRbehaviorLocalMACislearnt,Broadcom,Northstar/DonnerDataplanelearningdisabled,CPU,Broadcom,Northstar/DonnerDataplanelearningdisabled,CPU,IP:MAC:A,IP:MAC:B,E1/1,E1/1,Leaf1,Assumingthereisnocontrolplanelearningyet.Letsfocusondataplanelearningbehavior.Ifwedisabledataplanelearning,leafdoesntlearnremoteendpoints.(andMACBonLeaf1),Leaf1,Leaf2,Northstar/DonnerbasedToRbehaviorLocalMACislearnt,Broadcom,Northstar/DonnerDataplanelearningdisabled,CPU,Broadcom,Northstar/DonnerDataplanelearningdisabled,CPU,IP:MAC:A,IP:MAC:B,E1/1,E1/1,Leaf1,*BroadcomalwayslearnlocalMAC,whichmakesNorthstarlearnslocalMACviaCPU.,Leaf1,Leaf2,Northstar/DonnerbasedToRbehaviorServer1(standby)sendRST,IP:192.168