V4ACIH_L1L2PBR_mihiguch

MinakoHiguchi,TechnicalMarketingEngineer,INSBU,Lastupdate:03/21/2019,DRAFT,H-releaseL1/L2PBR,PriortoAPICversion4.1(H-release),PBRdestinationdevicemustbeL3(routedmode).StartingfromAPICversion4.1,PBRcanbeusedwithL1orL2device,forexampleinlineIPS,transparentFW,etc.Benefit:SelectiveandflexibleserviceinsertionInsertfirewallwithoutrelyingonBD/VLANstitching.Redirectspecifictrafficbasedoncontract(sourceEPG,destinationEPGandfiltermatching)RequireEX/FXleafonward(no1stgenleafsupport)L1/L2/L3servicedevicecanbemixedinaservicegraph.,Overview,L1deviceNoVLANtranslationonaservicedeviceBothofservicelegsusesameVLANTypicallycalled“inline”mode.L2deviceVLANtranslationonaservicedeviceBothofservicelegsusedifferentVLANDestinationMAClookupmayhappenTypicallycalled“transparent”or“bridged”mode.IncaseoftypicalL1/L2deviceabove,alltrafficacrossVLANsneedstogothroughL1/L2deviceasitspartoftrafficpathalways.ByusingL1/L2PBRinACI,selectivetrafficredirectionispossible.,TypicalL1/L2deviceinsertiondesign,VLANstitchingiscommonlyusedtoinsertL2device.,InlineiscommonlyusedtoinsertL1device.,ServicegraphismandatorytousePBR.L1/L2PBRissupportedwithunmanagedmodeonly.Asitspartofcontract,PBRisappliedonunicast-trafficonly.L1/L2servicedevicemustbeinphysicaldomain.L1/L2servicedeviceBDmustbededicatedBDthatcannotbesharedwithotherendpoints.L1/L2devicemustbetwo-armmode.ExternalandinternallegsofL1/L2servicedevicemustbeindifferentBD.ExternalandinternallegsofL1devicemustbeconnectedtodifferentleafnodes.Perportvlanisnotsupported.L2devicedoesnthavethisconsideration.Active/Standbymodeonly.NoActive/ActivedeploymentsupportunlikeL3PBR.Upto2L1/L2destinationsperPBRdestinationgroupcanbeadded.Morethan2isnotsupported.TrackingisrequiredforActive/Standbymode.Asactive-activeisnotsupported,thresholdisnotapplicable.Downactionisdenywhentrackingisenabled.DownactionpermitcantbesetinHrelease(CSCvn36571).L2Ping(Ethertype0 x0721)isusedfortracking.L2Pingisexchangedbetweenleafnodes,whichisgoingthroughservicedevice.Thus,L1/L2deviceneedstopermitEthertype0 x0721.L1/L2PBRissupportedinMulti-pod,Multi-siteandRemoteLeaf.Multi-nodePBRissupported.L1/L2/L3servicedevicecanbemixedinaservicegraph.,Designconsiderations,HowL1/L2PBRworks,FromACIforwardingperspective,L1/L2/L3PBRservicehandlinghasnotmuchdifference.TrafficisredirectedPBRdestinationMACthatisstaticMACendpointprogrammedonaleafinterfaceconnectedtoanL1/L2deviceinterface.ThustrafficcanberedirectedtoleafinterfaceconnectedtoL1/L2device.MACaddressofstaticMACendpointisautomaticallygeneratedbyAPICbydefault.Itsconfigurable.RedirecttrafficdestinedtoaPBRdestination(staticMACendpoint)isalwayssenttoL2spineproxythenforwardtoservicedevice.Forthetrafficreturningfromservicedevice,packetsaretreatedasL3trafficandisalwaysrouted.L1/L2servicedeviceBDisnotautomaticallycreatedthroughservicegraph.IProutingmustbeenabled(butnoBDsubnetisrequired)DataplaneIPlearningisautomaticallydisabledsimilartoL3PBRcase,Forwardingbehavioroverview,L1/L2PBRPacketflowoverview,VRF1,ShadowEPGSvc-consumer,ShadowEPGSvc-provider,eth1,eth2,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,PBRNode(L2),IfservicedevicedoesdestinationMAClookup,FirewallneedstohaveMAC-AandMAC-BinitsMACaddresstable.AsMAC-AandBarenotusedassourceMAC,staticMACconfigurationisrequired(pleaseseepacketwalkstartingfromnextslide.),Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),StaticMACendpointisprogrammedMAC-AviaEth1/1.AsifMAC-AisconnectedtoEth1/1,Eth1/1,Eth1/1,StaticMACendpointisprogrammedMAC-BviaEth1/1AsifMAC-BisconnectedtoEth1/1,IP/MAClearningisdisabledonserviceBDs.,IP/MAClearningisdisabledonserviceBDs.,L1/L2PBRPacketflowoverview(incoming),VRF1,ShadowEPGSvc-consumer,ShadowEPGSvc-provider,eth1,eth2,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,PBRNode(L2),FirewallMACtable,1:trafficfromClienttoWebSrcclass:ClientDestClass:WebSrcIP:SrcMAC:Client-MACDestIP:DestMAC:Leaf-MAC,4:trafficcomesbackSrcclass:Svc-providerDestClass:WebSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-A,6:traffictoWebSrcIP:SrcMAC:BD2-MACDestIP:DestMAC:Web-MAC,2:policyapplied(PBR)SrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-ASegmentID:Svc-Con-BD,5:RoutingtoproviderToRNewbehavioronBDRegardlessdestinationMAC,trafficisroutedbasedonDestinationIP.(BDsubnetIPisnotrequired)DL(DontLearnBit):1,Eth1/1,Eth1/1,*TTLisdecreasedasitsrouting,3:traffictoServicenodeSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-A,Svc-Con-BD,Svc-Prov-BD,L1/L2PBRPacketflowoverview(return),VRF1,ShadowEPGSvc-consumer,ShadowEPGSvc-provider,eth1,eth2,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,PBRNode(L2),FirewallMACtable,1:trafficfromWebtoClientSrcclass:WebDestClass:ClientSrcIP:SrcMAC:Web-MACDestIP:DestMAC:Leaf-MAC,4:trafficcomesbackSrcclass:Svc-consumerDestClass:ClientSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-B,5:traffictoWebSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:Client-MAC,2:trafficredirectedSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-BSegmentID:Svc-Prov-BD,Eth1/1,Eth1/1,5:RoutingtoconsumerToRRegardlessdestinationMAC,trafficisroutedbasedonDestinationIP.(BDsubnetIPisnotrequired)DL(DontLearnBit):1,*TTLisdecreasedasitsrouting,Svc-Con-BD,Svc-Prov-BD,3:traffictoServicenodeSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-B,L1/L2PBRPacketflowoverview(incoming)ConsumerandproviderinsameSubnet,VRF1,ShadowEPGSvc-consumer,ShadowEPGSvc-provider,eth1,eth2,BD1(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,PBRNode(L2),1:trafficfromClienttoWebSrcclass:ClientDestClass:WebSrcIP:SrcMAC:Client-MACDestIP:DestMAC:Web-MAC,3:trafficcomesbackSrcclass:Svc-providerDestClass:WebSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-A,5:traffictoWebSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:Web-MAC,2:trafficredirectedSrcIP:SrcMAC:Leaf-MACDestIP:DestMAC:MAC-A,4:RoutingtoproviderToRRegardlessdestinationMAC,trafficisroutedbasedonDestinationIP.(BDsubnetIPisnotrequired)DL(DontLearnBit):1,Eth1/1,Eth1/1,*TTLisdecreasedasitsrouting,Svc-Con-BD,Svc-Prov-BD,FirewallMACtable,Tracking(Active-Standby),VRF1,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,StaticMACendpointsprogrammedNode1-Prov-MACviaEth1/1Node2-Prov-MACviaEth1/2,StaticMACendpointsprogrammedNode1-Con-MACviaEth1/1Node2-Con-MACviaEth1/2,Eth1/1,Eth1/1,Eth1/2,Eth1/2,Active,Standby,FirewallMACtable,Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),Tracking(Active-Standby)L2pingpathusingactivedeviceissuccess,VRF1,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,Eth1/1,Eth1/1,Eth1/2,Eth1/2,L2Ping(Success)Source:Node1-Con-MACDestination:Node1-Prov-MAC,L2Ping(Success)Source:Node1-Prov-MACDestination:Node1-Con-MAC,Active,Standby,StaticMACendpointsprogrammedNode1-Prov-MACviaEth1/1Node2-Prov-MACviaEth1/2,StaticMACendpointsprogrammedNode1-Con-MACviaEth1/1Node2-Con-MACviaEth1/2,Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),FirewallMACtable,Tracking(Active-Standby)L2pingpathusingstandbydeviceissuccess,VRF1,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,Eth1/1,Eth1/1,Eth1/2,Eth1/2,L2Ping(Failed)Source:Node2-Con-MACDestination:Node2-Prov-MAC,L2Ping(Failed)Source:Node2-Prov-MACDestination:Node2-Con-MAC,Active,Standby,Standbydevicedoesntforwardpacket,Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),StaticMACendpointsprogrammedNode1-Prov-MACviaEth1/1Node2-Prov-MACviaEth1/2,StaticMACendpointsprogrammedNode1-Con-MACviaEth1/1Node2-Con-MACviaEth1/2,Tracking(Active-Standby)Notifytootherleafnodes,VRF1,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,Eth1/1,Eth1/1,Eth1/2,Eth1/2,PeriodicSystem-widebroadcastonserviceleafs,announcingNode1isActive.,Active,Standby,Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),Configurationexample,AssumingthereareEPGsandBDsfortheEPGsCreateServiceBDsPBRconfigurationCreateanL4-L7deviceCreateaServiceGraphtemplateCreateaPBRpolicyApplyServiceGraphOptionalconfigurationChangeMACaddressofPBRdestinationTransparentmodeASAspecificconfiguration,Configurationstep,TenantNetworkingBridgeDomains,CreateServiceBDs,IPRouting:enabledNoneedBDsubnetNoneedtochangeIPData-planelearningknob.DataplaneIPlearningisautomaticallydisabled,ShadowEPGSvc-consumer,ShadowEPGSvc-provider,Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),2ServiceBDsarerequired,CreateServiceBDs,ShadowEPGSvc-consumer,ShadowEPGSvc-provider,BD2(54/24),EPGWeb,BD1(54/24),EPGClient,BDsforconsumerandproviderEPGsmusthaveBDsubnet,Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),TenantServicesL4-L7Devices,1:CreateanL4-L7device,Managed:uncheckSeryiceType:OtherDeviceType:PHYSICALFunctionType:L2,Configureconcretedevicesasusual.ConcreteinterfaceinformationwillbeusedforstaticMACEPprogrammingforPBR,L2-ASA-1,L2-ASA-2,G0/0Vlan-711consumer,G0/1Vlan-712provider,G0/1,G0/0,TenantServicesL4-L7ServiceGraphTemplates,2:CreateaServiceGraphtemplate,RouteRedirect:True,TenantPoliciesProtocolL4-L7Policy-BasedRedirect,3:CreateaPBRpolicy,DestinationType:L1,L2orL3CreateIPSLAMonitoringPolicyifneeded,SLAFrequency:60secisdefaultSLAType:L2Ping,3:CreateaPBRpolicy,MAC:optional(itsautomaticallygeneratedbydefault)CreateRedirectHealthGroupifneeded.,L2-ASA-1,G0/0(L2-ASA-consumer1),G0/1(L2-ASA-provider1),Health-groupL2-ASA-1,3:CreateaPBRpolicy,SelectconcreteInterfacefromlist.,Addstandbydeviceifneeded.2L1/L2destinationscanbeaddedincaseofactive/standbywithtracking.Morethan2isnotsupportedandwilltriggerafault.,L2-ASA-1,G0/0(L2-ASA-consumer1),G0/1(L2-ASA-provider1),Health-groupL2-ASA-1,3:CreateaPBRpolicy,SelectconcreteInterfacefromlist.,Inthisexample,weusedifferentRedirectHealthGroups,L2-ASA-1,G0/0(L2-ASA-consumer1),G0/1(L2-ASA-provider1),Health-groupL2-ASA-1,L2-ASA-2,G0/0(L2-ASA-consumer2),G0/1(L2-ASA-provider1),Health-groupL2-ASA-2,3:CreateaPBRpolicy,IPandMACareautomaticallygenerated.MACisconfigurable,butIPisnot.ThoughIPisnotusedforL2pingheader,itsthekeyforalldestinations.WeidentifyadestinationusingtheIPwhilepublishingthetrackinginformation.,CreatePBRpolicyfortheotherlegaswell,3:CreateaPBRpolicy,L2-ASA-1,G0/0(L2-ASA-consumer1),G0/1(L2-ASA-provider1),Health-groupL2-ASA-1,L2-ASA-2,G0/0(L2-ASA-consumer2),G0/1(L2-ASA-provider1),Health-groupL2-ASA-2,CreatePBRpolicyfortheotherlegaswell,3:CreateaPBRpolicy,TenantServicesL4-L7ServiceGraphTemplates,4:Applyservicegraph,Inthisexample,ConsumerEPG:ClientProviderEPG:WebCreateanewcontract,Redirect,provider,consumer,4:Applyservicegraph,Redirect,provider,consumer,Svc-Con-BD,Svc-Prov-BD,PBRpolicy:L2-ASA-consumer,PBRpolicy:L2-ASA-provider,consumer,provider,DeviceSelectionpolicyiscreatedduring“ApplyServiceGraphtemplate”wizard,DeviceSelectionPolicy,Svc-Con-BD,Svc-Prov-BD,PBRpolicy:L2-ASA-consumer,PBRpolicy:L2-ASA-provider,consumer,provider,CheckthereisnofaultonDeployedGraphInstance,DeployedGraphInstance,ConfigureMACaddressforPBRdestination(optional),VRF1,BD2(54/24),EPGWeb,/24MAC:Web-MAC,BD1(54/24),EPGClient,/24MAC:Client-MAC,Inthisexample:StaticMACendpointsprogrammed02:C7:7E:00:00:10viaASA1G0/002:C7:7E:00:00:20viaASA2G0/0,Inthisexample:StaticMACendpointsprogrammed02:C7:7E:00:00:11viaASA1G0/102:C7:7E:00:00:21viaASA2G0/1,G0/0,G0/1,G0/0,G0/1,L2-ASA-1,L2-ASA-2,ConfigureFirewallMACtable,Svc-Con-BDUnicastRouting(noIP),Svc-Prov-BDUnicastRouting(noIP),vlan-711,vlan-712,ConfigureMACaddressforPBRdestination(optional),IfservicedevicedoesdestinationMAClookup,theseMACaddressesmustbeconfiguredinMACaddresstableofthedevice.Pleaseseenextslide.,Permitethertype0 x0721forL2pingDisableMAC-learningConfigurethemac-addresstable.InternalserviceEPMACisconfiguredtobeforwardedtoexternalinterface.ExternalserviceEPMACisconfiguredtobeforwardedtointernalinterface.,TransparentmodeASAspecificconfiguration,mac-learnexternalIfdisablemac-learninternalIfdisable,access-listPermit-Ethethertypepermit721access-groupPermit-EthininterfaceexternalIfaccess-groupPermit-EthininterfaceinternalIf,mac-address-tablestaticexternalIf02c7.7e00.0011mac-address-tablestaticinternalIf02c7.7e00.0010mac-address-tablestaticexternalIf02c7.7e00.0021mac-address-tablestaticinternalIf02c7.7e00.0020,firewalltransparentinterfaceGigabitEthernet0/0bridge-group1nameifexternalIfsecurity-level0interfaceGigabitEthernet0/1bridge-group1nameifinternalIfsecurity-level100interfaceBVI1ipaddress00,Inthisexample,Consumerside:externalIf(G0/0)Providerside:internalIf(G0/1),FirewallMACtable,BVIIPisnotactuallyusedintrafficflowinthisexample.,TheseareMACAddressesofPBRdestination.PleaseseethepreviousslidewheretocheckMACs,Troubleshooting,CLIsonconsumer/providerLeaf,Pod1-Leaf1#showzoning-rulescope2195457+-+-+-+-+-+-+-+-+-+-+|RuleID|SrcEPG|DstEPG|FilterID|Dir|operSt|Scope|Name|Action|Priority|+-+-+-+-+-+-+-+-+-+-+|4126|0|16389|implicit|uni-dir|enabled|2195457|permit|any_dest_any(16)|4127|0|0|implicit|uni-dir|enabled|2195457|deny,log|any_any_any(21)|4128|0|0|implarp|uni-dir|enabled|2195457|permit|any_any_filter(17)|4129|0|15|implicit|uni-dir|enabled|2195457|deny,log|any_vrf_any_deny(22)|4130|49156|16388|default|uni-dir|enabled|2195457|permit|src_dst_any(9)|4131|0|49154|implicit|uni-dir|enabled|2195457|permit|any_dest_any(16)|4132|49155|32770|default|uni-dir|enabled|2195457|permit|src_dst_any(9)|4133|0|16387|implicit|uni-dir|enabled|2195457|permit|any_dest_any(16)|4136|0|49153|implicit|uni-dir|enabled|2195457|permit|any_dest_any(16)|4134|16388|32770|default|bi-dir|enabled|2195457|redir(destgrp-5)|src_dst_any(9)|4135|32770|16388|default|uni-dir-ignore|enabled|2195457|redir(destgrp-4)|src_dst_any(9)|+-+-+-+-+-+-+-+-+-+-+,32770,16388,Redirect,provider,consumer,49156,49155,Zoning-ruleisupdatedwhenaservicegraphisassociatedtoacontract.PBRisappliedonconsumer/providerleaf.,CheckZoning-ruleonconsumer/providerleaf.,CLIsonconsumer/providerLeaf,Pod1-Leaf1#showsysteminternalpolicy-mgrstatsCCPod1-Leaf1#showserviceredirinfo=LEGENDTL:Threshold(Low)|TH:Threshold(High)|HP:HashProfile|HG:HealthGrp=ListofDestGroupsGrpIDNamedestinationHG-nameoperStoperStQualTLTHHPTracking=4destgrp-4dest-111a:55d2:bdb1:49e3:8713:a41c:bcd6L1-L2-PBR:L2-Aenabledno-oper-grp00symmetricyesdest-3d91:2b73:cd34:4fa3:9a98:6879:9269:7ec7-vxlan-2195457L1-L2-PBR:L2-ASA-15destgrp-5dest-18ea:ee89:375d:4dcd:b531:1287:c6d4L1-L2-PBR:L2-Aenabledno-oper-grp00symmetricyesdest-1355:bf04:58cf:48c6:a8a3:b33a:76d1:1c98-vxlan-2195457L1-L2-PBR:L2-ASA-1ListofdestinationsNamebdVnidvMacvrfoperStoperStQualHG-name=dest-3d91:2b73:cd34:4fa3:9a98:6879:9269:7ec7-vxlan-2195457vxlan-1599073602:C7:7E:00:00:11L1-L2-PBR:VRF1enabledno-oper-destL1-L2-PBR:L2-ASA-1dest-111a:55d2:bdb1:49e3:8713:a41c:bcd6:542e-vxlan-2195457vxlan-1599073602:C7:7E:00:00:21L1-L2-PBR:VRF1disabledstandby-tracked-as-downL1-L2-PBR:L2-ASA-2dest-1355:bf04:58cf:48c6:a8a3:b33a:76d1:1c98-vxlan-2195457vxlan-1589244602:C7:7E:00:00:10L1-L2-PBR:VRF1enabledno-oper-destL1-L2-PBR:L2-ASA-1dest-18ea:ee89:375d:4dcd:b531:1287:c6d4:b38d-vxlan-2195457vxlan-1589244602:C7:7E:00:00:20L1-L2-PBR:VRF1disabledstandby-tracked-as-downL1-L2-PBR:L2-ASA-2ListofHealthGroupsHG-NameHG-OperStHG-DestHG-Dest-OperSt=L1-L2-PBR:L2-ASA-2disableddest-18ea:ee89:375d:4dcd:b531:1287:c6d4:b38d-vxlan-2195457downdest-111a:55d2:bdb1:49e3:8713:a41c:bcd6:542e-vxlan-2195457downL1-L2-PBR:L2-ASA-1enableddest-3d91:2b73:cd34:4fa3:9a98:6879:9269:7ec7-vxlan-2195457updest-1355:bf04:58cf:48c6:a8a3:b33a:76d1:1c98-vxlan-2195457up,Trackingthoughactiveservicenodeissuccessful.Inthisexample,L2-ASA-1isactive.L2-ASA-2isstandby.,CheckTrackingstatusonconsumer/providerleaf,CLIsonserviceLeaf,Pod1-Leaf2#vshCiscoiNX-OSDebugShellThisshellshouldonlybeusedforinternalcommandsandexistsforlegacyreasons.Usershoulduseibashinfrastructureasthiswillbedeprecated.Pod1-Leaf2#showsysteminternalslamonrecordslink-layer-SLAMONRECORDs:-vnid:2195457ip:111a:55d2:bdb1:49e3:8713:a41c:bcd6:542estate:DOWNvnid:2195457ip:1355:bf04:58cf:48c6:a8a3:b33a:76d1:1c98state:UPUptime:00:02:34vnid:2195457ip:18ea:ee89:375d:4dcd:b531:1287:c6d4:b38dstate:DOWNvnid:2195457ip:3d91:2b73:cd34:4fa3:9a98:6879:9269:7ec7state:UPUptime:00:02:35,CheckTrackingstatusonserviceleaf,WeidentifyadestinationusingtheIPwhilepublishingthetrackinginformation.,CLIsonserviceLeaf,CheckTrackingstatusonservice;eaf,WeidentifyadestinationusingtheIPwhilepublishingthetrackinginformation.,Pod1-Leaf2#showsysteminternalepmendpointvrfL1-L2-PBR:VRF1VRF:L1-L2-PBR:VRF1:Contextid:8:Vnid:2195457MAC:02c7.7e00.0011:NumIPs:0Vlanid:23:Vlanvnid:9204:VRFname:L1-L2-PBR:VRF1BDvnid:15990736:VRFvnid:2195457PhyIf:0 x1a008000:TunnelIf:0Interface:Ethernet1/9Flags:0 x806:sclass:49155:Refcount:4EPCreateTimestamp:03/21/201900:49:31.7803