新一代云数据中心网络架构

1、新一代云数据中心网络架构议题设计架构方案概述服务保障企业的应用发展趋势：小型化、分布式存储资源计算资源1009080706050403020100性能和可靠性指数Mainframe / RISC / FTEnterprise x86 platforms企业sHyperscale网络更多的参与到计算过程中企业需要新一代的网络架构Extranet接入1Extr接入2安全控制安全控制anet安全控制安全控制Extranet1Extranet2主机生产 业务 B安全控制安全控制生产 业务 A前置3安全控制前置1控制安全前置2安全控制安全控制办公 业务安全控制网管1 网管2安全控制广域 网区核心骨分行分

2、行1主网银备网银安全控制安全控制InternetInternet高速交换核心安全控制安全控制传统网络架构的问题：应用系统及服务服务器的基础架构 存储的基础架构网络基础架构干网通讯的应用2 基础设施灾备中心 (电力, 冷却,布线, 及物理安全)片面提高硬件性能却难以实现云计算的资源池和多云扩展性现有的基础架构相互难以配合的服务不一致的安全策略 相互矛盾的优化方案不一致的业务永续/容灾计划资源利用率低物理资源无法复用重复人力物力投入缺乏灵活性僵化隔离的基础架构任何变动牵涉复杂效率 15,. 25% - Gartner 40% - IDC部署新业务平均8周 - CIO Survey11接2安全控制E

3、xtranet接入安全控制Extranet入安全控制安全控制Extranet1Extranet2主机生产 业务 B安全控制安全控制生产 业务 A前置3安全控制前置1安全控制前置2安全控制安全控制办公 业务安全控制域 区广 网心骨干网核分行2备网银安全控制安全控制InternetInternet灾备中心安全控制高速心交换核安全控制网管1网管2安全控制传统数据中心的业务分区竖井VLAN 110L3L2安全控制主网银VLAN 1120VLAN 2130分行1传统的分区式网络架构导致资源竖井限制了Oracle RAC、HACMP等的集群化并行处理限制了VMware、Hyper-V等虚拟环境需要自由灵活

4、的虚机调度范围 限制了同部门服务器跨区延展（跨区的VLAN延展）传统的分区扩展的方案导致严重问题数据中心技术发展传统生成树传统跨机箱捆绑传统TRILL传统VXLAN环路无负载均衡根本无法扩展不能横向扩展难于管理不支持三层路由 不支持分布式网关不支持三层路由 不支持分布式网关企业的新一代网络架构NG Fabric自动质量按需扩展健壮可靠弹性应用优化应用感知敏捷部署安全可控NGFabric面向云计算、大数据和应用自动化的New Gen FabricCatalystN7K/6K/5K/ 3K/2KN9K/3K/2K传统技术、stp、vssVPC、FEX、Fabricpach(VXLAN)DFAVPC

5、FEXVXLANVXLAN EVPN、ACI企业数据中心所必须具备的弹性Pay as You Grow：动态横向扩展VLAN Anywhere：二层处处可达高可靠：广播控制、差错隔离、松耦合高性能：低延迟，多链路负载均衡易管理：扁平架构，线路清晰11WAN/Core边缘终结所有的主机二层协议：包括MAC学习, ARP, IGMP, LLDP, DHCP内部严格的故障隔离：Fabric内没有生成树、ARP广播、未知单播洪泛等等， 象传统的路由网络一样可扩展VLAN Anywhere：任何服务器可在任意VLAN，VLAN可延展到全网Gateway Anywhere：任意位置都可做任意VLAN路由，

6、同一VLAN网关一致Routing Anywhere：多链路负载均衡，横向按需扩展我们提供的Fabric技术：NG FabricGW IP: 10.1.1.1 GW MAC: 0011:2222:3333核心是增强型的VXLAN横向扩展高可用性动态接入象三层路由网络一样的扩展和隔离性，实现统一的二层/三层转发Cisco的NG Fabric带来优势灵活拓扑APIC新一代数据中心平行扩展示意图（横向扩展，即插即用）13东西流量优化40G Bidi40G干线具备更低的价格，更简单的管理传统40G新型40G端口布线成本成本70%75%应用的可视性：应用健康监测HYPERVISORHYPERVISORH

7、YPERVISORApplication Delivery Controller FirewallHEALTH SCORE96%LATENCY 5Microsecond(s)DROP COUNT 25 Packets DroppedVISIBILITY7VMs3PhysicaliVTEPeVXLANIPPayloadFlagsPolicy GroupVNID应用标识的实现应用的可视性：基千健康状态的资源调整VMsPhysicalApplication Delivery Controller Firewall15Microsecond(s)HEALTH SCORE5926%LATENCY150D

8、ROP COUNT 32550 Packets DroppedVISIBILITY16816以应用为中心对各类软硬件资源实现优化调度Access configuration, VLAN, VSAN, Security, and HardeningOperating System Configuration OS Type, Patch Level, SettingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configuration:

9、worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique service ID, Application revisions, and Storage settingsApplication resources: Server, Storage, NetworkSecurity, OSUplink port configuration, VLAN, VSAN, QoS, and EtherChannelsServer port configuration including LAN

10、 and SAN settingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configuration: worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique user ID (UUID), firmware revisions,and RAID controller settingsService profile as

11、signed to server, chassis slot, or poolUplink port configuration, VLAN, VSAN, QoS, and EtherChannelsServer port configuration including LAN and SAN settingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configuration: worldwide names (WWNs), VSAN

12、s, and bandwidth constraints;and firmware revisionsUnique user ID (UUID), firmware revisions,and RAID controller settingsService profile assigned to server, chassis slot, or poolUplink port configuration, VLAN, VSAN, QoS, and EtherChannelsServer port configuration including LAN and SAN settingsNetwo

13、rk interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configuration: worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique user ID (UUID), firmware revisions,and RAID controller settingsService profile assigned to server, chas

14、sis slot, or poolUplink port configuration, VLAN, VSAN, QoS, and EtherChannelsServer port configuration including LAN and SAN settingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configuration: worldwide names (WWNs), VSANs, and bandwidth const

15、raints;and firmware revisionsUnique user ID (UUID), firmware revisions,and RAID controller settingsService profile assigned to server, chassis slot, or pool策略组合为 Application Profile形 成 Application Network Profile, Server Profile, Storage Profile等在融合一体化的资 源池内自动调配应 用所需的各项资源234Storage SMEServer Network

16、 SMESME专项主题专家(SME)根据应用需求定义 专项策略1Security SMEApp.OSSMESMEApplication ProfileServer ProfileNetwork ProfileStorage ProfileSecurity Profile17Access configuration, VLAN, VSAN, Security, and HardeningOperating System Configuration OS Type, Patch Level, SettingsNetwork interface card (NIC) configuration: M

17、AC address, VLAN, and QoS settings;host bus adapter HBA configuration: worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique service ID, Application revisions, and Storage settingsApplication resources: Server, Storage, NetworkSecurity, OSUplink port configuration, VL

18、AN, VSAN, QoS, and EtherChannelsServer port configuration including LAN and SAN settingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configuration: worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique user ID (U

19、UID), firmware revisions,and RAID controller settingsService profile assigned to server, chassis slot, or poolUplink port configuration, VLAN, VSAN, QoS, and EtherChannelsServer port configuration including LAN and SAN settingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS se

20、ttings;host bus adapter HBA configuration: worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique user ID (UUID), firmware revisions,and RAID controller settingsService profile assigned to server, chassis slot, or poolUplink port configuration, VLAN, VSAN, QoS, and Eth

21、erChannelsServer port configuration including LAN and SAN settingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configuration: worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique user ID (UUID), firmware revisio

22、ns,and RAID controller settingsService profile assigned to server, chassis slot, or pool策略组合为 Application Profile形 成 Application Network Profile, Server Profile, Storage Profile等在融合一体化的资 源池内自动调配应 用所需的各项资源234Storage SMEServer Network SMESME专项主题专家(SME)根据应用需求定义 专项策略1Security SMEApp.OSSMESMEApplication

23、ProfileServer ProfileStorage ProfileSecurity Profile以应用为中心对各类软硬件资源实现优化调度Uplink port configuration, VLAN, VSAN, QoS, and EtherChannelsServer port configuration including LAN and SAN settingsNetwork interface card (NIC) configuration: MAC address, VLAN, and QoS settings;host bus adapter HBA configurat

24、ion: worldwide names (WWNs), VSANs, and bandwidth constraints;and firmware revisionsUnique user ID (UUID), firmware revisions,and RAID controller settingsService profile assigned to server, chassis slot, or poolNetwork Profile18举例：Application Network Profile实现 “应用定义的网络”通过控制器APIC（Application Policy I

25、nfrastructure Controller）ADCAPPDBF/W ADCHYPERVISORHYPERVISORHYPERVISORCONNECTIVITYPOLICYSECURITYBANDWIDTHICIESRESERVPOLATIONAVAILABILITYSTORAGE ANDCOMPUTEAPPLICATION L4-L7SERVICESWEBSLAQoS SecurityLoadBalancing QOSAPPLICATIONNETWORK PROFILEExtensible SAcPriIpCting Model19举例：Application Network Profi

26、le实现 “应用定义的安全服务”无交换网洪泛攻击，安全稳定统一合规审计流量自动导入防 火墙、流量清洗灵活定义群组(默认硬件隔离)统一策略下发 灵活变迁移动差错隔离Policy EngineAPICWebApplicationDBStorageAppA1pplication 1A,2pp2ENABLING A DYNAMIC ENTERPRISE WITHOUT COMPROMISEAPPLICATION NETWORK PROFILEUnified DC by Weihang 2012 Cisco Systems, Inc. All rights reserved.20议题设计架构方案概述服务

27、保障21PRICEPOWER EFFICIENCYNEXUS 9000PORT DENSITYPROGRAMMABILITYPERFORMANCEPRERICFEORMANCE CINODSUTSTRY LEADING PRICE / LSITNREUCCATRUDRBEANDWITH1fo.9r 21GTbtops per slot1/0100GGTready and 10G to 40G migrationPOROORWTGERDRAEEMNFSMFIAITCYBIEILNITCY 2JSS0T%AOTNEH/OIXGFMHTELHREAAPRI T3LB6iAnCuPKxoPrCLt A

28、o4Nn0EtaGFiniRgeErNEfonr - bcDulEosSctIokGimNngerDaepnpssity 15% greater powerand cooling efficiencyMERCHANT+ ASIC APPROACHInnovation in Cisco ASICsNexus 9300：NG Fabric的理想LeafNexus 9396PQ960G Bandwidth48-port 1/10 Gb SFP+ and 12-port 40 Gb QSFP+2 RUNexus 9396TX (future)960G48-port 1/10 GBaseT & 12-p

29、ort 40 Gb QSFP+2 RUNexus 93128TX1,280G Bandwidth96-port 1/10 G-T and 8-port 40 Gb QSFP+3 RU拓扑结构设计Spine（Nexus 9508）LeafNexus 9396Border Leaf Nexus 9504ECMP40G x n x 210GInternetSpine（Nexus 9508）LeafNexus 9396LeafNexus 9396APIC APIC控制器APIC APIC控制器带外管理网10G10GRSLVSLVS共70台TOR交换机从双上连40G到1:1 无过载上连，自由调节ECMP

30、 64路负载均衡 或Service Scaling上千路超大规模负载均衡拓扑结构设计LeafNexus 9396r Leaf950440G xAPIC控APIC APIC控制器APIC带外管理网Spine（Nexus 9508） Spine（Nexus 9508）n x 2ECMPLeafBordeNexus 9396LeafNexus Nexus 939610G10G10G制器RSLVSLVSInternet共70台TOR交换机Application Health互连拓扑结构设计通过Service Border Leaf互连，管控域独立，差错隔离1664个40G/100G互连，无瓶颈共同接入

31、互联网Spine（Nexus 9508）Spine（Nexus 9508）40/100G x nInternetCisco 数据中心云计算基础架构领导者超过第二名14倍！28ACI解决了客户什么问题 第一步，简化部 署在过去，要上一个应用，需要很多服务器，于是网管人员得一台台交换机的去配置VLANQoSACL，费时间而且很僵化有了虚拟化以后，问题更麻烦了，如果虚拟机发生漂移，到了另外 一台交换机下面的物理机器上，那么网络也有做变动久而久之，就形成两种问题：上线一个新的业务软件，网络成为瓶颈，半天配不好；在虚拟化的环境下，因为怕虚拟机漂移，得把全网打通，安全管控又成了问题ACI引入了APIC集中

32、控制器，可以统一动态的配置全网所有的物理虚拟交换机更重要的是，ACI不再按照以前传统命令行的方式去配置网络，APIC控制器通过类似于Service Profile的东西，一开始就把每个应用的各种策略制定好（ACLQoSVLAN），以后不管是物理机器搬来搬去，还是虚拟机的漂移，这些策略都会跟着跑到对应的交换机上去，自动配好理解ACI的Profile概念，是理解接下来ACI的所有特点的基础29让我们回到一切的开始 UCS SERVICE PROFILESERVICE PROFILE 制定模板，简化部署服务器策略存储策略网络策略虚拟化策略应用程序策略管理人员制定好策略存储管理员服务器管理员网路管理员

33、利用这些策略生 成服务器特性模板基于模板生成服务器的硬 件特性参数将硬件特性参数指定到 各台机器登录到UCSM，管理160台服务器Server NameUUID, MAC, WWN Boot Information LAN, SAN Config Firmware PolicyServer Name UUID, MAC, WWNBoot Information LAN, SAN Config Firmware PolicyServer Name UUID, MAC, WWNBoot Information LAN, SAN Config Firmware PolicyServer NameUU

34、ID, MAC, WWN Boot Information LAN, SAN Config Firmware PolicyServer Name UUID, MAC, WWNBoot Information LAN, SAN Config Firmware Policy30传统模式是怎么配置网络的switch5(config)# switch5(config)# int eth 1/10 - 11switch5(config)# switch mode acc switch5(config)# switch acc vlan 444 switch5(config)# no shut switc

35、h5(config)# int eth 1/11 - 15 switch5(config)# switch mode acc switch5(config)# switch acc vlan 555switch5(config)# no shut switch5(config)# monitor session 1 source vlan 555switch5(config)# monitor session 1 desteth 1/16应用软件部门的同应事用程序有啥要求 网络部门“CLI” 命令行A correct network is defined with thenecessary s

36、ettings and安全控制部门Text-based languageVery VerboseSettings & Resources Allocation are done in one shotRedundant DefinitionsIn ONE StepDefine required settings Allocate network resources31在ACI的世界里，网络是怎么被配置的应用软件部门 Application requirements安全部门APICStep 1定义好策略网络部门Step 2分配网络资源通过APIC控制器，自动下发 配置到全网物理交换机和虚拟交换机32部署灵活性： 在任意地方任意网络架构下部署您的应用ACI APPLICATION-CENTRIC INFRASTRUCTUREADCAPPDBF/W ADCWEBCONNECTI VITYPOLICYSECURITQOSBANDWIDTH RES