VMware VSAN 与 WS16 S2D竞争分析报告

1、VMware VSAN 与 WS16 S2D竞争分析报告GoalsP0: Technical performance comparison cached workload performance ($/IOPs) on hybrid configurationsP1: OOB setup experience, troubleshooting/monitoringOut-of-scope: Operations/Day 100 experience, failover, cache saturation, all-flash configurationsStakeholders/ Audien

2、ce: Engineering (requestor), Marketing (consumer)MethodologyPlatform: 2x 8 node clusters w/ identical hardware, one running WS16 S2D the other VMware vSAN 6.2Configuration: followed published and/or internal best practicesS2D: MRT virtual disks, ReFS file system, VM workloads homed to node owning un

3、derlying virtual diskVSAN: 3-way mirror, 1 parity stripe, 4 disk groups per host (1 per SSD cache device)Workload: IOPS stress with Diskspd (via VMFleet) Cache-only testVSAN vs S2D: BackgroundHighlightsNo additional software required to enable/manage S2DBeing a novice with S2D at the start of this p

4、roject, but having familiarity with the failover clustering GUI/PS cmdlets, the learning curve to create/manage/operate S2D was very shortRDMA provided a notable boost in overall solution performance (with fall back to TCP/IP allowing for additional troubleshooting/maintenance options)Storage QoS po

5、licies with a non-zero minimum IOPS value (regardless of what the maximum value was) provided consistent performance normalization of the VM workloadsChallengesOOB experience to get from base failover cluster to S2D-enabled cluster was easy (single PowerShell cmdlet), getting to a point where the cl

6、uster can begin to service workloads takes an additional few PS cmdlets (virtual disk creation, etc.)Vastly different management experience depending on choice of base OS (Full/Core vs. Nano which lacked failover cluster/CSV PS modules, offered fewer Event Log providers, difficulty installing firmwa

7、re/drivers, etc.)Ongoing/ proactive health monitoring in S2D disjointed (e.g. many places to look); at load, SMAPI becomes unresponsive (e.g. Get-Physical disk takes several minutes to complete)RDMA RoCE configuration challenging (especially the TOR/Agg switches) and troubleshooting difficult (e.g.

8、Is it working properly?)VSAN vs S2D: S2D ObservationsHighlightsOOB experience with VSAN setup was very easy: 3 mouse clicksPurpose-built VSAN monitoring tool (VSAN Observer) ships free with the product, and is very informativePrescriptive component-level hardware support published for VSAN, and auto

9、mated checks for compliance performed daily by vCenterGranular proactive VSAN health monitoring/alerting can be configured from within the same GUI tool used for VSAN management (vCenter)Robust policy engine allows granular control of storage policy settings at the VM object level (erasure coding/mi

10、rroring, stripe width, # of failures to tolerate, QoS, etc.), and provides ongoing compliance checking/remediation of any objects that drift out of policyChallengesSeparate software package required to make VSAN function (vCenter)Holistic monitoring of ESXi/VSAN very disconnected, split between mult

11、iple tools (vCenter, VSAN Observer, vRealize Operations, ESXi management shell)Ongoing support for a wide range of legacy NIC, SSD, HDD, and IO Controller (RAID 0 and Pass-Through) components within VSAN hinders overall performance (e.g. VSAN extremely sensitive to queue depth on storage controller)

12、Once VSAN enabled on a given node, boot time for the node increases 5x (2 minutes to 10 minutes)E/W traffic between VSAN cluster nodes can only use IGMP Multicast, which is very inefficient and not supported on all network switch modelsWhen nodes come out of maintenance mode, no ability to automatic

13、ally fail resources previously owned by the node back over to itVSAN vs S2D: VSAN ObservationsVSAN vs S2D: Base HardwareCompute/Storage: Dell R730 xd8-node clustersCPU: 2x Intel Xeon E5-2660v3 (10C / 2.60 GHz)RAM: 256GB ECC DDR4 2133MHzStorage:Controller: HBA330 (S2D) / H730 (VSAN)2x Intel S3610 200

14、GB SSD (OS) FW: DL294x Intel S3710 800GB SSD (SBL/Cache) FW: 015012x Seagate Constellation ES 4TB HDD (Capacity) FW: GA6CNetwork:1x 1Gbps Host Management (Intel i350)RDMA: 2x 10Gbps Storage/Cluster (Mellanox ConnectX-3 Pro)Non-RDMA: 2x 10Gbps Intel X540 (onboard)TOR Switch (x2): Cisco Nexus 3172PQOS

15、: 6.0(2)U6(6)Agg Switch: Cisco Nexus 3132QOS: 6.0(2)U6(6)VSAN vs S2D: Total Cost of Acquisition (MSRP)ComponentMicrosoftVMwareWindows Server Guest$49,240$49,240ComputeIncluded$82,176StorageIncluded$87,920NetworkIncludedOptionalSystem Center/vCenterOptional$5,995Software Subtotal$49,240*$225,331Dell

16、R730 xd Hardware$180,000$180,000Total$229,240$405,331Delta57%-InformationHostsProcs per HostTotal ProcsHost configuration8216*Pricing and features consistent with HCI Standard QuickPath configuration.VSAN vs S2D: Test ConfigurationGoal: Maximum IOPS performanceWorkload (via VMFleet and Diskspd)20 VM

17、s per Node2x vCPU, 512MB-2GB dynamic RAM per VM2:1 Virtual to Physical Core ratio20GB IO working set per VMStorage QoSS2D: policy applied to each VHDX 10 min/50,000 max IOPSVSAN: no QoS on VSAN VMs (no official recommendation and only offers Max IOPS cap)Statistics averaged over 30 minutes after VHD

18、X/VMDK had been properly seasoned with writes (i.e. after steady state)OS Versions:VMWare: ESXi 6.0 Update 2 / vSAN 6.2S2D: Full server build 14387.1001.160711-1819 from RS1_SRV_HASVSAN vs S2D: 100/0 Results (IOPS)100/0 Random Read/Write, 4k Block, 32 OIOVSAN vs S2D: 70/30 Results (IOPS)70/30 Random

19、 Read/Write, 4k Block, 32 OIOVSAN vs S2D: 100/0 Results (Throughput)100/0 Sequential Read/Write, 512k Block, 4 OIOVSAN vs S2D: CPU/IOPS TrendsGoal: Model disk performance while leaving enough CPU capacity free on each node for application workloads and failover (i.e. model real-world runtime stresse

20、s rather than max IOPS values)IOPS value captured at various target points of Avg. CPU UtilizationTrendline added within Excel to project non-measured CPU/IOPS values based on linear regression/R2 modelStorageQoS used to tune per-VM performance up or down to reach the targeted CPU utilization number

21、s (automatic for S2D in sweep-cputarget VMFleet script, manual QoS adjustment in VSAN)VSAN vs S2D: 100/0 CPU/IOPS TrendsVSAN vs S2D: 90/10 CPU/IOPS TrendsVSAN vs S2D: 70/30 CPU/IOPS TrendsVSAN vs S2D: Performance SummaryWS16 S2D: the cost and 3x the performance of VSANAppendixBacklogP1Pivot the subs

22、equent testing to R730 xd nodes running NVMe/SSDThe majority of VMWares feature enhancements for VSAN have been exclusively in All-Flash for the past 2 releases, so this will give us a more current representation of comparative performance going forwardDetermine which VSAN features to light up for c

23、omparison (dedupe, compression, erasure coding, stretch clustering, etc.)Dialing in to 1ms latency (measured at CSVFS layer)Stress the cache (run at 75%, run at 150%, run at 75% increasing at 1GB/minute)Failure scenario: Run Fleet, lose 1 node, run for 2 hours, bring node back in compare operational

24、 experience and measure recovery/reconvergence times.P2Test passes on 16-node clusters (both Hybrid and All-Flash)VSAN vs S2D: Total Cost of Acquisition (MSRP)ComponentMicrosoftVMwareWindows Server Guest$49,240$49,240ComputeIncluded$82,176StorageIncluded$87,920NetworkIncluded$111,920System Center/vCenter$28,856$5,995Public Cloud BCDR (50 VMs)$2,700$38,500Software Subtotal$80,796$375,751Dell R730 xd Hardware$180,000$180,000Total$260,796$555,751Delta47% (i.e. the cost)-InformationHostsProcs per HostTotal ProcsHost configuration821