HPE 3Par自适应数据压缩功能

1、HPE 3Par 自适应数据压缩功能（Data Reduction ）Sizing considerations and updatesAgenda2Adaptive Data Reduction sizing conceptsWhen to use deduplication and compressionCapacity estimation toolsSizing with NinjaSTARS and compactionAdaptive Data Reduction Sizing Concepts3HPE 3PAR is the long-standing leader of cap

2、acity efficiencyHardware-accelerated data reduction technologiesDeduplicationPrevent storing duplicate dataCompressionReduce data footprintData PackingPack odd-sized data togetherZero DetectRemove zeros inlineThin ProvisioningDont store free spaceWhy not use Dedup + Compression (DECO) for everything

3、?We do expect customers to try to use only DECO volumesMany already use dedup for everythingThe primary competitors do deduplication+compressionThe mind set is SSDs are expensive so they want to wring every last byte out of them5Best practice will be to choose the appropriate volume type for the dat

4、a setA small time investment in understanding the data will pay big dividends operationallyVolume Type Positioning6FullDeduplicatedDeduplicated + CompressedCompressedThinProvisioning TypePerformanceSpace SavingsSelective Adaptive Data ReductionAllowing more efficient use of system resourcesDifferent

5、 data types have different requirementsFor each data type, enable the technologies that provide benefits and disable the technologies that dontOracle databaseCompressed(2:1)Exchange serverDeduplicatedCompressed(1.5:1)Compressed videoThin ProvisionedVDI environmentDeduplicatedCompressed(2:1+)A small

6、time investment in understanding the data will pay big dividends operationallyWhen to use whatFull and thinly provisioned volumesFully provisioned volumes are good for:Maximum performanceCustomers who dont want to overprovision storage (or manage utilization)Host compressed dataHost encrypted dataTh

7、in provisioned volumes are good for:Host compressed dataHost encrypted dataShould still be considered the default volume type8When to use whatDeduplicated volumesGood candidates for deduplication: Any data that has a high level of redundancyVDI - Persistent desktops can achieve excellent deduplicati

8、on ratiosVM - OS images from multiple VMs can benefit from dedup. The app data may or may not dedup.Home directory and file shares - Users often store copies of the same file so this may benefit from dedupPoor candidates for deduplication: Databases - Most databases do not contain redundant data blo

9、cksPreviously deduplicated, compressed or encrypted data will not compact furtherThis does not include self-encrypting drives where data is deduped before it is written9When to use whatCompressed volumesGood candidates for compression: Data with little redundancy will not dedup well but can benefit

10、from compressionDatabases Typically do not have redundant blocks but do have redundant data within blocksVM images with a lot of application data can benefit from compression of the application dataVDI with non-persistent desktops can achieve excellent compression ratiosPoor candidates for compressi

11、on: Compressed data data that is compressed at the host will not compress furtherEncrypted data - Host or SAN encrypted data will not benefit from storage compressionThis does not include self-encrypting drives where data is compressed before it is writtenBe careful with file data as it many contain

12、 compressed data such as jpegs and mp3s10When to use whatDeduplicated and Compressed volumes (DECO)Good candidates for DECO: VM images - OS images from multiple VMs can benefit from dedup and the application data will compressVDI Both persistent and non-persistent desktops can achieve excellent data

13、 reduction ratiosHome directory and file shares - Deduplication and compression can offer significant space savingsEmail applications such as ExchangePoor candidates for DECO: Databases - Most databases will not dedup. Compression only is best for databases.Deduplicated data - Data that has already

14、been deduplicated on the host will not dedup furtherData compressed or encrypted at the host or switch will not dedup or compress furtherThis does not include self-encrypting drives where data is deduped before it is written11Data ReductionData reduction sizing with dedup and compression combined122

15、:1Deduplication?+Compression2:1=HPE 3PAR StoreServ Data Reduction TechnologiesWorking together for optimal resultsWhen used together, duplicate pages are removed first and unique pages are then compressedDeduped data goes to the DDSCompressed data goes to the DDCExpress Index tablesDeduplicationComp

16、ressionIntel CPUData in cacheResulting data written to SSDExpress ScanUnique dataData PackingWhy does it depend?Example of storing data that is both 2:1 dedupable and 2:1 compressible14DDCDDSData Written = 10 blocksData Stored = 5 blocks( 2:1 )Data Written = 10 blocksData Stored = 3 blocks( 3.3:1 )D

17、atadedupdedup + compressionDDCDDSData3PAR Adaptive Data ReductionEstimator Tools153PAR Adaptive Data Reduction Tools for the field16NinjaSTARS Capacity & Performance Sizing tool for 3PARHelps design cost effective storage solutions that meet customers SLAsEnhanced to model the effects of compression

18、 and deduplication in 3PAR OS 3.3.1NinjaCrawler Dedup & Compression Estimation ToolScan actual customer volumes to identify duplicate data and determine footprint reduction with 3PAR deduplication and compression. Host based, array agnostic tool for windows and linux platforms (VMware in Q3)GetThinn

19、er HPE Best Practices validation and Dedup Ratio CalculatorSAF module supporting the eponymous 3PAR guarantee program.Provides recommendations for multiple types of workload and determine possible savings in capacity requirements. Deduplication only, compression will be added soonNinjaCrawler Estima

20、tor173PAR NinjaCrawlerWhat is it? a tool that reads actual raw data on customer volumes to determine the possible capacity savings that could be achieved with 3PAR data deduplication and compressionIt uses a 3PAR like algorithm (16K page based) to identify duplicate data on volumesUses lz4 compressi

21、on algorithm to determine data footprint reductionIts a host based tool so that does not depend on any storage array.Has low CPU and memory foot printDeduplicationAnalysisIdentify duplicate data on volumesCompressionAnalysisDetermine data footprint reductionnew3PAR NinjaCrawlerWhats in it for me?No

22、more guess-timation! You get measured deduplication and compression ratios based on actual customer data.Drive better customer conversations based on their unique IT environmentReinforce confidence in 3PAR data reduction technology and backs up our GetThinner guarantee programHow long does it take t

23、o run NinjaCrawler ?Short answer: the same as for any other “piece of string” question, It dependsThere are a couple of key driving factors (performance of targeted host and storage, tmp database, etc)But assuming an average read throughput of 300 MB/s, it would take:About 15 mins for 250GB worth of

24、 disk space to scanAbout one hour to scan a Terabyte, and over a month to scan a Petabyte(please dont) NinjaCrawler can report current dedup & compression ratios while scan is in progressWINDOWSLINUXVMWAREOTHERSWindows server 2016Windows server 2012Windows server 2008Windows 7, 8.x, 10RedHatUbuntuCe

25、ntOSOracle LinuxESXi 5.xESXi 6.xHP-UXIBM AIXOracle SolarisMacOS3PAR NinjaCrawler 20_coming on 3Q 2017futureavailable nowSupported Platforms:Internal Link: TME sharepoint Support/Feedback: getthinnerNinjaCrawlerSample output213PAR NinjaCrawler v2.0.1(c) 2017 Hewlett Packard Enterprise. All rights res

26、erved.| Drive | File | Total | Written | After | After | Overall | Dedupe | Compress | Path | System | Size | non-zero | Dedupe | Compress | Ratio | Ratio | Ratio | | | | | | | | | | C: | NTFS 4K | 465.2 GB | 399.4 GB | 338.0 GB | 222.9 GB | 2.09 | 1.18 | 1.79 | D: | NTFS 4K | 1.8 TB | 619.2 GB | 50

27、4.7 GB | 374.1 GB | 4.98 | 1.23 | 1.66 | H: | NTFS 64K | 140.8 GB | 99.3 GB | 80.4 GB | 60.0 GB | 2.35 | 1.23 | 1.65 | G: | NTFS 4K | 97.7 GB | 97.0 GB | 82.0 GB | 59.9 GB | 1.63 | 1.18 | 1.62 | Total | | 2.5 TB | 1.2 TB | 938.5 GB | 716.9 GB | 3.58 | 1.29 | 1.69 | The information provided herein is

28、 applicable to HPE 3PAR data reduction only. It should be considered for estimation purposes only and not be taken as a guarantee.C: NinjaCrawler C: D: H: G:NinjaCrawlerIdentify further data reduction opportunityDB (sqlite database) keeps a metadata record for every unique hashkey on the host volume

29、(s) and compression footprint.Assessment typically produces one database file per scanned host. (has to run locally on the machine) To enable host or application consolidation scenarios, and determine further reduction opportunity database files can be combined. Unlike data scan, analysis takes minu

30、tes22Windows 2012R2Hyper-V 3 volumes crawler.dbLinux RHEL 7MySQL - 2 volumes crawler.db+=ConsolidatedWin/linux - 5 volumes consolidated.db2.3:1reduction ratio1.4:1reduction ratio2.9:1total reduction ratio duration02h56mduration08h15mduration00h02mNinjaCrawler vs. 3PAR Adaptive Data ReductionDeduplic

31、ationExpress IndexingPrevent storing duplicate dataCompressionExpress ScanReduce data footprintData PackingMaintain EfficiencyPack multiple pages togetherEmulation in NinjaCrawlerTool computes unique hash key (crc-32)Use reference count for 16K pagesCapacity usage derived from page countEmulation in

32、 NinjaCrawlerUse same lz4 compression algorithmMeasure compression gain against 16K pagescalculates a DECO ratio. Compression only will be supported in a future update.The compression ratio is approximated and represents a best case scenario. (due to limitation in data packing emulation)Observed del

33、tas are typically 3 to 10%Near 100% accuracy Estimating Capacity Savings on an Existing 3PAR System243PAR Flash AdvisorEnabling a smooth transition to FlashAdaptive Optimization (AO)I/O Density ReportsAdaptive Data Reduction (ADR) Estimation and Dynamic OptimizationPowerful I/O reporting to determin

34、e the exact amount of flash tier needed for hot dataAdaptive Flash Cache SimulationDetermine benefits and amount of flash required in for optimal random read acceleration createflashcache sim xxG1101000110111010110100011011101011010001101110102. Online Dynamic Optimization to dedup/compression statu

35、s1. Estimate Savings 11010001101110101101000110111010Thin vvol1101000110111010SSD capacityestimation3PAR Compression / Deduplication EstimationDry run using SSMC / CLIPredicts the expected savings to be achieved when migrating to a compressed or deduped volumeCan be run on a 3PAR without any SSDs in

36、stalledRuns as a background task The estimation can take some timeDedup + compression (DECO) estimation currently is CLI onlyVOLUME TYPEcommand lineCompressedcheckvv -compr_dryrun Dedupedcheckvv -dedup_dryrun DECOcheckvv -dedup_compr_dryrun More information can be found in the new Adaptive Data Redu

37、ction white paper2017-01-18 14:59:25 CET Created task.2017-01-18 14:59:25 CET Started checkvv space estimation started with option -compr_dryrun. . .2017-01-18 15:09:35 CET Finished checkvv space estimation process finished Compression Estimate Results (MiB) ID Name User Data Estimated Ratio Total 8

38、39871 430703 1.95 17 bigvol2 839871 430703 1.95 Sizing for Adaptive Data ReductionNinjaSTARS update27Adaptive Data Reduction (ADR)Compression, Deduplication and DECO now available in NinjaSTARS28Dedup/Compression/Deco affects capacity and performanceWeightSpecify the weights in percent of SSD worklo

39、ads that will be using Adaptive Data Reduction Data Reduction RatioSpecify the data reduction as a ratio or in percent or use ratio calculator to determine expected benefits for known workloadsData Reduction TypeDeduplicationCompressionDeCoThinNew Adaptive Data Reduction (ADR) Calculatora built-in t

40、ool to estimate the benefits of deduplication and compression for 3PAR sizingEfficiency ratios based on lab testing and early access customer resultsRatios will be changed periodically based on customer experience / call home dataSupports DEDUP, COMPRESSION and DECOSimple to use:Add customer workloa

41、d typesEnter the capacity for each workloadGet a quick estimate of the potential savingsAddresses shortcomings of the current ADR Calculator spreadsheet:Partners accessible (thru NinjaSTARS)Workloads list and user inputs saved in NS project fileUse anytime, anywhere doesnt require HPE intranet conne

42、ction29Capacity & Performance with ThinPCapacity & Performance with CompressionNinjaSTARSAdaptive Data Reduction30Dedup/Compression ratio affects capacity and performanceData Reduction TypeWeightSpecify the percentage of SSD space that will be using Adaptive Data Reduction Performance and usable cap

43、acity are proportional to the amount of SSD space percentage and the Data Reduction TypeADR Estimated Performance Impact 31* 9450 Based on preliminary performance data subject to changeI/O DensityEven if the data reduction technologies have no impact to performancei.e. The system had the same maximu

44、m performance regardless of volume typeFull provisioned volumes would be the fastest and deduplicated-compressed volumes would be the slowestAs the effective capacity of a system is increased by data reduction there are fewer IOPS per TB of space available to applicationsWe have seen customer system

45、s that have great dedup ratios but have run out of system IOPS32Customers are increasingly asking for IOPS per TB sizingIf they dont then you need to add it to the conversation when competitors quote high data reduction ratiosWhats new in NinjaSTARS ?Use I/O Density (IOPS per TB) in your sizingNew o

46、ption in Flash Usable CapacityIts not only about capacity and ratiosAs the effective capacity of a system is increased by data reduction, there are fewer IOPS per TB of space available to applications.right click on the chart to toggle the view between capacity and densityCustomers are increasingly

47、asking for IOPS per TB sizingIf they dont then you need to add it to the conversation when competitors quote high data reduction ratiosI/O DensityConsider the following scenarioWe start selling 3PAR AFAs as appliances. These are black boxes where all you can do is provision volumes (possibly automat

48、ically from the host. e.g. VMware VVols). 34Option AOption BOption C250 TB500 TB1 PBQ: If all options are the same price which one will the customer choose?* No data reduction technologies* Assuming 2:1 data reduction* Assuming 4:1 data reductionI/O DensityConsider the following scenarioA customer is a service provider and offers VMs with 10 TB of storage and a 10k IOPS guaranteeThe systems are capable of 400k IOPS and data reduction does not impact system performance35Option AOption BOption C250 TB (1:1)500 TB (2:1)1 PB (4:1)25 VMs? 50 VMs? 100 VM