基于科学计算的Power scale系统

Whitepaper

DellPrecisionDataScienceWorkstationwithIsilon

©2020DellInc.oritssubsidiaries.

DellPrecisionIsilonH400

DataScienceWorkstationwith

ThiswhitepaperdemonstrateshowtheDellPrecision7920TowerDataScienceWorkstationwithNVIDIA®Quadro®RTX6000GPUsandDellEMCIsilonH400scale-outNAScanbeusedtoprovideanexcellentenvironmentforsmallteamsperformingdatascience,AI,anddeeplearning.Theresultsofindustry-standardimageclassificationtrainingbenchmarksusingTensorFlowareincluded.

March2020

2

Theinformationinthispublicationisprovided“asis.”DELLEMCCorporationmakesnorepresentationsorwarrantiesofanykindwith

respecttotheinformationinthispublication,andspecificallydisclaimsimpliedwarrantiesofmerchantabilityorfitnessforapurpose.

Use,copying,anddistributionofanyDELLEMCsoftwaredescribedinthispublicationrequiresanapplicablesoftwarelicense.

DELLEMC2,DELLEMC,theDELLEMClogoareregisteredtrademarksortrademarksofDELLEMCCorporationintheUnitedStates

andothercountries.Allothertrademarksusedhereinarethepropertyoftheirrespectiveowners.©Copyright2020DELLEMC

Corporation.Allrightsreserved.PublishedintheUSA.3/20.H18228

DELLEMCbelievestheinformationinthisdocumentisaccurateasofitspublicationdate.Theinformationissubjecttochangewithout

notice.DELLEMCisnowpartoftheDellgroupofcompanies.

3

TableofContents

Revisions 4

Executivesummary 4

Audience 4

Introduction 4

Solutionarchitecture 4

OVERVIEW 4

DELLPRECISION7920TOWERDATASCIENCEWORKSTATION 5

DELLEMCISILONH400SCALE-OUTNAS 5

Storagetiering 6

OneFScaching 7

Locksandconcurrency 7

KEYHARDWARECOMPONENTS 7

SOFTWAREVERSIONS 8

Deeplearningtrainingperformanceandanalysis 8

BENCHMARKMETHODOLOGY 8

BENCHMARKRESULTS 8

Solutionsizingguidance 9

Conclusions 9

Appendix–Benchmarkdetails 10

References 12

Revisions

Description

Initialrelease

Author

ClaudioFahey

Date

March2020

Executivesummary

ThiswhitepaperfocusesonhowtheDellPrecision7920TowerWorkstationwithNVIDIAGPUsandDellEMCIsilonscale-outNAS(NetworkAttachedStorage)acceleratesAIinnovationandcollaborationbyprovidingsharedaccesstoverylargedatasetswithhighperformanceandscalability.Thisisanexcellententry-levelsolutionforsmallteamsofupto5membersthatexpecttogrowandneedasystemthatcangrowwiththem.

Audience

ThisdocumentisintendedfororganizationsandsmalldatascienceteamsthatarelookingtoaccelerateAIinnovationandcollaboration.Solutionarchitects,systemadministrators,datascientists,dataengineers,andotherinterestedreaderswithinthoseorganizationsconstitutethetargetaudience.

Introduction

Anefficientdatascienceteamoftenrequirestheabilitytosharelargeamountsofdatawhileprovidinghighperformance,reliability,andseamlessaccessfrommultipleoperatingsystems.Anewteammayoftenstartsmallwithasfewastwomembers,buttheywillstillneedaccesstolargeamountsofdata.Further,asthedatascienceteamgrows,thecomputeandstorageneedswillincreaseaswell.

DellTechnologiesandNVIDIAoffermultipleGPU-acceleratedserversandsystemsfordatacenterenvironmentsincludingtheDellEMCPowerEdgeC4140,DellEMCDSS8440,NVIDIADGX-1™,andNVIDIADGX-2™.WealsohavecompletesolutionsthatcombinethesesystemswithnetworkingandDellEMCIsilonscale-outNAS,

ThisdocumentfocusesonthelateststepintheDellTechnologiesandNVIDIAcollaboration,anewAIreferencearchitecturethatcombinestheDellPrecision7920TowerDataScienceWorkstationwithNVIDIAGPUsandIsilonH400storageforsmalldatascienceteamsofupto5workstations.Thisisanexcellententry-levelsolutionforsmallteamsthatexpecttogrowandneedasystemthatcangrowwiththemwithouttheneedofdatamigration.Insuchusecases,DellDataScienceWorkstationsareidealenterprise-classhighperformancedevelopmentplatformsflexibleforAI/ML/DLmodelexperimentationanddevelopmentpriortotakingthemodelstoscaleinthebusiness’sdatacentercontainingevenlargerdatasetsandmoreGPUs.

Deeplearning(DL)isanareaofAIwhichusesartificialneuralnetworkstoenableaccuratepatternrecognitionofcomplexreal-worldpatternsbycomputers.Thesenewlevelsofinnovationhaveapplicabilityacrossnearlyeveryindustryvertical.Someoftheearlyadoptersincludeadvancedresearch,precisionmedicine,hightechmanufacturing,advanceddriverassistancesystems(ADAS)andautonomousdriving.Buildingontheseinitialsuccesses,AIinitiativesarespringingupinvariousbusinessunits,suchasmanufacturing,customersupport,lifesciences,marketing,andsales.GartnerpredictsthatAIaugmentationwillgenerate$2.9trillioninbusinessvalueby2021alone.Organizationsarefacedwithamultitudeofcomplexchoicesrelatedtodata,analyticskill-sets,softwarestacks,analytictoolkits,andinfrastructurecomponents;eachwithsignificantimplicationsonthetimetomarketandthevalueassociatedwiththeseinitiatives.

Insuchacomplexenvironment,itiscriticalthatorganizationsbeabletorelyonvendorsthattheytrust.Overthelastfewyears,DellTechnologiesandNVIDIAhaveestablishedastrongpartnershiptohelporganizationsacceleratetheirAIinitiatives.Ourpartnershipisbuiltonthephilosophyofofferingflexibilityandinformedchoiceacrossanextensiveportfolio.TogetherourtechnologiesprovidethefoundationforsuccessfulAIsolutionswhichdrivethedevelopmentofadvancedDLsoftwareframeworks,delivermassivelyparallelcomputeintheformofNVIDIAGPUsforparallelmodeltrainingandscale-outfilesystemstosupporttheconcurrency,performance,andcapacityrequirementsofunstructuredimageandvideodatasets.

TheresultsofindustrystandardDLimageclassificationbenchmarksusingTensorFlowareincludedinthiswhitepaper.

Solutionarchitecture

OVERVIEW

Figure1

illustratesthereferencearchitectureshowingthekeycomponentsthatmakeupthesolution.Notethatinacustomerdeployment,thenumberofworkstationsandIsilonstoragenodeswillvaryandcanbescaledindependentlytomeettherequirementsofthespecificworkload.Refertothe

Solutionsizingguidance

sectionfordetails.

5

DellPrecision7920TowerWorkstations

10GbEthernetswitch

DellEMCIsilonH400

(4nodesin1chassis)

Figure1:ReferenceArchitecture

DELLPRECISION7920TOWERDATASCIENCEWORKSTATION

Theworld’smostpowerfulworkstation,theDellPrecision7920TowerWorkstation,providesultimateperformanceandscalabilitytogrowalongsideyourAIinitiativesanddata.TheDellDataScienceWorkstationastestedforthisdocumentincludesanIntel®Xeon®Gold61348coreCPU,128GBCPURAM,twoNVIDIAQuadroRTX6000GPUswith24GBofGPURAMeach,aSamsungPM951NVMedrive,andaMellanoxConnectX4LXNIC.SoftwareincludesUbuntuLinuxandtheNVIDIAQuadroRTXGPUacceleratedNVIDIADataScienceSoftwarestack

DELLEMCISILONH400SCALE-OUTNAS

Anefficientdatascienceteamoftenrequirestheabilitytosharemassiveamountsofdatawhileprovidinghighperformance,reliability,andseamlessaccessfrommultipleoperatingsystems.TheDellEMCIsilonscale-outNASprovidesthiscriticalcapability.DellEMCIsilonhybridstorageplatforms,poweredbytheOneFSoperatingsystem,useahighlyversatileyetsimplescale-outstoragearchitecturetospeedaccesstomassiveamountsofdata,whiledramaticallyreducingcostandcomplexity.Thehybridstorageplatformsarehighlyflexibleandstrikethebalancebetweenlargecapacityandhigh-performancestoragetoprovidesupportforabroadrangeofenterprisefileworkloads.TheH400providesabalanceofperformance,capacityandvaluetosupportawiderangeoffileworkloads.Anditdeliversupto3GB/sbandwidthperchassisandprovidescapacityoptionsrangingfrom120TBto720TBperchassis.

EachH400chassis,shownin

Figure2,

containsfourstoragenodes,60SATAHDDdrivesandeight10GbEnetworkconnections.OneFScombinesupto252nodesin63chassisintoasinglehigh-performancefilesystemdesignedtohandlethemostintenseI/OworkloadssuchasDL.Asperformanceandcapacitydemandsincrease,bothcanbescaled-outsimplyandnon-disruptively,allowingapplicationsanduserstocontinueworking.

6

Figure2:IsilonH400chassis,containingfourstoragenodes

Inthesolutiontestedforthisdocument,fourH400nodes,inonechassis,wereused.

DellEMCIsilonH400hasthefollowingfeatures.

Highcapacitywiththeabilitytogrowasneeded:

•120TBto720TBrawHDDcapacityperchassis;upto45PBpercluster

•Upto3GB/sthroughputperchassis

TheabilitytorunAIin-placeondatausingmulti-protocolaccess:

•Multi-protocolsupportsuchasSMB,NFS,HTTP,andnativeHDFStomaximizeoperationalflexibilityThiseliminatesthecostlyneedtomigrate/copydataandresultsovertoaseparateAIstack.

Enterprisegradefeaturesout-of-box:

•Enterprisedataprotectionandresiliency

•Robustsecurityoptions

ThisenablesorganizationstomanageAIdatalifecyclewithminimalcostandrisk,whileprotectingdataandmeetingregulatoryrequirements.

Extremescale:

•SeamlesslytierbetweenAllFlash,Hybrid,andArchivenodesviaSmartPools

•Grow-as-you-goscalabilitywithupto45PBHDDcapacitypercluster

•Newnodescanbeaddedtoaclustersimplybyconnectingpower,back-endEthernetandfront-endEthernet

•Asnewnodesareadded,storagecapacity,throughput,IOPS,cache,andCPUgrow

•Upto63chassis(252nodes)maybeconnectedtoformasingleclusterwithasinglenamespaceandasinglecoherentcache

•Upto85%storageefficiencytoreducecosts

•Optionaldatade-dupandcompressionenablinguptoa3:1datareduction

OrganizationscanachieveAIatscaleinacost-effectivemanner,enablingthemtohandlemulti-petabytedatasetswithhighresolutioncontentwithoutre-architectureand/orperformancedegradation.

ThereareseveralkeyfeaturesofIsilonOneFSthatmakeitanexcellentstoragesystemforDLworkloadsthatrequireperformance,concurrency,andscale.Thesefeaturesaredetailedbelow.

Storagetiering

DellEMCIsilonSmartPoolssoftwareenablesmultiplelevelsofperformance,protection,andstoragedensitytoco-existwithinthesamefilesystemandunlockstheabilitytoaggregateandconsolidateawiderangeofapplicationswithinasingleextensible,ubiquitousstorageresourcepool.Thishelpsprovidegranularperformanceoptimization,workflowisolation,higherutilization,andindependentscalability–allwithasinglepointofmanagement.

SmartPoolsallowsyoutodefinethevalueofthedatawithinyourworkflowsbasedonpoliciesandautomaticallyalignsdatatotheappropriateprice/performancetierovertime.Datamovementisseamlessandwithfile-levelgranularityandcontrolviaautomatedpolicies,manualcontrolorAPI,youcantuneperformanceandlayout,storagetieralignmentandprotectionsettings–allwithminimalimpacttoyourend-users.

7

Quantity

14Uchassis(4nodes)

1

Workstation

128GBRAM

Storagetieringhasaveryconvincingvalueproposition,namelysegregatingdataaccordingtoitsbusinessvalueand

aligningitwiththeappropriateclassofstorageandlevelsofperformanceandprotection.InformationLifecycleManagementtechniqueshavebeenaroundforseveralyears,buthavetypicallysufferedfromthefollowinginefficiencies:complextoinstallandmanage,involveschangestothefilesystem,requirestheuseofstubfiles,etc.

DellEMCIsilonSmartPoolsisanextgenerationapproachtotieringthatfacilitatesthemanagementofheterogeneousclusters.TheSmartPoolscapabilityisnativetotheIsilonOneFSscale-outfilesystem,whichallowsforunprecedentedflexibility,granularity,andeaseofmanagement.Inordertoachievethis,SmartPoolsleveragesmanyofthecomponentsandattributesofOneFS,includingdatalayoutandmobility,protection,performance,schedulingandimpactmanagement.

AtypicalIsilonclusterwillstoremultipledatasetswithdifferentperformance,protection,andpricerequirements.Generally,filesthathavebeenrecentlycreatedandaccessedshouldbestoredinahottierwhilefilesthathavenotbeenaccessedrecentlyshouldbestoredinacoldtier.BecauseIsilonsupportstieringbasedonafile’saccesstime,thiscanbeperformedautomatically.Forstorageadministratorsthatwantmorecontrol,complexrulescanbedefinedtosetthestoragetierbasedonafile’spath,size,orotherattributes.

AllfilesonIsilonarealwaysimmediatelyaccessible(readandwrite)regardlessoftheirstoragetierandevenwhilebeingmovedbetweentiers.Thefilesystempathtoafileisnotchangedbytiering.Storagetieringpoliciesareapplied,andfilesaremovedbytheIsilonSmartPoolsjob,whichrunsdailyat22:00bydefault.

Formoredetails,see

StorageTieringwithDellEMCIsilonSmartPools.

OneFScaching

TheOneFScachinginfrastructuredesignispredicatedonaggregatingthecachepresentoneachnodeinaclusterintoonegloballyaccessiblepoolofmemory.Thisallowsallthememorycacheinanodetobeavailabletoeverynodeinthecluster.RemotememoryisaccessedoveraninternalinterconnectandhaslowerlatencythanaccessingharddiskdrivesandSSDs.

Forfilesmarkedwithanaccesspatternofconcurrentorstreaming,OneFScantakeadvantageofprefetchingofdatabasedonheuristicsusedbytheIsilonSmartReadcomponent.Thisgreatlyimprovessequential-readperformanceacrossallprotocolsandmeansthatreadscomedirectlyfromRAMwithinmilliseconds.Forhigh-sequentialcases,SmartReadcanveryaggressivelyprefetchahead,allowingreadsofindividualfilesatveryhighdatarates.

OneFSusesuptothreelevelsofreadcache,plusanNVRAM-backedwritecache.L1andL2readcachesuseRAMwhileL3usestheSSDsthatareavailableonallIsilonhybridnodes.

Formoredetails,see

OneFSSmartFlash.

Locksandconcurrency

OneFShasafullydistributedlockmanagerthatcoordinateslocksondataacrossallnodesinastoragecluster.Thelockmanagerishighlyextensibleandallowsformultiplelockpersonalitiestosupportbothfilesystemlocksaswellascluster-coherentprotocol-levellockssuchasSMBsharemodelocksorNFSadvisory-modelocks.Everynodeinaclusterisacoordinatorforlockingresourcesandacoordinatorisassignedtolockableresourcesbaseduponanadvancedhashingalgorithm.

EfficientlockingiscriticaltosupporttheefficientparallelI/OprofiledemandedbymanyiterativeAIandDLworkloadsenablingconcurrentfilereadaccessupintothemillions.

Formoredetails,seethe

OneFSTechnicalOverview.

KEYHARDWARECOMPONENTS

Table1

showsthekeyhardwarecomponentsastestedforthisdocument.

Component

Purpose

DellEMCIsilonH400

120TBHDD12.8TBSSD

256GBRAM

Four1GbE,eight10GbEinterfaces

Sharedstorage

DellPrecision7920TowerDataScienceWorkstation

Intel(R)Xeon(R)Gold61348-coreCPU@3.20GHz

8

each

PM951NVMeSAMSUNG1024GB

MellanoxConnectX4LXNIC

2NVIDIAQuadroRTX6000with24GBofRAM

Table1:HardwareComponents

SOFTWAREVERSIONS

Table2

showsthesoftwareversionsthatweretestedforthisdocument.

ComponentVersion

AIBenchmarkUtil

/claudiofahey/ai-benchmark-util/commit/ca7f5d2

DellEMCIsilon–OneFS

NVIDIADriver435.21

NVIDIACUDA10.0

MellanoxOFEDDriver4.7--ubuntu19.10-x86_64

Ubuntu19.10

DockerEngine19.03.3

NVIDIAGPUCloudTensorFlowImagenvcr.io/nvidia/tensorflow:19.09-py3

TensorFlow1.14.0

TensorFlowBenchmarks

/claudiofahey/benchmarks/commit/31ea13f

Table2:SoftwareVersions

Deeplearningtrainingperformanceandanalysis

BENCHMARKMETHODOLOGY

Inordertomeasuretheperformanceofthesolution,variousbenchmarksfromthe

TensorFlowBenchmarks

repositorywereexecuted.Thissuiteofbenchmarksperformstrainingofanimageclassificationconvolutionalneuralnetwork(CNN)onlabeledimages.Essentially,thesystemlearnswhetheranimagecontainsacat,dog,car,train,etc.Thewell-known

ILSVRC2012

imagedataset(oftenreferredtoasImageNet)wasused.Thisdatasetcontains1,281,167trainingimagesin144.8GB

1

.Allimagesaregroupedinto1000categoriesorclasses.ThisdatasetiscommonlyusedbyDLresearchersforbenchmarkingandcomparisonstudies.

TheindividualJPEGimagesintheImageNetdatasetwereconvertedto1024TFRecordfiles.TheTFRecordfileformatisaProtocolBuffersbinaryformatthatcombinesmultipleJPEGimagefilestogetherwiththeirmetadata(boundingboxforcroppingandlabel)intoonebinaryfile.ItmaintainstheimagecompressionofferedbytheJPEGformatandthetotalsizeofthedatasetremainedroughlythesame(148GB).Theaverageimagesizewas115KB.

AsmanydatasetsareoftensignificantlylargerthanImageNet,wewantedtodeterminetheperformancewithdatasetsthatarelargerthanthe256GBofcoherentsharedcacheavailableacrossthefour-nodeIsilonH400cluster.Toaccomplishthis,wesimplymade13exactcopiesofeachTFRecordfile,creatinga2.0TBdataset.Having13copiesoftheexactsameimagesdoesn’timprovetrainingaccuracyorspeedbutitdoesproducethesameI/Opatternforthestorage,network,andGPUs.HavingidenticalfilesdidnotprovideanunfairadvantageasIsilondeduplicationwasnotenabledandallimagesarereorderedrandomly(shuffled)intheinputpipeline.

Akeycriticalquestiononehaswhentryingtosizeasystemishowfastthestoragemustbesothatitisnotabottleneck.Toanswerthisquestion,wecopiedthe148GBdatasettotheworkstation'sNVMedriveandranthebenchmarkfromthishigh-speeddisk.Theimagerate(images/sec)measuredinthiswayaccountsforthesignificantpreprocessingpipelineaswellastheGPUcomputation.Todeterminethethroughput(bytes/sec)demandedbythisworkload,wesimplymultiplytheimages/secbytheaverageimagesize(115KB).Inthenextsection,resultsusingthismethodarelabeledLocalNVMe.

Priortoeachexecutionofthebenchmark,theL1,L2,andL3cachesonIsilonwereflushedwiththecommandisi_for_arrayisi_flush.Inaddition,theLinuxbuffercachewasflushedonallcomputesystemsbyrunningsync;echo3>/proc/sys/vm/drop_caches.However,notethatthetrainingprocesswillreadthesamefilesrepeatedlyandafterjustseveralminutes,muchofthedatawillbeservedfromoneofthesecaches.

BENCHMARKRESULTS

Thereareafewconclusionsthatwecanmakefromthebenchmarksrepresentedin

Figure3.

1AllunitprefixesinthisdocumentusetheSIstandard(base10)where1GBis1billionbytes.

9

Images/Sec

Images/Sec

Images/Sec

97

96

83

77

ResNet-50

Inception-V3

•Imagethroughputandthereforestoragethroughputscalelinearlyfrom1to2GPUs.

•ThereisnosignificantdifferenceinimagethroughputbetweenLocalNVMeandIsilon.

•Thehigheststoragethroughputdemandwas139MB/secandoccuredduringResNet-50with2GPUs.

•TheIsilonH400iseasilycapableofhandlingthisworkload.

1400

1200

1000

800

600

400

200

0

215

210

1

1

619

616

1

2

GPUs

1400

1200

1000

800

600

400

200

0

VGG-16

693

690

57

55

3

3

12

GPUs

1400

1200

1000

800

600

400

200

0

7

7

3

3

12

GPUs

IsilonH400LocalNVMe

Figure3:ModelDevelopment–TrainingBenchmarkResults

Solutionsizingguidance

TheDataSciencePrecisionWorkstationsolutiondescribedinthiswhitepaperisanentry-levelsolutionforsmalldatascienceteamsusingupto5workstations.TheIsilonH400isexpectedtoprovideapproximately3GB/secformostreadworkloadsand2GB/secformostwriteworkloads.Withupto5workstations(10GPUs),eachGPUwouldbeabletouse200-300MB/seconaverage.ThisisenoughformanyAIworkloads.However,beawarethatAI/DLworkloadsvarysignificantlywithrespecttothedemandforcompute,memory,disk,andI/Oprofiles,oftenbyordersofmagnitude.

Asteamsgrowordemandmorestorageperformanceandcapacity,Isiloncanbeeasilyexpandedbyaddingadditionalnodes.AdditionalH400nodescanbeaddedinincrementsoftwo.Ifahigherperformance-to-capacityratioisneeded,fourormorenodesofdifferenttypescanbeadded,andstoragetieringcanbeutilized.Isilonprovidesfaster(H500)andhigher-capacity(H5600)hybridnodesaswellasall-flashnodes(F800andF810)forextremeperformance.

AnunderstandingoftheI/OthroughputdemandedperGPUforthespecificworkloadandthetotalstoragecapacityrequirementscanhelpprovidebetterguidanceonIsilonnodecountandconfiguration.ItisrecommendedtoreachouttotheDellEMCaccountandSMEteamstoprovidethisguidanceforthespecificAIworkload,throughput,andstoragerequirements.

Conclusions

ThisdocumentpresentedascalablearchitectureforsmalldatascienceteamsbycombiningDellPrecision7920TowerDataScienceWorkstationswithsingleanddualNVIDIAQuadroRTX6000GPUs,andDellEMCIsilonH400scale-outNAS.OnecanexpectcomparableresultsfromsingleordualNVIDIAQuadroRTX8000GPUconfigurationsprovidinglargeramountsofGPURAM.WediscussedkeyfeaturesofIsilonthatmakeitapowerfulpersistentstoragesystemforAIsolutions.ThisnewreferencearchitectureextendsthecommitmentofDellTechnologiesandNVIDIAtomakingAIsimpleandaccessibletoeveryorganizationwithourunmatchedsetofjointofferings.TogetherweprovideourcustomerswithinformedchoicesandflexibilityinhowtheydeployDLatanyscale.

ItisimportanttopointoutthatAIalgorithmshaveadiversesetofrequirementswithvariouscompute,memory,I/O,anddiskcapacityprofiles.Thatsaid,thearchitectureandtheperformancedatapointspresentedinthiswhitepapercanbeutilizedasthestartingpointforbuildingAIsolutionstailoredtovariedsetsofresourcerequirements.Moreimportantly,allthecomponentsofthisarchitecturearelinearlyscalableandcanbeindependentlyexpandedtoprovideAIsolutionsthatcanmanagetensofPBsofdata.

10

Whilethebenchmarkspresentedhereprovideseveralperformancedatapoints,therearemanyotheroperationalbenefits

ofpersistingdataforAIonIsilon:

•TheabilitytorunAIin-placeondatausingmulti-protocolaccesswithouttheneedformigration

•Enterprisegradefeaturesout-of-box

•Seamlesslytiertomorecost-effectivenodes

•Scaleupto58PBpercluster(somenodemodelswillhavelowerlimits)

Insummary,DellEMCIsilon-basedstoragesolutionsdeliverthecapacity,performanceandhighconcurrencytoeliminatetheI/OstoragebottlenecksforAI.CombinedwiththeDellPrecision7920DataScienceWorkstation,thissolutionprovidesarock-solidfoundationforlargescale,enterprise-gradedatasciencesolutionswithafutureproofscale-outarchitecturethatmeetsyourAIneedsoftodayandscales