分布式系统报告

CS425/ECE428DistributedSystemsFall2018IndranilGupta(Indy)Lecture2-3:IntroductiontoCloudComputingAllslides©IG12TheHype!Forresterin2010–Cloudcomputingwillgofrom$40.7billionin2010to$241billionin2020.GoldmanSachssayscloudcomputingwillgrowatannualrateof30%from2013-2018Hadoopmarkettoreach$20.8Bbyby2018:TransparencyMarketResearchCompaniesandevenFederal/stategovernmentsusingcloudcomputingnow:3ManyCloudProvidersAWS:AmazonWebServicesEC2:ElasticComputeCloudS3:SimpleStorageServiceEBS:ElasticBlockStorageMicrosoftAzureGoogleCloud/ComputeEngine/AppEngineRightscale,Salesforce,EMC,Gigaspaces,10gen,Datastax,Oracle,VMWare,Yahoo,ClouderaAndmanymanymore!4TwoCategoriesofCloudsCanbeeithera(i)publiccloud,or(ii)privatecloudPrivatecloudsareaccessibleonlytocompanyemployeesPubliccloudsprovideservicetoanypayingcustomer:AmazonS3(SimpleStorageService):storearbitrarydatasets,payperGB-monthstoredAsof2018:1-4cperGBmonthAmazonEC2(ElasticComputeCloud):uploadandrunarbitraryOSimages,payperCPUhourusedAsof2018:0.5cperCPUhrto$5.4perCPUhr(dependingonstrength)Googlecloud:similarpricingasaboveGoogleAppEngine/ComputeEngine:developapplicationswithintheirappengineframework,uploaddatathatwillbeimportedintotheirformat,andrun5CustomersSaveTimeand$$$DavePower,AssociateInformationConsultantatEliLillyandCompany:“WithAWS,Powerssaid,anewservercanbeupandrunninginthreeminutes(itusedtotakeEliLillysevenandahalfweekstodeployaserverinternally)anda64-nodeLinuxclustercanbeonlineinfiveminutes(comparedwiththreemonthsinternally).…It'sjustshyofinstantaneous.”IngoElfering,VicePresidentofInformationTechnologyStrategy,GlaxoSmithKline:“WithOnlineServices,weareabletoreduceourIToperationalcostsbyroughly30%ofwhatwe’respending”JimSwartz,CIO,Sybase:“AtSybase,aprivatecloudofvirtualserversinsideitsdatacenterhassavednearly$US2millionannuallysince2006,Swartzsays,becausethecompanycansharecomputingpowerandstorageresourcesacrossservers.”100sofstartupsinSiliconValleycanharnesslargecomputingresourceswithoutbuyingtheirownmachines.6ButwhatexactlyISacloud?7WhatisaCloud?It’sacluster!It’sasupercomputer!It’sadatastore!It’ssuperman!NoneoftheaboveAlloftheaboveCloud=Lotsofstorage+computecyclesnearby8WhatisaCloud?Asingle-sitecloud(aka“Datacenter”)consistsofComputenodes(groupedintoracks)Switches,connectingtheracksAnetworktopology,e.g.,hierarchicalStorage(backend)nodesconnectedtothenetworkFront-endforsubmittingjobsandreceivingclientrequests(Oftencalled“three-tierarchitecture”)SoftwareServicesAgeographicallydistributedcloudconsistsofMultiplesuchsitesEachsiteperhapswithadifferentstructureandservices9ASampleCloudTopologySothen,whatisacluster?10“ACloudyHistoryofTime”1940195019601970198019902000TimesharingCompanies&DataProcessingIndustryGridsPeertopeersystemsClustersThefirstdatacenters!PCs(notdistributed!)Cloudsanddatacenters201211“ACloudyHistoryofTime”19401950196019701980199020002012CloudsGrids(1980s-2000s):GriPhyN(1970s-80s)OpenScienceGridandLambdaRail(2000s)Globus&otherstandards(1990s-2000s)TimesharingIndustry(1975):MarketShare:Honeywell34%,IBM15%,Xerox10%,CDC10%,DEC10%,UNIVAC10%Honeywell6000&635,IBM370/168,Xerox940&Sigma9,DECPDP-10,UNIVAC1108DataProcessingIndustry-1968:$70M.1978:$3.15BillionFirstlargedatacenters:ENIAC,ORDVAC,ILLIACManyusedvacuumtubesandmechanicalrelaysBerkeleyNOWProjectSupercomputersServerFarms(e.g.,Oceano)P2PSystems(90s-00s)ManyMillionsofusersManyGBperday12Trends:TechnologyDoublingPeriods–storage:12mos,bandwidth:9mos,and(whatlawisthis?)cpucomputecapacity:18mosThenandNowBandwidth1985:mostly56Kbpslinksnationwide2015:TbpslinkswidespreadDiskcapacityToday’sPCshaveTBs,farmorethana1990supercomputer13Trends:UsersThenandNowBiologists:1990:wererunningsmallsingle-moleculesimulationsToday:CERN’sLargeHadronColliderproducingmanyPB/year14PropheciesIn1965,MIT'sFernandoCorbatóandtheotherdesignersoftheMulticsoperatingsystemenvisionedacomputerfacilityoperating“likeapowercompanyorwatercompany”.PlugyourthinclientintothecomputingUtilityandPlayyourfavoriteIntensiveCompute&CommunicateApplicationHavetoday’scloudsbroughtusclosertothisreality?Thinkaboutit.15FourFeaturesNewinToday’sCloudsMassivescale.On-demandaccess:Pay-as-you-go,noupfrontcommitment.AndanyonecanaccessitData-intensiveNature:WhatwasMBshasnowbecomeTBs,PBsandXBs.Dailylogs,forensics,Webdata,etc.Humanshavedatanumbness:Wikipedia(large)compressedisonlyabout10GB!NewCloudProgrammingParadigms:MapReduce/Hadoop,NoSQL/Cassandra/MongoDBandmanyothers.HighinaccessibilityandeaseofprogrammabilityLotsofopen-sourceCombinationofoneormoreofthesegivesrisetonovelandunsolveddistributedcomputingproblemsincloudcomputing.16I.MassiveScaleFacebook[GigaOm,2012]30Kin2009->60Kin2010->180Kin2012Microsoft[NYTimes,2008]150KmachinesGrowthrateof10Kpermonth80KtotalrunningBingIn2013,MicrosoftCosmoshad110Kmachines(4sites)Yahoo![2009]:100KSplitintoclustersof4000AWSEC2[RandyBias,2009]40Kmachines8cores/machineeBay[2012]:50KmachinesHP[2012]:380Kin180DCsGoogle[2011,DataCenterKnowledge]:900K17Quiz:WhereistheWorld’sLargestDatacenter?18Quiz:WhereistheWorld’sLargestDatacenter?(2017)“TheCitadel”Nevada.7.2Millionsq.ft.(2015)InChicago!350EastCermak,Chicago,1.1MILLIONsq.ft.Sharedbymanydifferent“carriers”CriticaltoChicagoMercantileExchangeSee:/news/data-center-news/top-10-largest-data-centers-world/19Whatdoesadatacenterlooklikefrominside?AvirtualwalkthroughadatacenterReference:/cleantech/a-rare-look-inside-facebooks-oregon-data-center-photos-video/20ServersFrontBackInSomehighlysecure(e.g.,financialinfo)21PowerOff-siteOn-siteWUE=AnnualWaterUsage/ITEquipmentEnergy(L/kWh)–lowisgoodPUE=TotalfacilityPower/ITEquipmentPower–lowisgood (e.g.,Google~1.1)22CoolingAirsuckedinfromtop(also,Bugzappers)WaterpurifiedWatersprayedintoair15motorsperserverbank23Extra-FunVideostoWatchMicrosoftGFSDatacenterTour(Youtube)/watch?v=hOxA1l1pQIw

TimelapseofaDatacenterConstructionontheInside(Fortune500company)/watch?v=ujO-xNvXj3g

24II.On-demandaccess:*aaSClassificationOn-demand:rentingacabvs.(previously)rentingacar,orbuyingone.E.g.:AWSElasticComputeCloud(EC2):afewcentstoafew$perCPUhourAWSSimpleStorageService(S3):afewcentsperGB-monthHaaS:HardwareasaServiceYougetaccesstobareboneshardwaremachines,dowhateveryouwantwiththem,Ex:YourownclusterNotalwaysagoodideabecauseofsecurityrisksIaaS:InfrastructureasaServiceYougetaccesstoflexiblecomputingandstorageinfrastructure.Virtualizationisonewayofachievingthis(cgroups,Kubernetes,Dockers,VMs,…).OftensaidtosubsumeHaaS.Ex:AmazonWebServices(AWS:EC2andS3),OpenStack,Eucalyptus,Rightscale,MicrosoftAzure,GoogleCloud.25II.On-demandaccess:*aaSClassificationPaaS:PlatformasaServiceYougetaccesstoflexiblecomputingandstorageinfrastructure,coupledwithasoftwareplatform(oftentightlycoupled)Ex:Google’sAppEngine(Python,Java,Go)SaaS:SoftwareasaServiceYougetaccesstosoftwareservices,whenyouneedthem.OftensaidtosubsumeSOA(ServiceOrientedArchitectures).Ex:Googledocs,MSOfficeondemand26III.Data-intensiveComputingComputation-IntensiveComputingExampleareas:MPI-based,High-performancecomputing,GridsTypicallyrunonsupercomputers(e.g.,NCSABlueWaters)Data-IntensiveTypicallystoredataatdatacentersUsecomputenodesnearbyComputenodesruncomputationservicesIndata-intensivecomputing,thefocusshiftsfromcomputationtothedata:CPUutilizationnolongerthemostimportantresourcemetric,insteadI/Ois(diskand/ornetwork)27IV.NewCloudProgrammingParadigmsEasytowriteandrunhighlyparallelprogramsinnewcloudprogrammingparadigms:Google:MapReduceandSawzallAmazon:ElasticMapReduceservice(pay-as-you-go)Google(MapReduce)Indexing:achainof24MapReducejobs~200Kjobsprocessing50PB/month(in2006)Yahoo!(Hadoop+Pig)WebMap:achainofseveralMapReducejobs300TBofdata,10Kcores,manytensofhours(~2008)Facebook(Hadoop+Hive)~300TBtotal,adding2TB/day(in2008)3Kjobsprocessing55TB/daySimilarnumbersfromothercompanies,e.g.,Yieldex,,etc.NoSQL:MySQLisanindustrystandard,butCassandrais2400timesfaster!28TwoCategoriesofCloudsCanbeeithera(i)publiccloud,or(ii)privatecloudPrivatecloudsareaccessibleonlytocompanyemployeesPubliccloudsprovideservicetoanypayingcustomerYou’restartinganewservice/company:shouldyouuseapubliccloudorpurchaseyourownprivatecloud?29SinglesiteCloud:toOutsourceorOwn?Medium-sizedorganization:wishestorunaserviceforMmonthsServicerequires128servers(1024cores)and524TBSameasUIUCCCT(CloudComputingTestbed)cloudsite(boughtin2009,nowdecommissioned)Outsource(e.g.,viaAWS):monthlycostS3costs:$0.12perGBmonth.EC2costs:$0.10perCPUhour(costsfrom2009)Storage=$0.12X524X1000~$62KTotal=Storage+CPUs=$62K+$0.10X1024X24X30~$136KOwn:monthlycostStorage~$349K/MTotal~$1555K/M+7.5K(includes1sysadmin/100nodes)using0.45:0.4:0.15splitforhardware:power:networkand3yearlifetimeofhardware30SinglesiteCloud:toOutsourceorOwn?Breakevenanalysis:morepreferabletoownif:-$349K/M<$62K(storage)-

$1555K/M+7.5K<$136K(overall)BreakevenpointsM>5.55months(storage)M>12months(overall)Asaresult StartupsusecloudsalotCloudprovidersbenefitmonetarilymostfromstorage31AcademicClouds:EmulabAcommunityresourceopentoresearchersinacademiaandindustry.Verywidelyusedbyresearcherseverywheretoday./

Acluster,withcurrently~500serversFoundedandownedbyUniversityofUtah(ledbyLateProf.JayLepreau)Asauser,youcan:GrabasetofmachinesforyourexperimentYougetroot-level(sudo)accesstothesemachinesYoucanspecifyanetworktopologyforyourclusterYoucanemulateanytopologyAllimages©Emulab32Acommunityresourceopentoresearchersinacademiaand industry/Currently,~1077nodesat~500sitesacrosstheworldFoundedatPrincetonUniversity(ledbyProf.LarryPeterson),butownedinafederatedmannerbythesitesNode:DedicatedserverthatrunscomponentsofPlanetLabservices.Site:Alocation,e