课程资源云计算cclecture3商业平台和开源

App Amazon(AmazonAlibaba(Aliyun234的 FileTheChubbylockserviceforloosely-coupleddistributed5 6 7提供数据结构化MapReduce数据源

对BigTable中数使用BigTable 记录MapReduce其它锁服务和信 8

9 AmazonMachineImage ElasticBlock

AmazonAWS微软WindowsSQL 云端操作系统Windows WindowsAzure WindowsAzure SQL 数据中心AuthorityContainer

SQLSQLSQL SQLAzure数据库

WindowsAzure •IOE+Greenplum+Hadoop+

愿景

))( (

Linux集群 混

M 混 文件 ，单集群>

FuxiMasterHA，AppMaster

..

Fuxi ..

..

…… ……

Met

飞天集群

100（目前单日平均 HadoopCommon:ThecommonutilitiesthatsupporttheotherHadoopmodules.HadoopDistributedFileSystem(HDFS™):Adistributedfilesystemthatprovideshigh-throughputaccesstoapplicationdata.HadoopYARN:Aframeworkforjobschedulingandclusterresourcemanagement.HadoopMapReduce:AYARN-basedsystemforparallelprocessingoflargedatasets.HadoopOzone:AnobjectstoreforHadoop.HadoopSubmarine:Alearningenginefor

Ambari:Aweb-basedtoolforprovisioning,managing,andmonitoringApacheHadoopclusterswhichincludessupportforHadoopHDFS,HadoopMapReduce,Hive,HCatalog,HBase,ZooKeeper,Oozie,PigandSqoop.AmbarialsoprovidesadashboardforviewingclusterhealthsuchasheatmapsandabilitytoviewMapReduce,PigandHiveapplicationsvisuallyalongwithfeaturestodiagnosetheirperformancecharacteristicsinauser-friendlymanner.Avro:AdataserializationCassandra:Ascalablemulti-masterdatabasewithnosinglepointsoffailure.Chukwa:AdatacollectionsystemformanagingdistributedHBase:Ascalable,distributeddatabasethatsupportsstructureddatastorageforlargetables.Hive:AdatawarehouseinfrastructurethatprovidessummarizationandadhocMahout:AScalablemachinelearninganddataminingPig:Ahigh-leveldata-flowlanguageandexecutionframeworkforparallelcomputation.Spark:AfastandgeneralcomputeengineforHadoopdata.Sparkprovidesasimpleandexpressiveprogrammingmodelthatsupportsawiderangeofapplications,includingETL,machinelearning,streamprocessing,andgraphcomputation.Tez:Ageneralizeddata-flowprogrammingframework,builtonHadoopYARN,whichprovidesapowerfulandflexibleenginetoexecuteanarbitraryDAGoftaskstoprocessdataforbothbatchandin ctiveuse-cases.TezisbeingadoptedbyHive™,Pig™andotherframeworksintheHadoopecosystem,andalsobyothercommercialsoftware(e.g.ETLtools),tore ceHadoopMapReduceastheunderlyingexecutionengine.ZooKeeper:Ahigh-performancecoordinationservicefordistributedapplications.

与机器学习等需要迭代的mapreduce的算法 WalrusWEucalypt