大数据平台建设v20chs

大数据平台建设2014年EMC—某著名企业（南阳站）高端客户研讨会Source:EMCDigitalUniversewithResearchandAnalysisbyIDC,TheDigitalUniverseofOpportunities:RichDataandtheIncreasingValueoftheInternetofThings,April2014.20204.4

ZETTABYTES44

ZETTABYTES10x

MORE数字化宇宙20142013ZETTABYTE

=

1,000,000,000,000,000,000,000bytesFirstExabyteQuarterFirstExabyteYearQ3Q2FirstExabyteMonth200920102011201220132007200820052006FirstExabyteShippedEMC’sExabyte旅程Source:EMCDigitalUniversewithResearchandAnalysisbyIDC,TheDigitalUniverseofOpportunities:RichDataandtheIncreasingValueoftheInternetofThings,April2014.2013202028

MILLION

ITPROS

WORLDWIDE36

MILLION

ITPROS

WORLDWIDE8

MILLIONMORE

OFYOU数字化宇宙2014Source:EMCDigitalUniversewithResearchandAnalysisbyIDC,TheDigitalUniverseofOpportunities:RichDataandtheIncreasingValueoftheInternetofThings,April2014.2020230

GBPERITPRO1230

GBPERITPRO5x

MORE数字化宇宙20142013大数据无处不在Video&

GraphicsGenetic

SequencingSatellite

PhotosComputer

ModelingSeismic

ExplorationMusic/AudioHealthRecordMedical

Imaging大数据地图定义大数据湖MachinelatencyInteractivereportsBatchprocessingHumaninteractionsMillisecondsSecondsSeconds,MinutesMinutes,Hours分布式流计算，MPPDBReal-timeanalytic/MPP/SQLonHadoopNear-real-timeanalytic不同的响应时间的要求决定了不同的功能实现MapReduceofflineanalytic/Pivotal的大数据平台服务能力企业级Hadoop商用平台1横向扩展的MPP数据仓库2基于分布式内存的实时数据平台3大数据平台咨询以及架构规划4大数据平台实施运维以及优化5PivotalDataLake融合架构CentralizedManagementSystemmonitoringSystemmanagementUnifiedDataManagementTierDatamgmt.servicesMDMRDMAuditandpolicymgmt.ProcessingTierWorkflowManagementDistillationTierHDFSstorageUnstructuredandstructureddataIn-memoryMPPdatabaseUnifiedSourcesFlexibleActionsReal-time

ingestionMicrobatch

ingestionBatchingestionReal-timeinsightsInteractiveinsightsBatchinsightsWorld’sLeadingExpertsPivotalLabs–PivotalDataLabsOnDemandServicesPivotalDataDispatchBATCHBATCHNEARTIMENEARTIMEHAWQGreenplumDBPivotalHDREALTIMEREALTIMEGemFireXDGemFirePivotal的大数据平台企业数据架构分析数据集市MPPDatabase实时数据库In-MemoryDB实时分析系统In-MemoryObject企业数据仓库RDBMS数据缓冲沉淀系统报表与展现DataVisualization数据采集系统Stream/CEPAnalytic

DataMartsOperational

IntelligenceRun-Time

ApplicationsEnterpriseData

WarehouseDataStaging

PlatformTraditionalBI/ReportingDataVisualizationDataIngestionSystemPivotal全套对应方案分布式计算之MPPShared-Nothing架构演进共享磁盘例如：InformixXPSOracleRACDB2pureScaleDBSAN/共享磁盘DBDBDB网络SAN/FC完全共享例如：SMP服务器DB磁盘无共享例如：GreenplumDBDBDBDB网络磁盘磁盘磁盘磁盘MasterMPP注：蓝灰色表示共享资源Mainframe,MiniComputerTerminalsLAN/InternetClient/Server

PCMobileCloudBigDataSocial

MobileDevices1STPLATFORM2NDPLATFORM3RDPLATFORMMILLIONSO