雅虎云计算介绍

1、Cloud Computing Yahoo!What well cover today Why Cloud? Scale and Abstraction; Quality and Agility Yahoo!s unique footprint Yahoo!s Cloud Strategy Overview of the Yahoo! Cloud vision and portfolio Deep dive on Horizontal & Functional Cloud Services The Yahoo! Open Strategy Marrying Yahoo!s “Open

2、Strategy”, its platforms and ethic with external Cloud servicesAgility & InnovationQuality & StabilityWhy Cloud? Benefits for Yahoo!Higher Agility & Stability while maintaining Scale Abstraction Enable developers to focus on their applications, not infrastructure Accelerating innovation

3、Adding new features and products at an ever faster rate Increasing Scale & Availability More robustly, more globally, more completely, for a given budgetCloud is pushing up the Operation Excellence Curve Yahoo!s Unique Cloud: Unprecedented Scale Massive user base and engagement 500M+ unique user

4、s per month Hundreds of petabyte of storage Hundreds of billions of objects Hundred of thousands of requests/sec Global Tens of globally distributed data centers Serving each region at low latencies Challenging Users Rapidly extracting value from voluminous data Downtime is not an option (outages co

5、st $millions) Variable usage patternsYahoo! Cloud ServicesROI & InnovationUsersApplicationsFunctional CloudServicesHorizontal Cloud Services Physical LayerY!OS, BOSS, YQL, APT, Analytics, Storage, Batch, Edge Serving,Yahoo! Cloud Services: Focus on PaaS offeringsROI & InnovationUsersApplicat

6、ionsFunctional CloudServicesHorizontal Cloud Services Physical LayerIaaSPaaSSaaSFrom Infrastructure to Shareholders benefit Horizontal Cloud Focus on open source and collaborative R&D with industry, academia and government Functional Cloud Focus on developing open strategy frameworks, tools and

7、services for developers (at Yahoo! and beyond) Combined Together Leverage our unique scale, assets and data to drive disruptive innovations in the market and expand Yahoo!s competitive differentiationYahoo! Cloud Strategy in Action:The Front Page Case Study Horizontal Cloud Storage & Hadoop Anal

8、yze extremely large content data sets Functional Cloud Content Optimization Rate content items based on various parameters Applications Yahoos Front Page Display “high rating” items to the right users Benefit consumers and advertisers and grow Yahoo!s revenueYahoo! Cloud Strategy in Action:The Inqui

9、sitor Case Study Horizontal Cloud Hadoop Analyze large search-index data sets Functional Cloud - BOSS Expose the data in a structured, open, flexibleand “cloud like” way Applications - iPhoneTM Inquisitor Leverage BOSS to provide innovative consumer experience Benefit consumers and grow Yahoo!s reve

10、nueHorizontal Cloud ServicesROI & InnovationUsersApplicationsFunctional CloudServicesHorizontal Cloud Services Physical LayerHorizontal Cloud Services Optimized for Yahoo!-scale Yahoo!-internal focus Data processing and serving environments Drive faster innovation and agility Shorter product dev

11、elopment cycles Reduce labor and costs for infrastructure Multi-year effort Strategic investment across the companyHorizontal Cloud Services: Conceptual ViewCommon Approaches to QA, Production Engineering,Performance Engineering, Datacenter Management, and OptimizationID & Account ManagementShar

12、ed InfrastructureProvisioning & Virtualization (Xen)Simple APIs Operational StorageStructured, unstructuredBatch Storage & ProcessingHadoop, PIGEdge Content ServicesCaching, ProxiesOnline ServingWeb, DataSecurity and AuthenticationMetering, BillingMonitoring & QoSHorizontal Cloud Service

13、s: Use CasesAds OptimizationContent OptimizationSearch IndexImage/Video Storage &DeliveryMachine Learning (e.g. Spam filters)AttachmentStorageYahoo! Distribution of Hadoop Hadoop in a nutshell Open source distributed file system & parallel execution environment to process massive amounts of

14、data Started in 2005, became top-level Apache project in 2008 Simple Design for Horizontal Scaling on commodity HW Yahoo! Distribution of Hadoop Source distribution of Yahoo!s implementation of Hadoop(Based entirely on code found in the Apache Hadoop) Tested and deployed at Yahoo!s massive scale Ben

15、efit the larger ecosystem , Increase pace of innovation http:/ runs the largest Hadoop Clusters in the World 25,000+ nodes Clusters of up to 4,000 nodes 4 Tiers of clusters Development & Testing, POCs, Science & Research, Production Terasort Benchmarks 62 seconds to sort One Terabyte (run on

16、 1,500 nodes) 16.25 hours to sort One Petabyte (run on 3,700 nodes) Webmap application 490 TB shuffling 280 TB outputCase Study - Search AssistDatabase for Search Assist is built using Hadoop. 3 years of log-data, 20-steps of map-reduce Leverage Hadoops scalability, load balancing and resiliency Sim

17、plified access, flexibility for rapid innovation (from C+ to Python)Before HadoopAfter HadoopTime26 days20 minutesDevelopment Time2-3 weeks2-3 days16Functional Cloud ServicesROI & InnovationUsersApplicationsFunctional CloudServicesHorizontal Cloud Services Physical LayerFunctional Cloud Services

18、 Provides functional capabilities for applications Help developers to accomplish integrated web experiences in a faster and easier way Provides common set of functional “building blocks” “Powered by” the horizontal cloud services Abstracts infrastructure services from the Application E.g. Storage, C

19、ompute, Serving, Robustness and Scalability Self-Served, Global, Managed, Elastic and MeteredFunctional Cloud Services: YQL & BOSS A single endpoint service that enables developers to query, filter and combine data across Yahoo! and beyond http:/ Yahoo! Search infrastructure and technology to de

20、velopers and companies to help them build their own search experiences Build your Own Search Service http:/ Query LanguageBuild your Own Search Service (BOSS) Yahoo!s open search web services platform Serving hundreds of millions of users across the Web. Goal: foster innovation in the search industr

21、y Build and launch web-scale search products that utilize the entire Yahoo! Search index. Access to Yahoo!s investments in crawling and indexing, ranking and relevancy algorithmsYahoo! Query Language (YQL) Single endpoint service to query, filter and combine data across Yahoo! and beyond The “Internet API” SQL-like SELECT syntax for getting the right data Quickly discover available