云计算与虚拟化

云计算与虚拟化肖臻 研究员/北京大学信息科学技术学院北京大学信息科学技术学院计算机网络与分布式系统实验室2009.3.27Copyright Zhen Xiao. All rights reserved. 什么是云计算 ？什么是云计算 ？ 云计算就像法国大餐 一 样正被人们津津乐道 。云计算就像法国大餐 样正被人们津津乐道- Ben Pring (Gartner资深分析师 ) 新瓶装旧酒 ？新瓶装旧酒 ？ 分布式计算 G id ti i di t ib t d ti ?rid computing is dis ribu e compu ng 网格计算 Batch-oriented, scientific computation Utility computing 煤气、水、电 买、租、服务？Copyright Zhen Xiao. All rights reserved. 核心思想 On demand computing核心思想 ： - 随需而变的资源配置： Highly variable resource demands for Internet services Provisioning for the typical load is insufficient Provisioning for the highest load is wasteful Pitfall of relying on past usage for future flash crowd (Anthrax, Boeing Web sites) Manual resource provisioning is slow and expensive 技术挑战：虚拟化是核心！ 双向透明：用户和 application provider都无需知道计算由谁来提供 just throw it onto any server 怎样实现应用的迁移与复制 ？怎样实现应用的迁移与复制 ？Copyright Zhen Xiao. All rights reserved. Server Consolidation Server ConsolidationCopyright Zhen Xiao. All rights reserved. Adaptive to Changing Demand APP1 APP1APP1APP1 APP2 APP2APP1 APP1APP3 APP3CPUSchedulingAPP2APP2 APP2APP2APP2 APP2APP1 APP1APP3 APP3NetworkCopyright Zhen Xiao. All rights reserved. 虚拟化实例 Dynamic virtual cluster虚拟化实例 ： 问题 Cluster基础架构和应用程序需求之间的矛盾 共享的 Cluster环境 vs. 用户不同的环境需求 动态 Cluster（ COD: Cluster On Demand) 的 Deployment和 Reclustering开销较大 多任务共享 Cluster环境的调度问题多任务共享 环境的调度问题 异构的 Clusters，难以共同处理任务 解决问题的思路： 基于虚拟机的动态 Cluster Cluster节点构成： Virtual Machine 利用 Suspend/Resume、 ballooning等减小同一物理节点上分属不同 virtual cluster的 VM之间的转移代价Copyright Zhen Xiao. All rights reserved. Dynamic virtual cluster From Physical cluster to Virtual cluster clusterApplication 1 Application 2ppVM1.1 VM1.2 VM2.1 VM2.2 VM3.1 VM3.2 VM4.1 VM4.2 VM5.1 VM5.2Node 1 Node 2 Node 3 Node 4 Node 5Copyright Zhen Xiao. All rights reserved. 虚拟化开销freebenchBonnie+200000250000)202550000100000150000Throughput(KB/sNative LinuxDom0DomU51015score NativeDom0DomU0putc write(2) rewrite getc read(2)0Analyzer FourInARow Mason pCompress2 Gmean (int) PiFFT DistRay Neural Gmean(float) GeometricMeandbench500600700Bonnie+200000250000s)200300400Throughput(MB/s)NativeDom0DomU50000100000150000Throughput(KB/sNativeDom0Dom1010010 20 50 100clients0putc write(2) rewrite getc read(2)Copyright Zhen Xiao. All rights reserved. Dynamic virtual cluster 我们的目标 ： Dynamic virtual cluster： virtual 减小同一物理节点上分属不同 virtual cluster的VM之间的转移代价之间的转移代价 Suspend/Resume 资源的动态配置， e.g. ballooning 评价现有 VMM在该环境下的功能和性能 ， 确定评价现有 在该环境下的功能和性能 ， 确定改进的方向Copyright Zhen Xiao. All rights reserved. Dynamic virtual cluster Reclustering virtual cluster Using VM suspend/resumeApplication 1 Application 2VM4.2VM1.1 VM2.1 VM5.2VM3.1VM3.2Node 4Node 1 Node 2 Node 5Node 3VM3.2VM2.2VM1.2 VM4.1 VM5.1Copyright Zhen Xiao. All rights reserved. Suspend/Resume的执行时间的执行时间suspend/resume30.00db h20 0025.00enc _savedbench_restoredbench_pausedbench_unpausefbh15.00.time(s)freebench_savefreebench_restorefreebench_pausefreebench_unpause5.0010.00idle_saveidle_restoreidle_pauseidle_unpause0.00256 512 1024 2048memory(MB)Copyright Zhen Xiao. All rights reserved. Dynamic virtual cluster Reclustering virtual cluster Using dynamic resource reallocation across VMsApplication 1 Application 2Ballooning: 0.1sVM3.1 VM3.2VM4.1 VM4.2 VM5.1 VM5.2VM1.1 VM1.2 VM2.1 VM2.2Node 3 Node 4Node 1 Node 2 Node 5Copyright Zhen Xiao. All rights reserved. Configuration of our demo environmentCopyright Zhen Xiao. All rights reserved. Realization of Hot-backup: challenges of Hot 1. How to synchronize memory and cpu state between Primary Server and Backup Server?2. When synchronizing memory and cpu state, disk and network state should also be taken into consideration. Then how can we guarantee that two VMs (one is the primary VM and the other backup) s device states also match? 3. Synchronizing two VMs will bring some performance degradation. How can we minimize such a lost? lost?Copyright Zhen Xiao. All rights reserved. Hot-backup: our realization Copyright Zhen Xiao. All rights reserved. Write memory procedure before hot-backupMain ServerWrite memory procedure before Virtual Machine VMM Page table for virtual machineAll physcical memorywritepage base address can write0x293a1000 10x293a2000 1Address:0x293a20000xa09b0000 1At thi ti i t l hi t bl i s time, v r ua mac ne s page a e s configured like above so that virtual machine can write a page without any side-effect side .Copyright Zhen Xiao. All rights reserved. Write memory procedure when hot-backupingWrite memory procedure when Main ServerOnline Virtual Machine VMM Page table for virtual machineAll physical memorywriteemptyempty0x293a2000dirty page log tablepage base address can write0x293a1000 00x293a2000 0Address:0x293a2000trapLog dirty page number0xa09b0000 0Page fault handlerDo write operationGet dirty page contentsHot-backup daemonget dirty page numbersDirty pages contentsBk SSddit ttBackup erverHot-backup daemonBackup Virtual MachineSend dir y page con en s to backup serverVMMAll physical memorypage base address can writePage table for virtual machineDirty pages contentshypercall0x293a1000 00x293a2000 0 Address:0x293a2000Synchronize page using hypercallCopyright Zhen Xiao. All rights reserved. Synchronize cpu state stateMain Server Running VMsSuspended VMs VMsVirtual machine X under hot-backupcpu states.nput VM X to dd ttcpu states.nsuspended stateBackup ServerSuspended VMssynchronizeVirtual machine Xcpu states.(n-1)Copyright Zhen Xiao. All rights reserved. Synchronize network state stateMain Servern=1 Generating checkpoint nVirtual network card vifx.0pktXnetfilterpkt1pkt2Checkpoint n acked,n=n+1Packet queuecheckpoint nGenerated, suspend VM topktn.send checkpoint n,wait for ackNULLpktXget vcpu statesWait for ackpeth0Release packets As the above figure shows, before checkpointN is acknowledged by backup server, we block all packets generated by domU in a queue built in netfilter and once all generated in checkpointN is acknowleged, we can immediately transfer all the packets blocked in the queue.Copyright Zhen Xiao. All rights reserved. Synchronize disk state Write disk log when VM Rollback disk image disk log when is runningMain Server NFS ServerVM di k i before back-up VM gets ready to runNFS ServerVirtual machineBlock backend s magesector XReqest:Write sector Xget sector Xs original dataWrite sector XVM disk imagesector a1sector a2Image log filesector a1sector a2Rollback disk dataImage log fileSector dataWrite original data to log filesector ansector an In practice, we generate a log file for each checkpoint, for example, if next checkpoint to be generated is checkpoint K then we generate log K an