高级可配置电源接口发展现状.ppt

,UpdatesinACPIBased,MemoryHot-Plug,Agenda,.5.6.,ACPI&MemoryHot-PlugMemoryHot-PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot-PlugFuturework,2,ACPI&MemoryHot-PlugACPI:AdvancedConfigurationandPowerInterfaceACPIisaninterfacespecificationofOperatingSystem-directedmotherboarddeviceconfigurationandPowerManagement.-ACPISpecification5.0,Methods(dynamic),ACPITables(static),ACPIBIOS(Firmware)ACPIRegistersHardware,Kernel(Software)ACPIFrameworkRunTimeBootTime,OSlayerframework.EventhandlingAPI,Staticinfousedonlyatboottime.DSDTSRAT,Eventdrivenmodel.EventregistersControlregisters3,Dynamicmethodsusedatruntime._EJ0,_STASCI(SystemControlInterrupt),Kernel,MemoryHot-PlugSubsystem,4,Hardware,Methods(dynamic),ACPIRegisters,Eventinfo,Calleventhandler,ACPITables(static)ACPIBIOS,GenerateSCI(SystemControlInterrupt),CallAPICallACPIMethod,Hardwareoperation,InstalleventhandlerACPIFrameworkReadACPITables,ACPI&MemoryHot-PlugACPIandMemoryHot-Plug,BoottimeprocessRuntimeprocess,Calldevicedependentcode,Hot-Plughappens,MemoryDeviceDriver,MemoryAffinity,SRAT,StaticinformationofNUMAarchitecture.,MemoryAffinityMemoryAffinity,MainlyusefulinformationMemoryrange,PXM(proximitydomain)Hotpluggableflag,ACPI&MemoryHot-PlugStaticconfigurationSRAT:SystemResourceAffinityTable,5,Agenda,.5.6.,ACPI&MemoryHot-PlugMemoryHot-PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot-PlugFuturework,6,Memorymanagementbackground,Kernelspace,directmapping,(64TB)hole,modulemappingspacekerneltextmappingholevirtualmemorymap(1TB)holevmalloc/ioremapspacehole,Physicalspace,BlockX,movablepages(used),page_structsofBlockX,7,4.Virtualmemorymapping,pages.5.Memoryblocktobehot-plugged.,1.Userprocessespagetables.2.Kerneldirectmappingpagetables.3.Virtualmemorymapping,pagetables.,1,5,3,2,4,process(128TB)process(128TB),Userprocessspace(128TB)process(128TB),MemoryHot-addProcess,Physicalspace,Kernelspace,modulemappingspacekerneltextmappingholevirtualmemorymap(1TB),holevmalloc/ioremapspaceholedirectmapping(64TB)hole,Empty,page_structsofBlockX,8,4,3,2,1,process(128TB)process(128TB),Userprocessspace(128TB)process(128TB),BlockX,pages,(invalid),1.Initializedirectmapping.2.Initializevirtualmemory,mapping.3.Onlinememoryblock.,4.Establishuserspacemapping.,MemoryHot-removeProcess,Kernelspace,modulemappingspacekerneltextmappingholevirtualmemorymap(1TB)hole,vmalloc/ioremapspaceholedirectmapping(64TB)hole,Empty,9,5,4,process(128TB)process(128TB),Userprocessspace(128TB)process(128TB),3.Reestablishnewuserspacemapping.,Physicalspace,page_structsofBlockXNewpage_structs,pages(invalid)Newpage,Newpage2BlockX,1.Unmapuserspacepages.2.Offlinepagesandmigratepages.,13,4.Freekerneldirectmapping.5.Freevirtualmemory,mapping.,10,directmapping(64TB),KernelspaceUserspace,(128TB),Physicalspace,Userpage,Userpage,Kernelpageusermapping,ProblemKernelpagescannotbehot-removedKernelpage,variable,KernelpageUserpage1.migrate,Kernelpageva=pa+offset(1-1mapped),Userpage,Kernelpagenotmigratable,not,hot-removablehot-removable,2.hot-remove,Agenda,.5.6.,ACPI&MemoryHot-PlugMemoryHot-PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot-PlugFuturework,11,AvoidAllocatingHot-pluggableMemory,start_kernel,Thekernelcannotrecognizewhichmemoryishot-pluggable,free_all_bootmemThekernelcanrecognizethehot-pluggablememory,ParsingSRAT,memblockallocator,buddyallocator,Boottime,12,AvoidAllocatingHot-pluggableMemory(BeforeParsingSRAT),How?,free_all_bootmem,ParsingSRAT,start_kernelAllocatememoryjustbehindthekernelimage:,-,Thenodekernelresidesinisun-hot-pluggableIntroduceanewbottom-upmodeformemblockallocator,Boottime,13,Top-downV.S.Bottom-upTop-downallocationmode,NODE0NODE1,NODE0NODE1,-,Memoryatlowaddressesisprecious(e.g.forDMAdevices)Fornon-memory-hot-plugusers,-,Inmostcases,memoryallocatedbeforeparsingSRATwontbetoomuch,soitcouldhighlylikelybeinthesamenodewithkernelimageFormemory-hot-plugusers,Bottom-upallocationmodeKernelimage,Kernelimage,14,newallocation,AvoidAllocatingHot-pluggableMemory(AfterParsingSRAT),start_kernel,free_all_bootmem,ParsingSRAT,Markhot-pluggablememoryandskiptheminfollowedallocations:,-,IntroduceMEMBLOCK_HOTPLUGflagformemblockallocatorChangebacktotop-downmode,How?,Boottime,15,start_kernel,SummaryAllocatememoryjustbehindthekernelimagebyusingbottom-upallocationmode,free_all_bootmemSkipallocatingmemorywhichismarkedwithMEMBLOCK_HOTPLUGflag,ParsingSRAT,memblockallocator,buddyallocator,Boottime,16,17,BootOption:movable_nodeAboot-timeswitchtoenablemovablenodefunctionality,HigherprioritythankernelcoreandmovablecorebootoptionMakesuremovablenodefunctionalitycanbeconfigured,Agenda,.5.6.,ACPI&MemoryHot-PlugMemoryHot-PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot-PlugFuturework,18,PagesPinnedbyKernelShort-livedPinscma,fs/exec,security,nfs,events,net/ceph,lotsofdirversLong-livedPins(pinnedinalllifecycle)KVMRealmodeidentityEPTpagetableAPICaccesspageAIOEventRingbuffer,19,20,Short-livedPinsJustfordatacopyingSolution:NohandlingMemory-offlineretrytimeout(120s)isenough,21,Long-livedPins:KVMRealmodeeptidentitypagetableNeededforCPUsthatdonotallowenteringguestmodewithpagingdisable.Populatedwithptesthatcoverentireguestsmemory.APICaccesspageUsedbyCPUdirectlytocatchMMIOaccesstoanAPIC.,22,Guest,gva,gpa,Host,page2,Guest,Pagetable,EPT,hpa1,hpa2,page1,kvm_arch|-ept_identity_pagetable,|-apic_access_page,Long-livedPins:KVMWhypinned:Forconvenience,notnecessary.VMCS:APIC_ACCESS_ADDR,3.UpdateAPIC_ACCESS_ADDRpointerinVMCS.,3,2,1,2.Updatelocalcacheforpagestruct.,1.UpdateEPTentry.,23,Host,hpa1page1,hpa2page2,Long-livedPins:KVMEPTidentitypagetable:UnpindirectlyGuest,gpa,EPT,1.Guestrequiresapage,2.MMUsearchesEPTforahpa,3.MMUreturnsEPTviolationsincepagehasbeenmigrated,4.KVMhandlesEPTviolationandfindthenewpage,5.KVMupdatesEPT6.MMUreturnsnewhpa,2,1,6,3,4,5,Long-livedPins:KVMAPICaccesspage,24,hpa1,hpa2,page1page2L0(HostMode),L2Guest,GuestModeVMCS,L1Guest,VMCS,TwokindsofVM-Entry/Exit:VM-Entry/ExitinGuestmodeareemulatedbyKVM.L1L2VM-Entry/Exit,VM-Entry/ExitbetweenHostandGuestmodeareprovidedbyCPU.L0L1VM-Entry/ExitL0L2VM-Entry/Exit,Twocasestohandle:CPUisrunningL1Guest.CPUisrunningL2Guest.,25,L0(HostMode),hpa1page1,hpa2page2,L2Guest,VMCS,L1Guest,VMCS,Long-livedPins:KVMAPICaccesspageCPUisrunningL1Guest:GuestMode,PageMigration:,1.Trytounmappage2.MMUnotifierworks,3.Unmapandmigratepage4.Pagemigrated,KVM:,1.HandlerenforcesaL1-L0VM-Exit2.HandlermakesvcpurequesttoupdateL1VMCS,3.NextL0-L1VM-Entry:GUPwaitsforpagemigration4.UpdateL1VMCS5.UpdateL2VMCSin,nextL1-L2VM-Entry,L0(HostMode),hpa1page1,hpa2page2,L2Guest,VMCS,L1Guest,VMCS,Long-livedPins:KVMAPICaccesspageCPUisrunningL2Guest:GuestMode,PageMigration:,1.Trytounmappage2.MMUnotifierworks,3.Unmapandmigratepage4.Pagemigrated,KVM:,1.HandlerenforcesaL2-L0VM-Exit2.HandlermakesvcpurequesttoupdateL2VMCS,3.NextL0-L2VM-Entry:GUPwaitsforpagemigration4.UpdateL2VMCS5.UpdateL1VMCSin,nextL2-L1VM-Exit26,Long-livedPins:AIO,27,AIOEventRingBufferUsedbykerneltonotifyuserspacethatAIOhascompleted.,Long-livedPins:AIOWhypinned:UnabletoknowwhenAIOcompletes.,Userprocess:AIOeventhandledAIORingBufferKernel:AIOcompleted,AIO:1.AIOpinsringpages2.AIOinprogress3.AIOcompletes4.AIOunpinringpages28,PageMigration:1.Offlinememory2.Pagemigration|-migratepages()3.Pagemigrationfails,29,Long-livedPins:AIONeednewsplution,WhycannotuseMMUnotifier:1.Nowaytogetthepage(havetorepin),GUPmaysleepiniointerruptcontext,2.NowaytonotifyAIOtorepinthepage,NosuchMMUnotifierafterpagemigrationcompletes,new,page,oldpage,3)Migratepage,AIO:,1.AIOpinsringpages2.AIOinprogress3.aio_migrate(),1)lock2)unpingringpages3)migrateringpages4)repinringpages,5)unlock4.AIOcompletes,5.AIOunpinringpages30,PageMigration:,1.Offlinememory2.Pagemigrationstarts,3.Pagemigrationinprogress4.Pagemigrationends,AIOcomplete1)BlockedbylockAIORingBuffer,5)Unlockedwhenmigrationends,2)Unpin,4)Repin,Long-livedPins:AIOSolution:Implem