高级可配置电源接口发展现状.ppt

UpdatesinACPIBased MemoryHot Plug Agenda 1 2 3 4 5 6 ACPI MemoryHot PlugMemoryHot PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot PlugFuturework 2 ACPI MemoryHot Plug ACPI AdvancedConfigurationandPowerInterfaceACPIisaninterfacespecificationofOperatingSystem directedmotherboarddeviceconfigurationandPowerManagement ACPISpecification5 0 Methods dynamic ACPITables static ACPIBIOS Firmware ACPIRegistersHardware Kernel Software ACPIFrameworkRunTimeBootTime OSlayerframework Eventhandling API Staticinfousedonlyatboottime DSDT SRAT Eventdrivenmodel Eventregisters Controlregisters 3 Dynamicmethodsusedatruntime EJ0 STA SCI SystemControlInterrupt Kernel MemoryHot PlugSubsystem 4 Hardware Methods dynamic ACPIRegisters Eventinfo Calleventhandler ACPITables static ACPIBIOS GenerateSCI SystemControlInterrupt CallAPICallACPIMethod Hardwareoperation InstalleventhandlerACPIFrameworkReadACPITables ACPI MemoryHot Plug ACPIandMemoryHot Plug BoottimeprocessRuntimeprocess Calldevicedependentcode Hot Plughappens MemoryDeviceDriver MemoryAffinity SRAT StaticinformationofNUMAarchitecture MemoryAffinityMemoryAffinity Mainlyusefulinformation Memoryrange PXM proximitydomain Hotpluggableflag ACPI MemoryHot Plug Staticconfiguration SRAT SystemResourceAffinityTable 5 Agenda 1 2 3 4 5 6 ACPI MemoryHot PlugMemoryHot PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot PlugFuturework 6 Memorymanagementbackground Kernelspace directmapping 64TB hole modulemappingspacekerneltextmappingholevirtualmemorymap 1TB holevmalloc ioremapspacehole Physicalspace BlockX movablepages used page structsofBlockX 7 4 Virtualmemorymapping pages 5 Memoryblocktobehot plugged 1 Userprocesses pagetables 2 Kerneldirectmappingpagetables 3 Virtualmemorymapping pagetables 1 5 3 2 4 process 128TB process 128TB Userprocessspace 128TB process 128TB MemoryHot addProcess Physicalspace Kernelspace modulemappingspacekerneltextmappingholevirtualmemorymap 1TB holevmalloc ioremapspaceholedirectmapping 64TB hole Empty page structsofBlockX 8 4 3 2 1 process 128TB process 128TB Userprocessspace 128TB process 128TB BlockX pages invalid 1 Initializedirectmapping 2 Initializevirtualmemory mapping 3 Onlinememoryblock 4 Establishuserspacemapping MemoryHot removeProcess Kernelspace modulemappingspacekerneltextmappingholevirtualmemorymap 1TB hole vmalloc ioremapspaceholedirectmapping 64TB hole Empty 9 5 4 process 128TB process 128TB Userprocessspace 128TB process 128TB 3 Reestablishnewuserspacemapping Physicalspace page structsofBlockXNewpage structs pages invalid Newpage Newpage2BlockX 1 Unmapuserspacepages 2 Offlinepagesandmigratepages 13 4 Freekerneldirectmapping 5 Freevirtualmemory mapping 10 directmapping 64TB KernelspaceUserspace 128TB Physicalspace Userpage Userpage Kernelpageusermapping Problem Kernelpagescannotbehot removedKernelpage variable KernelpageUserpage1 migrate Kernelpageva pa offset 1 1mapped Userpage Kernelpagenotmigratable not hot removablehot removable 2 hot remove Agenda 1 2 3 4 5 6 ACPI MemoryHot PlugMemoryHot PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot PlugFuturework 11 AvoidAllocatingHot pluggableMemory start kernel Thekernelcannotrecognizewhichmemoryishot pluggable free all bootmemThekernelcanrecognizethehot pluggablememory ParsingSRAT memblockallocator buddyallocator Boottime 12 AvoidAllocatingHot pluggableMemory BeforeParsingSRAT How free all bootmem ParsingSRAT start kernelAllocatememoryjustbehindthekernelimage Thenodekernelresidesinisun hot pluggableIntroduceanewbottom upmodeformemblockallocator Boottime 13 Top downV S Bottom upTop downallocationmode NODE0NODE1 NODE0NODE1 Memoryatlowaddressesisprecious e g forDMAdevices Fornon memory hot plugusers Inmostcases memoryallocatedbeforeparsingSRATwon tbetoomuch soitcouldhighlylikelybeinthesamenodewithkernelimageFormemory hot plugusers Bottom upallocationmodeKernelimage Kernelimage 14 newallocation AvoidAllocatingHot pluggableMemory AfterParsingSRAT start kernel free all bootmem ParsingSRAT Markhot pluggablememoryandskiptheminfollowedallocations IntroduceMEMBLOCK HOTPLUGflagformemblockallocatorChangebacktotop downmode How Boottime 15 start kernel SummaryAllocatememoryjustbehindthekernelimagebyusingbottom upallocationmode free all bootmemSkipallocatingmemorywhichismarkedwithMEMBLOCK HOTPLUGflag ParsingSRAT memblockallocator buddyallocator Boottime 16 17 BootOption movable node Aboot timeswitchtoenablemovablenodefunctionality Higherprioritythankernelcoreandmovablecorebootoption Makesuremovablenodefunctionalitycanbeconfigured Agenda 1 2 3 4 5 6 ACPI MemoryHot PlugMemoryHot PlugProcessBootMemoryHandlingPinnedPagesMigrationQEmumemoryHot PlugFuturework 18 PagesPinnedbyKernel Short livedPins cma fs exec security nfs events net ceph lotsofdirvers Long livedPins pinnedinalllifecycle KVM RealmodeidentityEPTpagetable APICaccesspage AIO EventRingbuffer 19 20 Short livedPins Justfordatacopying Solution Nohandling Memory offlineretrytimeout 120s isenough 21 Long livedPins KVM Realmodeeptidentitypagetable NeededforCPUsthatdonotallowenteringguestmodewithpagingdisable Populatedwithptesthatcoverentireguest smemory APICaccesspage UsedbyCPUdirectlytocatchMMIOaccesstoanAPIC 22 Guest gva gpa Host page2 Guest Pagetable EPT hpa1 hpa2 page1 kvm arch ept identity pagetable apic access page Long livedPins KVMWhypinned Forconvenience notnecessary VMCS APIC ACCESS ADDR 3 UpdateAPIC ACCESS ADDRpointerinVMCS 3 2 1 2 Updatelocalcacheforpagestruct 1 UpdateEPTentry 23 Host hpa1page1 hpa2page2 Long livedPins KVM EPTidentitypagetable UnpindirectlyGuest gpa EPT 1 Guestrequiresapage 2 MMUsearchesEPTforahpa 3 MMUreturnsEPTviolationsincepagehasbeenmigrated 4 KVMhandlesEPTviolationandfindthenewpage 5 KVMupdatesEPT6 MMUreturnsnewhpa 2 1 6 3 4 5 Long livedPins KVMAPICaccesspage 24 hpa1 hpa2 page1page2L0 HostMode L2Guest GuestModeVMCS L1Guest VMCS TwokindsofVM Entry Exit VM Entry ExitinGuestmodeareemulatedbyKVM L1L2VM Entry Exit VM Entry ExitbetweenHostandGuestmodeareprovidedbyCPU L0L1VM Entry Exit L0L2VM Entry Exit Twocasestohandle CPUisrunningL1Guest CPUisrunningL2Guest 25 L0 HostMode hpa1page1 hpa2page2 L2Guest VMCS L1Guest VMCS Long livedPins KVMAPICaccesspageCPUisrunningL1Guest GuestMode PageMigration 1 Trytounmappage2 MMUnotifierworks 3 Unmapandmigratepage 4 Pagemigrated KVM 1 HandlerenforcesaL1 L0VM Exit2 HandlermakesvcpurequesttoupdateL1VMCS 3 NextL0 L1VM Entry GUPwaitsforpagemigration 4 UpdateL1VMCS5 UpdateL2VMCSin nextL1 L2VM Entry L0 HostMode hpa1page1 hpa2page2 L2Guest VMCS L1Guest VMCS Long livedPins KVMAPICaccesspageCPUisrunningL2Guest GuestMode PageMigration 1 Trytounmappage2 MMUnotifierworks 3 Unmapandmigratepage 4 Pagemigrated KVM 1 HandlerenforcesaL2 L0VM Exit2 HandlermakesvcpurequesttoupdateL2VMCS 3 NextL0 L2VM Entry GUPwaitsforpagemigration 4 UpdateL2VMCS5 UpdateL1VMCSin nextL2 L1VM Exit26 Long livedPins AIO 27 AIOEventRingBuffer UsedbykerneltonotifyuserspacethatAIOhascompleted Long livedPins AIOWhypinned UnabletoknowwhenAIOcompletes Userprocess AIOeventhandledAIORingBufferKernel AIOcompleted AIO 1 AIOpinsringpages2 AIOinprogress 3 AIOcompletes4 AIOunpinringpages28 PageMigration 1 Offlinememory2 Pagemigration migratepages 3 Pagemigrationfails 29 Long livedPins AIONeednewsplution WhycannotuseMMUnotifier 1 Nowaytogetthepage havetorepin GUPmaysleepiniointerruptcontext 2 NowaytonotifyAIOtorepinthepage NosuchMMUnotifierafterpagemigrationcompletes new page oldpage 3 Migratepage AIO 1 AIOpinsringpages2 AIOinprogress 3 aio migrate 1 lock2 unpingringpages3 migrateringpages 4 repinringpages 5 unlock 4 AIOcompletes 5 AIOunpinringpages30 PageMigration 1 Offlinememory2 Pagemigrationstarts 3 Pagemigrationinprogress 4 Pagemigrationends AIOcomplete1 BlockedbylockAIORingBuffer 5 Unlockedwhenmigrationends 2 Unpin 4 Repin Long livedPins AIOSolution I