性能和功耗基准测试

10/24/05ELEC65001PerformanceAndPowerBenchmarkingKhushbooShethDepartmentofElectricalandComputerEngineering

210/24/05ELEC6500PerformanceReducingResponseTime(ExecutionTime)-thetimebetweenthestartandthecompletionofatask.Totaltimerequiredforthecomputertocompleteatask,includingdiskaccesses,memoryaccesses,I/Oactivities,operatingsystemsoverhead,CPUexecutiontime,etc.IncreasingThroughput-thetotalamountofworkdoneinagiventime.

310/24/05ELEC6500PerformancePerformanceandExecutiontimerelationforacomputerXcanbegivenasPerformanceX=1--------------ExecutiontimeXIfcomuterXisntimesfasterthancomputerYthentheexecutiontimeonYisntimeslongerthanitisonX:PerformanceX=ExecutionTimeY=n------------------------------------PerformanceYExecutionTimeX410/24/05ELEC6500ExecutionTimeElapsedTime-totaltimetocompleteatask,includingdiskaccesses,memoryaccesses,I/Oactivities,operatingsystemsoverhead,etc.CPUTime–thetimetheCPUspendscomputingforthetaskanddoesnotincludetimespentwaitingforI/Oorrunningotherprograms(responsetime=elapsedtimenottheCPUtime)UserCPUTime–theCPUtimespentintheprogramSystemCPUTime–theCPUtimespentintheoperatingsystemperformingtasksonbehalfoftheprogram

510/24/05ELEC6500ComputingCPUExecutionTimeComputersareconstructedusingaclockthatrunsataconstantrateanddetermineswheneventtakeplaceinthehardware.Thesediscretetimeintervalsarecalledclockcycles.Clockrateistheinverseofclockperiod.CPUExecutiontime=CPUclockcycles*ClockcycleforaprogramforaprogramtimeCPUclockcycles=Instructions*AverageclockcyclesforaprogramperinstructionCPUTime=Instructioncount*CPI*ClockcycletimeSeconds=Instruction*Clockcycles*Seconds--------------------------------------------ProgramProgramInstructionClockcycles610/24/05ELEC6500EvaluatingPerformanceThe

computermaybeevaluatedusingasetofBENCHMARKS–programsspecificallychosentomeasuretheperformance.Thebenchmarksformaworkloadthattheuserhopeswillpredicttheperformanceoftheactualworkload.“Synthetic”benchmarks–speciallycreatedprogramsthatimposetheworkloadonthecomponent“Application”benchmarks–runactualreal-worldprogramsonthesystem.ApplicationBenchmarksusuallygiveamuchbettermeasureofrealworldperformanceonagivensystem,syntheticbenchmarksstillhavetheirusefortestingoutindividualcomponents

likeaharddiskornetworkingdevice.

710/24/05ELEC6500TypesOfBenchmarksRealProgramWordprocessingsoftwareToolsoftwareofCDAUser`sapplicationsoftware(MIS)KernelContainskeycodesNormallyabstractedfromactualprogramPopularkernel-LivermoreloopLinpackbenchmark(containsbasiclinearalgebrasubroutinewritteninFORTRANlanguage)ResultsarerepresentedinMFLOPSToyBenchmarkUsercanprogramitanduseittotestcomputer`sbasiccomponents.810/24/05ELEC6500TypesofBenchmarksSyntheticBenchmarkProcedureforprogrammingsyntheticbenchmarkTakestatisticsofalltypeofoperationsfromplentyofapplicationprogramsGetproportionofeachoperationWriteaprogrambasedontheproportionaboveItsresultsarerepresentedinKWIPS(KiloWhetstoneInstructionsPerSecond).NotsuitableformeasuringpipelinecomputersTypesofSyntheticBenchmarksWhetstone–isabenchmarkforevaluatingthepowerofcomputers.ItwasfirstwritteninAlgol60attheNationalPhysicalLaboratoryintheUnitedKingdom.Itoriginallymeasuredcomputingpowerinunitsofkilo-WIPS.Resultsforavarietyoflanguages,compilersandsystemarchitectureshavebeenobtainedandmodernworkstationstypicallyachievemorethan1,000,000kWIPS.Itprimarilymeasuresthefloatingpointarithmeticperformance.910/24/05ELEC6500TypesofBenchmarksTypesofSyntheticBenchmarksDhrystone–isabenchmarkinventedin1984byReinholdP.Weicker.Itcontainsnofloatingpointoperations,thusthenameisapunonthethenpopularWhetstonebenchmarkforfloatingpointoperations.Theo/pfromthebenchmarkisthenumberofDhrystonespersecond(thenumberofiterationsofthemaincodelooppersecond).OnecommonrepresentationoftheDhrystonebenchmarkistheDMIP-DhrystoneMIPS-obtainedwhentheDhrystonescoreisdividedby1,757(thenumberofDhrystonespersecondobtainedontheVAX11/780,a1MIPSmachine).TheDhrystonebenchmarkcontainsmainlyintegerandstringoperations.Butlikemostsyntheticbenchmarks,theDhrystonebenchmarkisnotparticularlyusefulinmeasuringtheperformanceofreal-worldcomputersystemsandhasfallenintodisusereplacedbybenchmarksthatmorecloselyresembletypicalactualusage.1010/24/05ELEC6500SPECTheStandardPerformanceEvaluationCorporation(SPEC)isanon-profitorganizationthataimstoproducefair,impartialandmeaningfulbenchmarksforcomputers.SPECwasfoundedin1988andisfinancedbyitsmemberorganizationswhichincludeallleadingcomputer&softwaremanufacturers.SPECbenchmarksarewidelyusedtodayinevaluatingtheperformanceofcomputersystems.Thebenchmarksaimstotestreal-lifesituations.SPEC_WEB,forexample,testswebserversperformancebyperformingvarioustypesofparallelHTTPrequests,andSPEC_CPUtestsCPUperformancebymeasuringtheruntimeofseveralprogramssuchasthecompilergccandthechessprogramcrafty.Thevarioustasksareassignedweightsbasedontheirperceivedimportance;theseweightsareusedtocomputeasinglebenchmarkresultintheend.SPECbenchmarksarewritteninaplatformneutralprogramminglanguage(usuallyCorFORTRAN)andtheinterestedpartiesmaycompilethecodeusingwhatevercompilertheypreferfortheirplatform,butmaynotchangethecode.ManufacturershavebeenknowntooptimizetheircompilerstoimproveperformanceofthevariousSPECbenchmarks.

1110/24/05ELEC6500VariousCurrentSPECBenchmarksSPECCPU2000,combinedperformanceofCPU,memoryandcompilerCIN2000(“SPECint”),testingintegerarithmetic,withprogramssuchascompilers,interpreters,wordprocessors,chessprograms,etc.CFP2000(“SPECfp”),testingfloatingpointperformance,withphysicalsimulations,3Dgraphics,imageprocessing,computationalchemistry,etc.SPECWEB99,webserverperformance,measuredbysettingupanetworkofclientmachinesthatstresstheserverwithparallelrequests.SPECHPC2002,testinghighendparallelcomputingsystemswithapplicationssuchasweatherpredictionandcomputationalchemistry.SPECJVM98,performanceofajavaclientserverrunningajavavirtualmachine.SPECMAIL2001,performanceofamailserver,testingSMTPandPOPprotocolSPECSFS97_R1,NFSfileserverthroughputandresponsetime.1210/24/05ELEC6500PowerBenchmarkingThepowerbenchmarkingofacomputerisfundamentallythenotionofdetermininghowmuchenergythecomputerisconsuminginordertoaccomplishsomemeasureofwork.TheBDTI(BerkeleyDesignTechnologyInc.),EEMBC(EDNEmbeddedMicroprocessorBenchmarkConsortium),andSPEC(StandardPerformanceEvaluationCorp)benchmarkorganizationssupportbenchmarksuitesthathighlightaprocessor'sperformancewhenperformingapplicationspecifictasks.ResearchersatBDTIandEEMBCarebothworkingonhowtoextendtheirbenchmarksuitestomeasureandcompareaprocessor’senergyefficiencyasopposedtopowerconsumptionwhenperformingapplicationspecifictasks.1310/24/05ELEC6500PowerBenchmarkStrategyThereare3primaryareasofinterestwhenbenchmarkingthecharacteristicsof“lowpower”systemsemployingpowermanagementtechniquestoachievelowpowergoals.First–actualpowerconsumptionofthesystemundertypicaluserconditions,presumablyunderpowermanagementspectrum.Second–systemoperabilityorusabilityunderpowermanagementconditions.Itsclearthatonecouldachieveremarkablepowercharacteristicsatthecostofsystemperformanceandtheresponsetime.Third–impactofpowermanagementtechniquesonsystemreliability.Anappropriatebenchmarkingstrategyforpowermanagedsystemsmustaddressthesethreeareasinordertopostulateanoverallsystemfiguremerit,lowpowerwithoutsacrificingsystemoperabilityorreliability.Itwouldbeonethatcharacterizesthesystempowerconsumptionwhilethesystemwascarryingoutsomeusefultask.1410/24/05ELEC6500PowerBenchmarkingTheprimaryinterestinpowerbenchmarkingispowerconsumedoverthecourseofexercisingagivenapplicationorinthecaseofmulti-taskingenvironments,multipleapplicationsrunningsimultaneously.Specifically,whatisthesystempowerconsumptionasanapplicationisexercisedandthesystemtransitionsthroughvariouspowermanagedpowerstates.Thisisfundamentallyaquestionofsystemexpectationsfromboththeapplicationandenduser’sperspectiveandhowapowermanagementfacilitymightbeabletoexploittheseexpectationstoreducethesystempowerconsumption.Ifaspecificsystemcomponentisn’tbeingusedandisunlikelytobeusedintheimmediatetimeframe,itslevelofreadinessmightbecompromisedinordertoreduceitsandultimatelythesystempowerconsumption.AWordProcessingapplicationbeingusedinEDITmodemightnotaccessthesystemFixedDiskforanextendedperiodoftime.ThepowermanagementfacilitymightrecognizethisasaflagthatsuggeststhattheFixedDiskisunlikelytobecalleduponintheneartimeframe.Basedonthisdeterminationthepowermanagementfacilitymightexploitthissystemexpectationasanopportunitytotransitionthefixeddisktoalowerpowerstate.Thisscenariocouldprogresstoapointwhenthefixeddiskisactuallycompletelypoweredoff,itslowestpowerstateandloweststateofreadiness.1510/24/05ELEC6500PowerBenchmarkingTheenergyconsumptionofasystemistherefore,theaggregatepowerdissipatedbyitscomponentsovertime,atvaryingpowerstates.Intermsoftime,itistheenergyrequiredtoexecuteagiventasktocompletion.Thiscanbereflectedatthesystemlevelasthesummationoftheenergyrequirementsofeachsubtaskandcanbecomputedbythefollowingexpression:

m

Pt=II(Pn)*Tc

1------3600where,Pt=“TaskEnergy”inwatthours,WHrs.m=no.ofpowertransitionsoccurringduringthetaskPn=SegmentedpowerdissipationduringagivenpowerstateTc=“TaskCycle”time,timerequiredtocompletethetaskPn=Tsn*Ps