外文资料--Performance comparison of dynamic web platforms.pdf

PerformancecomparisonofdynamicwebplatformsVarshaAptea,1,TonyHansenb,,PaulReeserbaDepartmentofComputerScienceandEngineering,IndianInstituteofTechnologyBombay,Mumbai400076,IndiabATTLabs,200LaurelAve,Middletown,NJ07748,USAReceived7August2002accepted7August2002AbstractOverthelastfewyears,theWorldWideWebhastransformeditselffromastaticcontentdistributionmediumtoaninteractive,dynamicmedium.TheWebisnowwidelyusedasthepresentationlayerforahostofonlineservicessuchasemailandaddressbooks,ecards,ecalendar,shopping,banking,andstocktrading.AsaconsequenceHyperTextMarkupLanguageHTMLfilesarenowtypicallygenerateddynamicallyaftertheserverreceivestherequest.FromtheWebsiteproviderspointofview,dynamicgenerationofHTMLpagesimpliesalesserunderstandingoftherealcapacityandperformanceoftheirWebservers.FromtheWebdeveloperspointofview,dynamiccontentimpliesanadditionaltechnologydecisiontheWebprogrammingtechnologytobeemployedincreatingaWebbasedservice.SincetheWebisinherentlyinteractive,performanceisakeyrequirement,andoftendemandscarefulanalysisofthesystems.Inthispaper,wecomparefourdynamicWebprogrammingtechnologiesfromthepointofviewofperformance.ThecomparisonisbasedontestingandmeasurementoftwocasesoneisacasestudyofarealapplicationthatwasdeployedinanactualWebbasedservicetheotherisatrivialapplication.Thetwocasesprovideuswithanopportunitytocomparetheperformanceofthesetechnologiesattwoendsofthespectrumintermsofcomplexity.Ourfocusinthispaperisonhowcomplexvs.simpleapplicationsperformwhenimplementedusingdifferentWebprogrammingtechnologies.Thepaperdrawscomparisonsandinsightsbasedonthisdevelopmentandperformancemeasurementeffort.q2002PublishedbyElsevierScienceB.V.KeywordsDynamicWebCommonGatewayInterfaceFastCGICþþJavaServletsJavaServerPagesPerformanceComparison1.IntroductionTheWorldWideWebWWWfirstemergedadecadeagoasamediumtorenderhypertextdocumentsthatwerestoredontheInternet,onauserscomputer,usingspecialsoftwarethebrowserandanewprotocolHTTPHyperTextTransferProtocol.Forthefirstfewyears,theWWWgrewprimarilyasanewmediuminwhichstaticcontentcouldbepublished,andinformationshared.ThecontentwaspublishedintheformofHTMLHyperTextMarkupLanguagefiles,whichwereservedbyWebservers,onrequestsfrombrowsers.However,overthelastfewyearstheWWWhastransformeditselffromastaticcontentdistributionmediumtoaninteractive,dynamicmedium.ContentontheWebisnowoftenpersonalized,andthereforedynamicallygenerated.TheWebisnowwidelyusedasthepresentationlayerforahostofonlineservicessuchasemail,ecards,ecalendar,andaddressbooks,shopping,banking,andstocktrading.Asaconsequence,theHTMLfilesthatarerenderedbytheclientsbrowserarenowtypicallygenerateddynamicallyaftertheWebserverhasprocessedtheusersrequest.ThisdynamicgenerationofHTMLfileshasnothappenedwithoutanassociatedperformancecost.JustwhenInternetusersweregettingaccustomedtoclickandwaitondialuplinesduetographicsrichWebsites,dynamicallygeneratedcontentstartedproliferatingontheWeb.Nowusersmustwaitnotonlyforthenetworkdelaybutalsofortheserversideprocessingdelayassociatedwithservingarequestdynamically.Inmanycases,thisisturningouttobethelargestcomponentofthedelay.FromtheWebsiteproviderspointofview,dynamicgenerationofHTMLpagesimpliesalesserunderstandingoftherealcapacityoftheirWebservers.ThevendorprovidedhitspersecondcapacityoftheWebserverisnolongerenough,asthisonlypertainstostaticHTMLfiles.ComputerCommunications262003888–898www.elsevier.com/locate/comcom01403664/03/seefrontmatterq2002PublishedbyElsevierScienceB.V.PIIS01403664020022191ThisworkwasdonewhilethisauthorwaswiththeNetworkDesignandPerformanceAnalysisDepartment,ATTLabs,Middletown,NJ07748,USA.ThisauthorhaspreviouslypublishedunderhermaidennameofVarshaMainkar.Correspondingauthor.Emailaddressmainkaratt.comT.Hansen.FromtheWebdeveloperspointofview,dynamicWebcontentimpliesanadditionaltechnologydecisiontheWebprogrammingtechnologytobeemployedincreatingaWebbasedserviceorproduct.Thisdecisionisbasedonseveralfactors.Amongthefactorsconsideredareeaseofprogramming,richnessoffeatures,maintainability,reliability,andperformance.SincetheWebisinherentlyinteractive,performanceisakeyrequirement,andoftendemandscarefulanalysisofthesystems.Inthispaper,wecomparetheperformanceoffourWebprogrammingtechnologies,namelyJavaServlets,JavaServerPages,CGI/CþþandFastCGI/CþþasmallerversionofthisstudywaspresentedinRef.1.ThestudywasmotivatedbyarealneedtomakeatechnologychoicefordevelopingsoftwarethatwouldsupportaWebbasedservice.Astudyofexistingliteratureshowedvaryingconclusionsabouttheperformancesuperiorityofonetechnologyovertheother.ProponentsoftheJavaplatformconsistentlyclaimsuperiorperformanceofservlettechnologymainlyduetotheeliminationoftheoverheadofprocesscreation2,3.Ontheotherhand,astudydoneinRef.4thatcomparesCGI,FastCGIandservletsshowedservletstobetheslowestofthethree.AsimilarcomparativestudyhasbeendoneinRef.5,focusingonWebtodatabaseapplications,whichshowedJavaservletstobebetterthanCGIprograms.Furthermore,although4brieflyprovidessomeexplanationabouttheresults,anindepthanalysisandinsightabouttheresultswasnotavailableintheliteraturethatwesurveyed.Oneconclusionfromthevariabilityoftheresultswasalsothattheperformanceofthetechnologydependsontheapplication.Inthispaper,thecomparisonisbasedontwocasesoneisacasestudyofacomplexapplicationthatwasdeployedinanactualWebbasedservicetheotherisatrivialapplication.Themethodologyofperformanceanalysiswasstresstestingandmeasurement.Performancemeasurementratherthanmodelingmademostsenseinthiseffort,sinceaquickturnaroundofresultswasnecessaryandtheaccuracyofresultswasrequiredtobehigh.Thetwocasesi.e.thecomplexandthetrivialprovideduswithanopportunitytocompareperformanceofthesetechnologiesattwoendsofthespectrumofapplications,intermsofcomplexity.Theperformanceorderofdifferenttechnologiesisseldomabsoluteitdependsgreatlyonthenatureoftheapplication.Ourfocusinthispaperisonhowcomplexvs.simpleapplicationsperformwhenimplementedusingdifferentWebprogrammingtechnologies.Amongthepaperssurveyed,webelievethisistheonlypaperpresentingasystematicbottleneckanalysisofeachofthetechnologies,andtheonlyonethatdemonstratesthattheperformancerankingcouldbereversed,basedontheapplication,whileprovidinginsightonwhythishappens.ThemainobservationsfromthisworkwereasfollowsIngeneral,FastCGIoutperformedCGI,whileJSPoutperformedJavaservlets.Inthecaseofacomplexapplication,theCGIbasedtechnologiesoutperformedtheJavabasedtechnologiesbyafactorof3–4£,withJavaperformancelimitedbysoftwarebottlenecksintheJVM.Inthecaseofatrivialapplication,therelativeperformancewasreversed,asJavaoutperformedCGIbyafactorof2–3£.TherestofthepaperisorganizedasfollowsinSection2weprovidethemotivationforconductingsuchacomparativestudy,andinSection3wedescribebrieflythetechnologiesthatwecompared.Section4describestheperformancemeasurementandanalysismethodology,Section5describesthecasestudytestingresultsindetail,andSection6describestheresultsoftestingatrivialapplication.Finally,Section7summarizesourresults,andSection8providessomeconcludingremarks.2.MotivationTheapplicationcontextforthiscasestudywasanewWebbasedmessagingservice.SuchaservicewouldinvolveabackendthatwouldincludethecoremessagingrelatedserversIMAP,POP,LDAP,SMTP,andafrontend,orapresentationlayerthatwouldserveasmiddlewarebetweentheWebbrowserandthemessagingserver.Thecoreserverswerechosentobeestablishedmessagingproducts.ThepresentationlayerwouldconsistofaWebserverofftheshelfandanewdynamicpagegenerationenginetobedeveloped,whichwould†Readdatasentbytheuser,throughtheHTTPprotocol†DothenecessaryprotocolconversionstoIMAP,LDAP,†CarryouttheactionencodedintheHTTPrequest,usingtheappropriateprotocolwiththebackendservers†FormattheresultasaWebpage,and†ReturntheresulttotheusersWebbrowserthroughtheWebserverOnewaytowritesuchaprogramthatgeneratesWebpagesdynamicallyistocodethelogic,andthenembedprintstatementsintheprogramthatwriteoutstaticHTMLcode,alongwithprintingotherstringvariablesthathavebeenpopulateddynamically.Abetterandmorepopularwayisusingtemplates.ThemiddlewarewestudiedusedatemplatebaseddesignforproducingdynamicWebpages.ThismethodinvolveswritingWebpagetemplateswhichhavestaticHTMLcodethatspecifythedesignoftheWebpage,interspersedwithscriptingtagsthatareinterpretedbyatemplateserverprogram.Thesetagsinstructthetemplateserverprogramontheactionthatistobetakene.g.populatingafieldwiththesubjectlineofamessageonreadingthetag.Thetemplateserverprogram,therefore,V.Apteetal./ComputerCommunications262003888–898889†Parsesthespecifictemplateaccessedbyauserrequest†Interpretsthetagsandcommunicateswiththebackendservers†PopulatesthefieldsthataretobydynamicallypopulatedTheresultisadynamicallygeneratedWebpage.ThearchitectureofsuchasystemisshowninFig.1.Giventheacceleratedtimetomarketgoalsandlimiteddevelopmenttime,thenaturalchoiceoftechnologyforthetemplateserverwastheoneperceivedtobepowerful,featurerichandyeteasytouseanddeploynamely,Javaservlets.Aninitialeffortwasdonetoquantifytheperformanceofthispagegenerationengine,toseewhetheritcouldmeettheexpectedperformancerequirements.Weconductedaseriesofstresstestsusingacommercialloaddrivertogeneraterepeatedrequeststotheservletengineatvariouslevelsofconcurrencysimulatedusers.ThetestconfigurationconsistedofaWindowsNTserverrunningtheloadgenerationscriptsdriver,aSolarisserverrunningthefrontendsoftware,andaSolarisserverrunningthebackendapplicationforthistest,aPOP3/IMAP4mailserverandanLDAPdirectoryserver.HardwareCPU,memory,disk,I/Oandsoftwareresourceconsumptionsweremeasuredonallmachines.Inaddition,endtoenduserperceivedresponsetimesweremeasured.Thedriverscriptsemulatedaprescribednumberofconcurrentusersrepeatedlygeneratingthesamerequeste.g.readmessage,sendmessage,etc..Thenumberofconcurrentsimulateduserswasvariedfrom1to20.Thenumberofrepeatedrequestsperuserateachconcurrencyleveltypically2000wassufficienttoachievestatisticalstability.Thetestswereruninstressmodethatis,assoonasauserreceivesaresponse,itimmediatelysubmitsthenextrequesti.e.withnegligibleclientdelay.EachoftheNsimulatedusersdoessoindependentlyandinparallel.Asaresult,theconcurrencyleveli.e.thenumberofrequestsinthesystemequalsNatalltimes.Theresultsofthestresstestsforaparticularrequesttypereada20KBmessageareshowninFigs.2and3.Inparticular,Fig.2plotstheendtoendresponsetimenormalizedonthelefthandaxis,andthefrontendCPUutilizationontherighthandaxis,asafunctionoftheconcurrencylevel.Ascanbeseen,theresponsetimecurvebeginstoridealongalinearasymptoteshownbythedottedlineafteronlysevenconcurrentusers.Thatis,responseFig.1.Templateserverarchitecture.Fig.2.Resultsofinitialstresstestsoftemplateserver,plottedvs.concurrentusers.V.Apteetal./ComputerCommunications262003888–898890timeincreasesproportionallywiththenumberofusers,indicatingsaturationinaclosedsystem6.Additionally,CPUutilizationlevelsoffafter11usersat65–70indicatinganonCPUsystembottleneck.Equivalently,Fig.3plotsendtoendresponsetimeasafunctionofthroughputrequests/s.Ascanbeseen,themaximumsystemthroughputpeaksatabout2requests/sec,andthendegradesunderoverloadtoabout11/2requests/s.Inotherwords,therewasactuallyadropincapacityof25aconcurrencypenalty,likelyduetocontextswitching,object/threadmanagement,garbagecollection,etc.Asizinganalysisbasedontheexpectedcustomergrowthandusagebehavior,togetherwiththeseinitialcapacityresults,suggestedthattheresultinghardwarecostswouldbeprohibitivelylarge.Itwasalsoclearthatthescalabilityofthisapplicationwaspoor.TheresourceconsumptionresultsdemonstratedthattheapplicationcouldnotmakefulluseoftheresourcesavailabletoitespeciallyCPU.Thefirstquestiontobeansweredwaswhetherornottheapplicationwasimplementedinthemostoptimalmanner.Theinitialphaseoftheperformanceenhancementeffortwasinthisdirection.SeveralkeyoptimizationswereperformedontheJavaservletcode,andseveralcriticalbottleneckswerediscoveredandresolvedintheendtoendarchitecturedescribedinaseparatepaper6.TheresultingimprovementinthereadrequestthroughputisshowninFig.4.Sizinganalysisontheimprovedandoptimizedcodeshowedthattheresourcerequirementswerestillquitesubstantial,andtheapplicationstillhitaprematurewallintermsofscalability.TheCPUcontinuedtoleveloff,nowatabout90.Asaconsequencetothisanalysis,anadditionaleffortwaslaunchedtoreassessthechoiceofthedynamictechnologyitself.Theplanwastoanalyzeapplicationsthathadthesamefunctionality,butwereimplementedindifferentlanguagesortechnologies,withinidenticalenvironments.Thetechnologieschosen,includingtheinitialversion,wereJavaservlets,JavaServerPagesJSP,CommonGatewayInterfaceCGIwithCþþprograms,andFastCGIwithCþþprograms.ThefollowingsectiongivesabriefintroductiontotheseWebprogrammingtechnologies.Fig.3.Equivalentloadtestresultsplottedvs.requests/s,fromstresstests.Fig.4.Resultsafteroptimization.V.Apteetal./ComputerCommunications262003888–898891