Loadrunner_结果分析向导.doc

1 LoadrunnerLoadrunnerLoadrunner ResultsResultsResults AnalysisAnalysisAnalysis ContentsContentsContents Analysis Summary Page 3 Running Vusers 5 Running Vusers 5 Hits per Second 6 Throughput 7 Hits per Second Throughput 8 Average Transaction Response Time 9 Transactions per Second Passed Failed 10 Total Transactions per Second 11 Transaction Summary Graph 12 Transaction Performance Summary 13 Transaction Response Time Under Load 14 Transaction Response Time Under Load 14 Transaction Response Time Percentile 15 HTTP Status Code Summary 16 HTTP Responses per Second 17 Retries per Second 18 Connections 19 Connections per Second 20 Error Statistics 21 Error Statistics by Description 22 2 Errors per Second 23 Errors per Second by Description 24 Errors per Second by Description 24 Errors per Second Running Vusers 25 Glossary 26 3 AnalysisAnalysisAnalysis SummarySummarySummary PagePagePage The Analysis Summary page is divided into three main sections the statistics summary transaction summary and HTTP responses summary The statistics summary lists overall statistics of interest in the scenario You can compare maximum running Vusers and the total duration of the scenario here the duration is 17 08 38 17 54 35 or 46 minutes against throughput and hits statistics Based on Vusers and duration was the performance of the server application acceptable given the throughput and hits second statistics You will have to have some knowledge of your server database capabilities and configuration to make this decision The next section of the Analysis Summary is the Transaction Summary containing statistics on individual transactions their response times minimum average maximum and standard deviation The 90 Percent statistic indicates the maximum response time of ninety percent for the transactions in that row Based on the statistics above you could say that 90 of the A MainPage AFI transactions had a maximum response time of 94 845 seconds Passed and failed transactions are also listed in this section Transactions with the Z prefix contain the sum of all statistics for transactions used in that script The total passed transactions in the Z transaction will be at most equal to the single transaction with the fewest passes excluding vuser init end transactions This is because the Z transaction 4 records statistics for the entire script flow not just a single transaction If a Vuser successfully executes all the steps in a script from start to finish only then will the number of passes for the Z transaction increase However if failures occur at any point in the script the number of failures for the transaction in which the failure occurred will increase and the number of failures for the Z transaction will increase as well In the statistics above the total number of passes in the Z transaction was 776 corresponding to the single transaction with the fewest passes E ResultsDisplay whereas the total number of failures for Z is the sum of failures for the entire group The final section in the Analysis Summary is the HTTP Responses Summary which contains the total and per second statistics for all HTTP responses recorded during the test The most common response codes will be HTTP 200 indicating success For a list of all HTTP response codes and their meanings see the attached document in the glossary Back to Contents 5 RunningRunningRunning VusersVusersVusers The Running Vusers graph displays the number of virtual users currently executing scripts in the test scenario A Vuser performs a sequence of tasks defined in a test script until the test has completed its duration the Vuser has executed the script for a required number of iterations or the test controller manually stops the Vuser Vusers will usually be added to the scenario in what is called a gradual ramp up unlike the graph above where all Vusers are added to the scenario simultaneously After the number of Vusers reaches a level set in the stress test plan that load is maintained for a set duration before a gradual ramp down is initiated The ramp down is usually faster than the ramp up because removing users from the scenario will almost never adversely affect server performance Whenever a problem occurs during a test you should always be aware of the number of Vusers currently running in the scenario The purpose of a stress test is to diagnose problems in an application under a simulated load so in an unstable application it is typical to see a decrease in some performance measure when the number of Vusers passes a certain level A sudden negative shift in performance usually means the number of running Vusers has increased The graph above follows a typical test schedule with a ramp up of 10 users per minute to a peak load of 250 users then the test was run at peak load for 10 minutes and a ramp down of approximately 25 users per minute Back to Contents 6 HitsHitsHits perperper SecondSecondSecond The Hits per Second graph shows the number of hits on the Web server over time This graph can display the entire scenario or the last 60 180 600 or 3600 seconds You can compare this graph to the Transaction Response Time graph to see how the number of hits affects transaction performance An increase in hits per second indicates an increased load on the server so comparing hits per second with transaction response time and throughput can give you a good idea of how the server is performing under increased stress When a web server is functioning correctly hits per second will often mirror the number of successful HTTP response codes In the graph above there is a spike in hits per second around 25 minutes into the test This can be explained by comparing hits per second with the results of other graphs like running Vusers throughput and transaction response time Back to Contents 7 ThroughputThroughputThroughput The Throughput graph shows the amount of data measured in bytes transmitted by the Web server during each second of the scenario run Throughput is measured in bytes second and represents the amount of data transmitted by the web server over the network at any given time Ideally throughput should increase as hits per second increases which indicates that there is sufficient bandwidth to process user requests You can gain information on server performance by comparing Throughput and Hits per Second If the server is performing as expected as hits per second increase then throughput should also increase This is expected behavior because an increase in hits will require the server to transmit more data back to the users However if the number of hits per second increases and throughput stays constant or decreases this signifies that the server cannot process the increase in requests hits per second resulting in an increase in transaction response time The graph above exhibits a general decrease in throughput during the entire test with a slight but noticeable decrease just before 25 minutes corresponding to the spike in hits per second in the previous graph To compare these statistics it is best to overlay the graphs Back to Contents 8 HitsHitsHits perperper SecondSecondSecond ThroughputThroughputThroughput This graph contains an overlay of hits per second and throughput We can see here that the spike in hits per second coincides with a decrease in throughput at around 25 minutes If a server is operating below peak load then hits per second usually mirrors throughput The result in the graph above is most likely due to an increase in user load because more users will cause an increase in hits per second and when the server can no longer process the number of requests coming in there will be a decrease in throughput Referring back the Running Vusers graph we know that the peak load of 250 users was reached just before 25 minutes resulting in an expected increase in hits per second and a decrease in throughput indicating that the load on the server was approaching its maximum We can guess from this graph that transaction response time and possibly errors per second will spike around 25 minutes as well Back to Contents 9 AverageAverageAverage TransactionTransactionTransaction ResponseResponseResponse TimeTimeTime The Average Transaction Response time graph shows the average response time of individual transactions over time Response time is the amount of time between a client request and server response Often response time will correspond to the number of Vusers currently running in the test scenario The most common cause for an increase in response time is an increase in running Vusers This occurs because as more Vusers enter the test scenario the amount of work the server must perform increases With an increased load it takes more time for the server to respond to an individual request so average response time increases as well In the graph above we should expect an increase in response time corresponding to the change in behavior in other graphs at around 25 minutes Looking at the response time for the Z transaction is the easiest way to analyze general behavior and here total response time increased to an average of around 110 seconds by 10 minutes into the scenario At 10 minutes the number of running Vusers was 120 so it is likely that the server was operating at high capacity from this time onwards Back to Contents 10 TransactionsTransactionsTransactions perperper SecondSecondSecond Passed Failed Passed Failed Passed Failed The Transactions per Second graph shows the number of successful unsuccessful transactions over time A transaction represents a single step or group of steps executed by a Vuser For example the process of entering and submitting login information on a web page could be grouped as a single transaction The number of transactions per second should correspond to the number of Vusers running in the test scenario at a given time and typically as more Vusers are added to the scenario the number of transactions per second will increase If the number of transactions per second decreases with an increase in Vusers then there is probably some problem occurring in the scenario You should first compare the number of running Vusers with transactions per second to verify that an increase in Vusers is the likely cause of the increase as well as hits per second transaction response time and throughput Taking all these statistics into account will help you pin point the cause of the problem Back to Contents 11 TotalTotalTotal TransactionsTransactionsTransactions perperper SecondSecondSecond Total transaction per second displays the total number of passed and failed transaction during the test scenario Here we have a clearer picture of the general status of all transaction and like the previous more specific transactions per second graph above the total failed transactions in red in this graph indicate the server was operating with a user load exceeding its maximum capacity The red line in the graph above indicates the total number of failed transactions The spike occurs as expected at around 25 minutes indicating that the server was exceeding its maximum capacity for user load Note the sharp decrease in errors just before 35 minutes which directly corresponds to the beginning of the ramp down decrease in Vusers in the test Back to Contents 12 TransactionTransactionTransaction SummarySummarySummary GraphGraphGraph The Transaction Summary graph displays a record of all passed failed and stopped transactions from the test scenario Passed transactions are those that were completed successfully failed transactions are those that were not completed because of an unexpected occurrence a page failed to load login failed etc and stopped transactions were those that did not reach a point at which a pass fail condition could be determined By analyzing this graph you can see the ratio of passed failed transactions and by determining which transactions need to receive further attention A high number of failures for a single transaction indicate that there is some type of problem occurring at that point in the test scenario which could be caused by the page related to that transaction or a server issue Again analyzing other results will help determine the specific cause of transaction failures For example the right most transaction above failed 91 47 of the time which is obviously an unacceptable ratio By looking at other graphs above you can see which statistic most accurately explains the failures for this transaction Single transactions that had both passes and failures suggest that a problem occurred at some point during the scenario so a comparison with the number of running Vusers would be useful as well as response time and throughput Note that this is the Z transaction and contains the sum of all failed transactions which will always be larger than any individual number of failed transactions Back to Contents 13 TransactionTransactionTransaction PerformancePerformancePerformance SummarySummarySummary The transaction performance summary graph is similar to the transaction summary graph except it gives the minimum average and maximum response times for all transactions in the test scenario The right most transaction in the graph above is the Z transaction which should have the highest values in the graph because it is a sum of all transactions Here no single transaction exhibits an unusually high maximum response time when compared to the other transactions but the transaction 2nd from the left has an average response time of 30 2 seconds If more information is available about the resources required by this transaction which database operations are performed which servers is it hitting then you may find an explanation for the higher average response time In this case the transactions with response times of zero are the initialization and termination transactions recorded when a Vuser enters or exits the test scenario and can be ignored The initialization and termination actions can contain multiple actions if required by the test plan For example a scenario that would require transactions in all three sections would be logging in to the application once initialization executing the action section for a set number of iterations action or Z and logging out of the application termination Back to Contents 14 TransactionTransactionTransaction ResponseResponseResponse TimeTimeTime Under Under Under Load Load Load The Transaction Response Time Under Load graph is a combination of the Running Vusers and Average Transaction Response Time graphs It indicates transaction times relative to the number of Vusers running at any point during the test scenario This graph helps you view the general impact of Vuser load on response time and is most useful when analyzing a scenario with a gradual load With a gradual load you can detect at which quantity of Vusers an adverse change in transaction response time occurred By comparing the test schedule with the statistics from this graph you can determine whether a spike in response time coincided with an increase in Vusers or if you need to look for the cause of the problem elsewhere In the graph above the average response time from the load of 0 20 users can be disregarded since the test began with 20 users The unusually high response times are due to resource usage by the Loadrunner test software itself and from 0 20 users this has no impact on the results of the test After user initialization average response time decreases sharply at 40 Vusers This is most likely due to the load of 40 users being redistributed among different web application servers and occurs again at 60 users Back to Contents 15 TransactionTransactionTransaction ResponseResponseResponse TimeTimeTime Percentile Percentile Percentile Transaction response time by percentile displays the percentage of transactions with a given response time In the graph above the average response time of transactions in the 90th percentile of response time the green line had response times greater than 200 seconds It is difficult to make any conclusions based on this graph other than general observations For instance transactions in the 90th percentile had response times higher than transactions in the 10th percentile It is also impossible to directly overlay this graph with any other because of the units of measurement Percent of transactions on the x axis making it difficult to draw comparative conclusions on based on these results Back to Contents 16 HTTPHTTPHTTP StatusStatusStatus CodeCodeCode SummarySummarySummary The HTTP status code summary contains a pie chart displaying the number of HTTP responses grouped by status code The majority of response codes in most tests will be 200 indicating success In the graph above about 50 of the response codes indicated success 200 25 indicated failure 500 and 25 indicated a requested page was not found 404 For a detailed listing of HTTP response codes and their meanings view the attached document in the glossary Back to Contents 17 HTTPHTTPHTTP ResponsesResponsesResponses perperper SecondSecondSecond The HTTP Responses per Second graph shows the number of HTTP status codes which indicate the status of HTTP requests The most common codes include 200 success 302 redirect 404 not found and 500 internal server error A higher number of HTTP responses indicates that the server is processing more requests and successfully sending the user an HTTP response In the graph above successful response codes and page not found response codes remained relatively steady throughout the test There was a spike in response code 503 indicating that a requested service was unavailable starting about 18 minutes into the test At 25 minutes the responses per second for code 503 are quite high at 16 per second indicating that the server has exceeded its maximum capacity at this point in the test a conclusion supported by other results Back to Contents Back to Contents 18 RetriesRetriesRetries perperper SecondSecondSecond The retries per seco