Lect 09-Testing the system.ppt

1、Testing the system,To ensure that the system does what the customer wants it to do.,2020/8/2,兰州大学信息学院,2,Contents,Function testing Performance testing Acceptable testing Software reliability, availability , and maintainability Installation testing Test documentation Testing safety-critical systems,20

2、20/8/2,兰州大学信息学院,3,Testing the System,Unit testing testers have overall control. Integrate component collaborate with other testers. Testing the system work with the entire development team.,2020/8/2,兰州大学信息学院,4,1 Principles of system testing,To ensure that the system does what the customer wants it t

3、o do.,2020/8/2,兰州大学信息学院,5,Principles of system testing,Sources of software faults :,Software faults can be inserted in a requirement , design ,or code component , or in the documentation , at any point during development or maintenance process.,2020/8/2,兰州大学信息学院,6,Principles of system testing,Misund

4、erstanding Communication mishap Poorly description Misinterpretation Improperly using of syntax and semantics Correction of a fault These actions may introduce faults.,2020/8/2,兰州大学信息学院,7,Principles of system testing,System testing process : Function testing: does the integrated system perform as pr

5、omised by the requirements specification? Performance testing: are the non-functional requirements met? Acceptance testing: is the system what the customer expects? Installation testing: does the system run at the customer site properly?,2020/8/2,兰州大学信息学院,8,Principles of system testing,Steps in the

6、testing process.,Success of each Step depends on it goal or objective.,2020/8/2,兰州大学信息学院,9,Principles of system testing,Process objectives Function test : checks that the integrated system performs its functions as specified in the requirements. Performance test : compares the integrated components

7、with the nonfunctional system requirements. Such as security, accuracy, speed, reliability, ,2020/8/2,兰州大学信息学院,10,Principles of system testing,A verified system : after function test and performance test, the system operates the way the SRS intend. A validated system : compare the system with the cu

8、stomers expectations by reviewing the requirements definition document. If the system meets the customers definitions, we calls it a validated system.,2020/8/2,兰州大学信息学院,11,Principles of system testing,Acceptance test : to ensure the customers that the system they requested is the system that was bui

9、lt for them. actual data set in actual environment. Installation test : to allow the users to exercise system functions and document additional problems that results from being at the actual site.,2020/8/2,兰州大学信息学院,12,Principles of system testing,System testing strategies : We often test a system in

10、 stages or pieces, based on spins or subsystems, function, or other decompositions that make testing easier to handle.,2020/8/2,兰州大学信息学院,13,Principles of system testing,Build or spin plan for gradual testing (for large project) Divide the test system into a nested sequence of subsystems. Incremental

11、 testing example: Routing calls within an exchange, Routing calls within an area code, Routing calls within a state, province, or district, Routing calls within a country, Routing international calls.,2020/8/2,兰州大学信息学院,14,Principles of system testing,A build plan / integration plan defines the subsy

12、stems to be tested and to describe how, where, when, and by whom the tests will be conducted. Spin: a level or subsystem of a build plan is called a spin, and the lowest level is called spin zero.,2020/8/2,兰州大学信息学院,15,Principles of system testing,Build plan for Telecommunications system,If a test of

13、 spin n succeeds and a problem arises in spin n+1, then the most likely source of the problem is related to the difference between spins n and n+1.,2020/8/2,兰州大学信息学院,16,Principles of system testing,How to create a spin plan? Critical functions first.,0: test the central computers general function. 1

14、: test the central computers message translation function. 2: test the central computers message assimilation function. 3: test each outlying computer in the stand-alone mode. 4: test each outlying computers message sending function. 5: test the central computers message receiving function.,2020/8/2

15、,兰州大学信息学院,17,Principles of system testing,Configuration management : System testing includes : System (function , performance , ) Several system configurations. System configuration : is a collection of system components delivered to a particular customer. e.x : a mathematical computation package: O

16、ne configuration for UNIX, one for Windows, and still another for Linux.,2020/8/2,兰州大学信息学院,18,Principles of system testing,Configuration management: the control of system differences to minimize risk and error.,2020/8/2,兰州大学信息学院,19,Principles of system testing,Versions and releases : A configuration

17、 for a particular system is sometimes called a version. The initial delivery of a software package may consist of several versions for each platform. A new release of the software is an improved system intended to replace the old one. Software systems are described as version n, release m, or as ver

18、sion n.m .,2020/8/2,兰州大学信息学院,20,Principles of system testing,Version n is sometimes intended to replace version n 1, and release m supersedes m 1. IBM(R) DB2(R) Universal Database Enterprise Server Edition,Version 8.1. Version may have two different means : For each type of platform or operating sys

19、tem. In the sequence of phased products.,XXXX Version n.m for xxxxx,2020/8/2,兰州大学信息学院,21,Principles of system testing,Phrased software development,2020/8/2,兰州大学信息学院,22,Principles of system testing,Regression testing : Regression testing identifies new faults that may have been introduced as current

20、ones are being corrected. A regression test is a test applied to a new version or release to verify that it still performs the same functions in the same manner as an older version or release.,2020/8/2,兰州大学信息学院,23,Principles of system testing,Steps of strict regression testing : ( Vm Vm+1) Inserting

21、 your new code. Testing functions known to be affected by the new code. Testing essential functions of m to verify that they still work properly. Regression Continuing function testing of m + 1. e.x : P392 Sidebar 9.1 telephone billing system.,2020/8/2,兰州大学信息学院,24,Principles of system testing,Deltas

22、, separate files, and conditional compilation. ways for control versions and releases. Keep different separate files for each different version or release. Difficult to update the separated files. Keep the common parts in a common place while different files for different versions should be keep sep

23、arately. The different file is called a delta. This way is useful for release.,2020/8/2,兰州大学信息学院,25,Principles of system testing,A single code component address all version, and conditional statements use the compiler to determine which statements apply to which version. This way is called Condition

24、al compilation.,2020/8/2,兰州大学信息学院,26,Principles of system testing,2020/8/2,兰州大学信息学院,27,Principles of system testing,Change control : when change to the software (code, documents,), some control should be done. Use “Check out” copies for mutually fixing. All members work with the most up-to-date docu

25、ments.,2020/8/2,兰州大学信息学院,28,Principles of system testing,Test team : Test team members are come from both developers and customers. The test team is often independent of implementation staff. The tests are familiar not only with the system specification, but also with testing methods and tools.,2020

26、/8/2,兰州大学信息学院,29,Principles of system testing,Team members Professional tester : organize and run the tests. Design test plans, test cases, and run the tests. Analysts : who are familiar with the requirements. System designers : who are familiar with the solution. A configuration management represen

27、tative : to arrange the changes. Users : for ease of use, and other human factors.,2020/8/2,兰州大学信息学院,30,2 Function testing,Whereas unit and integrated testing concentrated on components and their interactions, function testing ignores system structure and focuses on functionality. Use more closed bo

28、x than open method. Based on the systems functional requirements. (Develops test cases based on the requirements document),2020/8/2,兰州大学信息学院,31,Function testing,Purpose and roles : The set of actions associated with a function is called a thread. Function testing = thread testing. Logically, it shou

29、ld be easier to find the cause of a problem in a small set of components than in a large set.,2020/8/2,兰州大学信息学院,32,Function testing,A function test should: Have a high probability of detecting a fault. Use a test team independent of the designers and programmers. Know the expected actions and output

30、s. Test both valid and invalid input. Never modify the system just to make testing easier. Have stopping criteria.,Which is same with the unit test.,2020/8/2,兰州大学信息学院,33,Function testing,Functions may be defined in a nested manner. A water monitoring system is to identify : dissolved oxygen, tempera

31、ture, acidity and radioactivity. For ease of test, we view the monitoring as four separate functions rather than one big function.,2020/8/2,兰州大学信息学院,34,Function testing,Effective function tests should have a high probability of detecting a fault. Function testing may begin before the entire system i

32、s constructed. Test case design for function testing should based on the requirements documents.,2020/8/2,兰州大学信息学院,35,Function testing,ex : a word processing system can be tested by examining: Document creation Document modification Document deletion Whereas modification: P397/431 Adding a character

33、 Adding a word ,2020/8/2,兰州大学信息学院,36,Function testing,Cause-and-effect graphs : To create test cases automatically from the requirements by convert the graph to a decision table. The test cases should not be redundant. Use causes (inputs) and effects (outputs or transformations) to draw Boolean grap

34、h which reflecting the relationships.,Requirements formal specifications decision table test cases,2020/8/2,兰州大学信息学院,37,Function testing,steps of creating a cause-and-effect graph: The requirements are separated so each requirement describes a single function. The numbered causes and effect become n

35、odes of the graph. Placing causes on the left-hand side of the drawing and effect on the right. Draw the logical relationships.,2020/8/2,兰州大学信息学院,38,Function testing,Notation for cause-and-effect graphs.,2020/8/2,兰州大学信息学院,39,Function testing,e.x : a water-level monitoring system. Requirement : the s

36、ystem sends a message to the dam operator about the safety of the lake level.,2020/8/2,兰州大学信息学院,40,Function testing,Design description : Input: the syntax of the function is LEVEL(A,B). Where A is the height in meters of the water behind the dam, and B is the number of centimeters of rain in the las

37、t 24-hour period. Processing : the function calculates whether the water level is within a safe range, is too high, or is too low. Output : the screen shows one of the following message: 1. “LEVEL = SAFE” when the result is safe or low. 2. “LEVEL = HIGH” when the result is high. 3. “INVALID SYNTAX”,

38、2020/8/2,兰州大学信息学院,41,Function testing,Separate these requirements into five “causes”: The first five characters of the command “LEVEL.” The command contains exactly two parameters separated by a comma and enclosed in parentheses. The parameters A and B are real numbers such that the water level is c

39、alculated to be LOW. The parameters A and B are real numbers such that the water level is calculated to be SAFE. The parameters A and B are real numbers such that the water level is calculated to be HIGH.,2020/8/2,兰州大学信息学院,42,Function testing,Three effects: The message “LEVEL = SAFE” is displayed on

40、 the screen. The message “LEVEL = HIGH” is displayed on the screen. The message “INVALID SYNTAX” is printed out. Intermediate nodes: The command is syntactically valid. The operands are syntactically valid.,2020/8/2,兰州大学信息学院,43,Function testing,2020/8/2,兰州大学信息学院,44,Function testing,Define the decisi

41、on table: Put a row in the table for each cause or effect. Define the columns by examining each effect and listing all combinations of causes that can lead to that effect. The columns of the table correspond to the test cases.,2020/8/2,兰州大学信息学院,45,Function testing,Decision table for cause-and-effect

42、 graph,I the cause is invoked or true, S the cause is suppressed or false. X do not care. A the effect is absent, P the effect is present,2020/8/2,兰州大学信息学院,46,3 Performance testing,Performance testing addresses the nonfunctional requirements. To determine how well that functions performed as custome

43、rs required.,2020/8/2,兰州大学信息学院,47,Performance testing,Purposes and roles : Measured against the performance objectives set by the customers such as response time, accuracy of the result, Performance testing is designed and administered by the test team, and the results are provided to the customer.,

44、2020/8/2,兰州大学信息学院,48,Performance testing,Types of performance tests : Stress tests : evaluates system performance when stress to limits. Volume tests :handle of large amounts of data ( data structure) . Configuration tests : for various configurations. Compatibility tests : interfaces with other sys

45、tems. Regression tests : to replace an existing system. Security tests :security requirements. Timing tests : time to respond to a user or time to perform a function.,2020/8/2,兰州大学信息学院,49,Performance testing,Environmental tests : ability to perform at the installation site. Not accept test Quality t

46、ests : reliability, maintainability, and availability. Recovery tests :response to faults. Maintenance tests : diagnostic tools and helps to find the problem source. Documentation tests : write the right required documents. Human factors (usability) tests : user interfaces. And their combination.,20

47、20/8/2,兰州大学信息学院,50,4 Reliability, availability and maintainability,Involved testing belongs to performance testing. Different from other performance test: can not be directly measured.,2020/8/2,兰州大学信息学院,51,Reliability, availability and maintainability,Definitions : Idea condition: we want our softwa

48、re to function consistently and correctly over a long periods of time, to be available when we need it, and to be repaired quickly and easily if it does fail.,2020/8/2,兰州大学信息学院,52,Reliability, availability and maintainability,Reliability involves behavior over a period of time. Availability describe

49、s something at a given point in time. Maintainability involves how easy to fix a product.,2020/8/2,兰州大学信息学院,53,Reliability, availability and maintainability,Software reliability : is the probability that a system will operate without failure under given conditions for a given time interval. Reliabil

50、ity is measured over execution time, not clock time .,2020/8/2,兰州大学信息学院,54,Reliability, availability and maintainability,Software availability : is the probability that a system is operating successfully according to specification at a given point in time. Measured at points of clock time.,2020/8/2,

51、兰州大学信息学院,55,Reliability, availability and maintainability,Software maintainability : is the probability that a maintenance activity can be carried out within a stated time interval and using stated procedures and resources.,2020/8/2,兰州大学信息学院,56,Reliability, availability and maintainability,Failure s

52、everity (U.S. Military Standard MIL-STD-1629A): Catastrophic: cause death or system loss; Critical: cause injury or major system damage Marginal : cause minor injury or minor system damage; Minor: not serious.,2020/8/2,兰州大学信息学院,57,Reliability, availability and maintainability,Failure data :,Inter-fa

53、ilure times (left to right, in row),2020/8/2,兰州大学信息学院,58,Reliability, availability and maintainability,The long-term reliability growth. When the next failure will occur is uncertainty.,2020/8/2,兰州大学信息学院,59,Reliability, availability and maintainability,Uncertainty : Type-1 : reflecting uncertainty a

54、bout how the system will be used. At any point in time, the time to the next failure is uncertain. Type-2 : reflects our lack of knowledge about the effect of fault removal. When we fix a fault, we do not know if our corrections are complete and successful. And we are not certain about the degree to

55、 which our correction increases the softwares reliability.,2020/8/2,兰州大学信息学院,60,Reliability, availability and maintainability,Measuring reliability, availability, and maintainability : Record the interfailure times, or times to failure, as t1,t2, , ti-1. The average of these values is the Mean Time

56、to Failure (MTTF) Mean Time to Repair (MTTR) : the average time it takes to fix a faulty software component. Mean Time between Failures (MTBF) : combine MTTR and MTTF. MTBF = MTTF + MTTR.,2020/8/2,兰州大学信息学院,61,Reliability, availability and maintainability,Reliability : R = MTTF / (1+MTTF) Availabilit

57、y : A = MTBF / (1+MTBF) Maintainability : M = 1 / (1+MTTR),2020/8/2,兰州大学信息学院,62,Reliability, availability and maintainability,Reliability stability and growth : If the inter-failure times stay the same, then we have reliability stability. If the inter-failure times increase, we have reliability grow

58、th . Probability density function f of time t, written f(t). Distribution function , F(t) = f(t) (t1 t2) Reliability function R(t) = 1 F(t),2020/8/2,兰州大学信息学院,63,Reliability, availability and maintainability,Reliability prediction : Use historical information about failures and failure times to build

59、 simple predictive models for reliability. ti+2 = (ti+1 + ti)/2 ti+10 = (ti + ti+1 + + ti+9)/10 ti+20 = (ti + ti+1 + + ti+19)/20,2020/8/2,兰州大学信息学院,64,Reliability, availability and maintainability,Predicted mean time to failure (seconds),2020/8/2,兰州大学信息学院,65,Reliability, availability and maintainability,Any prediction system should have: A prediction model An inference procedure : F(ti) = f(t1,ti-1) A prediction procedure,2020/8/2,兰州大学信息学院,66,Reliability, availability and maintainability,Some prediction models The Jelinski-Moranda Model The Littlewood Model Importance of the operational enviro