创业管理(李雪灵)part2-blackbox

1、,Part II: Testing Fundamentals,farucalgary.ca,Software Testing,Contents,Types of software testing Examining the Specification Testing the Software with Blinders On Examining the Code Testing the Software with X-Ray Glasses,Part II: Testing Fundamentals,Types of software testing,Types of testing,One

2、possible classification is based on the following four classifiers: C1: Source of test generation. C2: Lifecycle phase in which testing takes place C3: Goal of a specific testing activity C4: Characteristics of the artifact under test,C1: Source of test generation,C2: Lifecycle phase in which testin

3、g takes place,C3: Goal of specific testing activity,C4: Artifact under test,Part II: Testing Fundamentals,Examining the Specification,Contents,Getting Started Performing a High-Level Review of the Specification Low-Level Specification Test Techniques,Getting Started,The Software Development Process:

4、 big-bang, code-and-fix, waterfall, and spiral. In each model, except big-bang, the development team creates a product specification from the requirements document to define what the software will become. The only way to assure that the end product is what the customer required and to properly plan

5、the test effort is to thoroughly describe the product in a specification.,Getting Started,An excerpt from the Windows Calculator (see Figure 4.1) : The Edit menu will have two selections: Copy and Paste. These can be chosen by one of three methods: pointing and clicking to the menu items with the mo

6、use, using access-keys (Alt+E and then C for Copy and P for Paste), or using the standard Windows shortcut keys of Ctrl+C for Copy and Ctrl+V for Paste. The Copy function will copy the current entry displayed in the number text box into the Windows Clipboard. The Paste function will paste the value

7、stored in the Windows Clipboard into the number text box. As a tester youll also have a document as a testable item. You can use it to find bugs before the first line of code is written.,Getting Started,Black-Box and White-Box Testing Black-box testing is sometimes referred to as functional testing

8、or behavioral testing. white-box testing is sometimes called clear-box testing,Getting Started,Static and Dynamic Testing Static testing refers to testing something thats not running examining and reviewing it. Dynamic testing is what you would normally think of as testing running and using the soft

9、ware.,Getting Started,Static Black-Box Testing: Testing the Specification You can test a specification with static black-box techniques no matter what the format of the specification. It can be a written or graphical document or a combination of both. You can even test an unwritten specification by

10、questioning the people who are designing and writing the software.,Performing a High-Level Review of the Specification,The first step in testing the specification isnt to jump in and look for specific bugs. The first step is to stand back and view it from a high level. Examine the spec for large fun

11、damental problems, oversights, and omissions.,Performing a High-Level Review of the Specification,Pretend to Be the Customer Its important to understand the customers expectations. Remember that the definition of quality means meeting the customers needs. As a tester, you must understand those needs

12、 to test that the software meets them. Dont forget about software security when pretending to be the customer.,Performing a High-Level Review of the Specification,Research Existing Standards and Guidelines The difference between standards and guidelines is a matter of degree. A standard is much more

13、 firm than a guideline. Guidelines are optional but should be followed. Corporate Terminology and Conventions. Industry Requirements. Government Standards. Graphical User Interface (GUI). Security Standards.,Performing a High-Level Review of the Specification,Review and Test Similar Software One of

14、the best methods for understanding what your product will become is to research similar software. This could be a competitors product or something similar to what your team is creating. Some things to look for when reviewing competitive products include Scale. Complexity. Testability. Quality/Reliab

15、ility. Security.,Low-Level Specification Test Techniques,Specification Attributes Checklist A good, well-thought-out product specification, with all its ts crossed and its is dotted, has eight important attributes: Complete. Accurate. Precise, Unambiguous, and Clear. Consistent. Relevant. Feasible.

16、Code-free. Testable.,Low-Level Specification Test Techniques,Specification Terminology Checklist The spec may go on to clarify or elaborate on them, or it may leave them ambiguous in which case, youve found a bug. Always, Every, All, None, Never. Certainly, Therefore, Clearly, Obviously, Evidently.

17、Some, Sometimes, Often, Usually, Ordinarily, Customarily, Most, Mostly. Etc., And So Forth, And So On, Such As. Good, Fast, Cheap, Efficient, Small, Stable. Handled, Processed, Rejected, Skipped, Eliminated. IfThen(but missing Else).,Highlights of this chapter include,What is black-box and white-box

18、 testing How static and dynamic testing differ What high-level techniques can be used for reviewing a product specification What specific problems you should look for when reviewing a product specification in detail,Part II: Testing Fundamentals,Testing the Software with Blinders On,Contents,Dynamic

19、 Black-Box Testing: Testing the Software While Blindfolded Test-to-Pass and Test-to-Fail Equivalence Partitioning Boundary value analysis Decision Table Based testing Test generation from predicates State Testing Other Black-Box Test Techniques,Dynamic Black-Box Testing,Youre entering inputs, receiv

20、ing outputs, and checking the results. Another name commonly used for dynamic black-box testing is behavioral testing because youre testing how the software actually behaves when its used. To do this effectively requires some definition of what the software does namely, a requirements document or pr

21、oduct specification. Once you know the ins and outs of the software youre about to test, your next step is to start defining the test cases.,Dynamic Black-Box Testing,Test cases are the specific inputs that youll try and the procedures that youll follow when you test the software. Selecting test cas

22、es is the single most important task that software testers do. Improper selection can result in testing too much, testing too little, or testing the wrong things. Intelligently weighing the risks and reducing the infinite possibilities to a manageable effective set is where the magic is.,Test-to-Pas

23、s and Test-to-Fail,There are two fundamental approaches to testing software: test-to-pass and test-to-fail. When you test-to-pass, you really assure only that the software minimally works. Designing and running test cases with the sole purpose of breaking the software is called testing-to-fail or er

24、ror-forcing. A common class of test cases is one that attempts to force error messages. You know the ones like saving a file to a floppy disk but not having one inserted in the drive. These cases actually straddle the line between test-to-pass and test-to-fail.,Equivalence class partitioning,Test se

25、lection using equivalence partitioning allows a tester to subdivide the input domain into a relatively small number of sub-domains, say N1, as shown (next slide (a).,In strict mathematical terms, the sub-domains by definition are disjoint. The four subsets shown in (a) constitute a partition of the

26、input domain while the subsets in (b) are not. Each subset is known as an equivalence class.,Equivalence partitioning,Subdomains,The equivalence classes are created assuming that the program under test exhibits the same behavior on all elements, i.e. tests, within a class.,This assumption allow the

27、tester to select exactly one test from each equivalence class resulting in a test suite of exactly N tests.,Program behavior and equivalence classes,The entire set of inputs to any application can be divided into at least two subsets: one containing all the expected, or legal, inputs (E) and the oth

28、er containing all unexpected, or illegal, inputs (U). Each of the two subsets, can be further subdivided into subsets on which the application is required to behave differently (e.g. E1, E2, E3, and U1, U2).,Faults targeted,Equivalence class partitioning selects tests that target any faults in the a

29、pplication that cause it to behave incorrectly when the input is in either of the two classes or their subsets.,Faults targeted (cont.),Consider an application A that takes an integer denoted by age as input. Let us suppose that the only legal values of age are in the range 1.120. The set of input v

30、alues is now divided into a set E containing all integers in the range 1.120 and a set U containing the remaining integers.,Example 1,Further, assume that the application is required to process all values in the range 1.61 in accordance with requirement R1 and those in the range 62.120 according to

31、requirement R2. Thus E is further subdivided into two regions depending on the expected behavior.,Similarly, it is expected that all invalid inputs less than or equal to 1 are to be treated in one way while all greater than 120 are to be treated differently. This leads to a subdivision of U into two

32、 categories.,Example 1 (contd.),All integers,62-120,1.61,1,120,Example 1 (contd.),It is expected that any single test selected from the range 1.61 will reveal any fault with respect to R1. Similarly, any test selected from the region 62.120 will reveal any fault with respect to R2. A similar expecta

33、tion applies to the two regions containing the unexpected inputs.,Tests selected using the equivalence partitioning technique aim at targeting faults in the application under test with respect to inputs in any of the four regions, i.e. two regions containing expected inputs and two regions containin

34、g the unexpected inputs.,Example 1 (contd.),The effectiveness of tests generated using equivalence partitioning for testing application A, is judged by the ratio of the number of faults these tests are able to expose to the total faults lurking in A.,As is the case with any test selection technique

35、in software testing, the effectiveness of tests selected using equivalence partitioning is less than 1 for most practical applications. The effectiveness can be improved through an unambiguous and complete specification of the requirements and carefully selected tests using the equivalence partition

36、ing technique described in the following sections.,Effectiveness,Consider that wordCount method takes a word w and a filename f as input and returns the number of occurrences of w in the text contained in the file named f. An exception is raised if there is no file with name f.,This example shows a

37、few ways to define equivalence classes based on the knowledge of requirements and the program text.,Example 2,begin String w, f Input w, f if (not exists(f) raise exception; return(0); if(length(w)=0)return(0); if(empty(f)return(0); return(getCount(w,f); end,Using the partitioning method described i

38、n the examples above, we obtain the following equivalence classes.,Example 2 (contd.),Example 2 (contd.),Note that the number of equivalence classes without any knowledge of the program code is 2, whereas the number of equivalence classes derived with the knowledge of partial code is 6.,Of course, a

39、n experienced tester will likely derive the six equivalence classes given above, and perhaps more, even before the code is available,Example 2 (contd.),In some cases the equivalence classes are based on the output generated by the program. For example, suppose that a program outputs an integer. It i

40、s worth asking: “Does the program ever generate a 0? What are the maximum and minimum possible values of the output?”,These two questions lead to the following equivalence classes based on outputs:,Equivalence classes based on program output,E1: Output value v is 0. E2: Output value v is the maximum

41、 possible. E3: Output value v is the minimum possible. E4: All other output values.,Based on the output equivalence classes one may now derive equivalence classes for the inputs. Thus each of the four classes given above might lead to one equivalence class consisting of inputs.,Equivalence classes b

42、ased on program output (contd.),Equivalence classes for variables: range,Constraints,Classes,Constraints,Classes,Equivalence classes for variables: strings,Constraints,Classes,Equivalence classes for variables: enumeration,Constraints,Classes,Equivalence classes for variables: arrays,Arrays in Java

43、and records, or structures, in C+, are compound types. Such input types may arise while testing components of an application such as a function or an object.,While generating equivalence classes for such inputs, one must consider legal and illegal values for each component of the structure. The next

44、 example illustrates the derivation of equivalence classes for an input variable that has a compound type.,Equivalence classes for variables: compound data type,struct transcript string fName; / First name. string lName; / Last name. string cTitle 200; / Course titles. char grades 200; / Letter grad

45、es corresponding to course titles. ,In-class exercise: Derive equivalence classes for each component of R and combine them!,Equivalence classes for variables: compound data type: Example,One way to partition the input domain is to consider one input variable at a time. Thus each input variable leads

46、 to a partition of the input domain. We refer to this style of partitioning as unidimensional equivalence partitioning or simply unidimensional partitioning. This type of partitioning is commonly used.,Unidimensional partitioning,Another way is to consider the input domain I as the set product of th

47、e input variables and define a relation on I. This procedure creates one partition consisting of several equivalence classes. We refer to this method as multidimensional equivalence partitioning or simply multidimensional partitioning.,Multidimensional partitioning leads to a large number of equival

48、ence classes that are difficult to manage manually. Many classes so created might be infeasible. Nevertheless, equivalence classes so created offer an increased variety of tests as is illustrated in the next section.,Multidimensional partitioning,Consider an application that requires two integer inp

49、uts x and y. Each of these inputs is expected to lie in the following ranges: 3 x7 and 5y9.,For unidimensional partitioning we apply the partitioning guidelines to x and y individually. This leads to the following six equivalence classes.,Partitioning Example,For multidimensional partitioning we con

50、sider the input domain to be the set product X x Y. This leads to 9 equivalence classes.,Partitioning Example (contd.),E7: 7, y5,E8: x7, 5y9,E9: x7, y9,Partitioning Example (contd.),6 equivalence classes:,9 equivalence classes:,E1: x3, y5,E2: x3, 5y9,E3: x9,E4: 3x7, y5,E5: 3x7, 5y9,E6: 3x7, y9,E7: 7

51、, y5,E8: x7, 5y9,E9: x7, y9,Partitioning Example (contd.),1. Identify the input domain: Read the requirements carefully and identify all input and output variables, their types, and any conditions associated with their use.,Environment variables, such as class variables used in the method under test

52、 and environment variables in Unix, Windows, and other operating systems, also serve as input variables. Given the set of values each variable can assume, an approximation to the input domain is the product of these sets.,Systematic procedure for equivalence partitioning,2. Equivalence classing: Par

53、tition the set of values of each variable into disjoint subsets. Each subset is an equivalence class. Together, the equivalence classes based on an input variable partition the input domain. partitioning the input domain using values of one variable, is done based on the the expected behavior of the

54、 program.,Values for which the program is expected to behave in the “same way” are grouped together. Note that “same way” needs to be defined by the tester.,Systematic procedure for equivalence partitioning (contd.),The equivalence classes are combined using the multidimensional partitioning approac

55、h described earlier.,3. Combine equivalence classes: This step is usually omitted and the equivalence classes defined for each variable are directly used to select test cases. However, by not combining the equivalence classes, one misses the opportunity to generate useful tests.,Systematic procedure

56、 for equivalence partitioning (contd.),For example, suppose that an application is tested via its GUI, i.e. data is input using commands available in the GUI. The GUI might disallow invalid inputs by offering a palette of valid inputs only. There might also be constraints in the requirements that re

57、nder certain equivalence infeasible.,4. Identify infeasible equivalence classes: An infeasible equivalence class is one that contains a combination of input data that cannot be generated during test. Such an equivalence class might arise due to several reasons.,Systematic procedure for equivalence p

58、artitioning (contd.),Command temp causes CS to ask the operator to enter the amount by which the temperature is to be changed (tempch). Values of tempch are in the range -10.10 in increments of 5 degrees Fahrenheit. An temperature change of 0 is not an option.,The control software of BCS, abbreviate

59、d as CS, is required to offer several options. One of the options, C (for control), is used by a human operator to give one of three commands (cmd): change the boiler temperature (temp), shut down the boiler (shut), and cancel the request (cancel).,Boiler control example (BCS),The command file may contain any one of the three commands, together with the value of the temperature to be changed if the command is temp. The file name is obtained from variable F.,Selection of option C forces the BCS to examine variable V. If V is set to GUI,