外文翻译---软件测试策略

附 录 英文文献 SOFTWARE TESTING STEATEGIES A strategy for software testing integrates software test case design methods into a well-planned series of steps that result in the successful construction of software .As important ,a software testing strategy provides a rode map for the software developer, the quality assurance organization ,and the customer a rode map that describes the steps to be conducted as part of testing, when these steps are planned and then undertaken, and how much effort, time, and resources will be required. Therefore , any testing strategy must incorporate test planning, test case design, test execution, and resultant data collection . 一 INTEGRATION TESTING A neophyte in the software world might ask a seemingly legitimate question once all modules have been unit tested:“ IF they all work individually, why do you doubt that theyll work when we put them together?” The problem, of course, is“ putting them together” interfacing . Data can be lost across an interface; one module can have an inadvertent, adverse affect on another; subfunctions, when combiner, may not produce the desired major function; individually acceptable imprecision may be magnified to unacceptable levels; global data structures can present problems sadly, the list goes on and on . Integration testing is a systematic technique for constructing the program structure while conducting tests to uncover errors associated with interfacing. The objective is to take unit tested modules and build a program structure that has been dictated by design. There is often a tendency to attempt non-incremental integration; that is, to construct the program using a :“ big bang” approach. All modules are combined in advance .the entire program in tested as a whole. And chaos usually results! A set of errors are encountered. Correction is difficult because isolation of causes is complicated by the vast expanse of the entire program. Once these errors are corrected, new ones appear and the process continues in a seemingly endless, loop. Incremental integration is the antithesis of the big bang approach. The program is constructed and tested in small segments, where errors are easier to isolate and correct; interfaces are more likely to be tested completely; and a systematic test approach may be applied. In the sections that follow, a number of different incremental integration strategies are discussed. 1.1 Top-Down Integration Top-Down Integration is an incremental approach to construction of program structure. Modules are integrated by moving downward through the control hierarchy, beginning with the main control module (main program). Modules subordinate (and ultimately subordinate) to the main control module are incorporated into the structure in either a depth-first or breadth-first manner. Depth-first integration would integrate all modules on a major control path of the structure. Selection of a major path is somewhat arbitrary and depends on application-specific characteristics. For example, selecting the left hand path, modules M1,M2, and M5 would be integrated first. Next, M8 or (if necessary for proper functioning of M2) M6 would be integrated. Then, the central and right hand control paths are built. Breadth-first integration incorporates all modules directly subordinate at each level, moving across the structure horizontally. From the figure, modules M2, M3, and M4 would be integrated first. The next control level, M5, M6, and so on, follows. The integration process is performed in a series of steps: （ 1） The main control module is used as a test driver, and stubs are substituted for all modules directly subordinate to the main control module. （ 2） Depending on the integration approach selected (i.e., depth- or breadth-first), subordinate stubs are replaced one at a time with actual modules. （ 3） Tests are conducted as each module is integrated. （ 4） On completion of each set of tests, another stub is replaced with the real module. （ 5） Regression testing may be conducted to ensure that new errors have not been introduced. The process continues from step 2 until the program structure is built. The top-down integration strategy verifies major control or decision points early in the test process. In a well-factored program structure, decision making occurs at upper levels in the hierarchy and is therefore encountered first. If major control program do exist, early recognition is essential. If depth-first integration is selected, a complete function of the software may be implemented and demonstrated. For example, consider a classic transaction structure in which a complex series of interactive inputs path are requested, acquired, and validated via an incoming path. The incoming path may be integrated in a top-down manner. All input processing (for subsequent transaction dispatching) maybe demonstrated before other elements of the structure have been integrated. Early demonstration of functional capability is a confidence builder for both the developer and the customer. Top-down strategy sounds relatively uncomplicated, but in practice, logistical problems can arise. The most common of these problems occurs when processing at low levels in the hierarchy is required to adequately test upper levels, Stubs replace low-level modules at the beginning of top-down testing; therefore no significant data can flow upward in the program structure. The tester is left with three choices: 1 delay many tests until stubs are replaced with actual modules, 2 develop stubs that perform limited functions that simulate the actual module, or 3 integrate the software from the bottom of the hierarchy upward. The first approach (delay tests until stubs are replaced by actual modules) causes us to lose some control over correspondence between specific tests and incorporation of specific modules. This can lead to difficulty in determining the cause of errors and tends to violate the highly constrained nature of top-down approach. The second approach is workable, but can lead to significant overhead, as stubs become more and more complex. The third approach is called bottom-up testing. 1.2 Bottom-Up Integration Bottom-up integration testing, as its name implies, begins construction and testing with atomic modules (i.e., modules at the lowest level in the program structure). Because modules are integrated from the bottom up, processing required for modules subordinate to a given level is always available and the need for stubs is eliminated. A bottom-up integration strategy may be implemented with the following steps: 1 Low-level modules are combined into clusters (sometimes called builds) that perform a specific software subfunction. 1. A driver (a control program for testing) is written to coordinate test case input and output. 2 .The cluster is tested. 3.Drivers are removed and clusters are combined moving upward in the program structure. Modules are combined to form clusters 1,2, and 3. Each of the clusters is tested using a driver (shown as a dashed block) Modules in clusters 1 and 2 are subordinate to M1. Drivers D1 and D2 are removed, and the clusters are interfaced directly to M1. Similarly, driver D3 for cluster 3 is removed prior to integration with module M2. Both M1 and M2 will ultimately be integrated with M3, and so forth. As integration moves upward, the need for separate test drivers lessens, In fact, if the top tow levels of program structure are integrated top-down, the number of drivers can be reduced substantially and integration of clusters is greatly simplified. 1.3 Regression Testing Each time a new module is added as part of integration testing, the software changes. New data flow paths are established, new I/O may occur, and new control logic is invoked. These changes may cause problems functions that regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects. In a broader context, successful tests (of any kind) result in the discovery of errors, and errors must be corrected. Whenever software is corrected, some aspect of the software configuration (the program, its documentation, or the data that support it) is changes. Regression testing is the activity that helps to ensure that changes (due to testing or for other reasons) do not introduce unintended behavior or additional errors. Regression testing maybe conducted manually, be re-executing a subset of all test cases or using automated capture playback tools. Capture-playback tools enable the software engineer to capture test cases and results for subsequent playback and comparison. The regression test suite (the subset of tests to be executed) contains three different classes of test cases: 1.A representative sample of tests that will exercise all software functions. 2.Additional tests that focus on software functions that are likely to be affected by the change. 3.Tests that focus on the software components that have been changed. As integration testing proceeds, the number of regression tests can grow quite large. Therefore, the regression test suite should be designed to include only those tests that address one or more classes of errors in each of the major program functions. It is impractical and inefficient to re-execute every test for every program function once a change has occurred. 1.4 Comments on Integration Testing There has been much discussion of the relative advantages and disadvantages of top-down versus bottom-up integration testing. In general, the advantages of one strategy tend to result in disadvantages for the other strategy. The major disadvantage of the top-down approach is the need for stubs and the attendant testing difficulties that can be associated with them. Problems associated with stubs maybe offset by the advantage of testing major control functions early. The major disadvantage of bottom up integration is that “ the program as an entity does not exist until the last module is added” . This drawback is tempered by easier test case design and a lack of stubs. Selection of an integration strategy depends upon software characteristics and sometimes, project schedule. In general, a combined approach (sometimes called sandwich testing) that uses a top-down strategy for upper levels of the program structure, coupled with a bottom-up strategy for subordinate levels maybe the best compromise. As integration testing is conducted, the tester should identify critical modules. A critical module has one or more of following characteristics: 1 addresses several software requirements;2 has a high level of control (resides relatively high in the program structure); 3 is complex or error-prone(cyclomatic complexity maybe used as an indicator ); or 4 has definite performance requirements. Critical modules should be tested as early as is possible. In addition, regression tests should focus on critical module function. 二 SYSTEM TESTING 2.1 Recovery Testing Many computer-based systems must recover from faults and resume processing within a prespecified time. In some cases, a system must be fault tolerant; that is, processing fault must not cause overall system function to cease. In other cases, a system failure must be corrected whining a specified period of time or severe economic damage will occur. Recovery testing is a system test that forces the software to fail in variety of ways and recovery is properly performed. If recovery is automatic (performed by the system itself), re-initialization, checkpointing mechrequires human intervention, the mean time to repair is evaluated to determine whether it is within acceptable limits. 2.2 Security Testing Any computer-based system that manages sensitive information or cause actions that can improperly harm (or benefit) individuals is a target for improper or illegal penetration.Penetration spans a broad range of activities: hackers who attempt to penetrate systems for sport; disgruntled employees who attempt to penetrate for revenge; and dishonest individuals who attempt to penetrate for illicit personal gain. Security testing attempts to verify that protection mechanisms built into a system will in fact protect it from improper penetration. To quote Beizer:“ The systems security must, of course, be tested for invulnerability from frontal attack but must also be tested for invulnerability from flank or rear attack.” During security testing, the tester plays the role of the individual who desires to penetrate the system. Anything goes! The tester may attempt to acquires passwords through external clerical means, may attack the system with custom software designed to break down any defenses that have been constructed; may overwhelm the errors, hoping to penetrate during recovery; may browse through insecure data, hoping to find the key to system entry; and so on. Given enough time and resources, good security testing will ultimately penetrate a system. The role of the system designer is to make penetration cost greater than the value of the information that will be obtained. 2.3 Stress Testing During earlier software testing steps, while-box techniques resulted in through evaluation of normal program functions and performance. Stress tests are designed to confront programs with abnormal situations. In essence, the tester who performs stress testing asks:“ How high can we crank this up before it fail?” Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume. For example, 1 special tests maybe designed that generate 10 interrupts per second, when one or two is the average rate; 2 input data rates maybe increased by an order of magnitude to determine how input functions will respond; 3 test cases that require maximum memory or other resources maybe executed;4 test cases that may cause thrashing in a virtual operating system maybe designed; or 5 test cases that may cause excessive hunting for disk resident data maybe created. Essentially, the tester attempts to break the problem. A variation of stress testing is a technique called sensitivity testing. In some situations (the most common occur in mathematical algorithms) a very small rang of data contained within the bounds of valid data for a program may cause extreme and even erroneous processing or profound performance degradation. This situation is analogous to a singularity in a mathematical function. Sensitivity testing attempts to uncover data combinations within valid input classes that may cause instability or improper processing. 2.4 Performance Testing For real-time and embedded systems, software that provides required function but does not conform to performance requirements is unacceptable. Performance testing is designed to test run-time performance of software within the context of an integrated system. Performance testing occurs throughout all steps in the testing process. Even at the unit level, the performance of an individual module maybe assessed as while-box test are conducted. H0owever, it is not until all system elements are fully integrated that the true performance of a system can be ascertained. 软件测试策略 软件测试策略把软件测试案例的设计方法集成到一系列已经周密计划的步骤中去，从而使软件的开发得以成功地完成。同样重要的事，软件测试策略为软件开发人员、质量保证组织和客户提供了一个路线图，这个路线图描述了测试的步骤，以及当这些步骤在计划和实施的过程中，需要多少工作量、时间和资源。因此，任何测试策略都必须和测试计划、测试案例设计、测试执行和测试结果数据的收集与分析结合在一起。 一 集成测试 一个在软件世界里初出茅庐的年轻人可能在所有的模块都已经完成单元测试之后会问这样一个似乎很合理的问题：“如果他们每一个都能单 独工作的很好，那么你为什么要怀疑把它们放在一起就不能正常工作呢？”当然，这个问题就在于“把它们放在一起” -接口相连。数据可能在通过接口的时候丢失；一个模块可能对另一个模块产生无法预料的副作用；当子函数被连到一起的时候，可能不能达到预期的功能；在单个模块中可以接受的不精确性在连起来之后可能会扩大到无法接受的程度；全局数据结构可能也会存在问题。不幸的事，还有很多很多。 集成测试是通过测试发现和接口有关的问题来构造程序的系统化技术，他的目标是把通过了单元测试的模块拿来，构造一个设计总所描述的程序结构。 通常 存在进行非增量集成的倾向，也就是说，使用“一步到位”的方法来构造程序。所有的构件都预先结合在一起，整个程序作为一个整体来进行测试，其后的结果通常是混乱不看！会遇到许许多多的错误，错误的修正也是非常困难的，因为在整个程序的庞大区域中要分离出一个错误是很复杂的。一旦这些错误被修正之后，就马上会有新的错误出现，这个过程会继续下去，而且看上去似乎是个无限循环。 增量集成是一步到位的方法的对立面。程序先分成小的部分进行构造和测试，这个时候错误比较容易分离和修正，接口也更容易进行彻底地测试，而且也可以使用一种系统化的 测试方法。 1.1 自顶向下集成 自顶向下集成测试时一直构造结构的增量方法。模块集成的顺序是首先集成主控模块（主程序），然后按照控制层次结构向下进行集成。从属于 (和间接从属于 )主控模块的模块按照深度优先或者宽度优先的方式集成到整个结构中去。 深度优先集成首先集成结构中的一个主控路径下的所有模块。主控路径的选择是有些随意的，它依赖于应用程序的特性，例如，选择最左边的路径，模块M1、 M2 和 M5 将首先进行集成，然后是 M8 或者 M6（如果对 M2 的适当的功 能是必要的），然后，开始构造中间的右边的控制路径。宽度优先集成 首先沿着水平方向，把每一层中的所有直接从属于上一层模块的模块集成起来，模块 M2、M3和 M4首先进行集成，然后是下层的 M5和 M6，然后继续。 集成的整个过程有下列 5个步骤来完成： 1主控模块被利用作测试驱动程序，所有的桩模块替换为直接从属于主控模块的模块。 2 根据集成的实现方法（如深度或宽度优先），下层的桩模块一次一个地被替换为真正的模块。 3在每一个模块集成的时候都要进行测试。 4在完成了每一次测试之后，又一个桩模块被用真正的模块替换。 5可以用回归测试来保证没有引进新的错误。 整个过程号第 2步循环继续进行 ，直至这个系统结构被构造完成。 自顶向下集成策略在测试过程中的早期验证主要的控制和决策点。在一个好的因子化的程序结构中，决策的确定性往往发生在层次结构中的高层，因此首先会被逮到。如果主控制的确存在问题，尽早地发现它是很重要的。如果选择了深度优先集成，软件的某个完整的功能会被实现和证明，例如，考虑一个经典的事务性结构，在这个结构中，有一系列复杂的交互式输入要通过一条输入路径请求、获得和验证，这条输入路径就可以用自顶向下的方式来进行集成。所有输入处理（为了后续的事务分派）可以在其他结构元素被集成前展示出来。早期 的对功能性的战士对开发人员和客户来说都是会增加信心的。 自顶向下策略似乎相对来说不是很复杂，但是在实践过程中，可能会出现逻辑上的问题。最普通的这类问题出现在当高层测试需要首先对较低层次的足够测试后才能完成的时候。在自顶向下测试开始的时候，桩模块代替了底层的模块，因此，在程序结构中就不会有重要的数据项上传递，测试者只有下面的三种选择：一是把测试推辞到桩模块被换成实际的模块之后再进行，而是开发能够实现有限功能的用来模拟实际模块的桩模块，三是从层次结构的最高底部向上来对软件进行集成。 第一种实现方法（把测试推迟到 桩模块被换成实际的模块之后再进行）使我们失去了对许多在特定测试和特定模块组合之间的对应性的控制，这样可能导致在确定错误发生原因是的困难性，并且会违背自顶向下方法的高度受限的本质。第二种方法是可行的，但是会导致很大的额外开销，因为桩模块会变得越来越复杂。第三种方法，也就是自底向上测试。 1.2 自底向上集成 顾名思义，自底向上集成测试是从原子模块（比如在程序结构的最低层的模块）开始来进行构造和测试的，因为模块时自底向上集成的，在进行时要求所有从属于某个给定层次的模块总是存在的，而且也不再有使用桩模块的必要。 自底向上集成策略可以使用下列步骤来实现： 1低层构件被组合成能够实现软件特定子功能的簇（又使也称为结构）。 2写一个驱动程序（一个供测试用的控制程序）来协调测试案例的输入和输出。 3对簇进行测试。 4移走驱动程序，沿着程序结构的层次向上对簇进行组合。 这样的集成首先把所有的构件举集成三个簇 1， 2， 3，然后用 对每一个促使用驱动程序进行测试，在簇 1和簇 2中的构件从属于 M1,把驱动程序中的 D1和D2去掉，然后把这两个簇和 M1直接连在一起。类似地，驱动程序 D3也在模块M2集成之前去掉， M1和 M2最后都要和模块 M3一起进行集成，依此类推。 当集成在向上进行的过程中，对单独的测试驱动程序的需求减少了，事实上，如果程序结构的最上两层时自顶向下集成的，那么所需的驱动程序树木就会明显的减少，而且簇的集成也会明显简化。 1.3 回归测试 每当一个新的模块被当作集成测试的一部分加进来的时候，软件就发生了改变。新的数据流路径被 建立起来，薪的 I/O操作可能也会出现，还有可能激活了新的控制逻辑。这些改变可能会使原本工作得很正常的功能产生错误。在集成测试策略的环境中，回归测试是对某些已经进行过的测试的子集的重新执行，以保证上述改变不会传播无法预料的副作用。 在更广阔的环境里，（任何种类的）成功测试结果都是发现错误，而错误是要被修改的，