When to Automate? - 摘自《.NET 软件测试指南》

Not all testing situations benefit from writing your own test code. In fact, there are many times when it’s not a good idea to automate testing. So how do you decide whether and when to automate or not? The decision to automate requires analysis and the definition of boundaries between the automated test plan and the manual test plan. Using a programming language like Visual Basic or C# requires additional careful planning since writing test scrīpts is essentially software development in its own right and can eat up plenty of time in a schedule.

How do you determine what to test manually and what to test using automated test scrīpts? While experience is the best judge, there are also some basic questions you can ask yourself prior to embarking on an automated test project. The following three sections will help you determine whether your project is a good candidate for automated testing.51Testing软件测试网,Wi%ANM"v{

Project and Personnel Issues51Testing软件测试网g*y
?
uk
Too many times test managers undertake automated testing projects without fully considering the abilities and availability of their personnel. Proper staffing is critical to the success of any project. Here are some important things to consider:51Testing软件测试网Rr!EmD5Y3Rls

What is the scope of the automated testing?51Testing软件测试网5Mj6D:ib k] p!^7g9u
If your goal is to fully automate all tests, then your scope is unrealistic. If you are trying to incorporate automated testing into existing projects or into a new one, then it’s best to start with small, manageable goals. For example, you can ask your team to write some simple utilities to support your test project using Visual Basic or C#. This has the added advantage of checking their experience level as well. Not all testers/programmers have the same capabilities!51Testing软件测试网vq"IY+}g f[y

What is the automated testing skill level of your testing personnel?
a@(WK `S8g'M0 If automated testing is new to your personnel, then you need to allow time and budget for them to take classes and learn. You will need to add experienced automated testing personnel to your staffpriorto your first project. The level of experience will determine the level of automation you will be able to undertake. One introductory course in programming will not be enough to enable your testers to undertake a large project. However, they could possibly use some of Visual Studio’s tools and wizards to support a test project, and perhaps create and use some simple test utilities.51Testing软件测试网O'\zM{ L1H|#B

What is the availability of your technically skilled testers?51Testing软件测试网&VoXL&I,x8V gVM
If you do have technically skilled testers, are they actually available? Many projects start with some experienced members who get pulled off for other projects. This may seem like a no-brainer, but we have seen this situation occur too many times to think it is just an aberration.51Testing软件测试网)\.hZ-lV?8|X

Product Issues
Zy6\)ZD{)W\
o z$i0Not all applications to be tested are created equal. In general, when you write automated test code yourself, you are working behind the GUI. That is, you are harnessing just pieces of the software. You have to spend some time to determine if this is appropriate for your product. You must do a thorough analysis. The following questions are a short list of things to consider:51Testing软件测试网+I.a
iZ Hr#F3J

Is the feature set of the application you are testing relatively stable?
7c/kH2Z#v\-C;].w)]d0 If not, the scrīpts you write need to change as often as the application changes. You may find yourself spinning your wheels if you start too soon, using up precious budget. Automated testing works best for products that are relatively stable in structure and components.

Do you plan to test the UI? Is your product GUI–based?
KXe2|'|0 Some automated testing tools are geared specifically for the GUI. If your project is to test the application’s GUI, then certain automated testing tools, commercial or open source, may be a better choice than others..NET languages can be used for GUI testing to a certain extent, but they require a significant amount of coding to do so. For this reason, in most cases, we would not choose using .NET for extensive GUI–based testing.
M1c0TtA3XNh0 Does your product have areas where tests are run repetitively, greater than ten times per test?51Testing软件测试网hBA9d%CZ
 Any repetitive tasks are candidates for automated testing. Computers perform repetitive tasks well. For example, writing regression tests for high-priority bugs or developing a Build Verification Test (BVT) suite for verification of product robustness after each build are good examples of tests that will be required to be run many times.

Will your product need to be compatible with multiple platforms?
&~C#q%[7`q*bu;G$IY0 Most products need to run on the various versions of Windows: Windows XP, Windows 2K, Windows NT, Windows 95, and Windows 98. There are many other compatibility issues, of course. Automated testing scrīpts can be written to address some of these compatibility issues.51Testing软件测试网K^w*fR5T(z

Is your project size and budget large enough to support an automated test piece?51Testing软件测试网KLLl qY'H.g
 Last but certainly not least, you must consider the additional time and budget required for automated testing. Although automated testing adds a lot to a test project, it can, especially initially, be time-consuming and costly. On a relatively small test project, adding automated test capability may not be worth it.

Additional Test-Management Issues

 Here are a few more management-level questions to ask yourself and your team:51Testing软件测试网S(x5B-hj4P

Do you have Visual Studio .NET software available to the project?
$x_-BY*p;n
MK0 If not, can you purchase the proper number of licenses and have them in place in time?51Testing软件测试网vSr&a$I(x]H

Can you insert automated testing without affecting existing testing?51Testing软件测试网mwkM"H
 For example, installing Visual Studio .NET and investigating the integration of the test scrīpting with other tests, such as manual tests or commercial tools, takes time and planning. Can you do this without adversely affecting your total project time and budget?51Testing软件测试网7]$T/bG:D#d5I*B

Do you have enough time to analyze requirements as well as code, debug, and maintain test scrīpts?
$tlX4~1?%q*n;a+N0 Development of automated test scrīpts is software development and requires all of the same considerations. It’s easy to use up time and budget.

Who will manage the automated testing for each project and across projects?
Ov\1V ?ki+|,@0 An important consideration is to keep and maintain the work done on a project for future use. For example, scrīpts for logging test results (covered in Chapter 5) can be used in any project. Identify a group or an individual who will be responsible for ensuring that code that can be reused on other projects is maintained for future use.

Managing the testing process is a big topic and an important one. There are many excellent texts available so we won’t attempt to compete with them here; check Appendix C of this book for more information on this topic.

 51Testing软件测试网n7BsqR*Q&@b5m