老外讨论自动化测试开发（基本原则）

转载自：51CMM.COM，一个叫LAWST的会议记录，应该是专家关于自动化测试的讨论会，个人感觉很实际。另外，翻译者语言水平很高，很少看到翻译这么流畅的文章。 

个人认为，基本原则里的第一条很有指导性，自动化测试得带有目的，并且和所处的环境关系很大。这里说的环境可能包括公司结构，项目结构， 被测对象，人员以及技术。另外，本章还提到了进化（evolution），既然是进化就是一个不断死去的过程。我觉得做自动化测试开发时，应该注意的不是我们能做什么，而是我们不能做什么。

 General principles

 基本原则

1. These statements are not ultimate truths. In automation planning, as in so many other endeavors, you must keep in mind what problem are you trying to solve, and what context are you trying to solve it in. (Consensus)

这些论述并非终极真理。在规划自动化时，由于投入了这么多的努力，你必须时刻记住哪个问题是你试图要解决的，以及在何种环境下解决它。(一致同意)

2. GUI test automation is a significant software development effort that requires architecture, standards, and discipline. The general principles that apply to software design and implementation apply to automation design and implementation. (Consensus)

GUI级自动化测试是软件开发中的一项重大工作。它需要架构、标准和纪律。适用于软件设计和实现的一般原则同样适用于自动化设计和实现。(一致同意)

3. For efficiency and maintainability, we need first to develop an automation structure that is invariant across feature changes; we should develop GUI-based automation content only as features stabilize. (Consensus)

为了效率和可维护性，我们需要首先开发一个不随特性改变的自动化结构。只有在功能稳定的情况下，我们才应该开发基于GUI的自动化。(一致同意)

4. Several of us had a sense of patterns of evolution of a company’s automation efforts over time:

我们中的一些人认为一个公司的自动化工作可以采用演化模式来进行。

First generalization (7 yes, 1 no): In the absence of previous automation experience, most automation efforts evolve through:

第一代(7票赞成，1票反对)：在缺乏以前的自动化经验时，大多数自动化工作将经历以下三个过程：

a. Failure in capture /playback. It doesn’t matter whether we’re capturing bits or widgets (object oriented capture/replay);

截取/回放失败。我们截取的是比特还是窗口部件都无关紧要（面向对象的截取/重放）

b. Failure in using individually programmed test cases. (Individuals code test cases on their own, without following common standards and without building shared libraries.)

使用个人编程的测试用例失败。(个人编码的测试用例是按他自己的标准编写的，没有遵循一般标准以及构建共享库)

c. Development of libraries that are maintained on an ongoing basis. The libraries might contain scripted test cases or data-driven tests.

开发维护在正在进行的基础之上的函数库。这个库可以包括脚本测试用例或数据驱动测试。

Second generalization (10 yes, 1 no): Common automation initiatives failures are due to:

第二代(10票赞成，1票反对)：一般自动化的主动失败源于以下几点：

a. Using capture/playback as the principle means of creating test cases;

把截取/回放作为创建测试用例的原则性方法。

b. Using individually scripted tested cases (i.e. test cases that individuals code on their own, without following common standards and without building shared libraries);

使用个人编写脚本测试用例(个人编码的测试用例是按他自己的标准编写的，没有遵循一般标准以及构建共享库)

c. Using poorly designed frameworks. This is a common problem.

使用设计不好的框架。这是一个普遍问题。

5. Straight replay of test cases yields a low percentage of defects. (Consensus)

测试用例的直接重放会带来低缺陷(一致同意)

Once the program passes a test, it is unlikely to fail that test again in the future. This led to several statements (none cleanly voted on) that automated testing can be dangerous because it can gives us a falsely warm and fuzzy feeling that the program is not broken. Even if the program isn’t broken today in the ways that it wasn’t broken yesterday, there are probably many ways in which the program is broken. But you won’t find them if you keep looking where the bugs aren’t.

一旦一个程序通过了测试，那么以后就不太可能不通过该测试了。这种说法导出了一些论述(没有一个被明确赞成)，即自动化测试有危险，因为它给我们一种很舒服的错觉，认为程序没有问题。即使程序今天没有出现昨天出现的问题，但是依然还有很多情况可以使程序出问题。而如果你总是在没有bug的地方寻找，你就永远找不到它们。

6. Of the bugs found during an automated testing effort, 60%-80% are found during development of the tests. That is, unless you create and run new test cases under the automation tool right from the start, most bugs are found during manual testing. (Consensus)

在一次自动化测试工作中找到的bug里，60%到80%是在开发测试中被发现的。也就是说，除非你从一开始就用自动化工具创建并运行新的测试用例，不然大多数bug都会在手动测试中被发现。(一致同意)

Most of us do not usually use the automation tool to run test cases the first time. In the traditional paradigm, you run the test case manually first, then add it to the automation suite after the program passes the test. However, you can use the tool more efficiently if you have a way of determining whether the program passed or failed the test that doesn’t depend on previously captured output. For example:

我们大多数人都不是第一次使用自动化工具来运行测试用例。在传统规范中，你首先要手动运行测试用例，然后在程序通过测试后把它添加到自动化套件中。但是，如果你有一种办法不依靠先前的截取的输出结果就能确定程序是否通过了测试，那么你就能更加有效的使用工具。例如：

Run the same series of tests on the program across different operating system versions or configurations. You may have never tested the program under this particular environment, but you know how it should work.

要在不同的操作系统版本或配置上运行相同系列的测试。你可能从未在一种特殊配置环境下测试过程序，但是你知道它应该怎样工作。

Run a function equivalence test. [11] In this case, you run two programs in parallel and feed the same inputs to both. The program that you are testing passes the test if its results always match those of the comparison program.

要运行一个功能等价测试。这时，你可以并行地运行两个程序，并且给它们同样的输入。如果程序的结果总是与对照程序一样，就说明它通过了测试。

Instrument the code under test so that it will generate a log entry any time that the program reaches an unexpected state, makes an unexpected state transition, manages memory, stack space, or other resources in an unexpected way, or does anything else that is an indicator of one of the types of errors under investigation. Use the test tool to randomly drive the program through a huge number of state transitions, logging the commands that it executes as it goes. The next day, the tester and the programmer trace through the log looking for bugs and the circumstances that triggered them. This is a simple example of a simulation. If you are working in collaboration with the application programming team, you can create tests like this that might use your tool more extensively and more effectively (in terms of finding new bugs per week) than you can achieve on your own, scripting new test cases by hand.

要在测试中操作代码，这样就能在出现异常情况时生成日志，比如，程序到达了一个异常状态、做了一个异常状态转换、以异常方式管理内存、堆栈或其他资源、或者做了任何一种错误的事情。先通过大量的状态转换以及记录执行的命令使用测试工具随机地驱动程序。然后在第二天，测试人员和程序员通过跟踪日志来寻找bug和触发它们的环境。这是模拟出的一个简单例子。如果你正在与应用程序团队合作，那你就能像这样，相对自己手工编写新的测试用例脚本，更加扩展地、有效地(每周都可以找到新的bug)用你的工具创建测试。

7. Automation can be much more successful when we collaborate with the programmers to develop hooks, interfaces, and debug output. (Consensus)

当我们与程序员合作开发链接、接口和调试输出时，自动化会更加成功。(一致同意)

Many of these collaborative approaches don’t rely on GUI-based automation tools, or they use these tools simply as convenient test drivers, without regard to what I’ve been calling the basic GUI regression paradigm. It was fascinating going around the table on the first day of LAWST, hearing automation success stories. In most cases, the most dramatic successes involved collaboration with the programming team, and didn’t involve traditional uses (if any use) of the GUI-based regression tools.

很多这种协作方法并不依赖基于GUI的自动化工具，或者说，人们只是用这些工具做测试驱动，并没有考虑我提到的基于GUI的回归测试规范。在LAWST会议的第一天，我们兴高采烈地听着自动化的成功案例。但这其中，很多重大的成功都涉及到与编程团队的协作，而并没有涉及到传统的基于GUI回归测试工具的使用。

We will probably explore collaborative test design and development in a later meeting of LAWST.

我们将在今后的LAWST会议中探讨可协作的测试设计与开发。

8. Most code that is generated by a capture utility is unmaintainable and of no long term value. However, the capture utility can be useful when writing a test because it shows how the tool interprets a series of recent events. The script. created by the capture tool can give you useful ideas for writing your own code. (Consensus)

大多数用截取工具生成的代码是不可维护的，且没有长期价值。但是，在编写测试时，截取工具很有用，因为它能显示工具是如何翻译一系列最近发生事件的。而且，用截取工具生成的脚本能在你编写代码时给你有用的信息。(一致同意)

9. We don't use screen shots "at all" because they are a waste of time. (Actually, we mean that we hate using screen shots and use them only when necessary. We do find value in comparing small sections of the screen. And sometimes we have to compare screen shots, perhaps because we’re testing an owner-draw control. But to the extent possible, we should be comparing logical results, not bitmaps.) (Consensus)

我们根本不使用屏幕截图，因为这是在浪费时间。（事实上，我们的意思是我们不喜欢使用屏幕截图并只在必要时使用它。例如，我们发现它在对比屏幕小分区方面有一定价值。而且，我们有时不得不对比屏幕截图，或许是因为我们正在测试一个自绘制控件。但是更可能的情况是，我们会对比逻辑结果，而不是位图。）(一致同意)

10. Don't lose site of the testing in test automation. It is too easy to get trapped in writing scripts instead of looking for bugs. (Consensus)

不要在测试自动化时弄丢测试点。比起查找bug，在编写脚本时犯糊涂真是太容易了。(一致同意)