自动化测试之谜与事实/Software Test Automation--Myths and Facts

转载请保留：本文出自jenny2008wu的51Testing软件测试博客：http://www.51testing.com/?215302

3M}b;IH&L|%s0个人序言

    相信大家总能听到自动化测试的强劲的声音，你去面试，面试官总是问你，你会写代码吗？你会××××工具吗？似乎所有的一切都预示，自动化是一种趋势，这种趋势势必取代手工的测试。

    我不否认，测试中确实可以用到自动化，但是现在有多少人用过呢，有多少人又非常成功的实施过呢？归根结底，自动化还是要看适宜不适宜，没有最好的测试，只有最适宜的测试。

    本文的作者对自动化测试提出了一些比较中肯的观点，我把他译成中文，希望能给大家解解困惑，同时希望那些鼓吹自动化的人的呐喊声能够稍微收敛一些，因为自动化测试不仅仅是技术，而且首先需要的是，你对需求了解的深度，你的测试设计，对工具解，工具能否适合当前的项目？测试的流程做好了没有？还有计划.........                                                          M.N. Alam
F
wuGU0                                                                                IMI Systems Inc.51Testing软件测试网$H#f5m7M0{2|v
                                                                                   Dallas, TX
lzg aZ,D^0Introduction
s%L-P:|3O051Testing软件测试网(|H
vt#t
Today softwaretestautomation is becoming more and more popular in both C/S andwebenvironment. As the requirements keep changing (mostly new requirements are getting introduced on daily basis) constantly and thetestingwindow is getting smaller and smaller everyday, the managers are realizing a greater need for test automation. This is good news for us (people who do test automation). But, I am afraid this is the only good news.  
-`&^ p)D Z/|0********************************************************************************************************51Testing软件测试网!Pi!_/I'`1h
前言
GN6`(G8u'I5oN&`H9j0    如今，软件自动化测试在C/S及WEB环境中变得越来越普及。由于需求变化频繁（大多新的需求都是每日被介绍给大家）而测试窗口则变得越来越小，管理者意识到自动化测试是十分有必要的，对于我们从事自动化的人来说，这是个好消息，但是，我很担心，这或许只是个好的消息51Testing软件测试网xLT~c(?Q,dR
***********************************************************************************************************************
BMFwL-}(Z0Myths & Facts51Testing软件测试网:i$zOj0Xw$A3Hgp
A number of articles and books are written on different aspects of Software Test Automation. “Test Automation Snake Oil” by, James Bach is an excellent article on some of the myths of automation. I like to discuss some of these myths and will try to point out the facts about these myths. I also like to discuss some of my observations and hopefully point out possible solutions. These are based on my experience with a number of automation projects I was involved with.51Testing软件测试网o!`)pvN2f2|
51Testing软件测试网!f Kd3F:L5^.Wk
- Find more bugs:Some QA managers think that by doing automation they should be able to find more bugs. It’s a myth. Let’s think about it for a minute. The process of automation involves a set of written test cases. In most places the test cases are written by test engineers who are familiar with the application they are testing. The test cases are then given to the automation engineers. In most cases the automation engineers are not very familiar with the test cases they are automating. From test cases to test scrīpts, automation does not add anything in the process to find more bugs. The test scrīpts will work only as good as the test cases when comes to finding bugs. So, it’s the test cases that find bugs (or don’t find bugs), not the test scrīpts.51Testing软件测试网j)o0h,N3CAZh
***********************************************************************************************************************
B:hZD*|z9jeH0   迷及事实    51Testing软件测试网U&RO)`$N'M!Z$d
        许多文章或书籍里面都有提到软件自动化测试的各个方面，由 James Bach 写的“Test Automation Snake Oil”的文章，是篇揭开自动化测试之谜的最好的文章，我很想他讨论下这些迷并且尽量揭示这些迷背后的真实；我也很想阐述下我的观点，希望能指出这些问题的可行的解决办法。这些都基于我曾经参与的大量的自动化测试项目51Testing软件测试网5c|s-M:O7u
        --找到更多的BUG：一些QA经理认为，通过实施自动化他们应该可以找到更多BUG，这是一个谜。先考虑一分钟，自动化的过程包括一套已经写好的测试用例，在大多数情况下，这些测试用例都是由熟悉测试应用程序的测试工程师写的。这些测试用例给到自动化工程师。在大多数情况下，自动化测试工程师并不熟悉他们要实施自动的用例。从测试用例到测试脚本，自动化根本就没有为多找BUG而多添加任何一个过程。在发现BUG方面，测试脚本和测试用例的效果是一样的。因此，是测试用例在发现BUG，而不是测试脚本。51Testing软件测试网j7g#Hy$Rf
***********************************************************************************************************************51Testing软件测试网T)MR#M}Mq:S3hj
- Eliminate or reduce manual testers:In order to justify automation, some point out that they should be able to eliminate or reduce the number of manual testers in the long run and thus save money in the process. Absolutely not true. Elimination or reduction of manual testers is not any of the objectives of test automation. Here is why – as I have pointed out earlier that the test scrīpts are only as good as the test cases and . They are the ones who know the application inside out. If the word gets out (it usually does) that the number of manual testers will be reduced by introducing automation then, most if not all manual testers will walk out the door and quality will go with them as well.51Testing软件测试网2{LoE!CP
51Testing软件测试网(kh o Xp/iX
Observations51Testing软件测试网(Y$\%|(i_*a
I have met a number of QA managers who are frustrated with their automation. According to them the tool is not doing what it is supposed to do. Here is a true story, the client (I had the opportunity to work with them for some time) found out that the tool they have just bought does not support the application they are testing (I am not making it up). How can this happen! – It does happen more often than one would think. I will get back on this when I discuss possible solutions. A manager of one of the major telecom companies that I had a recent interview with told me that after three years and more than a million dollar he is still struggling with automation. This is pretty sad and I get the feeling that he is not alone.
IPX[5{*_wKr]0***********************************************************************************************************************
Zdes-U|0    ---替代手工测试或减少手工测试：为了认识自动化，一些人指出，从长远上来说，自动化测试应该可以替代手工测试或减少手工测试，由此，可以节约过程成本。很显然这不是事实。替代或减少大量的手工测试并不是自动化测试的目的，原因如下：
bJ?5{;m0     就像我开始指出的那样，测试脚本只是和测试用例一样好，他们都很熟悉这些应用程序，假如大量手工测试者随着对自动化的理解而减少，很多手工测试都将退出手工测试领域，而软件的质量也会随之一落千丈。
NBktv-N$\^0观点51Testing软件测试网U4X0aaV5pYxeV
     我见过很多QA经理，他们对自动化都很困惑，对于他们来说，工具并没有像预想的那样发挥作用。这有个真实的实例，客户（我曾经和他们工作一段时间）发现他们刚买的工具不支持他们正在测试的程序（这不是我瞎编的），怎么会发生这种事情呢！--这件事的发生超乎了大家的想象。当我讨论可能的解决方案的时候我会回过来解释这点。我最近面试的一个电信的经理告诉我，他们花了三年的时间，一百多万美元，他仍然在自动化里面挣扎。这是事实，而我也知道，不只是他一个人这样。51Testing软件测试网&rb&urJu
Solutions/Suggestions51Testing软件测试网+q U_-h Q
51Testing软件测试网0P Cd4C_.r`
Let’s discuss some of the reasons for this frustration and some of the solutions to this problem.
~6H?v9T:{
K ?h051Testing软件测试网ps,^c?m(H3Dr2mw
- Unrealistic expectations:Most managers have their first encounter with any automation tool when they look at the demo and everything looks nice and simple.
r3s?%Ez-?*xZ0But everything is not so nice and simple when you try to use the tool with your application. The vendors will only tell you the things you want to hear (how easy to use, how simple to set up, how it will save time and money, how it will help you find more bugs etc.). This builds a false set of hopes and expectations.51Testing软件测试网(}q'xD o1D
51Testing软件测试网(j3qbRPB}+i
- Lack of planning:A great deal of planning is required from selection to implementation of the tool. “Evaluating Tools” by Elisabeth Hendrickson is a very good article on step by step process of selecting a tool. She talks about “Tool Audience” as one of the steps. This would be an ideal way to select a tool. It may not happen in every place because of the everyday workload of the people involved. But the participation of the users in the process is very important, because they are the ones who will use the tool day in and day out. I am almost certain that what happened to one of my clients (the tool they have bought did not support the application they were testing) would not have happened if the users were involved in the selection process.  51Testing软件测试网A4~o Udq#r2`:^1a8n G

s^dT\M ubv0- Lack of a process:Lack of a process may also contribute to failure of automation. Most places do have some kind of process in place. In most cases (although it differs from place to place) developers write code against a set of requirements. If the requirement does not call for a change in GUI then, there should not be any change in GUI. But if the GUI keep changing constantly from one release to another without any requirement for that change then, there is a problem in the process. You may have the best tool and the best (for your environment) architecture is in place and you will still have problems with your automation because of a faulty process.
2uAo4VN%f uQ;`l9j0方案及建议51Testing软件测试网G8?#?w'l RX;q;ch
       让我们讨论一下这个困惑的原因以及解决这些问题的方法。
pZ[/y6w0       --不现实的期望：许多经理第一次碰到自动化测试工具的时候，总是觉得很好，也很简单。51Testing软件测试网p{]4p$lO/W#n
       但是当这些工具用于到测试中来时，不是所有的事都是看上去很好且很简单，销售人员往往会告诉你想听的（用起来很简单，配置也很简单，节约很多时间和金钱，找到更多的BUG），这就建立一套错误的期望。51Testing软件测试网pTOn m6Xv;^Q]N
      --缺乏计划：从选择到配置工具，需要很多计划，由Elisabeth Hendrickson写的“Evaluating Tools”很好的介绍了怎样来一步步选择工具，这只和相关工作人有关，但是测试人员参与这个过程非常重要，因为他们是天天用这个工具的人，如果测试人员参与到选择工具的过程，我可以确定我上面提到的那个客户那样的情况不会发生。51Testing软件测试网s#y@o X.?
      ---缺乏流程：缺乏流程也是自动化失败的一个原因，很多地方都有一些流程。很多时候都是开发人员对着需求写代码。如果需求没有要求在界面上进行改变，他们就不应该改动界面。但是事实上，界面每个版本都在变化，而需求没有跟着变化，这就是流程中的问题。你可能有最好的工具，最好的架构师，但是因为错误的流程，你的自动化仍然会有问题。
gHeSi gQC ^0
*g YthP7{f0
1Ey2WF0KF0Conclusion51Testing软件测试网LtJ&qJ4s"z
I think there is a need to educate QA managers about the benefits and limitations of automation. There is a need to separate the facts from the fictions. But here is the problem, in most cases consultants are brought in to fix problems of prior attempts instead of initial setup. At this point the managers have already learned (painfully) the pitfalls of automation. In order to avoid this painful experience I would recommend (most automation engineers will agree with me) to spend more time up front doing research about the styles and techniques of automation and find an architecture that fits the environment. There is no doubt that automation adds a great value to overall QA process but, short of knowledge and understanding about automation and lack of planning can also cause a nightmare.51Testing软件测试网|U0b+}W2Tg&@2x
***********************************************************************************************************************51Testing软件测试网b)V7aZVA(Ve
总结51Testing软件测试网R'Y
Z6B D'^
     我认为十分必要让QA经理明白自动化的利益及限制。必须认清神话中的事实。但是还有一个问题，在很多情况下，建议只是对应问题而不是试图替代最终的构想。从这点上来说，经理们已经明白的自动化的陷阱。为了防止这种惨痛的经历，我要强调，要多花些时间来研究自动化工具的技术及类型，以此来找出一种适合环境的架构。毫无疑问，自动化为QA的过程增加了很多价值，但是，如果缺乏对自动化的了解、认识以及计划，那就会带来一场噩梦。51Testing软件测试网z|FL4Nj$Bs