Before this July and next self-dealing system, i 'd like to learn anything of RFT......
-
神码的老师来培训
2007-06-23 17:48:41
-
西瓜的计划july 21th-28th
2007-06-20 21:25:11
6月21日-6月28日（按周计）：
1、学习java基础(一至九），并且收集Java基础第二部分；
2、写完测试用例：wl-system；
3、选择wl-system中简单的一部分，录制脚本，建立回归测试库；
4、长线阅读：emma 第二章----？
5、短线阅读：news ，每一天都选择一两篇在线文章阅读；
6、每天都有运动时间：慢跑、或者网球；周末要去游泳;
督促自己啊！千万要有执行力啊！ -
小细节
2007-06-20 12:10:29
要学习遵守规定，比方说：中餐时间定在十二点，有些同事十一点五十就去了...but ！
me还是要到十二点离开座位，要严于律己！
提醒一下！
-
Domystifying the test of PeopleSoft enterprise application
2007-06-18 15:10:27
Today , I 'd like to read a new article about Rational Funtional Tester , named : Domystifying the testing of PeopleSoft enterprise applictation with ibm Rational Tester.
from
http://www.ibm.com/developerworks/rational/library/07/0424_weber_saracevic2/index.html
subtitle:
How to test deployments and upgrades easier and faster
Level: Intermediate
Donald Weber (weberdm@us.ibm.com), Sr. IT Specialist, IBM
Fariz Saracevic (fariz@us.ibm.com), Senior IT Architect, IBM24 Apr 2007
Learn the basics of how to use IBM® Rational® Functional Tester to test PeopleSoft deployments and upgrades.
Oracle's PeopleSoft Enterprise applications are designed to address the most complex business requirements. They provide comprehensive business and industry-specific solutions that enable organizations to significantly improve performance. PeopleSoft Enterprise applications offer Web services integration to fit seamlessly into a heterogeneous applications environment and a broad array of technology infrastructures. Simple configuration ensures that you can meet the most unique customer requirements. Rational Functional Tester is an automated, functional and regression testing tool. In this article, you will see how to use this product to test PeopleSoft Enterprise applications
Installation of IBM Rational Functional Tester V7.0. See Resources for where to download the trial version.
Recording against a PeopleSoft application
- To record a test scrīpt, select the red Record button, shown in Figure 1, on the Rational Functional Tester toolbar.
Figure 1. IBM Rational Functional Tester toolbar
- Next, select the scrīpt folder location, specify the scrīpt name, and click Finish.
- To start recording, you need to enable PeopleSoft as a new HTML application by defining the URL of the PeopleSoft application, as Figure 2 shows.
Figure 2. Editing the application list in the Start Application view
- Then specify the URL of the PeopleSoft application, and name the application. Optionally, you can select Browser (see Figure 3).
Figure 3. The application configuration tool
- Now, start recording the test by interacting with the application in the browser window that opens.
- Rational Performance Tester records each interaction and adds comments to the test scrīpt that describe the actions. The progress of the test recording is displayed in the recording window, which is visible in the lower-right corner of Figure 4.
Figure 4. PeopleSoft application under test
Figure 4 also illustrates that Rational Functional Tester can identify individual objects within the PeopleSoft application. In this case, the object is the Worklist. You can use the Worklist object to create verification points that gauge your test progress and status.
To determine whether a test passes or fails, verification points are inserted into the scrīpt. Rational Functional Tester recognizes the objects that are selected for the verification points, as shown in Figure 4. The Home Worklist object has been selected for a verification point. At run time, a data verification point compares the selected object's primary data (in this case, its text attributes) with the object that it captured during recording time.
If you use a property verification point, it allows the user to select the specific property that is of interest in the test.
Figure 5. The Verification Point and Action Wizard
- When the test has completed, stop the recording, and Rational Performance Tester will generate the test scrīpt, as you see in Figure 6. This screen capture also illustrates the object-oriented nature of the Rational Functional Tester scrīpt. The steps in the scrīpt capture the GUI-level interactions by using the PeopleSoft object model.
Figure 6. The recorded test scrīpt
Playing back a test of a PeopleSoft application
When you run the recorded scrīpt, the browser opens the PeopleSoft application, executes the recorded actions, and displays the results -- all automatically (Figure 7). In this scenario, the test scrīpt contains two verification points: a data verification point and a property verification point.
-
那些花儿：今晚打电话给？
2007-06-11 23:05:10
-
翻译：Using Rational Functional Tester V7.0 to test Mozilla Firefox applications
2007-06-08 20:58:49
The Preface of My Diary
Except for my formal-work , the study-task i should finish today is to translate this article which is from http://www.ibm.com/developerworks/rational/library/06/1219_kelly/index.html#author
into Chinese
the reason why i'd like to spend my time doing this is not only for its careful descrīption , but also an indelible impression would be made on me when writting words by words ....
people who is interested in it could check the original text by the link above.i'd like to share all of the things i know to you ,although there is not much.
then , the words below is what i have done today
.......文章题目：
使用 ibm RFT 测试基于mozilla firefox 浏览器的应用
（using ibm rational functional tester to test mozilla firefox application）
级别:入门
IBM RFT支持基于mozilla firefox浏览器HTML应用，本文就这一功能如何应用到自动化测试进行探讨。
FRT 提供了一种自由的选择，java脚本和开发环境或者是visual basic.net脚本和开发环境，都可行。这意味着在开发自动化测试过程中，你可以像杠杆一样充分整合不同的开发者、开发语言、平台。
我们将使用RFT7.0,在Mozilla firefox 2.0浏览器上，测试一个HTML应用。选择java环境。不过，别担心，如果你使用的是.net，操作过程基本上是一致的。
note：测试中的作者使用的是RFT7.0，Microsoft® Windows® XP Professional operating system (SP2)，Mozilla Firefox V2.0 。对于其它的操作系统和浏览器，RFT也是一样的原理。
标题一、测试运行在Mozilla Firefox version 2.0上的HTML应用
本文通过例子来道方法，可以一箭双雕，既可以告诉大家测试的方法，也可以展示RFT最基本的特点。
例子：假设你要在www.bookpool.com上定购一本关于java的书。
1.启动RFT， 打开一个已有的工程；
2.点击click a functional test scrīpt, 输入一个名字来命名这个脚本；
Figure 1. Record a Functional Test scrīpt screen display
3.点击finish ,开始录制。录制的界面，见图二，将打开；.
Figure 2. Recording window
4.点击start application ,以打开启动应用的界面，见下图三
Figure 3. Start Application window
5.为了将book.com url 添加到应用列表中，点击edit application list;
6.在application configuration tool界面中，见下图4，点击edit add;
Figure 4. Application Configuration Tool window
7.在add application 界面中，选择所测试的应用的类型，在本例中我们选择HTML application，然后点击next
Figure 5. Add Application window
8.在add application 的select html application 界面中，url栏中输入www.bookpool.com，然后点击finish
Figure 6. Add the URL in the Select HTML application view
9.在application configuration tool的edit application information 界面中，你可以看到bookpool 的url已经存在于application的列表中；
10.务必要在brower 栏目中选择mozilla firefox；
Figure 7. Edit Application Information window
- 点击finish;
12.界面自动返回到start application 界面，在application name 中选择url，然后点击ok，网页bookpool.com将会自动在浏览器中打开；
13.在网站的搜索栏中输入java，以搜索出一本关于java 的书，见下图8；
Figure 8. Search the Web site for a book about Java
14.在显示出搜索结果的页面，我们希望将列表中的第一本书添加到购物车中，于是点击add to basket,见图9；
Figure 9. Add the first book from the search results to your shopping basket
15.你应该可以看到书已经放到了购物车中，但是为了确保它确实在购物车中了，我们需要输入一个检查点(a verification poin)。首先，点击insert verification point or the action command。你将会看到verification point and action wizard界面打开，见图10；Figure 10. Verification Point and Action Wizard screen
16.使用object finder来选择列表中，关于那本书的相关数据。你将会看到一圈红色的边边围在外圈。见图11
Figure 11. Use the Object Finder to select the book that you want to order
17.在verification point and action wizaird界面中，选择perform propertiese verification poin,见图12，然后点击next;(
插一句:faint,还不知道为什么要选择这个，而不是其它........!)
Figure 12. Verification Point and Action Wizard
18.在insert properties verification point command 界面中，见图13，确定include children栏目中选择的是all，然后点击next；（再插一句：faint ，还不清楚include childern是什么意思？........
）
Figure 13. Insert Properties Verification Point Command window
19.在接下来的界面中，见图14，你将需要选择一些性质，以把验证点包含进去。（原话：The next screen prompts you to select the properties to include in the verification point. ）你可以通过操纵test object 树，到其旁边包含有所选择书的html的表格，来检查。（Navigate through the Test Objects tree to the table that contains the HTML for the book that you selected, and check that box. ）；（i am sorry,自己没搞懂）20.然后检查property 列表中的内容；（Then check the box in the Property list. ）（
sorry！）Figure 14. Verification Point Data window
- 点击finish，关掉浏览器，停止录制；
RFT 现在应当生成了一个脚本，它看起来应该很像下面的这一个listing1；
Listing 1. scrīpt generated by Rational Functional Tester
import resources.AddToCartHelper; import com.rational.test.ft.*; import com.rational.test.ft.object.interfaces.*; import com.rational.test.ft.object.interfaces.SAP.*; import com.rational.test.ft.object.interfaces.siebel.*; import com.rational.test.ft.scrīpt.*; import com.rational.test.ft.value.*; import com.rational.test.ft.vp.*; /** * Descrīption : Functional Test scrīpt * @author Michael */ public class AddToCart extends AddToCartHelper { /** * scrīpt Name : AddToCart * Generated : Nov 5, 2006 2:53:22 PM * Descrīption : Functional Test scrīpt * Original Host : WinNT Version 5.1 Build 2600 (S) * * @since 2006/11/05 * @author Michael */ public void testMain(Object[] args) { startApp("www.BookPool.com"); // Window: firefox.exe: Bookpool Discount Computer Books. Welcome! texttext().click(atPoint(78,10)); bookpoolDiscountComputerBooksW().inputChars("Java"); httpGBookpoolComHpSearch_btnGi().click(atPoint(24,11)); // Window: firefox.exe: Bookpool: Books Found - Mozilla Firefox cellwin().click(atPoint(0,0)); // Window: firefox.exe: Bookpool: Shopping Basket - Mozilla Firefox cellwin2().performTest(Cell_standardVP()); bookpoolShoppingBasketMozillaF(ANY,MAY_EXIT).click(CLOSE_BUTTON); } }接下来，运行这个刚刚录制好的脚本。
1.就在仍然打开着的脚本界面中，点击工具条中的run functional test scrīpt，会打开一个select log
界面，如下图15；
Figure 15. Select Log window
- 2.点击finish 来开始运行。当脚本在运行的过程中，你会看到playback界面，这个界面能让你
对当脚本由于某些原因停顿时会发生什么，有个概念；（号绕口，sorry
，原文：This window can give you an idea of
what's happening if the scrīpt appears to be stalled for some reason.
）
Figure 16. Playback window
当脚本执行完毕后，你的桌面上会打开一个浏览器，来显示测试运行的结果，见图17；
Figure 17. Results of the test run
(接下来一段就不罗嗦了......真不是想偷懒
)If you want a more detailed walkthrough of recording and playing back a scrīpt, read the Getting Started with Functional Tester Cheat Sheet under Help > Cheat Sheets. And don’t forget: You can always get help in the Functional and GUI Testing forum on developerWorks (also listed in Resources).
Learn
- "Run and Analyze Software Tests with Rational ClearQuest Test Management" (developerWorks tutorial, October 2006).
- In the Rational Functional Tester resource area you'll find technical documentation, how-to articles, education, downloads, product information, and more.
- Browse the technology bookstore for books on these and other technical topics.
Get products and technologies- Visit the Rational products' trial downloads area.
Discuss- Check out the Functional and GUI Testing Rational discussion forum.
- Check out developerWorks blogs and get involved in the developerWorks community.
Michael Kelly is currently an independent consultant and provides custom training in the IBM Rational testing tools. He consults, writes, and speaks on topics in software testing. He is currently serving as the Program Director for the Indianapolis Quality Assurance Association and is a Director at Large for the Association for Software Testing. He can be reached by email at Mike@MichaelDKelly.com.
-
文档命名方法（备案）
2007-06-07 18:05:17
-
测试完成后的工作内容
2007-06-07 17:32:18
-
从5月1号以来
2007-06-06 20:11:58
-
5月1日
2007-04-30 17:51:21
-
黑盒测试心得（正交方法）
2007-04-30 15:11:39
在使用正交法写测试用例之前，我的情况：
1、菜鸟
2、刚接触测试
3、本科不是计算机
4、需要写一个大模块的测试用例
剥离开个由于个人天性的差异导致的能找到bug的几率，问问自己，写用例有哪些方法？对于n多个界面、每个界面n多个按钮、n多个填写框，怎样排列组合最有效？需要测试哪些值？怎样界定边界条件？
带着这些疑问，我开始了一些小试验，最后找到一种方法，是一种简单有效的方法，帮助我写出了一份漂亮的测试用例。当然这份用例的50－60%，是使用了正交策略，其它的帮助来源于大家的帮助和自己的偶尔动脑筋。
对于正交试验方法来写测试用例，起初是一种猜测：如果软件测试也是一种工艺或试验，就像医学试验或者其它试验，我们想搞清楚的单因素对整体的影响，想搞清楚不同因素组合起来对整体的影响，那么是不是会有一种试验方法，可以用它来找到各个因素的组合，但是这种组合是最简单的。
于是，在网络上，找到了这样一篇文章，它介绍了怎样用正交试验方法来找到反应温度(A)，反应时间(B)，用碱量(C)，对一种化工产品的转化率的影响。（见附件，这篇文章真的很有意思，能够让人联想到软件测试
）
特别是文章中的图，把这些因素对应到空间上的点，然后让我们看到，这样的选择确实在空间上覆盖效率是最高的～
（如果学习是探险，思考是漫游，是不是很好玩？）
（接下来，还是网络上的知识，互联网络真的是时代的大进步！只有一点，如果维普之类的网站也可以自由拜访就太好了！）
其实早就有人想到了这样做，并且有位老兄还写了一个比较详细的测试方案，里面还讲了正交方法的故事
。但是操作起来，却充满疑惑，如果真得必须从如何构造正交表学习起，我的测试用例要写到什么时候？
（再接下来，还是网络上的知识，互联网络真的是时代的大进步！只有一点，如果维普之类的网站也可以自由拜访就太好了！罗唆！）
有一位可爱的同学，写了个小软件，可以模拟正交表操作，很好用！
只是，这个家伙把导出功能限制了一下，真是太可爱了，竟然说要给他的账户汇上一些大元才给这个其实也可以不需要的功能！
pitera_test@yahoo.com.cn
邮箱中有文章和软件，在草稿中名称《正交方法文章和小工具》
密码：pitera_test
-
such a bug (补充思考)
2007-04-29 17:33:22
-
such a bug
2007-04-29 16:52:00
今天要做一个小结
我负责测试的模块有三个：基础数据管理、报表中心、报表中心（二），仅仅通过跑这三个模块的测试用例，没有发现一个重大的bug,那个bug必须通过跑权限用例，才能测试出来，但是在页面上，它是报表中心里的。郁闷。
五一期间，要对自己这一段时间的测试心得做一个小结。提醒自己 -
转载文章之每日构造与冒烟测试
2007-04-29 08:53:56
-
四处寻找的文章之Michael Bolton
2007-04-26 13:51:36
Michael Bolton is an independent test consultant. He provides testing related training and mentoring, frequently contributes to discussions on various mailing lists, writes a newsletter on testing and plays mandolin. WhatIsTesting interviewed him recently to discuss various testing topics under the sun including testing education, testing certification, CMM and things that he is currently working on. You can visit his site http://www.developsense.com.
If you want to ask him more questions or you want to send any feedback on this interview, please send it to webmaster @ whatistesting.com.
1. Michael, please tell us something about your self, the path your career has taken and things that you do in your spare time.
Testing and computer stuff in general are my second career, actually. In high school, I was on a math-and-science track, but in my last couple of years I had a couple of great English teachers and one math teacher that really didn't inspire me at all, so I switched focus. At University of Toronto, I worked on English major and a Theatre (that's the way we spell it here in Canada) minor. After the kind of post-post-secondary jobs that you'd expect from a liberal arts major--washing dishes at a comedy club was one--I became a theatre stage manager, specializing in touring shows for kids. You wouldn't believe how that prepares one for dealing with software projects.
（还真能跳）
I worked for Quarterdeck from 1990 to 1998, the first four years in Toronto and the latter four as a program manager in Los Angeles. (I was born in New York City to Canadian parents, so I'm a citizen of both countries and relocating was easier for me than it would be for most people.) Quarterdeck made DESQview, a multitasker for DOS, and QEMM, a memory manager. Those products were pretty successful, and since they were technically excellent and interacted with so many other products, Quarterdeck was a fantastically educational place to work. As a friend said, if you didn't learn five really interesting new things, it was a slow day.（真的有这么好的地方？） On top of DESQview and QEMM, in the early 90s we had DESQview/X, the only complete X Window implementation for DOS, where we turned DOS and Windows into X Clients. Most people didn't understand the advantages of that sort of thing then (and the rest have forgotten about them now), and it took a lot of horsepower to run effectively--you needed 64MB of RAM and one of those newfangled Pentium chips. So DESQview/X didn't do well commercially, but it was brilliant technically, and it did lead Quarterdeck into the Web very early on. That was seen to be a Good Thing, because as Windows got memory management, QEMM's market niche eroded. Quarterdeck was the first company to licence Mosaic for commercial release. Our developers rewrote it into a wonderful product, but we released it too late to make a difference in the Netscape/Explorer space. We released a bunch of other Internet tools soon after that, but things were pretty noisy in the marketplace by then. I was managing QEMM and CleanSweep and some other utility software at the time, but Quarterdeck was a small enough company that we could all get exposed to everything--the strengths, which were mostly technical and the weaknesses, which were mostly business problems. Quarterdeck grew far too quickly by buying other companies with Internet-based paper, which led to the company's demise. I point out with some pride that Quarterdeck, as an Internet company, flamed out in 1998, fully three years before it became fashionable to do that. Quarterdeck was always ahead of its time. (laughs)
One of the best things about Quarterdeck, for me, was that it introduced me to Cem Kaner. Program managers at Quarterdeck were essentially the chief quality advocates for each product. I was promoted from testing to be the second program manager there, and everyone after that came out of the testing group too. Cem came to teach his Black Box Testing course after I had moved on from the testing department, but I was lucky enough to be able to sit in on the course. He was able to take all of the things that I had been thinking about in terms of testing and quality and put them into contexts that made sense to me. Cem was--and still is--extraordinarily generous with advice and mentorship, and he also pointed me towards Jerry Weinberg and James Bach, who are also huge influences on my work and my thinking.
So now I'm an independent consultant, and have been for six years. I test software, and I teach other people how to test software, and I write about doing both. Most of my writing eventually makes it to my Web site, www.developsense.com（阅读地图）.
That was the career path part. For fun, I play mandolin at Irish sessions around Toronto (and in other cities when I visit), I work on trying to keep up with repairs and reconstruction of our family home, Pandora's House, and I like bicycling--Toronto's a wonderful city for that. I enjoy cooking, and reading about food and cooking--those things have taught me a lot about testing, too. But the most fun and the most educational thing in my life these days is my first daughter Ariel, three weeks old today.
2.You publish a testing newsletter. Can you please tell us something about it, why did you start it and what are your plans regarding its future?
I started writing the newsletter mostly because I wanted to spread around some ideas about testing and about the world in general that I think are important. Although I do get the occasional flash of insight myself, most of the ideas I'd like to spread are from other people--so I'm really trying to be a curator of things that I've found to be interesting or useful. Other people write blogs for the same purpose. I wanted to do something a little more formal than that; a schedule drives me to write in a more regular way than a blog would. It's also a way of marketing my services, and, I hope, adding to my credibility. I'll keep doing it as long as I get positive feedback, and as long as I keep learning things as part of the exercise. If you want to learn about something, in fact, I recommend writing about it.
3. To which school of testing such as Process oriented, exploratory testing etc. do you belong? Or are you unhappy with this classification?
I'm perfectly happy--and proud--to be a member of the Context-Driven School of Software Testing; our manifesto is at www.context-driven-testing.com（阅读地图）. The principal tenet is that there are no practises or techniques that can be called "best"; there are things that might be more or less useful or appropriate depending on the context in which they're used. The whole idea is to do things that serve the testing mission and that serve the project. We testers already have plenty to do; we don't have time for practises that /don't/ serve the project.
I'm a reformed process-oholic. Quarterdeck was pretty typical of hot, publicly-held technology companies in the mid-to-late nineties; product release schedules driven by the need for quarterly results, lots of mergers and acquisitions, lots of rapid expansion and catastrophic layoffs. A bunch of us hoped that formal processes would help to foster a bit of coherence. Some processes did help us in the development department to get organized to some degree, which wasn't a bad thing. But general systems thinking will tell you what came next: the processes helped us to deal with an increasingly absurd number of products and projects to the point where, as soon as we were able to cope, the business piled even more projects on.
So we tried to get the business people to follow processes too. We wanted them to follow /our/ processes. Robert Heinlein, I think, said "Never try to teach a pig to sing; it wastes your time and annoys the pig." No, I'm not calling the business people pigs--far from it--but I am saying that some of them had a set of priorities and imperatives that baffled and frustrated a lot of the technologists, and vice versa. To the extent that we could get good work done--and we did do some very good work, under heavy pressure and tight deadlines--it always came down to negotiation and collaboration between people on both sides with skills, understanding, and good will. Rules or guidelines that got in the way would get pitched if we could collaborate to make the project happen. The big lessons there were that people, not processes, saved projects on a regular basis; that processes might be great or terrible, depending on the context in which we chose to use them; that processes and documentation have to serve the needs of project and the business; and that neither people nor processes can save a company that's too deeply in the mire. In order to get something done, you need motivated people. If you need to dig your way out of a hole, your processes won't ever pick up the shovel.
4. What is your take on quality certifications such as ISO and CMM? How do you see them being used or abused, especially in relation to testing?
I mostly see them as being abused, again mostly because some believe that processes, rather than people, are most important to the success of an organization. I recently worked on a project that incorporated software from a company that claimed CMM Level
5. I suspect that at one stage, one project or one department managed to garner that assessment from somebody, and now the whole company claims it. The software was terrible, just rotten. Fixes were shoddy in all kinds of ways; half or more of them failed.
In the early days, the ISO 9000 series of standards were designed for manufacturing contexts--repeated production of large volumes of tangible stuff--and I think they're more appropriate there. Software development is not that orderly; it's a creative, intellectual process, not a deterministic, manufacturing process. As for the CMM, its original context was for defense-related development projects, where human lives are at stake, and where time, money, and manpower are all available in quantity.
Are CMM practises appropriate for every software development context? Certainly not; no way. No single process model can fit every organization, and it's inappropriate to think so, in my view. To the extent that the CMM is ever appropriate, it's tailored for patient, affluent kinds of environments; I think CMM processes would ruin most companies that had to respond quickly to conditions in the commercial software market or the Web world. The trouble is that, to some influential people who are working in a chaotic circumstance, the CMM can have a seductive appeal. Those people will try to impose processes on their organizations--they'll teach many pigs to sing--but what if you need a critter to help you dig up truffles? The singing would be interesting, but it wouldn't help to find more truffles.
Jerry Weinberg defines quality as "value to some person", and he goes on to note that the next question is "who is that person"? The next step from there is to note that decisions about quality are political decisions. The CMM clearly has value to some people. So some questions, if you're interested in the CMM for your organization: Would it bring value to your organization? At what cost? Do you have the political power to see it through?
Let me give a thought experiment: there's a restaurant in Toronto called Susur. The chef, Susur Lee, is a brilliant, creative artist, a world-famous chef. The meals are very expensive, but they're exquisite and innovative. Pretty much the menu changes every night, so the cooks have to be well-trained, knowledgeable, and flexible. There's a much smaller and less expensive restaurant a few doors down--not as exciting, but still really nice and a lot cheaper. They have a standard menu and some specials every night. Just down the street from there, there's a McDonald's. Where are you going to eat? What factors go into your decision? Now ask: of the three, which restaurant uses processes that are closest to the CMM style?
5. What is your opinion on various tester certification schemes and examinations?
Do any of the certification schemes or examinations involve sitting a tester in front of a piece of software, watching while she actively investigates it, and making assessments about her thought processes? Or do the exams ask you to fill in the blanks or check boxes, using terms that appear in someone's Body of Knowledge? That is, do the certifications test your ability to think and to perform well as a tester, or do they merely test your memory? I have a very good memory, so I can assure you that having a good memory can make you appear to be much more clever than you really are.
I've spoken to a bunch of certified testers who tell me things like, "Well, I sat the test, and it was mostly multiple choice; I knew the answers that they wanted, so I put those down even when I didn't agree with them." That kind of conversation led me to describe certification recently: You go to a room in a hotel, you pay a couple hundred dollars to someone you've never met, you stay with them for a couple of hours, you say a bunch of things you don't mean, and at the end they tell you "Thanks--you were great." That a business model with a long pedigree, don't you think?
As a hiring manager, I would want to certify my employees by my own criteria--resumes, letters of reference, interviews, auditions--and not by criteria set down by someone else. On the other hand, if you're applying for a job, some certification or another might be important to the hiring manager. If the manager wanted a certification that I didn't find credible, I don't think I'd want to work for that kind of manager anyway.
6. Can you please suggest how testers should educate themselves in all matters pertaining to testing?
I'd say that the first thing is to educate yourself in all matters that interest you, whether they pertain to testing or not. I don't think that testing is a body of knowledge so much as a point of view, or a way of thinking: that is, thinking critically. I should point out that I mean thinking as a literary critic would, trying to comprehend something, rather than merely dissing it. A tester at some point should learn to think scientifically, too. I have some video of Richard Feynman summing it up: "When we want to discover a new law of nature, we form a hypothesis or a /guess/. Then we make a prediction and perform an experiment. If the hypothesis doesn't fit with the results of the experiment, then the hypothesis is /wrong/--it doesn't matter how beautiful it is, or how elegant, or who wrote it up: if it doesn't fit with observation, then the hypothesis is wrong." If you're tester, people will regularly approach you with a piece of software, and a theory that it works correctly in all circumstances. If you're a tester, a huge part of the job is to come up with experiments that challenge that theory, and to use observations to drive new theories.
I mentioned that I was interested in cooking. Testers who like to cook would do well to read Cook's Illustrated Magazine, which takes a very scientific, testing-oriented approach to cooking. In each article, the authors tell you what they were striving for in terms of taste and texture and presentation, and if they say that they table cream is better than the whipping cream in a given recipe, it's because they tested it.
I think testing and humour something important in common. Jonathan Miller (a British fellow who has been a comedian and doctor and TV presenter and stage director) once gave a lecture that I saw, in which he said that humour allows us to change our perspective on things, "to alter our categories", as he put it. Comedians are iconoclasts, they rely on jarring conventional views of things, and they also listen closely to the way people express themselves. Those are important skills for testers. When we see or hear things in a new light, we laugh, and I think we can learn from that new perspective. A lot of the people I admire are funny. George Carlin, the American standup comedian, is a great example of someone who observes things and relates them to us in new ways. The Europeans would love him; it's a shame he's not better known outside North America. (You can find out more about him at http://www.georgecarlin.com; I should give a strong caution to those who don't like strong language, though.) Richard Feynman, one of the great physicists of the 20th century, was a very funny guy. On the other hand, Newton was famously humourless, so you never know.
Similarly, testing is all about looking at things from as many perspectives as you can. That's a perfect place for me to pass on some advice from Jerry Weinberg, which is to think of at least three possible explanations for something that you've observed. If you can do that reliably, you're on your way to becoming a first-class tester.
Talk a lot with colleagues and peers. Swap stories, exchange techniques. Sign up for my newsletter (send mail to addme@developsense.com). Go to conferences and local testing groups; start one if you can't find one. For reading--read Kaner, Bach, and especially Jerry Weinberg. Oh, and Feynman.
9. What's your favourite testing joke?
Someone once posted a sign on a laser at CalTech that said, "Please do not look directly into laser with remaining eye."
7. Have you evevr been involved with testing in the XP world? Would you like to share your experiences with us?
I haven't been involved with much testing in the XP world--except that at Quarterdeck, our best developers were doing many XP-like things a full decade before Kent Beck codified XP. I loved that environment. I sometimes see some religiosity in the XP mailing lists, which worries me. But XP's values--collaboration, communication, courage, refactoring, simplicity, rapid and continuous feedback--those things are great. I seek them out, whether they're called XP or not.
8. What are the things you are currently working on?
Heh, heh. Marketing. For me, marketing is much harder than testing. I need people to introduce me to people and organizations that find testing harder than marketing. I think there are more of them than there are of us.
(laughs) That means writing the newsletter, working on the Web site, and so forth. Doing interviews for whatistesting.com. (laughs)
I'm preparing and offering presentations to the testing and development trade shows. I'm working with James Bach on a couple of workshops for Jerry Weinberg's Amplifying Your Effectiveness (AYE) conference (which is a fantastic and unusual conferences; check it out at http://www.ayeconference.com), and I'm trying to contribute to James' Rapid Software Testing course. Continuing to develop really good exercises seems to be the biggest challenge there. I'm also in the process of qualifying to teach Elisabeth Hendrickson's Creative Software Testing course.
I try to write at least a little every day, mostly on testing and quality issues. Since software is so pervasive, I think we need a wider awareness of testing's role, so I'd like to publish articles in places that don't usually talk about such issues.
And then there's that aforementioned daughter.
10. Any big ideas?
Some things seem to be percolating these days. One is that we're trying to automate large parts of our world, particularly the business world, with the goal of making things more efficient (note that efficient is usually just a code word for "cheaper" in modern-day business parlance). We're trying to replace expensive people with cheap machines, or with cheaper people aided by cheap machines. Most customers have needs that are "exceptional" to some degree; customers are individuals, and they have individual needs and preferences and circumstances. However, computers are pretty stupid and have no imagination, and processes are rarely designed to respond well to exceptional circumstances. When we force humans to follow business processes that are being modelled and controlled by poorly-designed, buggy software, will that really serve the customers? Are customers going to enjoy the experience?
The testing world contains an instance of that trend. There's a drive to push automated testing, but I'm not sure that the people behind that push really understand testing OR automation. As I said earlier, testing isn't an activity as much as it's a way of thinking. A human can observe all kinds of problems that a machine cannot. A human can change her behaviour based on observation and judgement; a machine cannot. Machines can do repetitive, deterministic tasks really quickly and accurately; so let's make sure we put automation to those tasks and those tasks only. XP and Agile Processes emphasize automation for unit testing, and that seems to be a good thing, as long as we continue to refine and develop the tests to reflect what the software has to do. But I think that automation shows its limits as it gets closer to modelling the end-user's task--for one thing, people rarely choose to automate the kinds of mistakes that people can quite reasonably be expected to make from time to time. So let's automate the low-level and high-volume stuff, but let's emphasize the value of the human capacity for thinking and inventing and observing and reporting things that matter.
That may be tough, because I frankly don't think businesses are very good at measuring or even considering cost and value if there's not a direct cash figure that can be calculated easily. Outsourcing is a great example: outsourcing is usually less expensive if you look only at the cost of labour. However, many companies today have no assets to speak of, other than intellectual ones--the real assets are in the minds and experience of their staff. Economics suggests that we can't put an accurate value on intellectual capital; there aren't good ways to measure its worth. That's not the same thing as intellectual capital having /no/ value, but without a yardstick, nobody bothers measuring, and the value implicitly gets set to zero. And yet it seems pretty clear to me that software companies that don't place value on knowledge, or that don't treat their people well are vulnerable losing their most valuable assets. Companies that work hard to retain their employees, to keep their minds engaged, to keep their skills sharp, and to keep knowledge in-house will do correspondingly better.
That's why, although I offer testing services, I prefer to provide training or mentorship. I can do expert testing for an organization, but in that case, a lot of what the organization /could/ learn leaves with me. If I train people, the knowledge stays with the organization, and what the people learn from their own subsequent testing stays with them and with the organization. Quarterdeck in its best days used and valued the intellects of its employees, and I can assure you that there's nothing more stimulating than that in the workplace. I hope I can pass on ways for other people to have that experience.
http://www.whatistesting.com/interviews/mbolton.htm -
四处找的文章之Q-Patterns
2007-04-26 13:06:30
（说明，读了这篇文章，对于Q-Patterns只有一个大致的了解，即：将所有可能遇到的问题进行归类×××的一种方法，这种方法的好处是～～～）
Q-Patterns
What are Q-Patterns?
1. Q-Patterns are a set of interrelated questions
（具体来说，interrelated questions 包含有哪些内容？）, grouped together.
2. These Questions（即指：interrelated questions） relate to some aspect of user/software requirements
（晕！）
3. They try to provide you with various alternatives which could be
used to provide a solution. For your needs either you could pick up a
solution from the questions OR based on the solution, pick up
required review comments, test cases etc. THUS you get a good set of
PRE-COOKED, pre-though out review comments, design strategies and
test cases etc.
4. You can add to the existing Q-Patterns to enhance them or create
new ones, based on your own experiences, needs etc.（开源的？在网络上共享？）
5. They follow a format different from GOF patterns（什么是GPF patterns?） but that may change. We have to think on this aspect and come up with a template generic enough to be useful to all types of Q-Patterns.
Understanding Q-Patterns by the help of an example
Password management
Passwords are used for security almost everywhere. But probably the
policies are not implemented in a similar manner by all. Even the
same company has two different policies of handling password in two
different products.
Some questions (without any attempt to categorize them...) that can
be asked are...
1. How many characters can the password accept? What are the maximum and the minimum number of characters?
2. What is the policy on password strength
2.1 It should contain some minimum numbers of numeric characters
2.2 It should contain some minimum number of characters long
3. Does the Password expire in X number of days? If yes then can the
user set the value of X?
4. Does the administrator have control over whose password does not
expire?
5. When the password is changed after expiry can it be reset to the
same password again? Or last N number of password can not be used?
6. Display: When password is typed (Either for login OR for first
time entry/changing) is password displayed in clear text or as some
other common character as * etc.? Or is it invisible?
7. When the password is shown as * to the user say, for changing, how
many characters are displayed? As many as the length of password OR a
fixed number of characters? Example: In Windows NT open control
panel/services. The password with which a service is running is
displayed as a fixed number of *s as compared to say Microsoft word's
password storage mechanism for protecting a document.
...
Some of these questions may have already been answered in the
specifications, but the chances are that there will be many unanswered
questions. If we could provide some structure to these questions which would help us categorize these questions, could provide us with framework where many more questions come to your mind naturally, it would enable us to write more complete test cases.
It is also amply evident from the questions that we are not trying to force a particular line of thought hence our Q-Patterns, based on these questions, will not be rigid depending on a particular implementation.Most important is the fact that we are not assuming any implementation. We are simply asking relevant questions. And these questions can help at the time of reviews or writing specs/design/test cases. (hence: [ hens ]
ad. 今后,从此,因此,所以
例句与用法:
1. We have no chance to meet each other a week hence.
我们今后一星期没有见面的机会了。
2. I fell off my bike yesterday hence the bruises.
我昨天骑自行车摔倒了--所以青一块、 紫一块的.
3. Handmade and hence expensive.
手工制造，因此很贵
4. A year hence it will be forgotten.
今后将被遗忘的一年 英英解释:副词hence:1. (used to introduce a logical conclusion) from that fact or reason or as a result同义词：therefore, thence, thus2. from this place3. from this time)
Q-Patterns FAQ
Q1. Why is a structure required for describing a Q-Pattern?
A structure is required to effectively capture the information in
represented in a UNIFORM manner for ease of RECORDING and SHARING.
Structure also promotes structured thoughts. At the same time a
structure also poses some restrictions and a possibility of stunting
the growth of new thoughts. A conscious effort may be required to
think "outside the box." This effort may not only help in bringing to
light new ideas but also in improving the structure.
Q2. What structure could possibly exist for Q-Patterns, since they
can be of
1. Different granularity
2. Different domains
3. Different concepts etc.
It is indeed very difficult to come up with a general structure that
fits all. The structure that I am proposing may be valid for the type
of Q-Patterns I have in mind. It may also be valid for some other
domains/concepts however, there is no guarantee. At best this is a
tentative format that I am proposing and this is likely to undergo
changes.
Q3. Why can we not use the existing GOF structure
A GOF Patterns typically has the following structure
Name: A single word or short phrase
Problem: A statement of the problem
Context: The environment where/for-which the problem has been
identified and a solution required.
Forces: Various conditions and constraints that influence the
solution.
Solution
Examples
Related Patterns
Known Uses
etc...
(For details you may refer to the Design Patterns book and also this
link http://www.hillside.net/patterns/patterns.html)
We use part of this structure.
Q4. What is the proposed format?
I propose the following format for documenting Q-Patterns
1. Name of the pattern
2. Intent/Explanation/definition
3. Questions related to:
a. Administration
b. Usage/User
c. UI
d. Security
e. Performance
i. Response Time (Fetch, insert/delete/update, Display)
ii. Concurrency
iii. Max and Min parameters (Data size etc.)
iv. Memory requirement
v. Disk Space
4. Examples
5. Associated patterns: Some times some Q-Patterns collaborate
together and these collaborating patterns should be preferably looked
at when looking at a given pattern.
6. Specialization: If the given patterns can be specialized, with
extra, context specific questions added and some specific questions
removed then that patterns can be called as "specialized pattern."
These patterns are specific to the given context.
We do not have
(a) Forces
(b) solution
as part of our template/structure because our questions indicate the forces and to
provide a solution is not the aim of Q-Patterns. To ask questions and
to enable one to choose to answer some of them is the intent.
Q5. What issues do you see with this structure?
I am not entirely satisfied with this structure. The reasons are
(a) Possibility of unwieldy collection of questions.
(b) Possibility of duplicating the questions in various
related/unrelated patterns
(c) Ambiguity in some heads/sections which makes it difficult to
place some questions in correct section.
(d) All the sections described in the proposed structure may not be
universally applicable.
Q6. What improvements can be made to the existing structure?
Some things that I have in mind are
1. Versioning the patterns: It is required for sharing the patterns
unambiguously.
2. Use of OO techniques like encapsulation and inheritance to take
care of some issues. This will give us a more robust and intuitive
structure for referencing other Q-Patterns.
3. Some classification of various Q-Patterns as Organizational Q-
Patterns, Design Q-Patterns, System Testing Q-Patterns etc.
Example of a Q-Pattern（看到这里似乎就清楚了些，但仅仅是“似乎”～）
Example: An incomplete example of a Q-Pattern is given here. I have not completed writing all questions yet :-)
NAME
Password Management
INTENT
The most general and common approach to authenticate a system or user is asking for a Password. Password authentication can be at different levels like user level, group level etc or at different stages like Operating System authentication, Application authentication etc.
QUESTIONS
If you are using Password authentication anywhere in your
spec/design/code/test you may ask following questions:
Administration
1. Can administrator reset the password?
2. Can administrator's password be reset?
3. What happens If the administrator forgets his password (any
default password is given or reinstallation would take place)?
4. Can administrator set the default password?
5. Can another administrator reset an administrator's password?
6. Can an administrator read the password of a user?
Usage
1. What's the maximum and minimum length of password?
2. Can we enter numbers in password?
3. Can blank password be used?
4. Where are passwords stored?
5. What is the default password (If any)?
6. Can one customize the default password?
7. Can Special characters (like #,$,ç,è,Þ,ß) be used
in password?
8. How is password change affected? Is original password required
before change password is allowed?
9. Is Confirm password used?
10. Is `Save Password' facility is there on the screen (so
that
user may not need to enter password every time she logs in)?
UI
11. Is password shown as stars (at the time of entering the
password, at the time of changing or resetting the password etc.)?
12. How many stars are shown for a password
• When it is being entered?
• When it is to be changed? (Note: Do not show same number of
stars as the number of characters.)
Security
1. How are passwords stored? Are they encrypted before storing? If
yes what is the encryption algorithm used?
2. Whether the password is case sensitive or not?
3. Whether the password can be cut and pasted?
4. Can a previously used password be used again? If `Yes'
then after how many changes?
5. Is there any expiry time for the password? What happens after
the date if user does not change the password during that period?
6. Is there any policy to count the number of password validations
in succession (e.g.. If user enters wrong password 4 times then she
is not able to enter password again in succession).
7. If application creates logs of all activities, then the logs of
password are created or not?
8. If logs of password are being made then the password is stored
in encrypted form or not?
Performance
1. Whether password is made up of single-byte characters (even if
multi-byte character set is being used in the application).
2. How much time will it take to authenticate the user after the
submission of password?
3. What is the maximum space required to store a password? Will
all the passwords require same space irrespective of size?
4. If wrong password is given, how much time will it take to give
the error message?
5. How many users can be authenticated at the same time?
Example: Various login screens and mechanisms (web based mail
systems, console based login etc.)
Associated patterns: Access Rights, Error messages
Specialization: Say, login for any particular web based mail system.
You can prune the question list to suite your needs, add more
questions to the specialized list OR enhance the parent Q-Pattern.
The Q-Pattern presented here is a simple one. Once we move to more
complex Q-Patterns and develop a web of collaborating Q-Patterns, the
whole concept will (hopefully) acquire a new and increased relevance
and usefulness.
文章来源http://www.whatistesting.com/qpatterns/qpatternintro.htm -
四处找的文章之Scott Barber
2007-04-26 10:36:19
Scott Barber is the person behind www.perftestplus.com（阅读地图“www.perftestplus.com”）, co-founder of WOPR (the Workshop On Performance and Reliability) and a prolific writer. In this interview he explains what performance testing is, how one should go about learning it, various resources that can help you and many more things.
If you want to ask him more questions or you want to send any feedback on this interview, please send it to webmaster @ whatistesting.com.
1. Scott, please tell us something about yourself.
I was born in Woodbury, NJ USA in 1971 and grew up in a typical American small town. As a child I was very active in school, sports and Boy Scouts. Growing up, the goals I had for myself in adulthood included:
Getting to travel and “see the world”
Get a good college education
Get out of small-town America
Be recognized as a success in my career (as with most young people in the USA my specific career choice changed many times as I grew up)
Be a father who is active in his children’s lives and activities.
Design and build my own home.
I left “small-town America” to attend Virginia Tech on a U.S. Army ROTC Scholarship to pursue a degree in Civil Engineering. After four years, I completed a Bachelor’s of Science in Civil Engineering and started my 4-year active duty commitment in the U.S. Army as a Second Lieutenant (2LT). I later went back to school at night to earn a Master’s Degree in Information Technology. My career has taken a rather wandering, but in retrospect an extremely beneficial, path to finally land in the area of Software Testing.
I spent 4 successful, but difficult years in the U.S. Army culminating as a Company Commander for the 101st Airborne Division, Air Assault Headquarters and Headquarters Company approximately 8 years before I was technically qualified to hold the position. As a junior Captain at the end of my 4 years, I was recruited into a government consulting company to be an Information Engineer on a team that was designing and building a system to replace the computer system I most often complained about while I was on active duty. I held this position for about 1½ years while becoming increasingly more technical before I went to a small private software development company with the intent of becoming an Oracle Database Administrator, but actually ended up spending 6 months as a Configuration Manager. During that time, I learned a ton about Databases, Configuration Management, development and networking. Most importantly, I learned that none of those were what I wanted to do day in and day out. 6 months later, that company was bought by a huge, publicly traded conglomerate that resulted in a significant reduction in pay, leading me to seek a new job. This is when I was recruited into a Performance Test Consulting job by an old college friend where I spent the next 4 years. This time as a consultant is where most of my performance testing experience comes from. After 4 years of consulting, my family and I decided it was time to move. As it turned out, we moved to Florida about the same time the consulting company went through a major re-organization leading me to re-think what I wanted to do next in my career. After a long and difficult 4 months of soul searching, I ended up taking a wonderful position as a Systems Test Engineer and Software Test Manager at my current company AuthenTec, Inc.
As far as my childhood goals are concerned, I have had a chance to travel more than I ever thought I would, but have plenty left to do. I achieved my educational goals, but have a passive interest in earning a Doctorate degree someday. After living in a wide variety of places, I have finally settled in a place that feel like “home” and am very content with my career progression so far.
I have two young boys who are the greatest joy in my life. As I write this, it is with thoughts of my older boy starting school tomorrow morning and my youngest learning to talk. I guess that means that the only childhood goal I have left is to design and build that house. Luckily, I continue adding goals, so I’m far from bored.
2. You specialize in performance testing. What drew you to it? How did you get started, and what led you to start writing about it?
One Sunday evening I got a call from my manager…
“Scott, be at the Marriott Hotel Conference Room at 8am tomorrow. Our CEO has an announcement to make.”
Being a good employee, I did as I was told and went to the meeting. As it turns out, our software development company was “merging” with a huge media company. We were going to become their “development branch”. As a result, we were giving up our bonuses, overtime, and our base pay was being reduced, but we were going to get stock options.
As one might imagine, I was less than enthusiastic about this. I called my friend of 10 years before I even left the parking lot of the hotel. I told him the circumstances and asked him if he’d help me update my resume, to which he responded:
“Dude, that sucks. Send me your resume I’ll take a look, but, we need performance engineers. You’d be a perfect fit.”
I said “Performance engineer? What’s that?!?”
He replied “Don’t worry, you’ll like it.”
He was right, I’ve loved every minute of it! Since then I have dedicated my professional career to this field. It has been a wonderful ride so far.
After a few years, people started approaching me to write about what I do. I never though of myself as much of a writer, so I hesitated and stalled. (Mind you, I was the Civil Engineer in college who avoided writing classes at ALL COSTS. When they finally told me that if I didn’t take a writing course I wouldn’t graduate, I went to the smallest, least reputable, community college I could find, took the course and transferred it!) Eventually, I agreed to create the Performance Engineering section of my company’s Best Practices. Some folks thought it was good and started selling me to other companies to write their procedures. Finally, my boss talked me into trying my hand at writing articles. I really didn’t think anyone would read them, but I wrote a few anyway. As it turns out, people did read them, and liked them.
3. What fascinates you in performance testing?
Most simply, I am fascinated by the diversity of it. As my meandering career path let me to this field, I had a lot of “false starts” in other branches of software development, I learned that regardless of my degree of success in those more narrow branches, I found myself becoming easily bored. As a technical tester, specifically as a performance tester of multi-user distributed systems, I get to apply aspects of all of those branches and more in new and unique ways on every project I do. This diversity from project to project is also what led me to embrace the Context-Driven School of software Testing and ultimately become friends with such software testing thought leaders as Cem Kaner and James Bach.
4. If one wants to gain expertise in performance testing what all should one do? What are the tools, books and sites that can be helpful?
There is very little consolidated information out there about the type of performance testing that most organizations need. There is no doubt that my site (http://www.perfetsplus.com) and articles are one of the most complete references for “Practical Performance Testing”. However, it is by no means complete. Some of the other places I have learned valuable lessons that I have applied to performance testing are:（以下标红色的全是阅读地图的线路）
Everything I can find written by Alberto Savoia
Lessons Learned in Software Testing, John Wiley & Sons, 2002 by Cem Kaner, James Bach and Brett Pettichord.
Testing Computer Software (2nd Ed.), International Thomson Computer Press, 1993 by Cem Kaner, Jack Falk, & Hung Quoc Nguyen.
Select Chapters from The Web Testing Handbook, STQE Publishers, 2001 by Steve Splaine and Stefan P. Jaskiel.
Research published by performance tool vendors.
Research published by testing and research organizations such as Nielsen//NetRatings, Keynote Systems, Inc., and IEEE
Online forums and magazines such as StickyMinds.com, QAForums.com and stpmag.com and performancetester.com.
Performance Engineering of Software Systems, Addison-Wesley, 1990 by Connie U. Smith, Ph.D.
Scaling for E-Business: technologies, models, performance, and capacity planning, Prentice Hall, 2000 by Daniel A. Menascé, Ph.D.
Speed Up Your Site, New Riders, 2003 by Andrew B. King
Anything by Cem Kaner, James Bach, Jerry Weinburg and/or Edward Tufte.
These days, I continue learning through networking with experts, being a technical editor for new performance related publications, beta testing tools, hosting and/or attending workshops and conferences and assisting with research being led by other experts and students in the field.
To really grow your career as a performance tester, I believe there are several things you can do that will gain you largest benefit.
Become a “Mid-Level Everything” – Developer, DBA, Network Admin, Systems Admin, Architect, Business Analyst, etc.
Become an expert in the following – your tools, performance analysis, relevant aspects of operational research and statistics, consolidating, interpreting, sorting, and graphically presenting complex information in an intuitive format.
Become skilled/knowledgeable in the following areas – group facilitation, product management and training.
Find a mentor. There are several exceptional performance testers out there who believe in Jerry Weinburg’s consulting principle
（阅读地图线路）“Give your best work away for free.” Befriending one (or more) of them to ask questions of, share ideas and brainstorm with is amazingly useful… to everyone involved.
5. What is your opinion about the state of performance testing tools? Are there some open source or free tools available?
Most tools on the market aren’t actually Performance Testing tools, they are Load Generation tools. No matter what the tool vendors claim, they do not make performance testing simple, they do not analyze results, and they do not pinpoint performance issues. They are virtually all grossly overpriced, and never believe the demo. That said, I have a favorite tool that I support heavily… with a long list of caveats. My favorite is my favorite because it allows me to create realistic loads by writing custom functions and procedures in C – rather than trying to guess how to make the silly GUI buttons duplicate the workload I have constructed.（算不算断章取意？）
The best OpenSource tool for general web based load generation is OpenSTA. As long as you are only testing websites, it is competitive with the major (pay) tools in terms of actual ability to generate load and collect data. There are plenty of other free niche tools for specific platforms and protocols, but they are only useful to small groups of people doing very specific types of testing.
6. How do you combine performance testing with load and stress testing
Actually, I see Performance Testing as a superset of testing that includes load and stress testing. I often refer to it as “Performance Related Testing”. In general, load testing is testing your application under expected load conditions to determine the overall performance. Stress testing is testing your application under unexpected or extreme conditions to determine failure modes so you can protect against them.（Scott Barber说的，一种关于 “Performance Testing、load testing、 Stress testing”的定义？）In my presentation “Introduction to Performance Testing; The Who, What, Where, When and Why.”（阅读地图线路） I describe various components of Performance Testing, and the process in general.
7. What is WOPR? How did it happen? Who all are involved in it and what is your role?
WOPR is the Workshop On Performance and Reliability (http://www.performance-workshop.org). Ross Collard and I co-founded this workshop about a year ago with the purpose of bringing together both Performance Testing Experts and “Enthusiastic Beginners” in a forum where we can share our experiences together in a format that encourages questions, critical thinking and learning. So far, we have had three very successful workshops and plan to continue having them in the spring and fall of every year. You can find out more from the website. The workshop is limited to 20 people per meeting and is by invitation only, but you can apply for an invitation by filling out the form on the website.
8. How is performance testing for consumer applications different from enterprise/server based applications?
There is actually less difference between consumer applications and enterprise/server based applications than there can be between any two individual applications in the same category. The real difference centers around performance expectations. One of the keys to effective performance testing is the ability to collect and classify requirements and then develop tests that verify those requirements.
9. Where do you think performance testing fits in the testing life cycle?
Should it be done early or should it be done late? What are the pros and cons and how do you decide?
There are actually at least 3 separate, but highly related, disciplines related to performance that should ideally be included in the Software Development Lifecycle – Software Performance Engineering (SPE), Performance Testing (what we have been talking about) and Capacity Planning. The only book I am aware of that addresses these three discipline as part of an integrated process is Improving .NET Application Performance and Scalability, Microsoft Press, 2004. In this book, Senior Author J.D. Meier does a fantastic job of taking the reader through the entire process. The only drawback is that much of the book is extremely Microsoft/.NET specific.
Performance Test planning should actually begin before the architecture has been determined or any code has been written. It is especially important to finalize performance testing planning before any machines have been configured for testing and the plan should continue to be revised as necessary until the application has reached it’s target load in production several times. Very few organizations do this. Most organizations create a plan shortly before load generation is scheduled to commence and stick to the letter of the plan regardless of the actual results or detected issues. By working this way, much of the value that can be gained through performance testing is lost.
The bottom line is this, Performance Testing late in the process alone, as most organizations do, can be highly effective, but only if all identified issues are relatively easy to resolve. Saving the testing for the end of the development cycle can be disastrous if the identified issues point back to poor architectural or design decisions.
10. You have written a lot of papers on performance testing. Do you plan to write a book on this subject?
I have written quite a number of papers, presentations and tried to tie them all together on my website. When I started doing performance testing, I found very little written on the topic and had to learn through trial and error, seeking out mentors and applying principles from other disciplines. I became frustrated with the lack of valuable information available and found many others who were looking to improve their performance testing. As a result I tried to share what I learned through these mediums so that others will have an easier time getting started than I did.
I have tentative plans for two books. The first is simply a consolidation and expansion of my existing articles. There are two main reasons that I would like to do this. First, I’d like to retrofit the articles to be even more generally applicable, include examples and code samples for various tools since they were initially written on contract for IBM Rational. The second reason is because many readers have requested hard copies, which is unfortunately prohibitively expensive on a case-by-case basis.
While there are some very good books on performance tuning, and other performance related topics and some very good chapters in various testing books about performance testing, there is no single “good” book for practical performance testing lessons. Writing a book is much more difficult than writing a book’s worth of articles, but I keep hoping that I’ll find the time, energy, resources and support to get it done in the not too distant future. While I certainly don’t think I know everything there is to know about performance testing, I do think sharing what I do know would be helpful to many performance testers both now and in the future.
11. What are the things you are currently working on?
Outside of the books and some “clean-up” articles, I have actually moved in a new direction with my new job. I am now in charge of testing fingerprint sensors and their associated software (API, drivers, etc) on both PC and embedded platforms. This is extremely challenging as the testing is a blend of four very different industries (Hardware, Software, Embedded/RealTime and Biometrics) that, to the best of my knowledge, has never been written about and as far as I can tell has rarely, if ever, been tried. The knowledge I have gained as a Performance Tester/Analyst has been extremely helpful in the endeavor, as has my relationships with various industry experts such as Cem Kaner, James Bach, and Alan Jorgensen, not to mention my Embedded Software Development Instructor friends at MIT.
Additionally, I am trying to take my User Community Modeling Language (UCML™) to the next level. It was created specifically to model Performance Workloads and the data associated to those workloads, but I have found it to be personally useful in modeling other application characteristics by simply adding a few symbols and thinking about it from a slightly different perspective. Soon, I hope to publish UCML™ v2.0 to see what other folks think.
-
四处找的文章之Rick Craig
2007-04-26 10:10:16
Rick Craig is a well known name in testing circles because of his book "Systematic Software Testing" as well as the quality of his tutorials and presentation.
In this interview he shares his ideas and advice with the testers.
If you want to ask him more questions or you want to send any feedback on this interview, please send it to webmaster @ whatistesting.com.
Q1. Rick, please tell us something about yourself, your background, your interests and things that you like to do in your free time.
I was born and raised in the Midwest and then moved to Annapolis, Maryland to attend the U.S. Naval Academy and then subsequently begin a career in the Marine Corps. I spent 12 years on active duty and then was recalled to active duty during the first Gulf War. I am now a Colonel in the Reserves.
Even though I am an artillery officer, I had the opportunity to attend the Marine Corps Computer Science School first as a student and then as an instructor. I got my first testing job around 1980. In 1982 I got the opportunity to work with a couple of contractors who were probably as close to testing experts as there were in those days. For 3 years they mentored me in testing, writing and presentation skills.
Then about that time Bill Perry invited me to do a key note at the First (Second?) International Testing Conference. Bill was a strong influence on my testing career at that point. I started working with Bill Hetzel and Dave Gelperin in the mid 80’s and they hired me to work for them fulltime in 1989 and I’m still with SQE.
I have a daughter, Crissy who is a Junior at the University of South Florida. I am the co-owner of a popular restaurant called Maddogs and Englishmen, which over the last 15 years has become an icon in Tampa Bay. (www.maddogs.com)
Q2. How did "Systematic Software Testing" happen?（阅读地图－>“Systematic Software Testing,from Rick Craig”） How much time did it take?
For years, the students in my seminars urged me to a write a book as a companion volume to my Test Management Course, but the problem was always finding the time…….. That problem was eased by asking my old friend Stefan Jaskiel, a technical writer to help me with the book. Even though almost all of the words in the book are mine, I doubt that I would ever have finished it without Stefan’s help and motivation.
The book took 2 months of dedicated work, plus a lot of work on airplanes over the course of a year. The book is also available in Chinese and will be out in Japanese this month.
Q3. Are there any incidents that might help future authors and that you would like to share?
Here are some tips that helped me write this first book:
1.Just do it!
2.Pick a time every day and work on the book even if it doesn’t seem like you’re making progress.
3.Get good reviewers. Listen to their input but at the end of the day, it’s your choice.
4.If you’re having trouble getting started, pick a coauthor.
5.Don’t try to make your book cover every topic under the sun………..
Q4. Are you working on any other book currently?
No, not right now. My travel and lecturing occupies most of my time.
Q5. What, in your experience, are the issues most testers face?
I think one of the major issues facing testers today (and every day in the past) is determining the value of the testing effort and determining when the task is done. I think that in some environments it is also difficult for testers to gain respect from developers and users.
Q6. Do you think there is a similarity in these issues across geographical locations or are
the issues different?
I see very little difference across geographical regions (including other countries)
Q7. What, in your experience, are the areas of improvement for most testers and how can they improve?
Bill Hetzel and I did some research in the early 90’s and the one thing that I learned that stood out for me was that the best teams were the best because they did the basic things well. That is not to say they didn’t employ complex, sophisticated techniques—they did. But it was the basic stuff done well that ensured their success. I think today’s testers need to get better at estimating schedules and risk and learn some of the basic test design techniques that have been available for years.
Q8. Where do you stand between pure exploratory testing and pure scrīpted testing? Has your understanding changed over time? In what way?
Well, my book is called Systematic Software Testing which would lead you to believe that I am in the scrīpted camp. And indeed, I am in favor of creating test cases that demonstrate the users can perform those tasks that are necessary for them to be successful in their jobs. It really doesn’t matter how many bugs we find (and hopefully fix)if the users cannot do their jobs. On the other hand, finding and removing bugs can ultimately help the user perform their jobs in a more productive manner. I don’t think anyone questions the effectiveness of techniques like exploratory testing in identifying bugs. I would contend that good testers have done “exploratory-like” testing for years. That is to say that the result of one test leads the tester into the next test….. Experts like James and Cem have given exploratory techniques credibility and made it possible for mainstream testers to benefit from them.
Q9. How effective do you think buddy testing is? How cost-effective do you think it is?
Buddy testing（阅读地图－>“Buddy testing”）, which is essentially working in teams and having the buddy create test scenarios before the code is written are very effective. Buddy testing, in effect, implements preventive testing at the unit level. Other techniques, like paired programming are also effective and enjoy much wider recognition.
Over the last 20 years or so, I’ve urged my clients and students to use buddy testing. I must admit that most organizations never receive enough buy-in to make it work. Literally, each programmer must learn twice as much code, which of course takes more time upfront. Those organizations that do try it though, are generally successful and in some cases greatly reduce the number of defects passing on to the test group and ultimately the users.
Q10. What is your take on certification of testers?
I think that certification is not necessarily a very good indicator of the skill or ability of a tester to do a good job. By the very nature of the training and certification testing, the focus is on terminology and “book learning”. Still without sounding too much like a consultant……I put myself firmly in the pro-certification camp. How so? If a company chooses to make certification a goal or challenge for their testers, it can be a motivator and offer a sense of accomplishment for the individual testers. Also, it can help a test manager standardize on the terminology used within the company and that alone is very valuable.
Q11. Is there any particular
certification that you recommend?
I’m pretty excited about the ISTQB that is just now rolling out in the States. It has some leverage in Europe and is created by a team of experts and practitioners.
Q12. What are your thoughts in a single standard testing terminology? Why do you think one does not exist? Well, testing was born of the need to validate software systems. The first testers tended to be developers and in some cases, users. Each group and each industry chose the words that made sense to them. When writing Systematic Software Testing, I spent quite a bit of time researching various terms and trying to come up with what was the most common usage. I can’t say that my efforts were always successful. I’m also not very optimistic that a common terminology will be obtained in the near future, although some progress is being made. I think, the internet, conferences, books, training programs, certification, etc. is gradually lessening this problem.
If you were your own interviewer what question(s) would you like to ask yourself? It is:
Q1. What is the one (ok two or three) skill that you look for most in a tester?
Good Communications skills. Creativity. Attention to detail -
四处找的文章之Elfriede Dustin
2007-04-26 09:03:23
（说明，现在处于“读”的时期，我想要多看看，看到一个怎样的程度呢？反过来知道是怎么回事，正过来还是知道是怎么回事！话不多说，就从今天开始，在这个小小的空间，记录下我所看过的网页内容或链接，留下足迹，遇到好风景，以后还可以再去看看～看到一个怎样的程度呢？～反过来觉得好，正过来还是觉得好）
Elfriede Dustin is author of various books on testing - "Automated Software Testing," "Quality Web Systems," and her latest book "Effective Software Testing."
Her website www.effectivesoftwaretesting.com was created to create a community of software professionals in the Washington DC area.
If you want to ask her more questions or you want to send any feedback on this interview, please send it to webmaster @ whatistesting.com
Q1. Elfriede please tell us something about yourself, your interests and things that you do in your free time.
Originally I am from Germany where I’ve received my business degree and then went on to get a Computer Science degree at KSU. In 1984 at one of my first jobs working at a US Government law center, we received a number of Wang computers and I lead the automation of the claims processes. Ever since then I have been hooked on computers, automation, and testing. Having lived in the suburbs of Washington, DC over a decade, I am now a naturalized American citizen. I enjoy spending any of my free time getting smarter in my chosen field of testing. Additionally, I enjoy spending time with my two daughters Jackie and Erika.
Q2. What prompted you to write the books that you wrote? What kind of response do you think these books have received?
In 1997 at a Rational users conference I presented the topic "Introducing Automated Testing to your Project." After the presentation I received uncountable repetitive inquiries from numerous sources throughout the world (even Belgium) about this same topic. It made me realize that there is a need for a book to put out this information and it prompted me to write the book "Automated Software Testing," together with my co-authors and former co-workers Jeff Rashka and John Paul. The books have been very well received, for example the book "Automated Software Testing" is in its 9th printing and has been translated into German, Russian, Japanese, Chinese, and is also available as a less expensive version for the testing audience in India.
Q3. What were your sources of experience that your book "Effective Software Testing" drew on?
"Effective Software Testing" is based on my many years of testing experience combined with the many years of software development experience of my former co-worker Douglas McDiarmid. The book also draws on the experiences of my former co-workers and co-authors Jeff Rashka, Program Manager, and John Paul, Developer and now CEO, as it also draws on the content of our books "Automated Software Testing" and "Quality Web Systems.”
Q4. Are you working on any other book?
I am in the process of writing a book on Security Testing for QA professionals, published by Symantec Press, written with two Symantec security experts. I feel there is a need for such a book that's specifically geared towards the testing and QA professionals. Not enough is known about how to approach security testing as a QA professional, often it's just a band-aid and too late in the development life-cycle.
My latest article Teamwork Tackles the Quality Goal was written for Software Test and Performance magazine, see www.stpmag.com for a free download. A summary of one of my five “Software Test and Performance conference” (see www.stpcon.com) presentations “Introducing Automated Software Testing” recently appeared in the SDMagazine newsletter. I am also preparing for a presentation at the ICSTest (see www.icstest.com) conference in Duesseldorf, Germany in April, while working full-time as an employee, i.e. an internal SQA consultant to GSS, Symantec. (Please note all opinions expressed here are my own and not those of my employer.)
Q5. Do you think there is a gap between what testing consultants and trainers teach and what the testers and test managers in the trenches face in their daily work? In what ways?
Yes, I definitely see a gap in what testing consultants and trainers teach and what testers and test managers face in the trenches. Theories are nice to have, but some theories are just not realistic and too time consuming or expensive to implement. In the trenches we constantly face ever shrinking budgets and schedules, and go-live dates and production deadlines which can’t be moved, and often the development schedule slips and cuts into the testing time.
In the trenches, so much depends on the developers’ skills. The most efficient and knowledgeable testers cannot succeed if developers implement bad software and/or ineffective software development life-cycles and processes are in place. If testing is the only quality phase implemented as part of a QA realm, it can at most be considered a band-aid often too late in the system development lifecycle to make much of a quality difference. Testing is only one piece of the Quality puzzle.
I don't see this here at Symantec, but I have seen it at other companies where there is still this general confusion by some management about what testers do. The one side of the management camp thinks that “anyone” can do the testing and no special skills are required and the other side of the management camp expects testers to “test quality into a system” and blames the tester if a defect makes it into production. When it comes to testing salaries and job requirements, generally a tester is paid less and technically less is expected of him. Generally there is still a gap between testers’ technical knowledge vs. the developer technical knowledge. This gap also rears itself in still little or no respect for the tester profession and testing still often being seen as the underdog and necessary evil. Based on this general mindset, I wrote my article Teamwork Tackles the Quality Goal on “integrating the “independent” testing team where I make the point that the tester has to be every bit as technical as the developer, working integrated with the development team, while maintaining the independent testing frame of mind. The testing profession has so much to offer, but so little about how it can really effectively be implemented is known or used.
Q6. Who in the testing community has influenced your thinking most? Actually, my Computer Science professor Dr. Gustafson at Kansas State University has influenced my thinking about testing and quality in general. I took his graduate level course on "Quality software engineering processes" which peaked my interest in Quality and testing and I learned about great quality concepts and influential thinkers such as Tom DeMarco, Capers Jones, Boris Beizer, and Gerald Weinberg.
Q7. According to you what are top three things that one should learn/practice/be good at to be an effective and efficient software tester?
To be an efficient software tester the person really has to enjoy testing, have a knack for it, be analytical, detail oriented, and be versatile, such as a) be technically savvy, b) understand the business, and c) never stop learning and improving on the testing efforts
Q8. What do you think is the current state of automation? Where do you see automation heading towards? Do you think a paradigm shift is required in order to have more automation? Do you think there are other areas where tool vendors need to focus in addition to what they focus on right now?
Current state of automation. During a recent presentation at the Software Test and Performance conference (www.stpcon.com) I surveyed and learned that 50 percent of my audience had test automation tools that have become "shelf-ware." Those people are now tasked with resurrecting the automation tools and efforts, which can be an exercise in futility for many reasons. They will have to keep in mind whether the tool is still compatible with the technology being used, i.e. has the technology changed since the tool was purchased. Also, if the automation didn't succeed in the first go around did they note the reason for failure and have those "lessons learned" been evaluated so they wouldn't be repeated on the second automation attempt, etc.
This inofficial survey result is just another example that shows not much progress is being made with the use of vendor provided automated testing tools, an issue we've already discussed in our book "Automated Software Testing" published in 1999, "When inefficient automated testing implementation hasn't shown significant ROI, and budget pressures materialize, planned expenditures for test tool licenses and related tool support may be scratched. Often the tools end up as shelf-ware."
Q9. Where do you see automation heading towards? Granted vendor provided tools continue to improve, vendors still have and will continue to have a difficult time to keep up with all the latest and greatest technologies and third party controls and widgets, which are usually the source for the biggest compatibility issues and user frustrations. People will continue to struggle with the vendor provided tools, others will turn to in-house developed automation efforts or to automation freeware. Here are some important tips before "jumping on the test automation bandwagon."
• Know the types of tools available along with the problem they are trying to solve. There is no one best tool on the market.
• Safeguard the integrity of the automated test scrīpts by using a configuration management tool to baseline them.
• Consider that automated testing is software development.
• Educate stakeholders about the test automation efforts, especially developers who need to understand the impact any code changes could have on it.
• Manage the expectations by not overselling test automation's return on investment.
• Do not let automated testing become a "side activity" to work on when time allows; ideally use technical testing talent whose sole focus is on test automation.
• Not all testers need to be "automators," since development expertise is required.
• Decide whether to buy or build a tool or whether to use freeware (or all of the above)
Q10. Do you think a paradigm shift is required in order to have more automation? Unit test automation is the most effective defect detection/test automation technique. Too many companies still focus all of their test automation efforts on system testing without realizing that unit test automation can have the most ROI. Wishful thinking: Wouldn't it be nice if self-testable development languages existed, such as components are developed automated related unit tests were automatically self-generated.
Q11. In absence of licenses most people can’t learn using tools from most of the tool vendors. But employers generally want tools knowledge. How do you think testers can break this vicious circle and gain automation knowledge?
I think employers need to be educated to understand that someone with a development background can easily pick up any automated testing tool, no matter which one. When I hire automated testers and they can show extensive experience in one tool's scrīpting language or any scrīpting language for that matter, chances are they will be able to pick up and learn another tool's scrīpting language. Testers need to remember that automation is software development, and it is important to have experience in one or more programming or scrīpting languages.
文章来源于http://www.whatistesting.com/interviews/edustin.htm
