Scott Barber - CTO of performance Test Plus
上一篇 / 下一篇 2011-02-28 14:42:26 / 个人分类：test performance
经常在SoftwareTest & Performance 上看到Scott Barber，今天看了他的采访内容。很了不起的一个性能测试技术出身的CTO，先看一直困惑我的Load Test和Stress Test的区别。
Loadtestingistestingyour application underexpectedload conditions to determine the overall performance.
Stresstestingistestingyour application underunexpectedorextremeconditions to determine thefailuremodes so you can protect aginst them.
I think stresstestingis a run-to-death test.
http://www.scottbarber.com/
Come Fromhttp://www.whatistesting.com/interviews/sbarber.htm
| 1. Scott, please tell us something about yourself. | I was borninWoodbury, NJ USAin1971 and grew upina typical American small town. As a child I was very activeinschool, sports and Boy Scouts. Growing up, the goals I had for myselfinadulthood included:
I spent 4 successful, but difficult yearsinthe U.S. Army culminating as a Company Commander for the 101st Airborne Division, Air Assault Headquarters and Headquarters Company approximately 8 years before I was technically qualified to hold the position. As a junior Captain at the end of my 4 years, I was recruited into a government consulting company to be an Information Engineer on a team that was designing and building a system to replace the computer system I most often complained about while I was on active duty. I held this position for about 1? years while becoming increasingly more technical before I went to a small privatesoftwaredevelopment company with the intent of becoming an Oracle Database Administrator, but actually ended up spending 6 months as a Configuration Manager. During that time, Ilearneda ton about Databases, Configuration Management, development and networking. Most importantly, Ilearnedthat none of those were what I wanted to do dayinand day out. 6 months later, that company was bought by a huge, publicly traded conglomerate that resultedina significant reductioninpay, leading me to seek a new job. This is when I was recruited into a Performance Test Consulting job by an old college friend where I spent the next 4 years. This time as a consultant is where most of my performancetestingexperience comes from. After 4 years of consulting, my family and I decided it was time to move. As it turned out, we moved to Florida about the same time the consulting company went through a major re-organization leading me to re-think what I wanted to do nextinmy career. After a long and difficult 4 months of soul searching, I ended up taking a wonderful position as a Systems Test Engineer andSoftwareTest Manager at my current company AuthenTec, Inc. As far as my childhood goals are concerned, I have had a chance to travel more than I ever thought I would, but have plenty left to do. I achieved my educational goals, but have a passive interestinearning a Doctorate degree someday. After livingina wide variety of places, I have finally settledina place that feel like “home” and am very content with my career progression so far. I have two young boys who are the greatest joyinmy life. As I write this, it is with thoughts of my older boy starting school tomorrow morning and my youngest learning to talk. I guess that means that the only childhood goal I have left is to design and build that house. Luckily, I continue adding goals, so I’m far from bored. |
2. You specializeinperformancetesting. What drew you to it? How did you get started, and what led you to start writing about it? | One Sunday evening I got a call from my manager… “Scott, be at the Marriott Hotel Conference Room at 8am tomorrow. Our CEO has an announcement to make.” Being a good employee, I did as I was told and went to the meeting. As it turns out, oursoftwaredevelopment company was “merging” with a huge media company. We were going to become their “development branch”. As a result, we were giving up our bonuses, overtime, and our base pay was being reduced, but we were going to get stock options. As one might imagine, I was less than enthusiastic about this. I called my friend of 10 years before I even left the parking lot of the hotel. I told him the circumstances and asked him if he’d help me update my resume, to which he responded: “Dude, that sucks. Send me your resume I’ll take a look, but, we need performance engineers. You’d be a perfect fit.” I said “Performance engineer? What’s that?!?” He replied “Don’t worry, you’ll like it.” He was right, I’ve loved every minute of it! Since then I have dedicated my professional career to this field. It has been a wonderful ride so far. After a few years, people started approaching me to write about what I do. I never though of myself as much of a writer, so I hesitated and stalled. (Mind you, I was the Civil Engineerincollege who avoided writing classes at ALL COSTS. When they finally told me that if I didn’t take a writing course I wouldn’t graduate, I went to the smallest, least reputable, community college I could find, took the course and transferred it!) Eventually, I agreed to create the Performance Engineering section of my company’s Best Practices. Some folks thought it was good and started selling me to other companies to write their procedures. Finally, my boss talked me into trying my hand at writing articles. I really didn’t think anyone would read them, but I wrote a few anyway. As it turns out, people did read them, and liked them. |
| 3. What fascinates youinperformancetesting? | Most simply, I am fascinated by the diversity of it. As my meandering career path let me to this field, I had a lot of “false starts”inother branches ofsoftwaredevelopment, Ilearnedthat regardless of my degree of successinthose more narrow branches, I found myself becoming easily bored. As a technical tester, specifically as a performance tester of multi-user distributed systems, I get to apply aspects of all of those branches and moreinnew and unique ways on every project I do. This diversity from project to project is also what led me to embrace the Context-Driven School ofsoftwareTestingand ultimately become friends with suchsoftwaretestingthought leaders as Cem Kaner and James Bach. |
4. If one wants to gain expertiseinperformancetestingwhat all should one do? What are the tools, books and sites that can be helpful? | There is very little consolidated information out there about the type of performancetestingthat most organizations need. There is no doubt that my site (http://www.perfetsplus.com) and articles are one of the most complete references for “Practical PerformanceTesting”. However, it is by no means complete. Some of the other places I havelearnedvaluablelessonsthat I have applied to performancetestingare:
To really grow your career as a performance tester, I believe there are several things you can do that will gain you largest benefit.
|
5. What is your opinion about the state of performancetestingtools? Are there some open source or free tools available? | Most tools on the market aren’t actually PerformanceTestingtools, they are Load Generation tools. No matter what the tool vendors claim, they do not make performancetestingsimple, they do not analyze results, and they do not pinpoint performance issues. They are virtually all grossly overpriced, and never believe the demo. That said, I have a favorite tool that I support heavily… with a long list of caveats. My favorite is my favorite because it allows me to create realistic loads by writing custom functions and proceduresinC – rather than trying to guess how to make the silly GUI buttons duplicate the workload I have constructed. The best OpenSource tool for general web based load generation is OpenSTA. As long as you are onlytestingwebsites, it is competitive with the major (pay) toolsinterms of actual ability to generate load and collect data. There are plenty of other free niche tools for specific platforms and protocols, but they are only useful to small groups of people doing very specific types oftesting. |
| 6.How do you combine performancetestingwith load and stresstesting | Actually, I see PerformanceTestingas a superset oftestingthat includes load and stresstesting. I often refer to it as “Performance RelatedTesting”. Ingeneral, loadtestingistestingyour application under expected load conditions to determine the overall performance. Stresstestingistestingyour application under unexpected or extreme conditions to determine failure modes so you can protect against them. Inmy presentation “Introduction to PerformanceTesting; The Who, What, Where, When and Why.” (availablehere) I describe various components of PerformanceTesting, and the processingeneral. |
7. What is WOPR? How did it happen? Who all are involvedinit and what is your role? | WOPR is the Workshop On Performance and Reliability (http://www.performance-workshop.org). Ross Collard and I co-founded this workshop about a year ago with the purpose of bringing together both PerformanceTestingExperts and “Enthusiastic Beginners”ina forum where we can share our experiences togetherina format that encourages questions, critical thinking and learning. So far, we have had three very successful workshops and plan to continue having theminthe spring and fall of every year. You can find out more from the website. The workshop is limited to 20 people per meeting and is by invitation only, but you can apply for an invitation by filling out the form. on the website. |
| 8. How is performancetestingfor consumer applications different from enterprise/server based applications? | There is actually less difference between consumer applications and enterprise/server based applications than there can be between any two individual applicationsinthe same category. The real difference centers around performance expectations. One of the keys to effective performancetestingis the ability to collect and classify requirements and then develop tests that verify those requirements. |
9. Where do you think performancetestingfitsinthetestinglife cycle? Should it be done early or should it be done late? What are the pros and cons and how do you decide? | There are actually at least 3 separate, but highly related, disciplines related to performance that should ideally be includedintheSoftwareDevelopment Lifecycle –SoftwarePerformance Engineering (SPE), PerformanceTesting(what we have been talking about) and Capacity Planning. The only book I am aware of that addresses these three discipline as part of an integrated process is Improving .NET Application Performance and Scalability, Microsoft Press, 2004. Inthis book, Senior Author J.D. Meier does a fantastic job of taking the reader through the entire process. The only drawback is that much of the book is extremely Microsoft/.NET specific. Performance Test planning should actually begin before the architecture has been determined or any code has been written. It is especially important to finalize performancetestingplanning before any machines have been configured fortestingand the plan should continue to be revised as necessary until the application has reached it’s target loadinproduction several times. Very few organizations do this. Most organizations create a plan shortly before load generation is scheduled to commence and stick to the letter of the plan regardless of the actual results or detected issues. By working this way, much of the value that can be gained through performancetestingis lost. The bottom line is this, PerformanceTestinglateinthe process alone, as most organizations do, can be highly effective, but only if all identified issues are relatively easy to resolve. Saving thetestingfor the end of the development cycle can be disastrous if the identified issues point back to poor architectural or design decisions. |
| 10. You have written a lot of papers on performancetesting. Do you plan to write a book on this subject? | I have written quite a number of papers, presentations and tried to tie them all together on my website. When I started doing performancetesting, I found very little written on the topic and had to learn through trial and error, seeking out mentors and applying principles from other disciplines. I became frustrated with the lack of valuable information available and found many others who were looking to improve their performancetesting. As a result I tried to share what Ilearnedthrough these mediums so that others will have an easier time getting started than I did. I have tentative plans for two books. The first is simply a consolidation and expansion of my existing articles. There are two main reasons that I would like to do this. First, I’d like to retrofit the articles to be even more generally applicable, include examples and code samples for various tools since they were initially written on contract for IBM Rational. The second reason is because many readers have requested hard copies, which is unfortunately prohibitively expensive on a case-by-case basis. While there are some very good books on performance tuning, and other performance related topics and some very good chaptersinvarioustestingbooks about performancetesting, there is no single “good” book for practical performancetestinglessons. Writing a book is much more difficult than writing a book’s worth of articles, but I keep hoping that I’ll find the time, energy, resources and support to get it doneinthe not too distant future. While I certainly don’t think I know everything there is to know about performancetesting, I do think sharing what I do know would be helpful to many performance testers both now andinthe future. |
11. What are the things you are currently working on? | Outside of the books and some “clean-up” articles, I have actually movedina new direction with my new job. I am nowincharge oftestingfingerprint sensors and their associatedsoftware(API, drivers, etc) on both PC and embedded platforms. This is extremely challenging as thetestingis a blend of four very different industries (Hardware,Software, Embedded/RealTime and Biometrics) that, to the best of my knowledge, has never been written about and as far as I can tell has rarely, if ever, been tried. The knowledge I have gained as a Performance Tester/Analyst has been extremely helpfulinthe endeavor, as has my relationships with various industry experts such as Cem Kaner, James Bach, and Alan Jorgensen, not to mention my EmbeddedSoftwareDevelopment Instructor friends at MIT. |
TAG:
标题搜索
日历
|
|||||||||
| 日 | 一 | 二 | 三 | 四 | 五 | 六 | |||
| 1 | 2 | 3 | 4 | 5 | 6 | ||||
| 7 | 8 | 9 | 10 | 11 | 12 | 13 | |||
| 14 | 15 | 16 | 17 | 18 | 19 | 20 | |||
| 21 | 22 | 23 | 24 | 25 | 26 | 27 | |||
| 28 | 29 | 30 | |||||||
我的存档
数据统计
- 访问量: 2933
- 日志数: 7
- 建立时间: 2010-09-27
- 更新时间: 2011-09-07
