软件测试

software test

Today, most mature companies treat software testing as a technical engineering profession. They recognize that having trained software testers on their project teams and allowing them to apply their trade early in the development process allows them to build better quality software. Unfortunately, there are still a few companies that don't appreciate the challenge of software testing and the value of well-done testing effort. In a free market society, these companies usually aren't around for long because the customers speak with their wallets and choose not to buy their buggy products. A good test organization (or the lack of one) can make or break a company.

Software testing is a critical job. With the size and complexity of today's software, it's imperative that software testing be performed professionally and effectively. Too much is at risk. We don't need more defective computer chips, crashed systems, or stolen credit card numbers.

To be an effective software tester, it's important to have at least a high-level understanding of the overall process used to develop software. If you write small programs as a student or hobbyist, you'll find that the methods you use are much different from what big companies use to develop software. The creation of a new software product may involve dozens, hundreds, even thousands of team members all playing different roles and working together under tight schedules. The specifics of what these people do, how they interact, and how they make decisions are all part of the software development process.

Testers or QA (Quality Assurance) Staff are responsible for finding and reporting problems in the software product. They work very closely with all members of the team as they develop and run their tests, and report the problems they find.

One key concept that software testers need to learn is how to reduce the huge domain of possible tests into a manageable set, and how to make wise risk-based decisions on what's important to test and what's not.

If you attempt to test everything, the costs go up dramatically and the number of missed bugs declines to the point that it's no longer cost effective to continue. If you cut the testing short or make poor decisions of what to test, the costs are low but you'll miss a lot of bugs. The goal is to hit that optimal amount of testing so that you don't test too much or too little.

The goal of a software tester is to find bugs.

You may run across product teams who want their testers to simply confirm that the software works, not to find bugs. Reread the case study about the Mars Polar Lander, and you'll see why this is the wrong approach. If you're only testing things that should work and setting up your tests so they'll pass, you will miss the things that don't work. You will miss the bugs.

If you're missing bugs, you're costing your project and your company money. As a software tester you shouldn't be content at just finding bugs you should think about how to find them sooner in the development process, thus making them cheaper to fix.

The goal of a software tester is to find bugs and find them as early as possible.

But, finding bugs, even finding them early, isn't enough. Remember the definition of a bug. You, the software tester, are the customer's eyes, the first one to see the software. You speak for the customer and must seek perfection.

The goal of a software tester is to find bugs, find them as early as possible, and make sure they get fixed.

It's important to note that "fixing" a bug does not necessarily imply correcting the software. It could mean adding a comment in the user manual or providing special training to the customers. It could require changing the statistics that the marketing group advertises or even postponing the release of the buggy feature. You'll learn throughout this book that although you're seeking perfection and making sure that the bugs get fixed, that there are practical realities to software testing. Don't get caught in the dangerous spiral of unattainable perfection.

There are several reasons why you might choose not to fix a bug:

There's not enough time. In every project there are always too many software features, too few people to code and test them, and not enough room left in the schedule to finish. If you're working on a tax preparation program, April 15 isn't going to move you must have your software ready in time.

It's really not a bug. Maybe you've heard the phrase, "It's not a bug, it's a feature!" It's not uncommon for misunderstandings, test errors, or spec changes to result in would-be bugs being dismissed as features.

It's too risky to fix. Unfortunately, this is all too often true. Software can be fragile, intertwined, and sometimes like spaghetti. You might make a bug fix that causes other bugs to appear. Under the pressure to release a product under a tight schedule, it might be too risky to change the software. It may be better to leave in the known bug to avoid the risk of creating new, unknown ones.

It's just not worth it. This may sound harsh, but it's reality. Bugs that would occur infrequently or bugs that appear in little-used features may be dismissed. Bugs that have work-arounds, ways that a user can prevent or avoid the bug, are often not fixed. It all comes down to a business decision based on risk.

As a new tester, you might believe that you can approach a piece of software, fully test it, find all the bugs, and assure that the software is perfect. Unfortunately, this isn't possible, even with the simplest programs, due to four key reasons:

The number of possible inputs is very large.

The number of possible outputs is very large.

The number of paths through the software is very large.

The software specification is subjective. You might say that a bug is in the eye of the beholder.

Multiply all these "very large" possibilities together and you get a set of test conditions that's too large to attempt.

To be an effective software tester, it's important to have at least a high-level understanding of the overall process used to develop software. If you write small programs as a student or hobbyist, you'll find that the methods you use are much different from what big companies use to develop software. The creation of a new software product may involve dozens, hundreds, even thousands of team members all playing different roles and working together under tight schedules. The specifics of what these people do, how they interact, and how they make decisions are all part of the software development process.

there are many different methods that can be used for developing software, and no model is necessarily the best for a particular project. There are four frequently used models, with most others just variations of these:

Big-Bang

Code-and-Fix

Waterfall

Spiral

In most cases, there is little to no formal testing done under the big-bang model. If testing does occur, it's squeezed in just before the product is released. It's a mystery why testing is sometimes inserted into this model, but it's probably to make everyone feel good that some testing was performed.

testing isn't specifically called out in the code-and-fix model but does play a significant role between the coding and the fixing.

As a tester on a code-and-fix project, you need to be aware that you, along with the programmers, will be in a constant state of cycling. As often as every day you'll be given new or updated releases of the software and will set off to test it. You'll run your tests, report the bugs, and then get a new software release. You may not have finished testing the previous release when the new one arrives, and the new one may have new or changed features. Eventually, you'll get a chance to test most of the features, find fewer and fewer bugs, and then someone (or the schedule) will decide that it's time to release the product.

From a testing perspective, the waterfall model offers one huge advantage over the other models presented so far. Everything is carefully and thoroughly specified. By the time the software is delivered to the test group, every detail has been decided on, written down, and turned into software. From that, the test group can create an accurate plan and schedule. They know exactly what they're testing, and there's no question about whether something is a feature or a bug.

If you're a tester, you'll like Spiral model. You'll get a chance to influence the product early by being involved in the preliminary design phases. You'll see where the project has come from and where it's going. And, at the very end of the project, you won't feel as rushed to perform all your testing at the last minute. You've been testing all along, so the last push should only be a validation that everything is okay.

Here's a list of the more important test deliverables:

The test plan describes the overall method to be used to verify that the software meets the product specification and the customer's needs. It includes the quality objectives, resource needs, schedules, assignments, methods, and so forth.

Test cases list the specific items that will be tested and describe the detailed steps that will be followed to verify the software.

Bug reports describe the problems found as the test cases are followed. These could be done on paper but are often tracked in a database.

What exactly is a software product? Many of us think of it as simply a program that we download from the Internet or install from a DVD that runs on our computer. That's a pretty good descrīption, but in reality, many hidden pieces go into making that software. There are also many pieces that "come in the box" that are often taken for granted or might even be ignored. Although it may be easy to forget about all those parts, as a software tester, you need to be aware of them, because they're all testable pieces and can all have bugs.

DON'T FORGET TO TEST ERROR MESSAGES.

Error messages are one of the most overlooked parts of a software product. Programmers, not trained writers, typically write them. They're seldom planned for and are usually hacked in while fixing bugs. It's also very difficult for testers to find and display all of them. Don't let error messages such as these creep into your software:

Error: Keyboard not found. Press F1 to continue.

Can't instantiate the video thing.

Windows has found an unknown device and is installing a driver for it.