Here we don't discuss what Test plan should  include, we just talk how to organize feature test in test plan.

I think the better way to organize is:

1. Do summarize introduction of the Moudle function
2. Break down features one by one
3. Design scenarios by feature one by one
4. Integrate scenarios for the module features(this may be optional)

• Error handling
• Feature 1 and feature 2 integrated scenarios

Automating what your test, this is our final goal. In my mind, if the thing can be done by machine, ok, let machine handle it. So if the new feature can be automated in short time, congratulations! Just do it.

Sometimes the automation work can not be done is short time, that's ok, we can make some semi-automation. even the semi-automation case, it can release a lot of efforts in testing. If we have time in feature testing or after feature testing, just continue your automation work.

Before automation, please make sure you are confident with your design, if you are not ready with the design, please don't start your automation work, or it will be hell for you to maintennance in the future.

What ablility need I have? Below I list some suggestion:

• Domain Expert
• Be familiar with one or two scrīpt language
• Be good at thinking

Verifying test result, it is our QA major task. Normally we veryify the test result by manual. It's very dull, more we often miss some issues to LTS. How can we do better with this. Making a tool to help us do this job, machine doesn't feel tired, also machine doesn't make mistake(it should be tested).

Below I will give an example. In our company almost every two weeks we need to do regression test for the old feature. unfortunately what we regress is checking the emails, and we need to check almost twenty emails. if we check email by manual, oh my god, it is almost impossible. But before I come to this domain, our QAs do this job by manual. I am sick of do the text checking day by day, so I decide to change this status. First we do some research and find we can leverage open source scrīpt ruby to help us do this job. Fortunately we have it done. Look, we needn't to do this dull work by manual, let the machine to help you.  

Before testing, I would like to think about the below three tools, whether can I create tools to help generate test data, whether can I create tools to help do data verification, whether can I leverage some automation cases to help test. Besides, I also think more about how much time there tools will take, normally the time on creating these tools should be very little, 2~3 days.

1. Tools for generating test data
2. Tools for Verifying test data
3. Tools for automating business flow.

Why need we think about the three tools, definitely, it can save a lot of test efforts for us. If we can make the three tools happen, then we can do a lot of ad hoc test using the saving time. Also the three tools improve test effectiveness and efficency.

Generating test data, it is very useful in system interface testing. for example, there is new system B, system B entry depends on system A result. under this condition, we can create tool to generate system A result, instead of going though system A to generate test data.

Note, even we have such generating test data tool, at last, we still do go through system A to generate some test data, so that we can cover end to end test.

To be continued...

Today, I have discussion about whether you can be replaced or not with an intern.

I think as a test engineer, we must have our specific character. In testing, I think two things are very important:

First, domain knowledge. we should at least master one or two domains.

Second, testing experience that is testing ablility. it includes programming ablility, test methodology, communication skills, common sense in your field etc.

What is Automation upsteam?

Upstream means push up. Normally automation testing will be taken after feature testing. Here we will push automation testing in feature testing. That is, we leverage automation cases to help us cover feature testing.

And this is not new concept, in another word, it is more like automation testing in application. let's me give you a sample, sometimes we generate some tool to generate test data, this we can regard it as semi- automation.

Then How to automate upstream? Below are some criteria:

First, you have set up an automation library, if new features come, what you do is just to set up the automation case by filling different module together.

Second, you must be domain expert. You should be very familiar with the domain. This is very important.

If the both criterias are meet, believe automation upstream is an easy work to do.

Below I will introduce some best practice.

I have set up an automation library, it contains almost more than 60 cases and 20 utilities. One day a new feature comes, after I analysis I find, one part feature I can take directly the automation cases to cover; the rest part, I just set up new cases by push some utilities together, and add some DB query.

Look, it is too simple.

Automation Upstream is not a dream. As I know in Microsoft, most of project, they takes this way to help do feature testing.

之前曾经开过一个blog, 后来没有坚持下来. 再后来, 就荒废了.

想想真的是可惜.

时间如梭,应该留点痕迹.就当写些成长日记吧.

测试也蛮长时间了,平常经常有些想法,苦于没有地方log. 就从这开始了.