cucumber学习笔记4
4 problems in using Cucumber:
Leaky Scenarios：
Race Conditions and Sleepy Steps：
1、Race Conditions and Sleepy Steps
4、Bored Stakeholders
2、Imperative Steps：
3、Duplication
4、Ubiquitous What?
Symptom|Problem
1 Some of our tests fail randomly|Flickering scenarios
2 We keep breaking tests unintentionally|Brittle features
3 Our features take too long to run|Slow features
4 Our stakeholders don’t read our features|Bored stakeholders
1、Flickering scenarios：
A flickering scenario fails occasionally and at random. The same scenario,run on the same codebase in the same environment, will mostly pass but sometimes fail.
Flickering scenarios are normally caused by one of the following underlying problems:
• Shared Environments
• Leaky Scenarios
• Race Conditions and Sleepy Steps
Shared Environments：
If it’s even slightly awkward to install the system in a new environment, the likelihood is that when the team starts to automate their tests, they’ll follow the path of least resistance and point their test scripts at this existing system
test environment.
So the environment is shared between not only the human
members of the team but the test scripts too. Suppose a developer gets a bug report and wants to reproduce it for himself. He logs in to the system test environment and clicks a few buttons, but he doesn’t realize the automated tests are running at the same time. As part of the steps to reproduce the bug,the developer unwittingly deletes a database record that the automated test relied on, and the automated test fails. This kind of situation is a classic cause of flickering scenarios.
The shared use of a single environment can also contribute to unreliable tests by causing heavy and inconsistent load on in-demand resources like databases. When the shared database is under excessive load, normally reliable tests will time out and fail.
How to resolve shared environment problem?
One-Click system setup：
To avoid flickering scenarios that result from using shared environments, the team needs a setup script. that will create a new instance of the system from scratch, at the click of a button.
Leaky Scenarios：
When one scenario depends upon state left behind by another earlier scenario in order for it to pass, you’ve created a dependency between the two scenarios.
When the state of the system is not reset between tests, we say that they allow state to leak between them. This is a major cause of brittle tests.
how to resolve？
independent scenarios ensures they put the system into a clean state and then add their own data on top of it. This makes them able to stand on their own, rather than being coupled to the data left behind by other tests or shared fixture data.
Test Data Builders：
FactoryGirla gem
Given /^I have been paid$/ do
Factory :pay_check
end
Given /^I have been paid 50 dollars$/ do
Factory :pay_check, :amount => 50
end
Race Conditions and Sleepy Steps：
If you write end-to-end integration tests for a reasonably complex system,you’ll eventually encounter this problem. Race conditions occur when two or more parts of the system are running in parallel, but success depends on a particular one of them finishing first.
In the case of a Cucumber test, your When step might cause the system to start some work that it runs in the background, such as generating a PDF or updating a search index. If this background task happens to finish before Cucumber runs your Then step, the scenario will pass. If Cucumber wins the race and the Then step executes before the background task is finished, the scenario will fail.
How to resolve？
a、use sleep steps
Just reduce the probability to fail,and it will make sceneraio slower,not a good choice.
b、Dealing with Message Queues and Asynchronous Components
to be learned in Chapter 9
2、Brittle Features
A necessary change in one part of the test suite or main codebase causes apparently unrelated scenarios to break.
Brittle features are normally caused by one of the following underlying
problems:
• Fixture Data
• Duplication
• Leaky Scenarios
• Tester Apartheid
1、Fixture Data：
A set of fixture data, even if it starts out relatively lean, will only tend to grow in size over time. As more and more scenarios come to rely on the data, each needing their own specific little tweaks, the size and complexity of the fixture data grows and grows.
You’ll start to feel the pain of brittle features when you make a change to the data for one scenario, but that change causes other scenarios to fail.
Resolution：Test Data Builders
2、Duplication：
Resolution：
a、Gherkin has the Background and Scenario Outline keywords you can use to reduce duplication.
3、Leaky Scenario
4、Tester Apartheid
When testers are left alone to build their own Cucumber tests,
they may lack the software engineering skill to keep their step definition and support code well organized.
Resolution：
Encouraging programmers and testers to work together when writing step definition and support code. The programmers can show the tester how to keep the code organized and factor out reusable components or libraries that other teams can use.
3、Slow Features
A slow feature run is normally caused by a combination of the following:
• Race Conditions and Sleepy Steps
• Lots of Scenarios
• Big Ball of Mud
1、Race Conditions and Sleepy Steps
2、Lots of Scenarios
Resolution：
a、you can keep your features organized using subfolders and tags.
b、some scenarios could be pushed down and expressed in fast unit tests instead.
3、Big Ball of Mud
The Big Ball of Mud is an ironic name given to the type of software design you see when nobody has really made much effort to actually do any software design. In other words, it’s a big, tangled mess.
4、Bored Stakeholders
This painful symptom is normally caused by a combination of the following underlying problems:
• Incidental Details
• Imperative Steps
• Duplication
• Ubiquitous What?
• Siloed Features
1、incidental Details：
Details that are mentioned in the scenario but that actually have no relevance to the
purpose of the scenario. This kind of irrelevant detail makes the scenario
harder to read.
Scenario: Check inbox
Given a User "Dave" with password "password"
And a User "Sue" with password "secret"
And an email to "Dave" from "Sue"
When I sign in as "Dave" with password "password"
Then I should see 1 email from "Sue" in my inbox
There is a lot of detail in this scenario,the passwords are just noise: the passwords of the users have nothing to do with what’s being tested and in fact are making it harder to read. For example, Sue has a different password than Dave. As you read the scenario, wondering whether this is relevant, you’re distracted from the main point of the scenario: to check that Dave can see Sue’s email.
2、Imperative Steps：
Scenario: Redirect user to originally requested page after logging in
Given a User "dave" exists with password "secret"
And I am not logged in
When I navigate to the home page
Then I am redirected to the login form
When I fill in "Username" with "dave"
And I fill in "Password" with "secret"
And I press "Login"
Then I should be on the home page
they’re easy to break: if the user-experience people decided to change the wording on the submit button from Login to Log in, the scenario will fail, for no good reason at all.
-->
Let’s raise the level of abstraction in this scenario and rewrite it using a more
declarative style.:
Scenario: Redirect user to originally requested page after logging in
Given I am an unauthenticated User
When I attempt to view some restricted content
Then I am shown a login form
When I authenticate with valid credentials
Then I should be shown the restricted content
3、Duplication
4、Ubiquitous What?
The point is to keep everyone on the team to use the same words,everywhere
5、Siloed Features
When testers and developers tuck their features away in source control, the rest of the team can feel as though their documentation had been locked away in a cupboardto which they don’t have the keys.
Relish is a service that was created by members of the Cucumber and RSpec
teams to provide an easy way to publish Cucumber features as documentation.
TAG:
