-
Notifications
You must be signed in to change notification settings - Fork 115
Parallel Tests
'Platform' here means a Sauce Labs platform 'Sauce Concurrency' here means a concurrent test slot on Sauce Labs' infrastructure. Each account has a limited number of concurrent test slots.
parallel_tests is a sweet gem that lets you run your specs, tests or features in parallel. It's highly configurable and takes away a lot of the concurrency pain on your behalf. You can use the parallel_tests gem as it stands.
The Sauce gem also integrates a modified version of the parallel_tests gem, to take full advantage of the concurrency and multiple platform support Sauce Labs provides. These are accessibly through Rake tasks.
##The Differences
The Sauce integration is currently targeted at RSpec and Cucumber, running on a local server you can spin up multiple copies of. It runs a copy of each test for each platform, and divides them up across all the concurrency available to your Sauce Labs account by default
The parallel_tests gem allows you to run TestUnit, RSpec and Cucumber, running on a local server you can spin up multiple copies of. It divides your test files across the cores available to your local machine, by default. It then runs each test file across each platform in serial.
For example, imagine you have 3 specs: "Alpha", "Beta", "Delta". Your local machine has 4 cores, and you've 4 concurrent test slots.
You've specified 4 platforms: 1. Chrome on Win7 2. Chrome on Linux 3. Firefox 18 on Win 8 4. Firefox 18 on OS X
| Tool | Core 1 | Core 2 | Core 3 | Core 4 | Sauce Concurrencies Used |
|---|---|---|---|---|---|
| parallel_tests | Alpha on 1, Alpha on 2, Alpha on 3, Alpha on 4 | Beta on 1, Beta on 2, Beta on 3, Beta on 4 | Delta on 1, Delta on 2, Delta on 3, Delta on 4 | 3 | |
| sauce parallel tasks | Alpha on 1, Beta on 1, Delta on 1 | Alpha on 2, Beta on 2, Delta on 3 | Alpha on 3, Beta on 3, Delta on 3 | Alpha on 4, Beta on 4, Delta on 4 | 4 |
Running parallel Cucumber and Capybara based tests on top of Sauce is very easy. This document covers using the parallel_tests gem for running multiple Cucumber features at once.
The parallel_tests gem provides two executables, parallel_rspec and parallel_cucumber, which use the gem's parallelization magic to invoke rspec or cucumber respectively.
If for example, you had a dual core computer, and a features/ directory with 10 .feature files in it, you could run:
% bundle exec parallel_cucumber features
And the command would run a two Cucumber processes at the same time, each executing one of the 10 feature files.
In order to run parallel tests effectively, your test code must be able to run at the same time as other tests.
- Avoid trying to run parallel tests on parallel browsers, instead make sure your tests pass on one browser first. When testing with Sauce, Chrome is the fastest browser, so if you run your tests with Chrome first then you can confidently say that the app isn't fundamentally broken before spending more time running the tests on slower browsers.
- Avoid forcing your tests to start up a server process, not only does this slow things down, but it also increases the likelihood that your tests will try to start a server at the same time and conflict. Instead use a remote staging environment, ideally one that closely represents production and can handle many clients simultaneously
- Avoid shared or hard-coded data in your tests, if you need an email address or some other form of fake data, generate it yourself (check out the Faker gem).