Paraspec

Paraspec is a parallel RSpec test runner.

It is built with a producer/consumer architecture. A master process loads the entire test suite and sets up a queue to feed the tests to the workers. Each worker requests a test from the master, runs it, reports the results back to the master and requests the next test until there are no more left.

This producer/consumer architecture enables a number of features:

  1. The worker load is naturally balanced. If a worker happens to come across a slow test, the other workers keep chugging away at faster tests.
  2. Tests defined in a single file can be executed by multiple workers, since paraspec operates on a test by test basis and not on a file by file basis.
  3. Standard output and error streams can be[*] captured and grouped on a test by test basis, avoiding interleaving output of different tests together. This output capture can be performed for output generated by C extensions as well as plain Ruby code.
  4. Test results are seamlessly integrated by the master, such that a parallel run produces a single progress bar with Fuubar across all workers.

[*] This feature is not yet implemented.

Usage

For a test suite with no external dependencies, using paraspec is trivially easy. Just run:

paraspec

To specify concurrency manually:

paraspec -c 4

To pass options to rspec, for example to filter examples to run:

paraspec -- -e 'My test'
paraspec -- spec/my_spec.rb

For a test suite with external dependencies, paraspec sets the TEST_ENV_NUMBER environment variable, like parallel_tests does. The test suite can then configure itself differently in each worker.

By default the master process doesn't have TEST_ENV_NUMBER set. To have that set to 1 use --master-is-1 option to paraspec:

paraspec --master-is-1

Advanced Usage

Formatters

Paraspec works with any RSpec formatter, and supports multiple formatters just like RSpec does. If your test suite is big enough for parallel execution to make a difference, chances are the default progress and documentation formatters aren't too useful for dealing with its output.

I recommend Fuubar and RSpec JUnit Formatter configured at the same time. Fuubar produces a very nice looking progress bar plus it prints failures and exceptions to the terminal as soon as they occur. JUnit output, passed through a JUnit XML to HTML converter like junit2html, is much handier than going through terminal output when a run produces 100 or 1000 failing tests.

Debugging

Paraspec offers several debugging aids. The first one is the terminal option:

paraspec -T

This option makes paraspec stay attached to the terminal it was launched in, making it possible to insert e.g. byebug calls in supervisor, master or worker code as well as anywhere in the test suite being executed and have byebug work. Setting this option also removes internal timeouts on interprocess waits and sets concurrency to 1, however concurrency can be reset with a subsequent -c option:

paraspec -T -c 2

Paraspec can produce copious debugging output in several facilities. The debugging output is turned on with -d/--debug option:

paraspec -d state   # supervisor, master, worker state transitions
paraspec -d ipc     # IPC requests and responses
paraspec -d perf    # timing & performance information

Bugs & Patches

Please report via issues and pull requests.

License

MIT