minitest-parallel_fork adds fork-based parallelization to Minitest. Each test/spec suite is run in one of the forks, allowing this to work correctly when using before_all/after_all/around_all hooks provided by minitest-hooks. Using separate processes via fork can significantly improve spec performance when using MRI, and can work in cases where Minitest's default thread-based parallelism do not work, such as when tests/specs modify the constant namespace.
gem install minitest-parallel_fork
Source code is available on GitHub at github.com/jeremyevans/minitest-parallel_fork
You can enable fork-based parallelism just by requiring
minitest/parallel_fork. One easy to do so without modifying the spec code itself is to use the
RUBYOPT environment variable. So if you execute your specs using:
You can switch to fork-based parallelism using:
RUBYOPT=-rminitest/parallel_fork rake spec
To control the number of forks, you can set the
NCPU environment variable:
NCPU=8 RUBYOPT=-rminitest/parallel_fork rake spec
If you don't set the
NCPU environment variable, minitest-parallel_fork will use 4 forks by default.
In some cases, especially when using external databases, you'll need to do some before fork or after fork setup. minitest/parallel_fork supports
before_parallel_fork is called before any child processes are forked:
. do DB.disconnect end
after_parallel_fork is called after each child process is forked, with the number of the child process, starting at 0:
. do |i| DB.opts[:database] += (i+1).to_s end
The above examples show a fairly easy way to use minitest-parallel_fork with an external database when using Sequel. Before forking, all existing database connections are disconnected, and after forking, the database name is changed in each child to reference a child-specific database, so that the child processes do not share a database and are thus independent.
There is also a hook for debugging.
on_parallel_fork_marshal_failure is called if there is an error unmarshalling data sent from the child process to the parent process. This can happen if one of the child processes exits unexpected during the test, before it reports results.
. do # Gather relevant logs for more debugging end
To use this with Rails/ActiveRecord, you probably want to use hooks similar to:
. do ActiveRecord::Base.connection.disconnect! end . do |i| db_config = Rails.application.config.database_configuration[Rails.env].clone db_config['database'] += (i+1).to_s ActiveRecord::Base.establish_connection(db_config) end
The speedup you get greatly depends on your specs. Here's some examples using Sequel's specs:
2 forks 4 forks spec_core: 1.25x - 1.36x 1.5x spec_model: 1.29x - 1.62x 1.72x - 2.02x spec_plugin: 1.57x - 1.76x 2.29x - 2.37x spec_sqlite: 1.75x - 1.86x 2.26x - 2.65x spec_postgres: 1.32x - 1.40x Untested
Jeremy Evans <[email protected]>