Class: CIRunner::Runners::MinitestRunner

Inherits:
Base
  • Object
show all
Defined in:
lib/ci_runner/runners/minitest_runner.rb

Overview

Runner responsible to detect parse and process a CI output log generated by Minitest.

Because minitest doesn’t have a CLI built-in, there is a lot of complications to re-run a selection of tests, especially when we need to run the tests in a subprocess.

In a nutshell, this Runner will try its best to get the file path of each failures. Without a custom repoter, Minitest will fail pointing to the right file a lot of the time. Therefore CI Runner tries a mix of possibilities (by looking at the stack trace, inferring the class name).

Once the logs have been parsed, we tell Ruby to load only the test files that failed. By default, loading those files would make Minitest run all tests included in those files, where we want to run only tests that failed on CI.

Minitest doesn’t have a way to filter by test name (the -n) isn’t powerful enough as two test suites can contain the same name. CI Runner launches a DRB server over a UNIX socket which allows allows the subprocess running minitest to know whether a test should ran. This is accomplished in combination with a Minitest plugin.

A vanilla rake task plugs the whole thing in order to not reinvent the wheel.

Constant Summary collapse

SEED_REGEX = 
Regexp.union(
  /Run options:.*?--seed\s+(\d+)/, # Default Minitest Statistics Repoter
  /Running tests with run options.*--seed\s+(\d+)/, # MinitestReporters BaseReporter
  /Started with run options.*--seed\s+(\d+)/, # MinitestReporters ProgressReporter
)
BUFFER_STARTS = 
/(Failure|Error):\s*\Z/

Instance Attribute Summary

Attributes inherited from Base

#failures, #gemfile, #ruby_version, #seed

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

#initialize, #parse!, #report

Constructor Details

This class inherits a constructor from CIRunner::Runners::Base

Class Method Details

.match?(ci_log) ⇒ Boolean

Returns Whether this runner detects (and therefore can handle) Minitest from the log output.

Parameters:

  • ci_log (String) —

    The CI log output

Returns:

  • (Boolean) —

    Whether this runner detects (and therefore can handle) Minitest from the log output.



40
41
42
43
44
 
# File 'lib/ci_runner/runners/minitest_runner.rb', line 40

def self.match?(ci_log)
  default_reporter = %r{(Finished in) \d+\.\d{6}s, \d+\.\d{4} runs/s, \d+\.\d{4} assertions/s\.}

  Regexp.union(default_reporter, SEED_REGEX).match?(ci_log)
end

Instance Method Details

#name ⇒ String

Returns See Runners::Base#report.

Returns:

  • (String) —

    See Runners::Base#report



47
48
49
 
# File 'lib/ci_runner/runners/minitest_runner.rb', line 47

def name
  "Minitest"
end

#start! ⇒ Object

Start a subprocess to rerun the detected failing tests.

Few things to note:

  • CI Runner is meant to be installed as a standalone gem, not Bundled (in a Gemfile). CI Runner doesn’t know what’s inside the loaded specs of the application and it’s possible (while unlikely) that “Rake” isn’t part of the application/gem dependencies. Therefore, when activativing Bundler, requiring “rake/testtask” would fail inside the Rakefile.

    To avoid this problem, requiring rake before Bundler gets activated (using the ruby -rswitch).

  • The CI Runner Minitest plugin will not be detected once the subprocess starts. (Again because CI Runner is not part of the application dependencies). Adding it to the LOAD_PATH manually is required.



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/ci_runner/runners/minitest_runner.rb', line 64

def start!
  super

  minitest_plugin_path = File.expand_path("../..", __dir__)
  rake_load_path = Gem.loaded_specs["rake"].full_require_paths.first

  code = <<~EOM
    Rake::TestTask.new(:__ci_runner_test) do |t|
      t.libs << "test"
      t.libs << "lib"
      t.libs << "#{rake_load_path}"
      t.libs << "#{minitest_plugin_path}"
      t.test_files = #{failures.map(&:path)}
    end

    Rake::Task[:__ci_runner_test].invoke
  EOM

  rakefile_path = File.expand_path("Rakefile", Dir.mktmpdir)
  File.write(rakefile_path, code)

  server = DRb.start_service("drbunix:", failures)

  env = { "TESTOPTS" => "--ci-runner=#{server.uri}" }
  env["SEED"] = seed if seed
  env["RUBY"] = ruby_path.to_s if ruby_path&.exist?
  env["BUNDLE_GEMFILE"] = gemfile_path.to_s if gemfile_path&.exist?

  execute_within_frame(env, "bundle exec ruby -r'rake/testtask' #{rakefile_path}")

  DRb.stop_service
end