Class: Minitest::Benchmark
- Defined in:
- lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb
Overview
Subclass Benchmark to create your own benchmark runs. Methods starting with “bench_” get executed on a per-class.
See Minitest::Assertions
Direct Known Subclasses
Constant Summary
Constants inherited from Test
Test::PASSTHROUGH_EXCEPTIONS, Test::SETUP_METHODS, Test::TEARDOWN_METHODS
Constants included from Assertions
Assertions::E, Assertions::UNDEFINED
Constants inherited from Runnable
Instance Attribute Summary
Attributes inherited from Runnable
Class Method Summary collapse
-
.bench_exp(min, max, base = 10) ⇒ Object
Returns a set of ranges stepped exponentially from
min
tomax
by powers ofbase
. -
.bench_linear(min, max, step = 10) ⇒ Object
Returns a set of ranges stepped linearly from
min
tomax
bystep
. -
.bench_range ⇒ Object
Specifies the ranges used for benchmarking for that class.
-
.io ⇒ Object
:nodoc:.
-
.run(reporter, options = {}) ⇒ Object
:nodoc:.
-
.runnable_methods ⇒ Object
:nodoc:.
Instance Method Summary collapse
-
#assert_performance(validation, &work) ⇒ Object
Runs the given
work
, gathering the times of each run. -
#assert_performance_constant(threshold = 0.99, &work) ⇒ Object
Runs the given
work
and asserts that the times gathered fit to match a constant rate (eg, linear slope == 0) within a giventhreshold
. -
#assert_performance_exponential(threshold = 0.99, &work) ⇒ Object
Runs the given
work
and asserts that the times gathered fit to match a exponential curve within a given errorthreshold
. -
#assert_performance_linear(threshold = 0.99, &work) ⇒ Object
Runs the given
work
and asserts that the times gathered fit to match a straight line within a given errorthreshold
. -
#assert_performance_logarithmic(threshold = 0.99, &work) ⇒ Object
Runs the given
work
and asserts that the times gathered fit to match a logarithmic curve within a given errorthreshold
. -
#assert_performance_power(threshold = 0.99, &work) ⇒ Object
Runs the given
work
and asserts that the times gathered curve fit to match a power curve within a given errorthreshold
. -
#fit_error(xys) ⇒ Object
Takes an array of x/y pairs and calculates the general R^2 value.
-
#fit_exponential(xs, ys) ⇒ Object
To fit a functional form: y = ae^(bx).
-
#fit_linear(xs, ys) ⇒ Object
Fits the functional form: a + bx.
-
#fit_logarithmic(xs, ys) ⇒ Object
To fit a functional form: y = a + b*ln(x).
-
#fit_power(xs, ys) ⇒ Object
To fit a functional form: y = ax^b.
-
#io ⇒ Object
:nodoc:.
-
#sigma(enum, &block) ⇒ Object
Enumerates over
enum
mappingblock
if given, returning the sum of the result. -
#validation_for_fit(msg, threshold) ⇒ Object
Returns a proc that calls the specified fit method and asserts that the error is within a tolerable threshold.
Methods inherited from Test
#capture_exceptions, #class_name, #clean, i_suck_and_my_tests_are_order_dependent!, make_my_diffs_pretty!, #neuter_exception, #new_exception, parallelize_me!, #run, #sanitize_exception, test_order, #with_empty_backtrace_filter, #with_info_handler
Methods included from Guard
#jruby?, #maglev?, #mri?, #osx?, #rubinius?, #windows?
Methods included from Test::LifecycleHooks
#after_setup, #after_teardown, #before_setup, #before_teardown, #setup, #teardown
Methods included from Reportable
#class_name, #error?, #location, #passed?, #result_code, #skipped?
Methods included from Assertions
#_synchronize, #assert, #assert_empty, #assert_equal, #assert_in_delta, #assert_in_epsilon, #assert_includes, #assert_instance_of, #assert_kind_of, #assert_match, #assert_mock, #assert_nil, #assert_operator, #assert_output, #assert_path_exists, #assert_predicate, #assert_raises, #assert_respond_to, #assert_same, #assert_send, #assert_silent, #assert_throws, #capture_io, #capture_subprocess_io, #diff, diff, diff=, #exception_details, #fail_after, #flunk, #message, #mu_pp, #mu_pp_for_diff, #pass, #refute, #refute_empty, #refute_equal, #refute_in_delta, #refute_in_epsilon, #refute_includes, #refute_instance_of, #refute_kind_of, #refute_match, #refute_nil, #refute_operator, #refute_path_exists, #refute_predicate, #refute_respond_to, #refute_same, #skip, #skip_until, #skipped?, #things_to_diff
Methods inherited from Runnable
#failure, inherited, #initialize, #marshal_dump, #marshal_load, methods_matching, #name, #name=, on_signal, #passed?, reset, #result_code, #run, run_one_method, runnables, #skipped?, #time_it, #whatever, with_info_handler
Constructor Details
This class inherits a constructor from Minitest::Runnable
Class Method Details
.bench_exp(min, max, base = 10) ⇒ Object
Returns a set of ranges stepped exponentially from min
to max
by powers of base
. Eg:
bench_exp(2, 16, 2) # => [2, 4, 8, 16]
35 36 37 38 39 40 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 35 def self.bench_exp min, max, base = 10 min = (Math.log10(min) / Math.log10(base)).to_i max = (Math.log10(max) / Math.log10(base)).to_i (min..max).map { |m| base ** m }.to_a end |
.bench_linear(min, max, step = 10) ⇒ Object
Returns a set of ranges stepped linearly from min
to max
by step
. Eg:
bench_linear(20, 40, 10) # => [20, 30, 40]
48 49 50 51 52 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 48 def self.bench_linear min, max, step = 10 (min..max).step(step).to_a rescue LocalJumpError # 1.8.6 r = []; (min..max).step(step) { |n| r << n }; r end |
.bench_range ⇒ Object
Specifies the ranges used for benchmarking for that class. Defaults to exponential growth from 1 to 10k by powers of 10. Override if you need different ranges for your benchmarks.
See also: ::bench_exp and ::bench_linear.
61 62 63 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 61 def self.bench_range bench_exp 1, 10_000 end |
.io ⇒ Object
:nodoc:
12 13 14 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 12 def self.io # :nodoc: @io end |
.run(reporter, options = {}) ⇒ Object
:nodoc:
20 21 22 23 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 20 def self.run reporter, = {} # :nodoc: @io = reporter.io super end |
.runnable_methods ⇒ Object
:nodoc:
25 26 27 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 25 def self.runnable_methods # :nodoc: methods_matching(/^bench_/) end |
Instance Method Details
#assert_performance(validation, &work) ⇒ Object
Runs the given work
, gathering the times of each run. Range and times are then passed to a given validation
proc. Outputs the benchmark name and times in tab-separated format, making it easy to paste into a spreadsheet for graphing or further analysis.
Ranges are specified by ::bench_range.
Eg:
def bench_algorithm
validation = proc { |x, y| ... }
assert_performance validation do |n|
@obj.algorithm(n)
end
end
83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 83 def assert_performance validation, &work range = self.class.bench_range io.print "#{self.name}" times = [] range.each do |x| GC.start t0 = Minitest.clock_time instance_exec(x, &work) t = Minitest.clock_time - t0 io.print "\t%9.6f" % t times << t end io.puts validation[range, times] end |
#assert_performance_constant(threshold = 0.99, &work) ⇒ Object
Runs the given work
and asserts that the times gathered fit to match a constant rate (eg, linear slope == 0) within a given threshold
. Note: because we’re testing for a slope of 0, R^2 is not a good determining factor for the fit, so the threshold is applied against the slope itself. As such, you probably want to tighten it from the default.
See www.graphpad.com/guides/prism/8/curve-fitting/reg_intepretingnonlinr2.htm for more details.
Fit is calculated by #fit_linear.
Ranges are specified by ::bench_range.
Eg:
def bench_algorithm
assert_performance_constant 0.9999 do |n|
@obj.algorithm(n)
end
end
127 128 129 130 131 132 133 134 135 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 127 def assert_performance_constant threshold = 0.99, &work validation = proc do |range, times| a, b, rr = fit_linear range, times assert_in_delta 0, b, 1 - threshold [a, b, rr] end assert_performance validation, &work end |
#assert_performance_exponential(threshold = 0.99, &work) ⇒ Object
Runs the given work
and asserts that the times gathered fit to match a exponential curve within a given error threshold
.
Fit is calculated by #fit_exponential.
Ranges are specified by ::bench_range.
Eg:
def bench_algorithm
assert_performance_exponential 0.9999 do |n|
@obj.algorithm(n)
end
end
153 154 155 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 153 def assert_performance_exponential threshold = 0.99, &work assert_performance validation_for_fit(:exponential, threshold), &work end |
#assert_performance_linear(threshold = 0.99, &work) ⇒ Object
Runs the given work
and asserts that the times gathered fit to match a straight line within a given error threshold
.
Fit is calculated by #fit_linear.
Ranges are specified by ::bench_range.
Eg:
def bench_algorithm
assert_performance_linear 0.9999 do |n|
@obj.algorithm(n)
end
end
193 194 195 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 193 def assert_performance_linear threshold = 0.99, &work assert_performance validation_for_fit(:linear, threshold), &work end |
#assert_performance_logarithmic(threshold = 0.99, &work) ⇒ Object
Runs the given work
and asserts that the times gathered fit to match a logarithmic curve within a given error threshold
.
Fit is calculated by #fit_logarithmic.
Ranges are specified by ::bench_range.
Eg:
def bench_algorithm
assert_performance_logarithmic 0.9999 do |n|
@obj.algorithm(n)
end
end
173 174 175 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 173 def assert_performance_logarithmic threshold = 0.99, &work assert_performance validation_for_fit(:logarithmic, threshold), &work end |
#assert_performance_power(threshold = 0.99, &work) ⇒ Object
Runs the given work
and asserts that the times gathered curve fit to match a power curve within a given error threshold
.
Fit is calculated by #fit_power.
Ranges are specified by ::bench_range.
Eg:
def bench_algorithm
assert_performance_power 0.9999 do |x|
@obj.algorithm
end
end
213 214 215 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 213 def assert_performance_power threshold = 0.99, &work assert_performance validation_for_fit(:power, threshold), &work end |
#fit_error(xys) ⇒ Object
Takes an array of x/y pairs and calculates the general R^2 value.
222 223 224 225 226 227 228 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 222 def fit_error xys = sigma(xys) { |_, y| y } / xys.size.to_f ss_tot = sigma(xys) { |_, y| (y - ) ** 2 } ss_err = sigma(xys) { |x, y| (yield(x) - y) ** 2 } 1 - (ss_err / ss_tot) end |
#fit_exponential(xs, ys) ⇒ Object
To fit a functional form: y = ae^(bx).
Takes x and y values and returns [a, b, r^2].
See: mathworld.wolfram.com/LeastSquaresFittingExponential.html
237 238 239 240 241 242 243 244 245 246 247 248 249 250 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 237 def fit_exponential xs, ys n = xs.size xys = xs.zip(ys) sxlny = sigma(xys) { |x, y| x * Math.log(y) } slny = sigma(xys) { |_, y| Math.log(y) } sx2 = sigma(xys) { |x, _| x * x } sx = sigma xs c = n * sx2 - sx ** 2 a = (slny * sx2 - sx * sxlny) / c b = ( n * sxlny - sx * slny ) / c return Math.exp(a), b, fit_error(xys) { |x| Math.exp(a + b * x) } end |
#fit_linear(xs, ys) ⇒ Object
Fits the functional form: a + bx.
Takes x and y values and returns [a, b, r^2].
281 282 283 284 285 286 287 288 289 290 291 292 293 294 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 281 def fit_linear xs, ys n = xs.size xys = xs.zip(ys) sx = sigma xs sy = sigma ys sx2 = sigma(xs) { |x| x ** 2 } sxy = sigma(xys) { |x, y| x * y } c = n * sx2 - sx**2 a = (sy * sx2 - sx * sxy) / c b = ( n * sxy - sx * sy ) / c return a, b, fit_error(xys) { |x| a + b * x } end |
#fit_logarithmic(xs, ys) ⇒ Object
To fit a functional form: y = a + b*ln(x).
Takes x and y values and returns [a, b, r^2].
See: mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html
259 260 261 262 263 264 265 266 267 268 269 270 271 272 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 259 def fit_logarithmic xs, ys n = xs.size xys = xs.zip(ys) slnx2 = sigma(xys) { |x, _| Math.log(x) ** 2 } slnx = sigma(xys) { |x, _| Math.log(x) } sylnx = sigma(xys) { |x, y| y * Math.log(x) } sy = sigma(xys) { |_, y| y } c = n * slnx2 - slnx ** 2 b = ( n * sylnx - sy * slnx ) / c a = (sy - b * slnx) / n return a, b, fit_error(xys) { |x| a + b * Math.log(x) } end |
#fit_power(xs, ys) ⇒ Object
To fit a functional form: y = ax^b.
Takes x and y values and returns [a, b, r^2].
303 304 305 306 307 308 309 310 311 312 313 314 315 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 303 def fit_power xs, ys n = xs.size xys = xs.zip(ys) slnxlny = sigma(xys) { |x, y| Math.log(x) * Math.log(y) } slnx = sigma(xs) { |x | Math.log(x) } slny = sigma(ys) { | y| Math.log(y) } slnx2 = sigma(xs) { |x | Math.log(x) ** 2 } b = (n * slnxlny - slnx * slny) / (n * slnx2 - slnx ** 2) a = (slny - b * slnx) / n return Math.exp(a), b, fit_error(xys) { |x| (Math.exp(a) * (x ** b)) } end |
#io ⇒ Object
:nodoc:
16 17 18 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 16 def io # :nodoc: self.class.io end |
#sigma(enum, &block) ⇒ Object
Enumerates over enum
mapping block
if given, returning the sum of the result. Eg:
sigma([1, 2, 3]) # => 1 + 2 + 3 => 6
sigma([1, 2, 3]) { |n| n ** 2 } # => 1 + 4 + 9 => 14
324 325 326 327 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 324 def sigma enum, &block enum = enum.map(&block) if block enum.inject { |sum, n| sum + n } end |
#validation_for_fit(msg, threshold) ⇒ Object
Returns a proc that calls the specified fit method and asserts that the error is within a tolerable threshold.
333 334 335 336 337 338 339 |
# File 'lib/rubypitaya/app-template/vendor/bundle/ruby/3.1.0/gems/minitest-5.17.0/lib/minitest/benchmark.rb', line 333 def validation_for_fit msg, threshold proc do |range, times| a, b, rr = send "fit_#{msg}", range, times assert_operator rr, :>=, threshold [a, b, rr] end end |