Module: Benchmark::Compare

Included in:
Benchmark
Defined in:
lib/benchmark/compare.rb

Overview

Functionality of performaing comparison between reports.

Usage:

Add x.compare! to perform comparison between reports.

Example:

> Benchmark.ips do |x|
  x.report('Reduce using tag')     { [*1..10].reduce(:+) }
  x.report('Reduce using to_proc') { [*1..10].reduce(&:+) }
  x.compare!
end

Calculating -------------------------------------
    Reduce using tag     19216 i/100ms
Reduce using to_proc     17437 i/100ms
-------------------------------------------------
    Reduce using tag   278950.0 (±8.5%) i/s -    1402768 in   5.065112s
Reduce using to_proc   247295.4 (±8.0%) i/s -    1238027 in   5.037299s

Comparison:
    Reduce using tag:   278950.0 i/s
Reduce using to_proc:   247295.4 i/s - 1.13x slower

Besides regular Calculating report, this will also indicates which one is slower.

x.compare! also takes an order: :baseline option.

Example:

> Benchmark.ips do |x|
 x.report('Reduce using block')   { [*1..10].reduce { |sum, n| sum + n } }
 x.report('Reduce using tag')     { [*1..10].reduce(:+) }
 x.report('Reduce using to_proc') { [*1..10].reduce(&:+) }
 x.compare!(order: :baseline)

end

Calculating ————————————-

Reduce using block    886.202k (± 2.2%) i/s -      4.521M in   5.103774s
  Reduce using tag      1.821M (± 1.6%) i/s -      9.111M in   5.004183s

Reduce using to_proc 895.948k (± 1.6%) i/s - 4.528M in 5.055368s

Comparison:

Reduce using block:   886202.5 i/s
  Reduce using tag:  1821055.0 i/s - 2.05x  (± 0.00) faster

Reduce using to_proc: 895948.1 i/s - same-ish: difference falls within error

The first report is considered the baseline against which other reports are compared.

Instance Method Summary collapse

Instance Method Details

#compare(*entries, order: :fastest) ⇒ Object

Compare between reports, prints out facts of each report: runtime, comparative speed difference.

Parameters:



56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/benchmark/compare.rb', line 56

def compare(*entries, order: :fastest)
  return if entries.size < 2

  case order
  when :baseline
    baseline = entries.shift
    sorted = entries.sort_by{ |e| e.stats.central_tendency }.reverse
  when :fastest
    sorted = entries.sort_by{ |e| e.stats.central_tendency }.reverse
    baseline = sorted.shift
  else
    raise ArgumentError, "Unknwon order: #{order.inspect}"
  end

  $stdout.puts "\nComparison:"

  $stdout.printf "%20s: %10.1f i/s\n", baseline.label.to_s, baseline.stats.central_tendency

  sorted.each do |report|
    name = report.label.to_s

    $stdout.printf "%20s: %10.1f i/s - ", name, report.stats.central_tendency

    if report.stats.overlaps?(baseline.stats)
      $stdout.print "same-ish: difference falls within error"
    elsif report.stats.central_tendency > baseline.stats.central_tendency
      speedup, error = report.stats.speedup(baseline.stats)
      $stdout.printf "%.2fx ", speedup
      if error
        $stdout.printf " (± %.2f)", error
      end
      $stdout.print " faster"
    else
      slowdown, error = report.stats.slowdown(baseline.stats)
      $stdout.printf "%.2fx ", slowdown
      if error
        $stdout.printf " (± %.2f)", error
      end
      $stdout.print " slower"
    end

    $stdout.puts
  end

  footer = baseline.stats.footer
  $stdout.puts footer.rjust(40) if footer

  $stdout.puts
end