Class: Bullshit::Clock

Inherits:
Object
  • Object
show all
Includes:
MoreMath
Defined in:
lib/bullshit.rb

Overview

A Clock instance is used to take measurements while benchmarking.

Constant Summary collapse

TIMES = 
[ :real, :total, :user, :system ]
ALL_COLUMNS = 
[ :scatter ] + TIMES + [ :repeat ]
TIMES_MAX = 
TIMES.map { |t| t.to_s.size }.max

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(bc_method) ⇒ Clock

Returns a Clock instance for CaseMethod instance bc_method.



76
77
78
79
80
81
 
# File 'lib/bullshit.rb', line 76

def initialize(bc_method)
  @bc_method = bc_method
  @times = Hash.new { |h, k| h[k] = [] }
  @repeat = 0
  @scatter = 0
end

Instance Attribute Details

#bc_method ⇒ Object (readonly)

Returns the benchmark method for this Clock instance.



158
159
160
 
# File 'lib/bullshit.rb', line 158

def bc_method
  @bc_method
end

#repeat ⇒ Object

Number of repetitions this clock has measured.



161
162
163
 
# File 'lib/bullshit.rb', line 161

def repeat
  @repeat
end

#slopes ⇒ Object (readonly)

Return all the slopes of linear regressions computed during data truncation phase.



168
169
170
 
# File 'lib/bullshit.rb', line 168

def slopes
  @slopes
end

#time ⇒ Object (readonly)

Last time object used for real time measurement.



164
165
166
 
# File 'lib/bullshit.rb', line 164

def time
  @time
end

Class Method Details

.repeat(bc_method) ⇒ Object

Use a Clock instance to measure the time necessary to do bc_method.case.iterations repetitions of bc_method.



85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
 
# File 'lib/bullshit.rb', line 85

def self.repeat(bc_method)
  clock = new(bc_method)
  bs = clock.case.batch_size.abs
  bs = 1 if !bs or bs < 0
  clock.case.iterations.times do
    bc_method.before_run
    $DEBUG and warn "Calling #{bc_method.name}."
    clock.inc_scatter
    clock.measure do
      bs.times { yield }
    end
    bc_method.after_run
  end
  clock
end

.scale_range(bc_method) ⇒ Object

Iterate over the range of the RangeCase instance of this bc_method and take measurements (including scattering).



128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/bullshit.rb', line 128

def self.scale_range(bc_method)
  clock = new(bc_method)
  my_case = clock.case
  bs = my_case.batch_size.abs
  bs = 1 if !bs or bs < 0
  for a in my_case.range
    begin
      my_case.args = (a.dup rescue a).freeze
      clock.inc_scatter
      my_case.scatter.times do
        bc_method.before_run
        $DEBUG and warn "Calling #{bc_method.name}."
        clock.measure do
          bs.times { yield }
        end
        bc_method.after_run
      end
    ensure
      my_case.args = nil
    end
  end
  clock
end

.time(bc_method) ⇒ Object

Use a Clock instance to measure how many repetitions of bc_method can be done in bc_method.case.duration seconds (a float value). If the bc_method.case.batch_size is >1 duration is multiplied by batch_size because the measured times are averaged by batch_size.



105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
 
# File 'lib/bullshit.rb', line 105

def self.time(bc_method)
  clock = new(bc_method)
  duration = clock.case.duration.abs
  if bs = clock.case.batch_size and bs > 1
    duration *= bs
  end
  until_at = Time.now + duration
  bs = clock.case.batch_size.abs
  bs = 1 if !bs or bs < 0
  begin 
    bc_method.before_run
    $DEBUG and warn "Calling #{bc_method.name}."
    clock.inc_scatter
    clock.measure do
      bs.times { yield }
    end
    bc_method.after_run
  end until clock.time > until_at
  clock
end

.times ⇒ Object

The times which should be displayed in the output.



349
350
351
 
# File 'lib/bullshit.rb', line 349

def self.times
  TIMES.map { |t| t.to_s }
end

.to_a ⇒ Object

Return column names in relation to Clock#to_a method.



200
201
202
 
# File 'lib/bullshit.rb', line 200

def self.to_a
  %w[ #scatter ] + TIMES + %w[ repeat ]
end

Instance Method Details

#<<(times) ⇒ Object

Add the array times to this clock’s time measurements. times consists of the time measurements in float values in order of TIMES.



172
173
174
175
176
177
178
 
# File 'lib/bullshit.rb', line 172

def <<(times)
  r = times.shift
  @repeat += 1 if @times[:repeat].last != r
  @times[:repeat] << r
  TIMES.zip(times) { |t, time| @times[t] << time.to_f }
  self
end

#analysis ⇒ Object

Returns a Hash of Sequence object for all of TIMES’s time keys.



181
182
183
184
185
186
187
 
# File 'lib/bullshit.rb', line 181

def analysis
  @analysis ||= Hash.new do |h, time|
    time = time.to_sym
    times = @times[time]
    h[time] = MoreMath::Sequence.new(times)
  end
end

#arithmetic_mean(time) ⇒ Object Also known as: mean

Returns the arithmetic mean of time.



332
333
334
 
# File 'lib/bullshit.rb', line 332

def arithmetic_mean(time)
  analysis[time.to_sym].mean
end

#autocorrelation(time) ⇒ Object

Return the array of autocorrelation values for time.



359
360
361
 
# File 'lib/bullshit.rb', line 359

def autocorrelation(time)
  analysis[time.to_sym].autocorrelation
end

#autocorrelation_plot(time) ⇒ Object

Returns the arrays for the autocorrelation plot, the first array for the numbers of lag measured, the second for the autocorrelation value.



365
366
367
368
369
370
 
# File 'lib/bullshit.rb', line 365

def autocorrelation_plot(time)
  r = autocorrelation time
  start = @times[:repeat].first
  ende = (start + r.size)
  (start...ende).to_a.zip(r)
end

#call_time_mean ⇒ Object

Seconds per call (mean)



306
307
308
 
# File 'lib/bullshit.rb', line 306

def call_time_mean
  mean(self.case.compare_time)
end

#call_time_median ⇒ Object

Seconds per call (median)



322
323
324
 
# File 'lib/bullshit.rb', line 322

def call_time_median
  median(self.case.compare_time)
end

#calls(call_time_type) ⇒ Object

Calls per second of the call_time_type, e. g. :call_time_mean or :call_time_median.



312
313
314
 
# File 'lib/bullshit.rb', line 312

def calls(call_time_type)
  __send__(call_time_type) ** -1
end

#calls_mean ⇒ Object

Calls per second (mean)



317
318
319
 
# File 'lib/bullshit.rb', line 317

def calls_mean
  call_time_mean ** -1
end

#calls_median ⇒ Object

Calls per second (median)



327
328
329
 
# File 'lib/bullshit.rb', line 327

def calls_median
  call_time_median ** -1
end

#case ⇒ Object

The benchmark case class this clock belongs to (via bc_method).



153
154
155
 
# File 'lib/bullshit.rb', line 153

def case
  @bc_method.case.class
end

#cover?(other) ⇒ Boolean

Return true, if other’s mean value is indistinguishable from this object’s mean after filtering out the noise from the measurements with a Welch’s t-Test. This mean’s that differences in the mean of both clocks might not inidicate a real performance difference and may be caused by chance.

Returns:

  • (Boolean) 


194
195
196
197
 
# File 'lib/bullshit.rb', line 194

def cover?(other)
  time = self.case.compare_time.to_sym
  analysis[time].cover?(other.analysis[time], self.case.covering.alpha_level.abs)
end

#detect_autocorrelation(time) ⇒ Object

Returns the q value for the Ljung-Box statistic of this time‘s analysis.detect_autocorrelation method.



282
283
284
285
286
 
# File 'lib/bullshit.rb', line 282

def detect_autocorrelation(time)
  analysis[time.to_sym].detect_autocorrelation(
    self.case.autocorrelation.max_lags.to_i,
    self.case.autocorrelation.alpha_level.abs)
end

#detect_outliers(time) ⇒ Object

Return a result hash with the number of :very_low, :low, :high, and :very_high outliers, determined by the box plotting algorithm run with :median and :iqr parameters. If no outliers were found or the iqr is less than epsilon, nil is returned.



292
293
294
 
# File 'lib/bullshit.rb', line 292

def detect_outliers(time)
  analysis[time.to_sym].detect_outliers(self.case.outliers_factor.abs)
end

#file_path(*args) ⇒ Object

Return the result of CaseMethod#file_path for this clock’s bc_method.



373
374
375
 
# File 'lib/bullshit.rb', line 373

def file_path(*args)
  @bc_method.file_path(*args)
end

#find_truncation_offset ⇒ Object

Find an offset from the start of the measurements in this clock to truncate the initial data until a stable state has been reached and return it as an integer.



392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
 
# File 'lib/bullshit.rb', line 392

def find_truncation_offset
  truncation = self.case.truncate_data
  slope_angle = self.case.truncate_data.slope_angle.abs
  time = self.case.compare_time.to_sym
  ms = analysis[time].elements.reverse
  offset = ms.size - 1
  @slopes = []
  ModuleFunctions.array_window(ms, truncation.window_size) do |data|
    lr = LinearRegression.new(data)
    a = lr.a
    @slopes << [ offset, a ]
    a.abs > slope_angle and break
    offset -= 1
  end
  offset < 0 ? 0 : offset
end

#geometric_mean(time) ⇒ Object

Returns the geometric mean of time.



344
345
346
 
# File 'lib/bullshit.rb', line 344

def geometric_mean(time)
  analysis[time.to_sym].geometric_mean
end

#harmonic_mean(time) ⇒ Object

Returns the harmonic mean of time.



339
340
341
 
# File 'lib/bullshit.rb', line 339

def harmonic_mean(time)
  analysis[time.to_sym].harmonic_mean
end

#histogram(time) ⇒ Object

Return the Histogram for the time values.



354
355
356
 
# File 'lib/bullshit.rb', line 354

def histogram(time)
  analysis[time.to_sym].histogram(self.case.histogram.bins)
end

#inc_scatter ⇒ Object

Increment scatter counter by one.



226
227
228
 
# File 'lib/bullshit.rb', line 226

def inc_scatter
  @scatter += 1
end

#max(time) ⇒ Object

Returns the maximum for the time (one of TIMES’ symbols).



266
267
268
 
# File 'lib/bullshit.rb', line 266

def max(time)
  analysis[time.to_sym].max
end

#measure ⇒ Object

Take a single measurement. This method should be called with the code to benchmark in a block.



232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
 
# File 'lib/bullshit.rb', line 232

def measure
  before = take_time
  yield
  after = take_time
  @repeat += 1
  @times[:repeat] << @repeat
  @times[:scatter] << @scatter
  bs = self.case.batch_size.abs
  if bs and bs > 1
    TIMES.each_with_index { |t, i| @times[t] << (after[i] - before[i]) / bs }
  else
    TIMES.each_with_index { |t, i| @times[t] << after[i] - before[i] }
  end
  @analysis = nil
end

#median(time) ⇒ Object

Returns the median of the time values (one of TIMES’ symbols).



271
272
273
 
# File 'lib/bullshit.rb', line 271

def median(time)
  analysis[time.to_sym].median
end

#min(time) ⇒ Object

Returns the minimum for the time (one of TIMES’ symbols).



261
262
263
 
# File 'lib/bullshit.rb', line 261

def min(time)
  analysis[time.to_sym].min
end

#percentile(time, p = 50) ⇒ Object

Returns the p-percentile of the time values (one of TIMES’ symbols).



276
277
278
 
# File 'lib/bullshit.rb', line 276

def percentile(time, p = 50)
  analysis[time.to_sym].percentile p
end

#sample_standard_deviation(time) ⇒ Object

Returns the sample standard deviation for the time (one of TIMES’ symbols).



250
251
252
 
# File 'lib/bullshit.rb', line 250

def sample_standard_deviation(time)
  analysis[time.to_sym].sample_standard_deviation
end

#sample_standard_deviation_percentage(time) ⇒ Object

Returns the sample standard deviation for the time (one of TIMES’ symbols) in percentage of its arithmetic mean.



256
257
258
 
# File 'lib/bullshit.rb', line 256

def sample_standard_deviation_percentage(time)
  analysis[time.to_sym].sample_standard_deviation_percentage
end

#sum(time) ⇒ Object

Returns the sum of measurement times for time.



301
302
303
 
# File 'lib/bullshit.rb', line 301

def sum(time)
  __send__ time
end

#take_time ⇒ Object

Takes the times an returns an array, consisting of the times in the order of enumerated in the TIMES constant.



217
218
219
220
221
222
223
 
# File 'lib/bullshit.rb', line 217

def take_time
  @time, times = Time.now, Process.times
  user_time = times.utime + times.cutime    # user time of this process and its children
  system_time = times.stime + times.cstime  # system time of this process and its children
  total_time = user_time + system_time      # total time of this process and its children
  [ @time.to_f, total_time, user_time, system_time ]
end

#to_a ⇒ Object

Returns the measurements as an array of arrays.



205
206
207
208
209
210
211
212
213
 
# File 'lib/bullshit.rb', line 205

def to_a
  if @repeat >= 1
    (::Bullshit::Clock::ALL_COLUMNS).map do |t|
      analysis[t].elements
    end.transpose
  else
    []
  end
end

#truncate_data(offset) ⇒ Object

Truncate the measurements stored in this clock starting from the integer offset.



379
380
381
382
383
384
385
386
387
 
# File 'lib/bullshit.rb', line 379

def truncate_data(offset)
  for t in ALL_COLUMNS
    times = @times[t]
    @times[t] = @times[t][offset, times.size]
    @repeat = @times[t].size
  end
  @analysis = nil
  self
end