Module: Scientist::Experiment

Included in:
Default
Defined in:
lib/scientist/experiment.rb

Overview

This mixin provides shared behavior for experiments. Includers must implement ‘enabled?` and `publish(result)`.

Override Scientist::Experiment.new to set your own class which includes and implements Scientist::Experiment’s interface.

Defined Under Namespace

Modules: RaiseOnMismatch Classes: MismatchError

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#raise_on_mismatches ⇒ Object

Whether to raise when the control and candidate mismatch. If this is nil, raise_on_mismatches class attribute is used instead.



10
11
12
 
# File 'lib/scientist/experiment.rb', line 10

def raise_on_mismatches
  @raise_on_mismatches
end

Class Method Details

.included(base) ⇒ Object 



70
71
72
 
# File 'lib/scientist/experiment.rb', line 70

def self.included(base)
  base.extend RaiseOnMismatch
end

.new(name) ⇒ Object

Create a new instance of a class that implements the Scientist::Experiment interface.

Override this method directly to change the default implementation.



16
17
18
 
# File 'lib/scientist/experiment.rb', line 16

def self.new(name)
  Scientist::Default.new(name)
end

Instance Method Details

#before_run(&block) ⇒ Object

Define a block of code to run before an experiment begins, if the experiment is enabled.

The block takes no arguments.

Returns the configured block.



80
81
82
 
# File 'lib/scientist/experiment.rb', line 80

def before_run(&block)
  @_scientist_before_run = block
end

#behaviors ⇒ Object

A Hash of behavior blocks, keyed by String name. Register behavior blocks with the ‘try` and `use` methods.



86
87
88
 
# File 'lib/scientist/experiment.rb', line 86

def behaviors
  @_scientist_behaviors ||= {}
end

#clean(&block) ⇒ Object

A block to clean an observed value for publishing or storing.

The block takes one argument, the observed value which will be cleaned.

Returns the configured block.



95
96
97
 
# File 'lib/scientist/experiment.rb', line 95

def clean(&block)
  @_scientist_cleaner = block
end

#clean_value(value) ⇒ Object

Internal: Clean a value with the configured clean block, or return the value if no clean block is configured.

Rescues and reports exceptions in the clean block if they occur.



103
104
105
106
107
108
109
110
111
112
 
# File 'lib/scientist/experiment.rb', line 103

def clean_value(value)
  if @_scientist_cleaner
    @_scientist_cleaner.call value
  else
    value
  end
rescue StandardError => ex
  raised :clean, ex
  value
end

#compare(*args, &block) ⇒ Object

A block which compares two experimental values.

The block must take two arguments, the control value and a candidate value, and return true or false.

Returns the block.



120
121
122
 
# File 'lib/scientist/experiment.rb', line 120

def compare(*args, &block)
  @_scientist_comparator = block
end

#context(context = nil) ⇒ Object

A Symbol-keyed Hash of extra experiment data.



125
126
127
128
129
 
# File 'lib/scientist/experiment.rb', line 125

def context(context = nil)
  @_scientist_context ||= {}
  @_scientist_context.merge!(context) if !context.nil?
  @_scientist_context
end

#ignore(&block) ⇒ Object

Configure this experiment to ignore an observation with the given block.

The block takes two arguments, the control observation and the candidate observation which didn’t match the control. If the block returns true, the mismatch is disregarded.

This can be called more than once with different blocks to use.



138
139
140
141
 
# File 'lib/scientist/experiment.rb', line 138

def ignore(&block)
  @_scientist_ignores ||= []
  @_scientist_ignores << block
end

#ignore_mismatched_observation?(control, candidate) ⇒ Boolean

Internal: ignore a mismatched observation?

Iterates through the configured ignore blocks and calls each of them with the given control and mismatched candidate observations.

Returns true or false.

Returns:

  • (Boolean) 


149
150
151
152
153
154
155
156
157
158
159
 
# File 'lib/scientist/experiment.rb', line 149

def ignore_mismatched_observation?(control, candidate)
  return false unless @_scientist_ignores
  @_scientist_ignores.any? do |ignore|
    begin
      ignore.call control.value, candidate.value
    rescue StandardError => ex
      raised :ignore, ex
      false
    end
  end
end

#name ⇒ Object

The String name of this experiment. Default is “experiment”. See Scientist::Default for an example of how to override this default.



163
164
165
 
# File 'lib/scientist/experiment.rb', line 163

def name
  "experiment"
end

#observations_are_equivalent?(a, b) ⇒ Boolean

Internal: compare two observations, using the configured compare block if present.

Returns:

  • (Boolean) 


168
169
170
171
172
173
174
175
176
177
 
# File 'lib/scientist/experiment.rb', line 168

def observations_are_equivalent?(a, b)
  if @_scientist_comparator
    a.equivalent_to?(b, &@_scientist_comparator)
  else
    a.equivalent_to? b
  end
rescue StandardError => ex
  raised :compare, ex
  false
end

#raise_on_mismatches? ⇒ Boolean

Whether or not to raise a mismatch error when a mismatch occurs.

Returns:

  • (Boolean) 


278
279
280
281
282
283
284
 
# File 'lib/scientist/experiment.rb', line 278

def raise_on_mismatches?
  if raise_on_mismatches.nil?
    self.class.raise_on_mismatches?
  else
    !!raise_on_mismatches
  end
end

#raised(operation, error) ⇒ Object

Called when an exception is raised while running an internal operation, like :publish. Override this method to track these exceptions. The default implementation re-raises the exception.



182
183
184
 
# File 'lib/scientist/experiment.rb', line 182

def raised(operation, error)
  raise error
end

#run(name = nil) ⇒ Object

Internal: Run all the behaviors for this experiment, observing each and publishing the results. Return the result of the named behavior, default “control”.



189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
 
# File 'lib/scientist/experiment.rb', line 189

def run(name = nil)
  behaviors.freeze
  context.freeze

  name = (name || "control").to_s
  block = behaviors[name]

  if block.nil?
    raise Scientist::BehaviorMissing.new(self, name)
  end

  unless should_experiment_run?
    return block.call
  end

  if @_scientist_before_run
    @_scientist_before_run.call
  end

  observations = []

  behaviors.keys.shuffle.each do |key|
    block = behaviors[key]
    observations << Scientist::Observation.new(key, self, &block)
  end

  control = observations.detect { |o| o.name == name }

  result = Scientist::Result.new self, observations, control

  begin
    publish(result)
  rescue StandardError => ex
    raised :publish, ex
  end

  if raise_on_mismatches? && result.mismatched?
    raise MismatchError.new(self.name, result)
  end

  if control.raised?
    raise control.exception
  else
    control.value
  end
end

#run_if(&block) ⇒ Object

Define a block that determines whether or not the experiment should run.



237
238
239
 
# File 'lib/scientist/experiment.rb', line 237

def run_if(&block)
  @_scientist_run_if_block = block
end

#run_if_block_allows? ⇒ Boolean

Internal: does a run_if block allow the experiment to run?

Rescues and reports exceptions in a run_if block if they occur.

Returns:

  • (Boolean) 


244
245
246
247
248
249
 
# File 'lib/scientist/experiment.rb', line 244

def run_if_block_allows?
  (@_scientist_run_if_block ? @_scientist_run_if_block.call : true)
rescue StandardError => ex
  raised :run_if, ex
  return false
end

#should_experiment_run? ⇒ Boolean

Internal: determine whether or not an experiment should run.

Rescues and reports exceptions in the enabled method if they occur.

Returns:

  • (Boolean) 


254
255
256
257
258
259
 
# File 'lib/scientist/experiment.rb', line 254

def should_experiment_run?
  behaviors.size > 1 && enabled? && run_if_block_allows?
rescue StandardError => ex
  raised :enabled, ex
  return false
end

#try(name = nil, &block) ⇒ Object

Register a named behavior for this experiment, default “candidate”.



262
263
264
265
266
267
268
269
270
 
# File 'lib/scientist/experiment.rb', line 262

def try(name = nil, &block)
  name = (name || "candidate").to_s

  if behaviors.include?(name)
    raise Scientist::BehaviorNotUnique.new(self, name)
  end

  behaviors[name] = block
end

#use(&block) ⇒ Object

Register the control behavior for this experiment.



273
274
275
 
# File 'lib/scientist/experiment.rb', line 273

def use(&block)
  try "control", &block
end