Class: LabCoat::Experiment

Inherits:
Object
  • Object
show all
Defined in:
lib/lab_coat/experiment.rb

Overview

A base experiment class meant to be subclassed to define various experiments.

Constant Summary collapse

OBSERVATIONS = 
%w[control candidate].freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(name) ⇒ Experiment

Returns a new instance of Experiment.



10
11
12
13
 
# File 'lib/lab_coat/experiment.rb', line 10

def initialize(name)
  @name = name
  @context = {}
end

Instance Attribute Details

#context ⇒ Object (readonly)

Returns the value of attribute context.



8
9
10
 
# File 'lib/lab_coat/experiment.rb', line 8

def context
  @context
end

#name ⇒ Object (readonly)

Returns the value of attribute name.



8
9
10
 
# File 'lib/lab_coat/experiment.rb', line 8

def name
  @name
end

Instance Method Details

#candidate ⇒ Object

Override this method to define the new aka “candidate” behavior. Only run if the experiment is enabled.

Returns:

  • (Object) —

    Anything.

Raises:



30
31
32
 
# File 'lib/lab_coat/experiment.rb', line 30

def candidate
  raise InvalidExperimentError, "`#candidate` must be implemented in your Experiment class."
end

#compare(control, candidate) ⇒ TrueClass, FalseClass

Override this method to define what is considered a match or mismatch. Must return a boolean.

Parameters:

Returns:

  • (TrueClass, FalseClass) 


38
39
40
 
# File 'lib/lab_coat/experiment.rb', line 38

def compare(control, candidate)
  control.value == candidate.value
end

#control ⇒ Object

Override this method to define the existing aka “control” behavior. This method is always run, even when ‘enabled?` is false.

Returns:

  • (Object) —

    Anything.

Raises:



24
25
26
 
# File 'lib/lab_coat/experiment.rb', line 24

def control
  raise InvalidExperimentError, "`#control` must be implemented in your Experiment class."
end

#enabled? ⇒ TrueClass, FalseClass

Override this method to control whether or not the experiment runs.

Returns:

  • (TrueClass, FalseClass)

Raises:



17
18
19
 
# File 'lib/lab_coat/experiment.rb', line 17

def enabled?
  raise InvalidExperimentError, "`#enabled?` must be implemented in your Experiment class."
end

#ignore?(_control, _candidate) ⇒ TrueClass, FalseClass

Override this method to define which results are ignored. Must return a boolean.

Parameters:

Returns:

  • (TrueClass, FalseClass) 


46
47
48
 
# File 'lib/lab_coat/experiment.rb', line 46

def ignore?(_control, _candidate)
  false
end

#publish!(result) ⇒ void

This method returns an undefined value.

Override this method to publish the ‘Result`. It’s recommended to override this once in an application wide base class.

Parameters:



66
 
# File 'lib/lab_coat/experiment.rb', line 66

def publish!(result); end

#publishable_value(observation) ⇒ Object

Override this method to transform the value for publishing. This could mean turning the value into something serializable (e.g. JSON).

Parameters:



58
59
60
 
# File 'lib/lab_coat/experiment.rb', line 58

def publishable_value(observation)
  observation.value
end

#raised(observation) ⇒ void

This method returns an undefined value.

Called when the control and/or candidate observations raise an error.

Parameters:



53
 
# File 'lib/lab_coat/experiment.rb', line 53

def raised(observation); end

#run!(**context) ⇒ Object

Runs the control and candidate and publishes the result. Always returns the result of ‘control`. It’s not recommended to override this method.

Parameters:

  • context (Hash) —

    Any data needed at runtime.

Returns:

  • (Object) —

    An ‘Observation` value.



81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
 
# File 'lib/lab_coat/experiment.rb', line 81

def run!(**context) # rubocop:disable Metrics/MethodLength, Metrics/AbcSize
  # Set the context for this run.
  @context = context

  # Run the control and exit early if the experiment is not enabled.
  unless enabled?
    control_obs = Observation.new("control", self) { control }
    raised(control_obs) if control_obs.raised?
    return control_obs.value
  end

  # Otherwise run the control and candidate in random order.
  observations = OBSERVATIONS.shuffle.map do |name|
    Observation.new(name, self) { public_send(name) }.tap do |observation|
      raised(observation) if observation.raised?
    end
  end

  # Compare and publish the results.
  result = if observations.first.name == "control"
             Result.new(self, observations.first, observations.last)
           else
             Result.new(self, observations.last, observations.first)
           end
  publish!(result)

  # Return the selected observations, control by default.
  select_observation(result).value.tap do
    # Reset the context for this run. Done here so that `select_observation` has access to the runtime context.
    @context = {}
  end
end

#select_observation(result) ⇒ LabCoat::Observation

Override this method to select which observation’s ‘value` should be returned by the `Experiment`. Defaults to the control `Observation`. This method is only called if the `Experiment` is enabled. This is useful for rolling out new behavior in a controlled way.

Parameters:

Returns:

  • (LabCoat::Observation) —

    Either the control or candidate ‘Observation` from the given `Result`.



73
74
75
 
# File 'lib/lab_coat/experiment.rb', line 73

def select_observation(result)
  result.control
end