Class: MachineLearningWorkbench::Optimizer::NaturalEvolutionStrategies::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb

Overview

Natural Evolution Strategies base class

Direct Known Subclasses

BDNES, SNES, XNES

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(ndims, obj_fn, opt_type, rseed: nil, mu_init: 0, sigma_init: 1) ⇒ Base

NES object initialization

Parameters:

  • ndims (Integer) —

    number of parameters to optimize

  • obj_fn (#call) —

    any object defining a #call method (Proc, lambda, custom class)

  • opt_type (:min, :max) —

    select minimization / maximization of obj_fn

  • rseed (Integer) (defaults to: nil) —

    allow for deterministic execution on rseed provided

Raises:

  • (ArgumentError) 


12
13
14
15
16
17
18
19
20
21
22
23
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 12

def initialize ndims, obj_fn, opt_type, rseed: nil, mu_init: 0, sigma_init: 1
  raise ArgumentError unless [:min, :max].include? opt_type
  raise ArgumentError unless obj_fn.respond_to? :call
  @ndims, @opt_type, @obj_fn = ndims, opt_type, obj_fn
  @id = NMatrix.identity(ndims, dtype: :float64)
  rseed ||= Random.new_seed
  # puts "NES rseed: #{s}"  # currently disabled
  @rng = Random.new rseed
  @best = [(opt_type==:max ? -1 : 1) * Float::INFINITY, nil]
  @last_fits = []
  initialize_distribution mu_init: mu_init, sigma_init: sigma_init
end

Instance Attribute Details

#best ⇒ Object (readonly)

Returns the value of attribute best.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def best
  @best
end

#id ⇒ Object (readonly)

Returns the value of attribute id.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def id
  @id
end

#last_fits ⇒ Object (readonly)

Returns the value of attribute last_fits.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def last_fits
  @last_fits
end

#mu ⇒ Object (readonly)

Returns the value of attribute mu.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def mu
  @mu
end

#ndims ⇒ Object (readonly)

Returns the value of attribute ndims.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def ndims
  @ndims
end

#obj_fn ⇒ Object (readonly)

Returns the value of attribute obj_fn.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def obj_fn
  @obj_fn
end

#opt_type ⇒ Object (readonly)

Returns the value of attribute opt_type.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def opt_type
  @opt_type
end

#rng ⇒ Object (readonly)

Returns the value of attribute rng.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def rng
  @rng
end

#sigma ⇒ Object (readonly)

Returns the value of attribute sigma.



5
6
7
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 5

def sigma
  @sigma
end

Instance Method Details

#cmaes_lrate ⇒ NMatrix, Float

Magic numbers from CMA-ES (TODO: add proper citation)

Returns:

  • (NMatrix) —

    scale-invariant utilities

  • (Float) —

    learning rate lower bound



58
59
60
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 58

def cmaes_lrate
  (3+Math.log(ndims)) / (5*Math.sqrt(ndims))
end

#cmaes_popsize ⇒ NMatrix, Integer

Magic numbers from CMA-ES (TODO: add proper citation)

Returns:

  • (NMatrix) —

    scale-invariant utilities

  • (Integer) —

    population size lower bound



64
65
66
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 64

def cmaes_popsize
  [5, 4 + (3*Math.log(ndims)).floor].max
end

#cmaes_utilities ⇒ NMatrix

Magic numbers from CMA-ES (TODO: add proper citation)

Returns:

  • (NMatrix) —

    scale-invariant utilities



44
45
46
47
48
49
50
51
52
53
54
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 44

def cmaes_utilities
  # Algorithm equations are meant for fitness maximization
  # Match utilities with individuals sorted by INCREASING fitness
  log_range = (1..popsize).collect do |v|
    [0, Math.log(popsize.to_f/2 - 1) - Math.log(v)].max
  end
  total = log_range.reduce(:+)
  buf = 1.0/popsize
  vals = log_range.collect { |v| v / total - buf }.reverse
  NMatrix[vals, dtype: :float64]
end

#interface_methods ⇒ Object

Declaring interface methods - implement these in child class!



106
107
108
109
110
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 106

[:train, :initialize_distribution, :convergence].each do |mname|
  define_method mname do
    raise NotImplementedError, "Implement in child class!"
  end
end

#lrate ⇒ Object

Memoized automatic magic numbers NOTE: Doubling popsize and halving lrate often helps



40
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 40

def lrate;   @lrate     ||= cmaes_lrate       end

#move_inds(inds) ⇒ NMatrix

Move standard normal samples to current distribution

Returns:

  • (NMatrix) —

    individuals



77
78
79
80
81
82
83
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 77

def move_inds inds
  # TODO: can we reduce the transpositions?
  # sigma.dot(inds.transpose).map(&mu.method(:+)).transpose
  multi_mu = NMatrix[*inds.rows.times.collect {mu.to_a}, dtype: :float64].transpose
  (multi_mu + sigma.dot(inds.transpose)).transpose
  # sigma.dot(inds.transpose).transpose + inds.rows.times.collect {mu.to_a}.to_nm
end

#popsize ⇒ Object

Memoized automatic magic numbers NOTE: Doubling popsize and halving lrate often helps



38
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 38

def popsize; @popsize   ||= cmaes_popsize * 2 end

#sorted_inds ⇒ Object

Sorted individuals NOTE: Algorithm equations are meant for fitness maximization. Utilities need to be matched with individuals sorted by INCREASING fitness. Then reverse order for minimization.

Returns:

  • standard normal samples sorted by the respective individuals’ fitnesses



89
90
91
92
93
94
95
96
97
98
99
100
101
102
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 89

def sorted_inds
  samples = standard_normal_samples
  inds = move_inds(samples).to_a
  fits = obj_fn.call(inds)
  # Quick cure for NaN fitnesses
  fits.map! { |x| x.nan? ? (opt_type==:max ? -1 : 1) * Float::INFINITY : x }
  @last_fits = fits # allows checking for stagnation
  sorted = [fits, inds, samples.to_a].transpose.sort_by(&:first)
  sorted.reverse! if opt_type==:min
  this_best = sorted.last.take(2)
  opt_cmp_fn = opt_type==:min ? :< : :>
  @best = this_best if this_best.first.send(opt_cmp_fn, best.first)
  NMatrix[*sorted.map(&:last), dtype: :float64]
end

#standard_normal_sample ⇒ Float

Box-Muller transform: generates standard (unit) normal distribution samples

Returns:

  • (Float) —

    a single sample from a standard normal distribution



27
28
29
30
31
32
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 27

def standard_normal_sample
  rho = Math.sqrt(-2.0 * Math.log(rng.rand))
  theta = 2 * Math::PI * rng.rand
  tfn = rng.rand > 0.5 ? :cos : :sin
  rho * Math.send(tfn, theta)
end

#standard_normal_samples ⇒ NMatrix

Samples a standard normal distribution to construct a NMatrix of

popsize multivariate samples of length ndims

Returns:

  • (NMatrix) —

    standard normal samples



71
72
73
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 71

def standard_normal_samples
  NMatrix.new([popsize, ndims], dtype: :float64) { standard_normal_sample }
end

#utils ⇒ Object

Memoized automatic magic numbers NOTE: Doubling popsize and halving lrate often helps



36
 
# File 'lib/machine_learning_workbench/optimizer/natural_evolution_strategies/base.rb', line 36

def utils;   @utilities ||= cmaes_utilities   end