Module: Measurable::KullbackLeibler

Included in:
Measurable
Defined in:
lib/measurable/kullback_leibler.rb

Instance Method Summary collapse

Instance Method Details

#kullback_leibler(p, q) ⇒ Object

call-seq:

kullback_leibler(p, q) -> Float

The Kullback-Leibler Divergence between the distributions p and q is a measure of their dissimilarity. However, it doesn’t obey the triangular inequality and isn’t symmetric, thus it isn’t a metric.

It is calculated as follows:

KL(p, q) = \sum_{i = q}^{N} p[i] * log(p[i] / q[i])

With distributions p and q represented as vectors of N elements summing to 1.0.

References:

Arguments:

  • p -> A probability distribution represented by a n-element Array.

  • q -> A probability distribution represented by a n-element Array.

Returns:

  • A measure of the difference between the probability distributions p and q.

Raises:

  • (ArgumentError) 


28
29
30
31
32
33
34
35
 
# File 'lib/measurable/kullback_leibler.rb', line 28

def kullback_leibler(p, q)
  # TODO: Change this to a more specific, custom-made exception.
  raise ArgumentError if p.size != q.size

  p.zip(q).reduce(0.0) do |acc, probs|
    acc += probs[0] * Math.log(probs[0] / probs[1])
  end
end