# Module: Measurable::KullbackLeibler

Included in:
Measurable
Defined in:
lib/measurable/kullback_leibler.rb

## Instance Method Summary collapse

• call-seq: kullback_leibler(p, q) -> Float.

## Instance Method Details

### #kullback_leibler(p, q) ⇒ Object

call-seq:

kullback_leibler(p, q) -> Float


The Kullback-Leibler Divergence between the distributions p and q is a measure of their dissimilarity. However, it doesn't obey the triangular inequality and isn't symmetric, thus it isn't a metric.

It is calculated as follows:

KL(p, q) = \sum_{i = q}^{N} p[i] * log(p[i] / q[i])


With distributions p and q represented as vectors of N elements summing to 1.0.

References:

Arguments:

• p -> A probability distribution represented by a n-element Array.

• q -> A probability distribution represented by a n-element Array.

Returns:

• A measure of the difference between the probability distributions p and q.

Raises:

• (ArgumentError)
  28 29 30 31 32 33 34 35 # File 'lib/measurable/kullback_leibler.rb', line 28 def kullback_leibler(p, q) # TODO: Change this to a more specific, custom-made exception. raise ArgumentError if p.size != q.size p.zip(q).reduce(0.0) do |acc, probs| acc += probs[0] * Math.log(probs[0] / probs[1]) end end