Module: Measurable::KullbackLeibler
 Included in:
 Measurable
 Defined in:
 lib/measurable/kullback_leibler.rb
Instance Method Summary collapse

#kullback_leibler(p, q) ⇒ Object
callseq: kullback_leibler(p, q) > Float.
Instance Method Details
#kullback_leibler(p, q) ⇒ Object
callseq:
kullback_leibler(p, q) > Float
The KullbackLeibler Divergence between the distributions p
and q
is a measure of their dissimilarity. However, it
doesn't obey the triangular inequality and isn't symmetric, thus it
isn't a metric.
It is calculated as follows:
KL(p, q) = \sum_{i = q}^{N} p[i] * log(p[i] / q[i])
With distributions p
and q
represented as vectors
of N elements summing to 1.0.
References:

Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Processing.
Arguments:

p
> A probability distribution represented by a nelement Array. 
q
> A probability distribution represented by a nelement Array.
Returns:

A measure of the difference between the probability distributions p and q.
28 29 30 31 32 33 34 35 
# File 'lib/measurable/kullback_leibler.rb', line 28 def kullback_leibler(p, q) # TODO: Change this to a more specific, custommade exception. raise ArgumentError if p.size != q.size p.zip(q).reduce(0.0) do acc, probs acc += probs[0] * Math.log(probs[0] / probs[1]) end end 