Class: EvalRuby::Metrics::NDCG

Inherits:
Base
  • Object
show all
Defined in:
lib/eval_ruby/metrics/ndcg.rb

Overview

Computes Normalized Discounted Cumulative Gain (NDCG). Measures ranking quality by comparing actual ranking to ideal ranking.

Examples:

NDCG.new.call(retrieved: ["a", "b", "c"], relevant: ["a", "c"])
# => 0.863

Instance Attribute Summary

Attributes inherited from Base

#judge

Instance Method Summary collapse

Methods inherited from Base

#initialize

Constructor Details

This class inherits a constructor from EvalRuby::Metrics::Base

Instance Method Details

#call(retrieved:, relevant:, k: nil, **_kwargs) ⇒ Float

Returns NDCG score (0.0-1.0).

Parameters:

  • retrieved (Array<String>) —

    retrieved document IDs in ranked order

  • relevant (Array<String>) —

    ground-truth relevant document IDs

  • k (Integer, nil) (defaults to: nil) —

    number of top results (nil for all)

Returns:

  • (Float) —

    NDCG score (0.0-1.0)



16
17
18
19
20
21
22
23
24
25
26
27
28
29
 
# File 'lib/eval_ruby/metrics/ndcg.rb', line 16

def call(retrieved:, relevant:, k: nil, **_kwargs)
  k ||= retrieved.length
  top_k = retrieved.first(k)

  dcg = top_k.each_with_index.sum do |doc, i|
    rel = relevant.include?(doc) ? 1.0 : 0.0
    rel / Math.log2(i + 2)
  end

  ideal_length = [relevant.length, k].min
  idcg = ideal_length.times.sum { |i| 1.0 / Math.log2(i + 2) }

  idcg.zero? ? 0.0 : dcg / idcg
end