Class: EvalRuby::Metrics::PrecisionAtK

Inherits:
Base
  • Object
show all
Defined in:
lib/eval_ruby/metrics/precision_at_k.rb

Overview

Computes Precision@K: the fraction of top-k retrieved documents that are relevant.

Examples:

PrecisionAtK.new.call(retrieved: ["a", "b", "c"], relevant: ["a", "c"], k: 3)
# => 0.667

Instance Attribute Summary

Attributes inherited from Base

#judge

Instance Method Summary collapse

Methods inherited from Base

#initialize

Constructor Details

This class inherits a constructor from EvalRuby::Metrics::Base

Instance Method Details

#call(retrieved:, relevant:, k: nil, **_kwargs) ⇒ Float

Returns precision score (0.0-1.0).

Parameters:

  • retrieved (Array<String>) —

    retrieved document IDs in ranked order

  • relevant (Array<String>) —

    ground-truth relevant document IDs

  • k (Integer, nil) (defaults to: nil) —

    number of top results (nil for all)

Returns:

  • (Float) —

    precision score (0.0-1.0)



15
16
17
18
19
20
21
22
 
# File 'lib/eval_ruby/metrics/precision_at_k.rb', line 15

def call(retrieved:, relevant:, k: nil, **_kwargs)
  k ||= retrieved.length
  top_k = retrieved.first(k)
  return 0.0 if top_k.empty?

  hits = top_k.count { |doc| relevant.include?(doc) }
  hits.to_f / top_k.size
end