Class: EvalRuby::Metrics::Base

Inherits:

Object

• Object
• EvalRuby::Metrics::Base

Defined in:

lib/eval_ruby/metrics/base.rb

Overview

Abstract base class for all evaluation metrics. Subclasses must implement #call.

Direct Known Subclasses

ContextPrecision, ContextRecall, Correctness, Faithfulness, MRR, NDCG, PrecisionAtK, RecallAtK, Relevance

Instance Attribute Summary

• #judge ⇒ EvalRuby::Judges::Base^? readonly

  The LLM judge instance.

Instance Method Summary

• #call(**kwargs) ⇒ Hash{Symbol => Object}

  Evaluates the metric.

• #initialize(judge: nil) ⇒ Base constructor

  A new instance of Base.

Constructor Details

#initialize(judge: nil) ⇒ Base

Returns a new instance of Base.

Parameters:

• judge (EvalRuby::Judges::Base, nil) (defaults to: nil) —

  LLM judge for evaluation

# File 'lib/eval_ruby/metrics/base.rb', line 12

def initialize(judge: nil)
  @judge = judge
end

Instance Attribute Details

#judge ⇒ EvalRuby::Judges::Base^? (readonly)

Returns the LLM judge instance.

Returns:

• (EvalRuby::Judges::Base, nil) —

  the LLM judge instance

# File 'lib/eval_ruby/metrics/base.rb', line 9

def judge
  @judge
end

Instance Method Details

#call(**kwargs) ⇒ Hash{Symbol => Object}

Evaluates the metric.

Parameters:

• kwargs (Hash) —

  metric-specific keyword arguments

Returns:

• (Hash{Symbol => Object}) —

  must include :score and :details keys

Raises:

• (NotImplementedError)

# File 'lib/eval_ruby/metrics/base.rb', line 20

def call(**kwargs)
  raise NotImplementedError, "#{self.class}#call must be implemented"
end