Class: EvalRuby::Result

Inherits:
Object
  • Object
show all
Defined in:
lib/eval_ruby/result.rb

Overview

Holds evaluation scores and details for a single sample.

Examples:

result = EvalRuby.evaluate(question: "...", answer: "...", context: [...])
result.faithfulness  # => 0.95
result.overall       # => 0.87

Constant Summary collapse

METRICS = 
%i[faithfulness relevance correctness context_precision context_recall].freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(scores: {}, details: {}) ⇒ Result

Returns a new instance of Result.

Parameters:

  • scores (Hash{Symbol => Float}) (defaults to: {})
  • details (Hash{Symbol => Hash}) (defaults to: {}) 


21
22
23
24
 
# File 'lib/eval_ruby/result.rb', line 21

def initialize(scores: {}, details: {})
  @scores = scores
  @details = details
end

Instance Attribute Details

#details ⇒ Hash{Symbol => Hash} (readonly)

Returns metric name to details mapping.

Returns:

  • (Hash{Symbol => Hash}) —

    metric name to details mapping



17
18
19
 
# File 'lib/eval_ruby/result.rb', line 17

def details
  @details
end

#scores ⇒ Hash{Symbol => Float} (readonly)

Returns metric name to score mapping.

Returns:

  • (Hash{Symbol => Float}) —

    metric name to score mapping



14
15
16
 
# File 'lib/eval_ruby/result.rb', line 14

def scores
  @scores
end

Instance Method Details

#overall(weights: nil) ⇒ Float?

Computes a weighted average of all available scores.

Parameters:

  • weights (Hash{Symbol => Float}, nil) (defaults to: nil) —

    custom weights per metric

Returns:

  • (Float, nil) —

    weighted average score, or nil if no scores available



44
45
46
47
48
49
50
51
 
# File 'lib/eval_ruby/result.rb', line 44

def overall(weights: nil)
  weights ||= METRICS.each_with_object({}) { |m, h| h[m] = 1.0 }
  available = @scores.select { |k, v| weights.key?(k) && v }
  return nil if available.empty?

  total_weight = available.sum { |k, _| weights[k] }
  available.sum { |k, v| v * weights[k] } / total_weight
end

#to_h ⇒ Hash

Returns scores plus overall.

Returns:

  • (Hash) —

    scores plus overall



54
55
56
 
# File 'lib/eval_ruby/result.rb', line 54

def to_h
  @scores.merge(overall: overall)
end

#to_s ⇒ String

Returns human-readable summary.

Returns:

  • (String) —

    human-readable summary



59
60
61
62
63
 
# File 'lib/eval_ruby/result.rb', line 59

def to_s
  lines = @scores.map { |k, v| "  #{k}: #{v&.round(4) || 'N/A'}" }
  lines << "  overall: #{overall&.round(4) || 'N/A'}"
  "EvalRuby::Result\n#{lines.join("\n")}"
end