Class: GuardrailsRuby::Checks::Relevance

Inherits:
GuardrailsRuby::Check show all
Defined in:
lib/guardrails_ruby/checks/relevance.rb

Instance Attribute Summary

Attributes inherited from GuardrailsRuby::Check

#options

Instance Method Summary collapse

Methods inherited from GuardrailsRuby::Check

check_name, direction, #initialize, lookup

Constructor Details

This class inherits a constructor from GuardrailsRuby::Check

Instance Method Details

#call(text, context: {}) ⇒ Object

This check requires LLM-based evaluation for real accuracy. For now, a simple heuristic: check if any words from the input appear in the output.



11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
 
# File 'lib/guardrails_ruby/checks/relevance.rb', line 11

def call(text, context: {})
  input_text = context[:input]
  return pass! unless input_text

  input_words = significant_words(input_text)
  return pass! if input_words.empty?

  output_lower = text.downcase
  overlap = input_words.count { |w| output_lower.include?(w) }
  ratio = overlap.to_f / input_words.size

  if ratio < 0.1 && text.length > 20
    fail! "Output may not be relevant to the input (low keyword overlap)",
      action: @options.fetch(:action, :warn)
  else
    pass!
  end
end