Class: GuardrailsRuby::Checks::ToxicLanguage

Inherits:
GuardrailsRuby::Check show all
Defined in:
lib/guardrails_ruby/checks/toxic_language.rb

Constant Summary collapse

TOXIC_PATTERNS =

Basic keyword-based detection; LLM-based detection can be added later

[
  /\b(kill|murder|attack)\s+(you|him|her|them)\b/i,
  /\b(threat(en)?|bomb|terroris[mt])\b/i,
  /\bi\s+(will|am going to)\s+(hurt|harm|destroy)\b/i
].freeze

Instance Attribute Summary

Attributes inherited from GuardrailsRuby::Check

#options

Instance Method Summary collapse

Methods inherited from GuardrailsRuby::Check

check_name, direction, #initialize, lookup

Constructor Details

This class inherits a constructor from GuardrailsRuby::Check

Instance Method Details

#call(text, context: {}) ⇒ Object 



16
17
18
19
20
21
22
23
24
25
 
# File 'lib/guardrails_ruby/checks/toxic_language.rb', line 16

def call(text, context: {})
  matched = TOXIC_PATTERNS.select { |p| text.match?(p) }

  if matched.any?
    fail! "Toxic language detected",
      matches: matched.map(&:source)
  else
    pass!
  end
end