Class: RubyLLM::Modes::Classifiers::Judge

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby_llm/modes/classifiers/judge.rb

Overview

The :judge backend: one RubyLLM.judge call with a single choice question over the available modes.

The state is the routing input as data (instructions, conversation, latest message); the mode descriptions are the choice options. The answer carries a probability per mode and a confidence reported by the judgment model with it: how concentrated the distribution is on one mode, not a self-report, and a different scale from the chat backend's. It is passed through as is. There is no free text, so reason is always nil.

model is passed to RubyLLM.judge as given (nil selects RubyLLM's default judgment model); provider only when given. judge: replaces RubyLLM.judge; it is called with the same arguments and must return a Judgment.

RubyLLM.judge is not in every RubyLLM release. Without it the backend cannot run, and the router says so when it is built.

Constant Summary collapse

QUESTION = 
"Which mode should answer the latest user message? Use the conversation\nonly to understand what the latest message refers to.\n".strip

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model: nil, provider: nil, judge: nil) ⇒ Judge

Returns a new instance of Judge.



36
37
38
39
40
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 36

def initialize(model: nil, provider: nil, judge: nil)
  @model = model
  @provider = provider
  @judge = judge
end

Instance Attribute Details

#model ⇒ Object (readonly)

Returns the value of attribute model.



30
31
32
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 30

def model
  @model
end

#provider ⇒ Object (readonly)

Returns the value of attribute provider.



30
31
32
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 30

def provider
  @provider
end

Class Method Details

.available? ⇒ Boolean

Returns:

  • (Boolean) 


32
33
34
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 32

def self.available?
  RubyLLM.respond_to?(:judge)
end

.questions(modes) ⇒ Object

One choice question whose options are the modes (Registration values), name to description.



67
68
69
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 67

def self.questions(modes)
  { mode: { type: :choice, instructions: QUESTION, options: modes.to_h { |mode| [ mode.name.to_s, mode.description.to_s ] } } }
end

.state(message:, history:, instructions: nil) ⇒ Object

The judgment state: instructions when present, the normalised history as a conversation, and the latest message.



57
58
59
60
61
62
63
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 57

def self.state(message:, history:, instructions: nil)
  state = {}
  state["instructions"] = instructions unless instructions.nil? || instructions.empty?
  state["conversation"] = history.map { |entry| transcript_entry(entry) } if history.any?
  state["latest_message"] = message.to_s
  state
end

Instance Method Details

#call(message:, history:, modes:, instructions:, inputs:) ⇒ Object 



42
43
44
45
46
47
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 42

def call(message:, history:, modes:, instructions:, inputs:)
  @resolved_model = nil
  state = self.class.state(message:, history:, instructions:)
  judgment = judge.call(state, questions: self.class.questions(modes), **model_options)
  decision_from(judgment)
end

#trace ⇒ Object

What ran, for Route#classifier: the model the provider reported once a judgment came back, else the declared model.



51
52
53
 
# File 'lib/ruby_llm/modes/classifiers/judge.rb', line 51

def trace
  { with: "judge", model: @resolved_model || model }
end