Class: Agentic::Verification::LlmVerificationStrategy

Inherits:
VerificationStrategy show all
Defined in:
lib/agentic/verification/llm_verification_strategy.rb

Overview

Verifies task results using an LLM

Instance Attribute Summary

Attributes inherited from VerificationStrategy

#config

Instance Method Summary collapse

Constructor Details

#initialize(llm_client, config = {}) ⇒ LlmVerificationStrategy

Initializes a new LlmVerificationStrategy

Parameters:

  • llm_client (LlmClient) —

    The LLM client to use for verification

  • config (Hash) (defaults to: {}) —

    Configuration options for the strategy



13
14
15
16
 
# File 'lib/agentic/verification/llm_verification_strategy.rb', line 13

def initialize(llm_client, config = {})
  super(config)
  @llm_client = llm_client
end

Instance Method Details

#verify(task, result) ⇒ VerificationResult

Verifies a task result using an LLM

Parameters:

  • task (Task) —

    The task to verify

  • result (TaskResult) —

    The result to verify

Returns:



22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/agentic/verification/llm_verification_strategy.rb', line 22

def verify(task, result)
  unless result.successful?
    return VerificationResult.new(
      task_id: task.id,
      verified: false,
      confidence: 0.0,
      messages: ["Task failed, skipping LLM verification"]
    )
  end

  # In a real implementation, we would send the task and result to the LLM
  # and analyze the LLM's assessment
  # For this stub, we'll simulate a response

  # Example verification prompt
  #   Task Description: #{task.description}
  #   Task Input: #{task.input.inspect}
  #   Task Result: #{result.output.inspect}
  #
  #   Verify if the result satisfies the task requirements.
  #   Consider correctness, completeness, and alignment with the task description.
  #   Provide your assessment with a boolean verdict (verified: true/false) and a confidence score (0.0-1.0).

  # In a real implementation, we would use the LLM client here
  # For this stub, we'll return a simulated verification result
  verified = rand > 0.1 # 90% chance of success for simulation purposes
  confidence = verified ? (0.8 + rand * 0.2) : (0.3 + rand * 0.3)
  message = verified ? "Result meets task requirements" : "Result does not fully satisfy task requirements"

  VerificationResult.new(
    task_id: task.id,
    verified: verified,
    confidence: confidence,
    messages: [message]
  )
end