Class: Aireview::JevStage

Inherits:
Object
  • Object
show all
Defined in:
lib/aireview/jev_stage.rb

Overview

The Critique stage with llm.critique.engine: jev. Jev decides keep or reject; the candidates it cannot judge (not enough context, too large) and, when Jev fails, all of them go to the LLM Critique with llm.jev.fallback: model, or are rejected / fail the run with fail. Duplicates are dropped once, over the merged verdicts of Jev and the LLM: neither side sees what the other kept.

Defined Under Namespace

Classes: Outcome

Instance Method Summary collapse

Constructor Details

#initialize(config:, critic:, logger: Logger.new($stderr)) ⇒ JevStage

Returns a new instance of JevStage.



18
19
20
21
22
 
# File 'lib/aireview/jev_stage.rb', line 18

def initialize(config:, critic:, logger: Logger.new($stderr))
  @config = config
  @critic = critic
  @logger = logger
end

Instance Method Details

#run(context:, candidates:, llm_critique:) ⇒ Object

llm_critique — the LLM Critique of the pipeline for a subset of the candidates; returns the kept ones, refined.



26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
 
# File 'lib/aireview/jev_stage.rb', line 26

def run(context:, candidates:, llm_critique:)
  @logger.info("Pipeline critique pass started (engine=jev, model=#{@config.jev_model})")
  begin
    result = @critic.assess(context: context, candidates: candidates, dedupe: false)
  rescue JevError => e
    return fall_back(e, candidates, llm_critique)
  end
  result.assessments.each { |assessment| log(assessment) }

  unverifiable = pick(candidates, result, :unverifiable)
  accepted = pick(candidates, result, :keep) + judge_unverifiable(unverifiable, llm_critique)
  accepted = drop_duplicates(in_generate_order(accepted, candidates), result.answers)
  @logger.info("Pipeline critique pass completed with #{accepted.size} kept candidate(s) (engine=jev, " \
               "model=#{result.model}, requests=#{result.requests})")
  Outcome.new(accepted: accepted, note: unverifiable.empty? || fail? ? :jev : :jev_partial)
end