Class: Aireview::ModelPool

Inherits:
Object
  • Object
show all
Defined in:
lib/aireview/model_pool.rb

Overview

A routing plan from a shared pool: models in order of priority (the first is the preferred one for Critique). Generate walks the pool from its start model downwards and round again; Critique takes the first live model not below the one that answered in Generate (rank: not_below_generate), self-critique by the same model is the last permitted option, lower only with allow_weaker. A stage with a model of its own does not use the pool: its chain is independent and the rank rule does not apply, as with rank: any.

Constant Summary collapse

CRITIQUE_RANKS = 
%w[not_below_generate any].freeze
DEFAULT_CRITIQUE_RANK = 
'not_below_generate'

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(items:, provider:, limits:, starts: {}, inherited_starts: [], rank: nil, allow_weaker: false, own_chains: nil, only_primary: false, stages: STAGES) ⇒ ModelPool

items — the raw llm.models; starts — the start per stage (a model name or nil); inherited_starts — stages whose start came from a layer below the pool (image defaults versus the project's LLM_MODELS): such a start missing from the pool is replaced by the first model with a warning, an explicit start outside the pool is a configuration error; own_chains — stages with a chain of their own; stages — the stages that go to an LLM in this run: without Critique (Jev decides with no fallback, --no-critique) the plan has no critique chain and no critique policy to validate or to sign. Ten named settings read better than a struct for its own sake.



73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/aireview/model_pool.rb', line 73

def initialize(items:, provider:, limits:, starts: {}, inherited_starts: [], rank: nil, allow_weaker: false, # rubocop:disable Metrics/ParameterLists
               own_chains: nil, only_primary: false, stages: STAGES)
  @stages = stages.map(&:to_s)
  @items = parse_items(items, provider)
  @limits = limits.transform_keys(&:to_s)
  @rank, @allow_weaker = critique_policy(rank, allow_weaker)
  @own_chains = own_chains || StageChains.new({})
  @only_primary = only_primary
  @warnings = []
  @starts = resolve_starts(starts.transform_keys(&:to_s), inherited_starts.map(&:to_s))
end

Instance Attribute Details

#warnings ⇒ Object (readonly)

Returns the value of attribute warnings.



21
22
23
 
# File 'lib/aireview/model_pool.rb', line 21

def warnings
  @warnings
end

Class Method Details

.item_candidate(item) ⇒ Object 



34
35
36
 
# File 'lib/aireview/model_pool.rb', line 34

def self.item_candidate(item)
  ModelCandidate.new(provider: item[:provider], model: item[:model], api_base: item[:api_base])
end

.item_name(item) ⇒ Object

The full name of a pool item, with the address of a server of your own.



39
40
41
 
# File 'lib/aireview/model_pool.rb', line 39

def self.item_name(item)
  item_candidate(item).to_s
end

.member?(items, provider, model) ⇒ Boolean

Whether a model is in the raw llm.models list — without building the plan, so that a CLI override does not trip over an invalid start of the old plan.

Returns:

  • (Boolean) 


26
27
28
29
30
31
32
 
# File 'lib/aireview/model_pool.rb', line 26

def self.member?(items, provider, model)
  return false if Aireview::Utils.blank?(model)

  Array(items).each_with_index.any? do |item, index|
    names(parse_item(item, index, provider)).include?(model.to_s)
  end
end

.names(item) ⇒ Object

How a model may be named in a start or a CLI override (ModelCandidate#names).



44
45
46
 
# File 'lib/aireview/model_pool.rb', line 44

def self.names(item)
  item_candidate(item).names
end

.parse_item(item, index, provider) ⇒ Object

Raises:



48
49
50
51
52
53
54
55
56
57
58
59
60
61
 
# File 'lib/aireview/model_pool.rb', line 48

def self.parse_item(item, index, provider)
  item = ModelCandidate.parse_item(item)
  name = "llm.models[#{index}]"
  raise ConfigError, "#{name} must be a model name or a hash with model" unless item.is_a?(Hash)
  raise ConfigError, "#{name}.model is required" if Aireview::Utils.blank?(item['model'])

  limit = item['max_prompt_chars']
  {
    provider: (item['provider'] || provider).to_s,
    model: item['model'].to_s,
    max_prompt_chars: limit.nil? ? nil : StageChains.positive_limit(limit, name),
    api_base: ModelCandidate.api_base(item['api_base'], "#{name}.api_base")
  }
end

Instance Method Details

#chain(stage) ⇒ Object

Raises:

  • (ArgumentError) 


97
98
99
100
101
102
103
 
# File 'lib/aireview/model_pool.rb', line 97

def chain(stage)
  stage = stage.to_s
  raise ArgumentError, "unknown LLM stage #{stage.inspect}" unless stage?(stage)
  return @own_chains.chain(stage) unless pool_stage?(stage)

  trim(pool(stage).rotate(index_of(@starts[stage] || @items.first[:model])))
end

#critique_chain(after:) ⇒ Object

The models not below the one that answered in Generate, in pool order (including that one), then — only with allow_weaker — the rest. An explicit critique start goes first only when it is permitted itself: Generate walks round the pool and may answer with a model above its own start, so this cannot be checked statically; a start that is not permitted does not bypass the ban on a weaker critique, it is skipped.



111
112
113
114
115
116
117
118
119
 
# File 'lib/aireview/model_pool.rb', line 111

def critique_chain(after:)
  return chain('critique') unless rank_applies?(after)

  models = pool('critique')
  limit = index_of(after)
  allowed = models.first(limit + 1)
  allowed += models.drop(limit + 1) if @allow_weaker
  trim(with_start_first(allowed, @starts['critique']))
end

#pool(stage = 'generate') ⇒ Object 



89
90
91
92
93
94
95
 
# File 'lib/aireview/model_pool.rb', line 89

def pool(stage = 'generate')
  limit = @limits.fetch(stage.to_s)
  @items.map do |item|
    ModelCandidate.new(provider: item[:provider], model: item[:model],
                       max_prompt_chars: item[:max_prompt_chars] || limit, api_base: item[:api_base])
  end
end

#pool? ⇒ Boolean

Returns:

  • (Boolean) 


152
153
154
 
# File 'lib/aireview/model_pool.rb', line 152

def pool?
  true
end

#pool_member?(model) ⇒ Boolean

Returns:

  • (Boolean) 


160
161
162
163
164
 
# File 'lib/aireview/model_pool.rb', line 160

def pool_member?(model)
  return false if model.nil?

  @items.any? { |item| match?(item, model) }
end

#pool_stage?(stage) ⇒ Boolean

Returns:

  • (Boolean) 


156
157
158
 
# File 'lib/aireview/model_pool.rb', line 156

def pool_stage?(stage)
  stage?(stage) && !@own_chains.stage?(stage)
end

#primary(stage) ⇒ Object 



121
122
123
 
# File 'lib/aireview/model_pool.rb', line 121

def primary(stage)
  chain(stage).first
end

#rule ⇒ Object

The critique selection rule in words, for --dry-run.



145
146
147
148
149
150
 
# File 'lib/aireview/model_pool.rb', line 145

def rule
  return nil unless stage?('critique') && all_stages_in_pool?
  return @rank if @rank == 'any' || !@allow_weaker

  "#{@rank}, weaker allowed"
end

#signature ⇒ Object

The order and policy of the pool go into the review key: they decide which model checks the findings. nil when a stage is outside the pool. Without Critique only the generate part: a critique setting nobody uses must not change the key, and the pool must stay in it.



135
136
137
138
139
140
141
142
 
# File 'lib/aireview/model_pool.rb', line 135

def signature
  return nil unless all_stages_in_pool?

  signature = {'models' => pool.map(&:to_s), 'generate_start' => @starts['generate']}
  return signature unless stage?('critique')

  signature.merge('critique_start' => @starts['critique'], 'rank' => @rank, 'allow_weaker' => @allow_weaker)
end

#stage?(stage) ⇒ Boolean

Returns:

  • (Boolean) 


85
86
87
 
# File 'lib/aireview/model_pool.rb', line 85

def stage?(stage)
  @stages.include?(stage.to_s)
end

#start_used?(stage) ⇒ Boolean

Returns:

  • (Boolean) 


166
167
168
 
# File 'lib/aireview/model_pool.rb', line 166

def start_used?(stage)
  !@starts[stage.to_s].nil?
end

#weaker?(critique_candidate, generate_candidate) ⇒ Boolean

Returns:

  • (Boolean) 


125
126
127
128
129
 
# File 'lib/aireview/model_pool.rb', line 125

def weaker?(critique_candidate, generate_candidate)
  return false unless pool_member?(critique_candidate) && pool_member?(generate_candidate)

  index_of(critique_candidate) > index_of(generate_candidate)
end