Module: OllamaChat::Compaction

Included in:
Chat
Defined in:
lib/ollama_chat/compaction.rb

Overview

Provides compaction support for chat sessions. Mixed into Chat.

Supplies configuration resolvers that derive concrete token budgets from the compaction: config block (ratios of num_ctx with absolute floors), the LLM-based summarization pipeline (+summarize_for_compaction+), and the tool_summary_line resolver for per-tool summary templates.

See Also:

  • for the `compaction:` block structure (`reserve`, `keep_recent`, `summary`).

Defined Under Namespace

Classes: Result

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.tool_summary_line(tool_name, result) ⇒ String

Generates a one-line summary for a tool call result.

Looks up the registered tool class and delegates to its summary_template(result:) class method. Falls back to a generic sentence if the tool is not registered or the template raises.

Parameters:

  • the registered name, e.g. 'read_file'.

  • the raw JSON result string from execute.

Returns:

  • a short natural-language description of the call.



22
23
24
25
26
27
28
 
# File 'lib/ollama_chat/compaction.rb', line 22

def tool_summary_line(tool_name, result)
  default_summary = "was called."
  klass           = OllamaChat::Tools.registered[tool_name.to_s]&.class
  klass&.summary_template(result:) || default_summary
rescue StandardError
  default_summary
end

Instance Method Details

#compact_ratio_tokens(name, tokens) ⇒ Integer

Resolves the effective token budget for a concern given a context size.

Computes max(ratio * tokens, min_tokens).floor, so the floor acts as a guaranteed minimum even for very small context windows.

Parameters:

  • the concern name, e.g. :reserve, :keep_recent, or :summary.

  • the context window size (num_ctx) to derive the budget from.

Returns:

  • the resolved token budget.



91
92
93
94
95
96
 
# File 'lib/ollama_chat/compaction.rb', line 91

def compact_ratio_tokens(name, tokens)
  [
    compact_ratio(name).to_f * tokens,
    compact_min_tokens(name).to_f,
  ].max.floor
end

#compact_with_retry ⇒ Boolean

Attempts compaction, prompting the user to retry on failure.

compact! is idempotent on failure: the LLM call (and thus any CompactionError) happens before the message list is mutated, so a retry always starts from a clean state.

Returns:

  • true if compaction succeeded, false if the user declined to retry.



106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/ollama_chat/compaction.rb', line 106

def compact_with_retry
  result   = messages.compact!
  session_sync
  if result
    report_compaction(result)
  else
    STDOUT.puts('Nothing to compact.')
  end
  return true
rescue OllamaChat::CompactionError => e
  STDERR.puts "⚠️  Compaction failed: #{e.message}"
  log(:error, "Compaction failed", data: {
    model: @model,
    ctx:   current_context_length,
    size:  messages.size,
  })
  retry if confirm?(
    prompt: '🔔 Retry compaction? (y/n) ',
    yes: /\Ay/i
  )
  false
end

#summarize_for_compaction(messages:, previous_summary: nil) ⇒ Array(String, Array<Hash>)

Generates a context summary for the given messages via LLM.

Serializes non-system messages into a structured prompt, calls the model via generate, and assembles the final summary with deterministic tool-call entries stored out-of-band.

Parameters:

  • the messages to summarize.

  • (defaults to: nil)

    the previous summary message for iterative re-compaction.

Returns:

  • the assembled content and the merged tool-call entries array.



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
 
# File 'lib/ollama_chat/compaction.rb', line 62

def summarize_for_compaction(messages:, previous_summary: nil)
  groups       = serialize_groups(messages)
  tool_entries = build_tool_entries(messages)

  es = OllamaChat::TokenEstimator.estimate(groups)
  log(:info, "Compaction: #{messages.size} messages, " \
    "#{es.tokens_formatted} (#{es.bytes_formatted}) groups")

  narrative_prev = ''
  old_tool_calls = []
  if previous_summary
    narrative_prev = extract_narrative(previous_summary.content)
    old_tool_calls = extract_old_tool_calls(previous_summary)
  end

  narrative = call_summarizer(groups:, previous_summary: narrative_prev)
  assemble_summary(narrative, tool_entries, old_tool_calls)
end