Class: RubyLLM::Chat

Inherits:
Object
  • Object
show all
Includes:
Enumerable, Support::Inspectable
Defined in:
lib/ruby_llm/chat.rb,
lib/ruby_llm/chat/tool_concurrency.rb

Overview

A Chat is a conversation with an AI model. It holds the messages exchanged so far, the tools the model may call, and the settings applied to each request. RubyLLM.chat is the usual way to create one.

chat = RubyLLM.chat
chat.ask "What's the best way to learn Ruby?"

Configuration methods return self, so calls chain:

chat = RubyLLM.chat(model: 'claude-sonnet-5')
chat.with_instructions("Be terse.").with_tools(Weather)

#ask runs the conversation loop, executing tools until the model answers or a call needs approval. #ask_later, #generate, #run_tools, and #step expose individual operations. Resume an approval pause with #approve or #deny followed by #complete.

A Chat is Enumerable over its messages.

Defined Under Namespace

Modules: ToolConcurrency

Constant Summary collapse

COMPACTION_OPTIONS =

The provider-neutral options #with_compaction accepts.

i[at instructions pause_after].freeze

Constants included from Support::Inspectable

Support::Inspectable::TRUNCATE_AT

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Support::Inspectable

#full_inspect, #inspect, #pretty_print

Constructor Details

#initialize(model: nil, provider: nil, protocol: nil, assume_model_exists: false, context: nil) ⇒ Chat

Creates a chat with model:, or with the configured default model when model: is nil. Most code calls RubyLLM.chat instead.

A model is identified by its name, an optional provider:, and an optional protocol:. Pass provider: to disambiguate models available from several providers, and protocol: to override the wire protocol the provider would otherwise pick for the model. With assume_model_exists: true the registry lookup is skipped, which requires provider:. Pass a Context as context: to use its configuration instead of the global one.



109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/ruby_llm/chat.rb', line 109

def initialize(model: nil, provider: nil, protocol: nil, assume_model_exists: false, context: nil)
  if assume_model_exists && !provider
    raise ArgumentError, 'Provider must be specified if assume_model_exists is true'
  end

  @context = context
  @config = context&.config || RubyLLM.config
  with_model(model, provider: provider, protocol: protocol, assume_model_exists: assume_model_exists)
  @temperature = nil
  @max_output_tokens = nil
  @messages = []
  @usage_entries = []
  @tools = {}
  @provider_tools = []
  @tool_prefs = { choice: nil, calls: nil }
  @concurrency = normalize_tool_concurrency(@config.tool_concurrency)
  @provider_options = {}
  @headers = {}
  @schema = nil
  @thinking = nil
  @citations = false
  @caching = nil
  @compaction = nil
  @end_user = nil
  @fallbacks = []
  @fallback_errors = Fallback::DEFAULT_ERRORS
  @callbacks = Hash.new { |callbacks, name| callbacks[name] = [] }
  @cancelled = false
  @cancellation_checker = nil
  @tool_call_decisions = {}
  @approval_checker = nil
end

Instance Attribute Details

#approval_checker=(value) ⇒ Object (writeonly)

Hooks installed by the Rails integration.



805
806
807
 
# File 'lib/ruby_llm/chat.rb', line 805

def approval_checker=(value)
  @approval_checker = value
end

#caching ⇒ Object (readonly)

The prompt caching options set with #with_caching, false when explicitly disabled, or nil when not configured.



71
72
73
 
# File 'lib/ruby_llm/chat.rb', line 71

def caching
  @caching
end

#cancellation_checker=(value) ⇒ Object (writeonly)

Hooks installed by the Rails integration.



805
806
807
 
# File 'lib/ruby_llm/chat.rb', line 805

def cancellation_checker=(value)
  @cancellation_checker = value
end

#citations ⇒ Object (readonly)

Whether #with_citations asked the provider for citations.



85
86
87
 
# File 'lib/ruby_llm/chat.rb', line 85

def citations
  @citations
end

#compaction ⇒ Object (readonly)

The context compaction options set with #with_compaction, false when explicitly disabled, or nil when not configured.



75
76
77
 
# File 'lib/ruby_llm/chat.rb', line 75

def compaction
  @compaction
end

#concurrency ⇒ Object (readonly)

The tool concurrency mode, or nil when tools run sequentially.



67
68
69
 
# File 'lib/ruby_llm/chat.rb', line 67

def concurrency
  @concurrency
end

#context ⇒ Object (readonly)

The Context this chat sends requests through, or nil for the global configuration.



89
90
91
 
# File 'lib/ruby_llm/chat.rb', line 89

def context
  @context
end

#end_user ⇒ Object (readonly)

The opaque per-user identifier set with #with_end_user, or nil.



79
80
81
 
# File 'lib/ruby_llm/chat.rb', line 79

def end_user
  @end_user
end

#fallback_errors ⇒ Object (readonly)

:nodoc:



91
92
93
 
# File 'lib/ruby_llm/chat.rb', line 91

def fallback_errors
  @fallback_errors
end

#fallbacks ⇒ Object (readonly)

The Fallback models tried in order when generation fails.



82
83
84
 
# File 'lib/ruby_llm/chat.rb', line 82

def fallbacks
  @fallbacks
end

#headers ⇒ Object (readonly)

Extra HTTP headers set with #with_headers.



54
55
56
 
# File 'lib/ruby_llm/chat.rb', line 54

def headers
  @headers
end

#max_output_tokens ⇒ Object (readonly)

The output cap set with #with_max_output_tokens, or nil.



61
62
63
 
# File 'lib/ruby_llm/chat.rb', line 61

def max_output_tokens
  @max_output_tokens
end

#messages ⇒ Object

The Message objects exchanged so far, including system instructions.



40
41
42
 
# File 'lib/ruby_llm/chat.rb', line 40

def messages
  @messages
end

#model ⇒ Object (readonly)

The Model the chat sends requests to.



34
35
36
 
# File 'lib/ruby_llm/chat.rb', line 34

def model
  @model
end

#provider ⇒ Object (readonly)

The Provider instance handling requests for the current model.



37
38
39
 
# File 'lib/ruby_llm/chat.rb', line 37

def provider
  @provider
end

#provider_options ⇒ Object (readonly)

Extra request options set with #with_provider_options, expressed in the provider's request vocabulary.



51
52
53
 
# File 'lib/ruby_llm/chat.rb', line 51

def provider_options
  @provider_options
end

#provider_tools ⇒ Object (readonly)

The provider tools enabled with #with_provider_tools, as an array of normalized entry Hashes.



47
48
49
 
# File 'lib/ruby_llm/chat.rb', line 47

def provider_tools
  @provider_tools
end

#schema ⇒ Object (readonly)

The normalized structured output schema set with #with_schema, or nil.



64
65
66
 
# File 'lib/ruby_llm/chat.rb', line 64

def schema
  @schema
end

#temperature ⇒ Object (readonly)

The sampling temperature set with #with_temperature, or nil to let the model use its default.



58
59
60
 
# File 'lib/ruby_llm/chat.rb', line 58

def temperature
  @temperature
end

#tool_prefs ⇒ Object (readonly)

:nodoc:



91
92
93
 
# File 'lib/ruby_llm/chat.rb', line 91

def tool_prefs
  @tool_prefs
end

#tools ⇒ Object (readonly)

The registered tools, as a Hash of tool name Symbols to Tool instances.



43
44
45
 
# File 'lib/ruby_llm/chat.rb', line 43

def tools
  @tools
end

#usage_entries ⇒ Object

:nodoc:



91
92
93
 
# File 'lib/ruby_llm/chat.rb', line 91

def usage_entries
  @usage_entries
end

#usage_recorder=(value) ⇒ Object (writeonly)

Hooks installed by the Rails integration.



805
806
807
 
# File 'lib/ruby_llm/chat.rb', line 805

def usage_recorder=(value)
  @usage_recorder = value
end

Instance Method Details

#add_completion(response, record_usage: false) ⇒ Object

Receives a completion produced out-of-band (e.g. by a batch), running the same callbacks as a synchronous completion so persistence works unchanged.



833
834
835
836
837
838
839
840
841
842
843
 
# File 'lib/ruby_llm/chat.rb', line 833

def add_completion(response, record_usage: false) # :nodoc:
  if response.ruby_llm_usage_entries.empty?
    record_out_of_band_usage(response)
  elsif record_usage
    response.ruby_llm_usage_entries.each { |entry| record_usage_entry(entry) }
  end
  run_callbacks(:before_message)
  add_message response
  run_callbacks(:after_message, response)
  response
end

#add_message(message_or_attributes) ⇒ Object

Appends a message to the conversation and returns it as a Message. Accepts a Message, an attribute Hash, or a record responding to to_llm.

chat.add_message(role: :user, content: "What's the capital of France?")



813
814
815
816
817
 
# File 'lib/ruby_llm/chat.rb', line 813

def add_message(message_or_attributes)
  message = coerce_message(message_or_attributes)
  messages << message
  message
end

#after_fallback ⇒ Object

Registers a callback that receives the Fallback attempt once it has succeeded or failed. Returns self.



698
699
700
 
# File 'lib/ruby_llm/chat.rb', line 698

def after_fallback(&)
  add_callback(:after_fallback, &)
end

#after_message ⇒ Object

Registers a callback that receives each assistant response and each tool result message once it has been appended. Returns self.

chat.after_message { |message| puts message.content }



670
671
672
 
# File 'lib/ruby_llm/chat.rb', line 670

def after_message(&)
  add_callback(:after_message, &)
end

#after_tool_result ⇒ Object

Registers a callback that receives each local tool's result after execution. Returns self.



685
686
687
 
# File 'lib/ruby_llm/chat.rb', line 685

def after_tool_result(&)
  add_callback(:after_tool_result, &)
end

#approve(tool_call) ⇒ Object

Records approval for tool_call, a ToolCall or its id, so the next #complete or #run_tools executes a local tool or records permission for the provider to execute a remote tool on the next request. Returns self.

chat.approve(tool_call)
chat.complete



256
257
258
 
# File 'lib/ruby_llm/chat.rb', line 256

def approve(tool_call)
  record_tool_call_decision(tool_call, true)
end

#ask(message = nil, with: nil) ⇒ Object Also known as: say

Adds message as a user message and runs the conversation loop, executing tools until the model answers or a call needs approval. Returns the latest assistant Message; check #awaiting_approval? before treating it as a final answer. Attach files with with:. A given block receives streamed Chunk objects as they arrive.

String attachments read local paths or fetch URLs. Only pass trusted, authorized sources; validate user uploads before calling this method. See Attachment.new.

chat.ask "What's the best way to learn Ruby?"
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.ask "Analyze these files", with: ["diagram.png", "report.pdf"]
chat.ask("Tell me a story") { |chunk| print chunk.content }



157
158
159
160
 
# File 'lib/ruby_llm/chat.rb', line 157

def ask(message = nil, with: nil, &)
  ask_later(message, with: with)
  complete(&)
end

#ask_later(message = nil, with: nil) ⇒ Object

Stages message as a user message without requesting a completion, leaving the chat ready for #complete, a single #step, or a provider-side batch via RubyLLM.batch. Accepts attachments with with: like #ask. Returns self.

chats = tickets.map { |t| RubyLLM.chat.ask_later(t.body) }
RubyLLM.batch(chats)

Raises PendingToolCallsError while the last response has unanswered tool calls: finish the round first, recording #approve or #deny decisions for calls that require approval.



175
176
177
178
179
 
# File 'lib/ruby_llm/chat.rb', line 175

def ask_later(message = nil, with: nil)
  raise_if_pending_tool_calls!
  add_message role: :user, content: message, attachments: with
  self
end

#awaiting_approval? ⇒ Boolean

Returns whether the conversation can make no progress without an approval decision: every remaining pending tool call requires approval and has none recorded. While true, #complete returns without executing them; record decisions with #approve or #deny, then call #complete again. Tool calls that need no approval still execute before the loop pauses.

Consults each pending tool's approval resolver when one is declared, so resolvers must be idempotent reads.

Returns:



278
279
280
281
282
283
284
 
# File 'lib/ruby_llm/chat.rb', line 278

def awaiting_approval?
  response = pending_tool_response
  return false unless response

  pending = pending_tool_calls(response)
  pending.any? && pending.all? { |_, tool_call| approval_pending?(tool_call) }
end

#before_fallback ⇒ Object

Registers a callback that receives the Fallback attempt after the current model fails and before the fallback model is tried. Returns self.



692
693
694
 
# File 'lib/ruby_llm/chat.rb', line 692

def before_fallback(&)
  add_callback(:before_fallback, &)
end

#before_message ⇒ Object

Registers a callback that runs before each assistant response or tool result is appended to the conversation. Callbacks are additive: every registered block runs. Returns self.



661
662
663
 
# File 'lib/ruby_llm/chat.rb', line 661

def before_message(&)
  add_callback(:before_message, &)
end

#before_request ⇒ Object

Registers a callback that receives the fully rendered request payload before it is sent and may mutate it in place. Runs after all RubyLLM formatting and #with_provider_options merging. Returns self.

chat.before_request { |payload| logger.debug payload }



708
709
710
 
# File 'lib/ruby_llm/chat.rb', line 708

def before_request(&)
  add_callback(:before_request, &)
end

#before_tool_call ⇒ Object

Registers a callback that receives each local ToolCall before the tool executes. Returns self.

chat.before_tool_call { |tool_call| puts tool_call.name }



679
680
681
 
# File 'lib/ruby_llm/chat.rb', line 679

def before_tool_call(&)
  add_callback(:before_tool_call, &)
end

#cache_until_here ⇒ Object

Marks the latest message as an explicit prompt cache boundary, asking the provider to cache everything up to this point. Returns self.

Raises ArgumentError if the chat has no messages.

Raises:



823
824
825
826
827
828
829
 
# File 'lib/ruby_llm/chat.rb', line 823

def cache_until_here
  message = messages.last
  raise ArgumentError, 'No messages to cache' unless message

  message.cache_until_here
  self
end

#cancel ⇒ Object

Cancels the current in-flight chat operation. The next cancellation checkpoint raises CancelledError and clears the flag so the chat can be reused.



303
304
305
306
 
# File 'lib/ruby_llm/chat.rb', line 303

def cancel
  @cancelled = true
  self
end

#cancelled? ⇒ Boolean

Returns whether this in-memory chat has been marked for cancellation.

Returns:



309
310
311
 
# File 'lib/ruby_llm/chat.rb', line 309

def cancelled?
  @cancelled
end

#compact ⇒ Object

Compacts the conversation's model context and returns an assistant Message. The message can have empty text and carries the provider's compacted context internally. Every earlier message remains in #messages, including on persisted Rails chats.

chat.ask "Remember these project requirements..."
chat.compact
chat.ask "Which requirement should we implement first?"

Uses the current instructions, headers, and request hooks. Records reported usage and runs the normal message callbacks. Raises Error when the provider has no manual compaction endpoint, and PendingToolCallsError until pending tool calls have been answered.



777
778
779
780
781
782
783
784
785
786
787
788
789
 
# File 'lib/ruby_llm/chat.rb', line 777

def compact
  raise_if_cancelled!
  raise_if_pending_tool_calls!
  usage_start = usage_entries.length
  payload = instrumentation_payload(streaming: false)
  RubyLLM.instrument('compaction.ruby_llm', payload, config: @config) do |event|
    result = provider_compaction
    record_out_of_band_usage(result) if usage_entries.length == usage_start
    record_generated_message(result, usage_start)
    record_completion_event(event, result)
    result
  end
end

#complete ⇒ Object

Runs the conversation loop until #complete? or #awaiting_approval? is true. Returns the last conversation Message, or nil for an empty chat. Used after #ask_later; #ask calls #complete for you.

When a pending tool call requires approval and no decision has been recorded, the loop pauses. Record #approve or #deny decisions, then call #complete again to continue.



233
234
235
236
 
# File 'lib/ruby_llm/chat.rb', line 233

def complete(&)
  step(&) until complete? || awaiting_approval?
  last_non_system_message || messages.last
end

#complete? ⇒ Boolean

Returns whether the chat has no pending response or tool execution: nothing is staged, or the model answered without requesting tools.

Returns:



240
241
242
243
244
245
246
247
 
# File 'lib/ruby_llm/chat.rb', line 240

def complete?
  last = last_non_system_message
  case last&.role
  when nil then true
  when :user, :tool then false
  else !last.tool_call?
  end
end

#cost ⇒ Object

Returns a Cost aggregating every provider attempt this chat has made, including retries and attempts that produced no message.

chat.cost.total



733
734
735
 
# File 'lib/ruby_llm/chat.rb', line 733

def cost
  Cost.aggregate(usage_entries.map(&:cost), complete: usage_entries.all?(&:cost_available?))
end

#count_tokens(message = nil) ⇒ Object

Counts input tokens for the conversation, including instructions, function tools, structured output, thinking, and attachments. Pass message to include it as a staged user message without mutating the chat. Returns an Integer.

chat.with_instructions("Be terse.").with_tools(Weather)
chat.count_tokens("What's the weather in Berlin?")

Provider tools, provider_options, compaction, and before_request hooks are not included. Raises Error when the provider has no token counting endpoint.



748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
 
# File 'lib/ruby_llm/chat.rb', line 748

def count_tokens(message = nil)
  request_messages = messages.dup
  request_messages << coerce_message(role: :user, content: message) unless message.nil?
  @provider.count_tokens(
    preprocessed_messages(request_messages),
    model: @model,
    tools: @tools,
    tool_prefs: @tool_prefs,
    thinking: resolved_thinking,
    schema: @schema,
    citations: @citations,
    caching: @caching,
    protocol: @protocol
  )
end

#deny(tool_call) ⇒ Object

Records denial for tool_call, a ToolCall or its id. The next #complete or #run_tools appends a structured denial result instead of executing a local tool, or sends a refusal for a remote tool on the next request. The model continues from there. Returns self.



265
266
267
 
# File 'lib/ruby_llm/chat.rb', line 265

def deny(tool_call)
  record_tool_call_decision(tool_call, false)
end

#each ⇒ Object

Yields each Message in the conversation. Returns an Enumerator when no block is given. Chat includes Enumerable, so the usual collection methods are available.



715
716
717
 
# File 'lib/ruby_llm/chat.rb', line 715

def each(&)
  messages.each(&)
end

#generate ⇒ Object

Requests one completion from the model, appends the response to the conversation, and returns it as a Message. Honors the fallbacks configured with #with_fallbacks. A given block receives streamed Chunk objects. Tool calls in the response are not executed; that is #run_tools.



186
187
188
189
190
191
192
 
# File 'lib/ruby_llm/chat.rb', line 186

def generate(&)
  raise_if_cancelled!

  return generate_once(&) if fallbacks.empty?

  with_model_restored { generate_with_fallbacks(&) }
end

#pending_approvals ⇒ Object

Returns the tool calls from the latest response that require approval and have no recorded decision, as an array of ToolCall objects. Pairs with #approve and #deny. ToolCall#remote? identifies provider-executed calls.

chat.pending_approvals.each { |tool_call| puts tool_call.name }
chat.approve(chat.pending_approvals.first)



293
294
295
296
297
298
 
# File 'lib/ruby_llm/chat.rb', line 293

def pending_approvals
  response = pending_tool_response
  return [] unless response

  pending_tool_calls(response).values.select { |tool_call| approval_pending?(tool_call) }
end

#raise_if_pending_tool_calls! ⇒ Object

Refuses to stage a user message onto an unfinished tool round, which providers reject. Called by #ask_later here and in the Rails integration before it persists anything.

Raises:



872
873
874
875
876
877
878
879
880
881
 
# File 'lib/ruby_llm/chat.rb', line 872

def raise_if_pending_tool_calls! # :nodoc:
  response = pending_tool_response
  return unless response

  names = pending_tool_calls(response).values.map(&:name).uniq
  raise PendingToolCallsError,
        "The last response has unanswered tool calls (#{names.join(', ')}). " \
        'Run complete, recording approve or deny decisions for calls that ' \
        'require approval, before asking again.'
end

#render ⇒ Object

Returns the request payload this chat would send to the provider for its next completion, with #before_request hooks applied. Useful for inspecting and testing request output.



848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
 
# File 'lib/ruby_llm/chat.rb', line 848

def render
  @provider.render(
    preprocessed_messages,
    tools: @tools,
    provider_tools: @provider_tools,
    tool_prefs: @tool_prefs,
    temperature: @temperature,
    max_output_tokens: @max_output_tokens,
    model: @model,
    provider_options: Support::Utils.deep_dup(@provider_options),
    schema: @schema,
    thinking: resolved_thinking,
    citations: @citations,
    caching: @caching,
    compaction: @compaction,
    end_user: @end_user,
    protocol: @protocol,
    before_request: @callbacks[:before_request]
  )
end

#run_tools ⇒ Object

Executes the tool calls pending in the latest response and appends their result messages, without asking the model to respond. Tool calls that already have results are skipped, so a chat reloaded mid-round resumes with only the remaining tools. Calls whose tool was declared with Tool.requires_approval only execute once #approve records a decision; denied calls receive a structured denial result, and undecided calls stay pending. Does nothing when no tool calls are pending. The chat is then ready for the next #generate, or the next batch round. Returns self.



203
204
205
206
207
208
209
 
# File 'lib/ruby_llm/chat.rb', line 203

def run_tools
  raise_if_cancelled!

  message = pending_tool_response
  execute_pending_tool_calls(message) if message
  self
end

#step ⇒ Object

Advances the conversation by one move: runs the pending tool calls if any are unanswered, otherwise generates the next response. Returns the Message that move produced, and nil once there is nothing left to do or the loop is parked on an approval.



215
216
217
218
219
220
221
222
223
224
 
# File 'lib/ruby_llm/chat.rb', line 215

def step(&)
  return if complete?

  raise_if_cancelled!
  return generate(&) unless pending_tool_response

  before = messages.length
  run_tools
  messages.last if messages.length > before
end

#thinking ⇒ Object

Returns the thinking options resolved for the current model, or nil when thinking was not configured or needs no provider control.



503
504
505
506
507
508
509
510
511
512
513
 
# File 'lib/ruby_llm/chat.rb', line 503

def thinking
  config = resolved_thinking
  return unless config

  {
    effort: config.effort,
    budget: config.budget,
    display: config.display,
    enabled: config.enabled
  }.compact
end

#tokens ⇒ Object

Returns token usage aggregated across every provider attempt this chat has made, including retries and attempts that produced no message.

chat.tokens.input



724
725
726
 
# File 'lib/ruby_llm/chat.rb', line 724

def tokens
  Tokens.aggregate(usage_entries.map(&:tokens))
end

#tool_options ⇒ Object

Returns the choice, calls, and concurrency set with #with_tool_options, with nil for anything left at the default.



95
96
97
 
# File 'lib/ruby_llm/chat.rb', line 95

def tool_options
  { choice: tool_prefs[:choice], calls: tool_prefs[:calls], concurrency: concurrency }
end

#with_caching(options = {}) ⇒ Object

Enables provider prompt caching. With no arguments the provider's default behavior applies; options such as ttl: apply where supported. Pass id: with a CachedContent (or its name) from RubyLLM.cache to attach an explicit content cache. Pass false to stop RubyLLM from sending cache controls or rendering explicit cache boundaries. A provider may still cache prompts implicitly. Passing nil raises ArgumentError. Returns self.

chat.with_caching
chat.with_caching(ttl: "1h")
chat.with_caching(id: cache)
chat.with_caching(false)



544
545
546
547
548
549
550
551
552
 
# File 'lib/ruby_llm/chat.rb', line 544

def with_caching(options = {})
  options = {} if options == true
  unless options == false || options.is_a?(Hash)
    raise ArgumentError, 'with_caching accepts true, false, or caching options'
  end

  @caching = options == false ? false : options.transform_keys(&:to_sym).freeze
  self
end

#with_citations(enabled = true) ⇒ Object

Enables document citations, so the model backs its claims with quotes from attached files. Pass false to disable. Passing nil raises ArgumentError. Returns self.

chat.with_citations
response = chat.ask "Who created Ruby?", with: "facts.txt"
response.citations.each { |citation| puts citation.cited_text }

Raises:



523
524
525
526
527
528
 
# File 'lib/ruby_llm/chat.rb', line 523

def with_citations(enabled = true)
  raise ArgumentError, 'with_citations accepts true or false' unless [true, false].include?(enabled)

  @citations = enabled
  self
end

#with_compaction(options = {}) ⇒ Object

Enables provider-side context compaction, so a long conversation keeps going instead of overflowing the context window. The provider condenses the earlier turns itself and returns a block that RubyLLM replays on later requests. With no arguments the provider's own defaults apply. The options are provider-neutral:

at

the input-token count that triggers compaction.

instructions

a custom prompt for the summary the provider writes.

pause_after

end the turn once compaction runs, instead of continuing straight into the answer.

Each provider applies the options it supports. Unsupported options are ignored with a debug log. Pass false to disable; passing nil raises ArgumentError. Returns self.

chat.with_compaction
chat.with_compaction(at: 50_000)
chat.with_compaction(at: 100_000, instructions: "Keep every decision.")
chat.with_compaction(false)

What a provider does when the threshold is crossed differs. Anthropic and OpenAI summarize the compacted span into an opaque block that replaces it; OpenRouter drops messages from the middle of the conversation instead, and has no threshold of its own.



578
579
580
581
582
583
584
585
586
 
# File 'lib/ruby_llm/chat.rb', line 578

def with_compaction(options = {})
  options = {} if options == true
  unless options == false || options.is_a?(Hash)
    raise ArgumentError, 'with_compaction accepts true, false, or compaction options'
  end

  @compaction = options == false ? false : normalize_compaction(options)
  self
end

#with_context(context) ⇒ Object

Rebinds the chat to context, a Context built with RubyLLM.context, so subsequent requests use its configuration. Pass nil to return to the global RubyLLM.config. Returns self.



605
606
607
608
609
610
 
# File 'lib/ruby_llm/chat.rb', line 605

def with_context(context)
  @context = context
  @config = context&.config || RubyLLM.config
  with_model(@model.id, provider: @provider.slug, protocol: @protocol, assume_model_exists: true)
  self
end

#with_end_user(end_user) ⇒ Object

Identifies the end user behind the conversation for the provider's abuse monitoring. Providers without an equivalent field omit it. Pass nil to remove it. Returns self.

chat.with_end_user("user-123").ask "Hello"

The value is sent as given, so use an opaque id such as a hash of your user id, never personal data.



597
598
599
600
 
# File 'lib/ruby_llm/chat.rb', line 597

def with_end_user(end_user)
  @end_user = end_user
  self
end

#with_fallbacks(*models, on: Fallback::DEFAULT_ERRORS) ⇒ Object

Sets fallback models to try, in order, when generation fails. on: selects the error classes that trigger a fallback; the default covers transient provider and network errors. Pass nil to remove all fallbacks and restore the default error classes. Returns self.

chat.with_fallbacks("gpt-4.1-mini", "claude-haiku-4-5")
chat.with_fallbacks(nil)



437
438
439
440
441
442
 
# File 'lib/ruby_llm/chat.rb', line 437

def with_fallbacks(*models, on: Fallback::DEFAULT_ERRORS)
  fallback_models = models.flatten.compact
  @fallbacks = fallback_models.map { |model| Fallback.build(model) }
  @fallback_errors = fallback_models.empty? ? Fallback::DEFAULT_ERRORS : Array(on).flatten.compact
  self
end

#with_headers(headers) ⇒ Object

Sets extra HTTP headers sent with completion requests, replacing any previously set headers; nil clears them. Returns self.

chat.with_headers('anthropic-beta' => 'fine-grained-tool-streaming-2025-05-14')



628
629
630
631
 
# File 'lib/ruby_llm/chat.rb', line 628

def with_headers(headers)
  @headers = headers.to_h
  self
end

#with_instructions(instructions, append: false, cache_until_here: false) ⇒ Object

Sets the system instructions for the conversation, replacing any existing system messages. With append: true the instructions are added alongside the existing ones. With cache_until_here: true the instruction becomes an explicit prompt cache boundary. Pass nil to remove all system instructions. Returns self.

chat.with_instructions "You are a helpful Ruby tutor."
chat.with_instructions "Use exactly one short paragraph.", append: true
chat.with_instructions nil



323
324
325
326
327
328
 
# File 'lib/ruby_llm/chat.rb', line 323

def with_instructions(instructions, append: false, cache_until_here: false)
  @messages.reject! { |message| message.role == :system } unless append
  @messages << Message.new(role: :system, content: instructions) unless instructions.nil?
  @messages.last.cache_until_here if instructions && cache_until_here
  self
end

#with_max_output_tokens(max_output_tokens) ⇒ Object

Caps the number of tokens the model may generate. Pass nil to remove the limit. Returns self.

chat.with_max_output_tokens(1000)



460
461
462
463
 
# File 'lib/ruby_llm/chat.rb', line 460

def with_max_output_tokens(max_output_tokens)
  @max_output_tokens = max_output_tokens
  self
end

#with_model(model_id, provider: nil, protocol: nil, assume_model_exists: false) ⇒ Object

Switches the chat to model_id and its provider. Pass provider: to disambiguate, and assume_model_exists: true to skip registry validation for custom or private models. Pass nil to return to the configured default model. Returns self.

protocol: overrides the wire protocol the provider would pick for the model, such as :responses or :chat_completions for OpenAI. It stays nil by default, meaning the provider chooses the protocol for each request. A bare #with_model resets the override to nil, just as it re-resolves the provider from the model.

Raises ModelNotFoundError if model_id is not in the registry and assume_model_exists: is false.

chat.with_model('claude-sonnet-5')
chat.with_model('gpt-5.6', protocol: :chat_completions)



421
422
423
424
425
426
427
 
# File 'lib/ruby_llm/chat.rb', line 421

def with_model(model_id, provider: nil, protocol: nil, assume_model_exists: false)
  model_id ||= @config.default_model
  @model, @provider = Models.resolve(model_id, provider:, assume_model_exists:, config: @config)
  @connection = @provider.connection
  @protocol = protocol
  self
end

#with_provider_options(provider_options) ⇒ Object

Sets options in the provider's request vocabulary, merged into the request payload as-is and overriding RubyLLM's defaults. Replaces any previously set provider options; nil clears them. Returns self.

chat.with_provider_options(service_tier: "flex")



618
619
620
621
 
# File 'lib/ruby_llm/chat.rb', line 618

def with_provider_options(provider_options)
  @provider_options = provider_options.to_h
  self
end

#with_provider_tools(*tools, **tools_with_options) ⇒ Object

Enables tools that run on the provider's servers, such as web search or code execution. Accepts portable alias Symbols, alias-with-options keywords whose options use the provider's own vocabulary, and raw Hashes passed to the provider verbatim, so provider tools RubyLLM has no alias for yet work without a gem update. Entries add to any tools enabled earlier; pass nil to clear them all. Returns self.

chat.with_provider_tools(:web_search)
chat.with_provider_tools(:web_search, :code_execution)
chat.with_provider_tools(web_search: { allowed_domains: ["ruby-lang.org"] })
chat.with_provider_tools({ type: "web_search_20260318", name: "web_search" })

The tool steps the model ran come back on Message#server_tool_calls, citations from search tools on Message#citations, and per-use billing counters on message.tokens.server_tool_use.

Raises UnsupportedServerToolError at request time when the provider has no provider-tool support or does not define a requested alias.



369
370
371
372
373
374
375
376
377
 
# File 'lib/ruby_llm/chat.rb', line 369

def with_provider_tools(*tools, **tools_with_options)
  if tools == [nil] && tools_with_options.empty?
    @provider_tools = []
    return self
  end

  @provider_tools += RubyLLM::Tools::ProviderTools.normalize(tools, tools_with_options)
  self
end

#with_schema(schema) ⇒ Object

Sets the schema for structured output. Accepts a JSON Schema Hash, a Schematist::Schema class or instance, or any object responding to to_json_schema. Returns self.

class PersonSchema < Schematist::Schema
string :name
integer :age
end

chat.with_schema(PersonSchema)
response = chat.ask("Generate a person named Alice who is 30 years old")
response.parsed # => {"name" => "Alice", "age" => 30}

Pass nil to remove the schema, returning the chat to plain text responses.



648
649
650
651
652
653
654
655
656
 
# File 'lib/ruby_llm/chat.rb', line 648

def with_schema(schema)
  schema_instance = schema.is_a?(Class) ? schema.new : schema

  @schema = normalize_schema_payload(
    schema_instance.respond_to?(:to_json_schema) ? schema_instance.to_json_schema : schema_instance
  )

  self
end

#with_temperature(temperature) ⇒ Object

Sets the sampling temperature for subsequent requests. Pass nil to return to the model's default sampling behavior. Returns self.

chat.with_temperature(0.2)



449
450
451
452
 
# File 'lib/ruby_llm/chat.rb', line 449

def with_temperature(temperature)
  @temperature = temperature
  self
end

#with_thinking(enabled = true, **options) ⇒ Object

Configures extended thinking for models that support it. With no arguments, RubyLLM uses the current model's registered default. Pass false to disable thinking, or tune it with effort: (+:low+, :medium, :high, :none, or a provider-specific tier such as :minimal, :xhigh, or :max, passed through as-is), budget: (a token count), and display: (+:summarized+ or :omitted, controlling whether providers that support it return readable thinking text). Accepts keywords or an options Hash. Passing nil raises ArgumentError. Returns self.

chat.with_thinking
chat.with_thinking(false)
chat.with_thinking(effort: :high)
chat.with_thinking(budget: 10_000)
chat.with_thinking(display: :summarized)

Raises:



481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
 
# File 'lib/ruby_llm/chat.rb', line 481

def with_thinking(enabled = true, **options) # rubocop:disable Metrics/PerceivedComplexity
  return with_thinking(**enabled.transform_keys(&:to_sym), **options) if enabled.is_a?(Hash)

  raise ArgumentError, 'with_thinking accepts false or thinking options' unless [true, false].include?(enabled)
  raise ArgumentError, 'with_thinking(false) does not accept options' if !enabled && options.any?
  raise ArgumentError, 'thinking options cannot be nil; pass false to disable' if options.value?(nil)
  if (unsupported = options.keys - THINKING_OPTIONS).any?
    raise ArgumentError,
          "with_thinking accepts #{format_option_keys(THINKING_OPTIONS)}, " \
          "got #{format_option_keys(unsupported)}"
  end

  @thinking = if enabled
                options.empty? ? Thinking::Config.default : Thinking::Config.new(**options)
              else
                Thinking::Config.disabled
              end
  self
end

#with_tool_options(**options) ⇒ Object

Configures how the model uses the registered tools. choice: constrains tool use to :auto, :none, :required, a tool name, or a Tool class. calls: limits how many tool calls one response may contain (+:many+ or :one). concurrency: runs tool calls concurrently: true or :threads for threads, :fibers for fibers. An omitted option is left unchanged; passing nil explicitly resets that option (+concurrency: nil+ returns to the configured default). Returns self.

chat.with_tools(Weather, Search).with_tool_options(choice: :required)
chat.with_tool_options(calls: :one, concurrency: :threads)
chat.with_tool_options(choice: nil)



392
393
394
395
396
397
398
399
400
401
402
 
# File 'lib/ruby_llm/chat.rb', line 392

def with_tool_options(**options)
  options.each do |option, value|
    case option
    when :choice then apply_tool_choice(value)
    when :calls then @tool_prefs[:calls] = value.nil? ? nil : normalize_calls(value)
    when :concurrency then @concurrency = normalize_tool_concurrency(value.nil? ? @config.tool_concurrency : value)
    else raise ArgumentError, "Unknown tool option: #{option}. Valid options are: choice, calls, concurrency"
    end
  end
  self
end

#with_tools(*tools) ⇒ Object

Registers tools, each a Tool class or instance, for the model to call. Configure how the model uses them with #with_tool_options. Pass nil to remove all registered tools. Returns self.

chat.with_tools(Weather, Search)
chat.with_tools(Weather).with_tool_options(choice: :required)

To replace the registered tools, clear them first:

chat.with_tools(nil).with_tools(NewTool)



341
342
343
344
345
346
347
348
 
# File 'lib/ruby_llm/chat.rb', line 341

def with_tools(*tools)
  @tools.clear if tools == [nil]
  tools.flatten.compact.each do |tool|
    tool_instance = tool.is_a?(Class) ? tool.new : tool
    @tools[tool_instance.name.to_sym] = tool_instance
  end
  self
end