Class: RubyLLM::Chat

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/ruby_llm/chat.rb

Overview

Represents a conversation with an AI model

Defined Under Namespace

Classes: Subscription

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model: nil, provider: nil, assume_model_exists: false, context: nil, tool_concurrency: nil, max_concurrency: nil) ⇒ Chat

rubocop:disable Metrics/ParameterLists



51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/ruby_llm/chat.rb', line 51

def initialize(model: nil, provider: nil, assume_model_exists: false, context: nil, # rubocop:disable Metrics/ParameterLists
               tool_concurrency: nil, max_concurrency: nil)
  if assume_model_exists && !provider
    raise ArgumentError, 'Provider must be specified if assume_model_exists is true'
  end

  @context = context
  @config = context&.config || RubyLLM.config
  model_id = model || @config.default_model
  with_model(model_id, provider: provider, assume_exists: assume_model_exists)
  @temperature = nil
  @messages = []
  @messages_mutex = Mutex.new
  @tools = {}
  @params = {}
  @headers = {}
  @schema = nil

  # Concurrent tool execution settings
  @tool_concurrency = tool_concurrency
  @max_concurrency = max_concurrency

  # Responses API state
  @responses_api_config = nil
  @responses_session = nil

  # Multi-subscriber callback system
  @callbacks = {
    new_message: [],
    end_message: [],
    tool_call: [],
    tool_result: []
  }
  @callback_monitor = Monitor.new

  # Extensibility hook for tool execution
  @around_tool_execution_hook = nil

  # Extensibility hook for LLM requests
  @around_llm_request_hook = nil
end

Instance Attribute Details

#headers ⇒ Object (readonly)

Returns the value of attribute headers.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def headers
  @headers
end

#max_concurrency ⇒ Object (readonly)

Returns the value of attribute max_concurrency.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def max_concurrency
  @max_concurrency
end

#messages ⇒ Object (readonly)

Returns the value of attribute messages.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def messages
  @messages
end

#model ⇒ Object (readonly)

Returns the value of attribute model.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def model
  @model
end

#params ⇒ Object (readonly)

Returns the value of attribute params.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def params
  @params
end

#responses_session ⇒ Object (readonly)

Returns the value of attribute responses_session.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def responses_session
  @responses_session
end

#schema ⇒ Object (readonly)

Returns the value of attribute schema.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def schema
  @schema
end

#tool_concurrency ⇒ Object (readonly)

Returns the value of attribute tool_concurrency.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def tool_concurrency
  @tool_concurrency
end

#tools ⇒ Object (readonly)

Returns the value of attribute tools.



48
49
50
 
# File 'lib/ruby_llm/chat.rb', line 48

def tools
  @tools
end

Instance Method Details

#add_message(message_or_attributes) ⇒ Message

Adds a message to the conversation history. Thread-safe: uses mutex to protect message array.

Parameters:

  • message_or_attributes (Message, Hash) —

    A Message object or hash of attributes

Returns:



459
460
461
462
463
464
465
 
# File 'lib/ruby_llm/chat.rb', line 459

def add_message(message_or_attributes)
  message = message_or_attributes.is_a?(Message) ? message_or_attributes : Message.new(message_or_attributes)
  @messages_mutex.synchronize do
    @messages << message
  end
  message
end

#around_llm_request {|Array<Message>| ... } ⇒ self

Sets a hook to wrap LLM API requests with custom behavior. Unlike event callbacks, only one around hook can be active at a time.

The block receives:

  • messages [Array]: The current conversation messages
  • send_request [Proc]: A block that executes the actual provider call

The block MUST call the block and return the response (can be modified). This allows intercepting, modifying messages, adding retry logic, etc.

Examples:

Inject ephemeral context (not stored in history)

chat.around_llm_request do |messages, &send_request|
  prepared = messages + [ephemeral_message]
  send_request.call(prepared)
end

Add retry logic with backoff

chat.around_llm_request do |messages, &send_request|
  retries = 0
  begin
    send_request.call(messages)
  rescue RubyLLM::RateLimitError => e
    sleep(2 ** retries)
    retry if (retries += 1) < 3
    raise
  end
end

Request logging

chat.around_llm_request do |messages, &send_request|
  start = Time.now
  response = send_request.call(messages)
  puts "LLM request took #{Time.now - start}s"
  response
end

Yields:

  • (Array<Message>) —

    Block called before each LLM request

Yield Parameters:

  • send_request (Proc) —

    Block that sends request to provider

Returns:

  • (self) —

    for chaining



400
401
402
403
 
# File 'lib/ruby_llm/chat.rb', line 400

def around_llm_request(&block)
  @around_llm_request_hook = block
  self
end

#around_tool_execution {|ToolCall, Tool, Proc| ... } ⇒ self

Sets a hook to wrap tool execution with custom behavior. Unlike event callbacks, only one around hook can be active at a time.

The block receives:

  • tool_call [ToolCall]: The tool call being executed (.name, .arguments, .id)
  • tool_instance [Tool]: The tool instance
  • execute [Proc]: Call this to execute the actual tool

The block must return the result (can modify or replace it). If the block doesn't call execute.call, it can return an alternative result (for caching, mocking, etc.).

Examples:

Logging and timing

chat.around_tool_execution do |tool_call, tool_instance, execute|
  start = Time.now
  result = execute.call
  puts "#{tool_call.name} took #{Time.now - start}s"
  result
end

Caching

chat.around_tool_execution do |tool_call, tool_instance, execute|
  cache_key = [tool_call.name, tool_call.arguments].hash
  Rails.cache.fetch(cache_key) { execute.call }
end

Yields:

  • (ToolCall, Tool, Proc) —

    Block called for each tool execution

Returns:

  • (self) —

    for chaining



356
357
358
359
 
# File 'lib/ruby_llm/chat.rb', line 356

def around_tool_execution(&block)
  @around_tool_execution_hook = block
  self
end

#ask(message = nil, with: nil) ⇒ Object Also known as: say 



93
94
95
96
 
# File 'lib/ruby_llm/chat.rb', line 93

def ask(message = nil, with: nil, &)
  add_message role: :user, content: build_content(message, with)
  complete(&)
end

#callback_count(event = nil) ⇒ Integer, Hash

Returns the number of callbacks registered for the specified event.

Parameters:

  • event (Symbol, nil) (defaults to: nil) —

    The event to count callbacks for, or nil for all events

Returns:

  • (Integer, Hash) —

    Count for specific event, or hash of counts for all events



424
425
426
427
428
429
430
431
432
 
# File 'lib/ruby_llm/chat.rb', line 424

def callback_count(event = nil)
  @callback_monitor.synchronize do
    if event
      @callbacks[event]&.size || 0
    else
      @callbacks.transform_values(&:size)
    end
  end
end

#clear_callbacks(event = nil) ⇒ self

Clears all callbacks for the specified event, or all events if none specified.

Parameters:

  • event (Symbol, nil) (defaults to: nil) —

    The event to clear callbacks for, or nil for all events

Returns:

  • (self) —

    for chaining



409
410
411
412
413
414
415
416
417
418
 
# File 'lib/ruby_llm/chat.rb', line 409

def clear_callbacks(event = nil)
  @callback_monitor.synchronize do
    if event
      @callbacks[event]&.clear
    else
      @callbacks.each_value(&:clear)
    end
  end
  self
end

#complete ⇒ Object 



438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
 
# File 'lib/ruby_llm/chat.rb', line 438

def complete(&)
  response = execute_llm_request(&)

  emit(:new_message) unless block_given?
  update_responses_session(response)
  parse_schema_response(response)

  if response.tool_call?
    execute_tool_call_sequence(response, &)
  else
    add_message response
    emit(:end_message, response)
    response
  end
end

#each ⇒ Object 



434
435
436
 
# File 'lib/ruby_llm/chat.rb', line 434

def each(&)
  messages.each(&)
end

#instance_variables ⇒ Object 



581
582
583
 
# File 'lib/ruby_llm/chat.rb', line 581

def instance_variables
  super - %i[@connection @config @messages_mutex @callback_monitor]
end

#message_history ⇒ Array<Message>

Returns a thread-safe, frozen snapshot of the message history. Use this for safe reading when concurrent operations may be modifying messages.

Returns:

  • (Array<Message>) —

    Frozen copy of the messages array



471
472
473
 
# File 'lib/ruby_llm/chat.rb', line 471

def message_history
  @messages_mutex.synchronize { @messages.dup.freeze }
end

#on_end_message {|Message| ... } ⇒ self

Registers a callback for when a message is complete. Multiple callbacks can be registered and all will fire in registration order.

Yields:

  • (Message) —

    Block called with the completed message

Returns:

  • (self) —

    for chaining



304
305
306
307
 
# File 'lib/ruby_llm/chat.rb', line 304

def on_end_message(&)
  subscribe(:end_message, &)
  self
end

#on_new_message { ... } ⇒ self

Registers a callback for when a new message starts being generated. Multiple callbacks can be registered and all will fire in registration order.

Yields:

  • Block called when a new message starts

Returns:

  • (self) —

    for chaining



294
295
296
297
 
# File 'lib/ruby_llm/chat.rb', line 294

def on_new_message(&)
  subscribe(:new_message, &)
  self
end

#on_tool_call {|ToolCall| ... } ⇒ self

Registers a callback for when a tool is called. Multiple callbacks can be registered and all will fire in registration order.

Yields:

  • (ToolCall) —

    Block called with the tool call object

Returns:

  • (self) —

    for chaining



314
315
316
317
 
# File 'lib/ruby_llm/chat.rb', line 314

def on_tool_call(&)
  subscribe(:tool_call, &)
  self
end

#on_tool_result {|ToolCall, Object| ... } ⇒ self

Registers a callback for when a tool returns a result. Multiple callbacks can be registered and all will fire in registration order.

Yields:

  • (ToolCall, Object) —

    Block called with the tool call and its result

Returns:

  • (self) —

    for chaining



324
325
326
327
 
# File 'lib/ruby_llm/chat.rb', line 324

def on_tool_result(&)
  subscribe(:tool_result, &)
  self
end

#once(event, tag: nil) { ... } ⇒ Subscription

Subscribes to an event that automatically unsubscribes after firing once.

Examples:

chat.once(:end_message) { |msg| setup_initial_state(msg) }

Parameters:

  • event (Symbol) —

    The event to subscribe to

  • tag (String, nil) (defaults to: nil) —

    Optional tag for debugging/identification

Yields:

  • The block to call when the event fires (once)

Returns:

  • (Subscription) —

    An object that can be used to unsubscribe before it fires



280
281
282
283
284
285
286
287
 
# File 'lib/ruby_llm/chat.rb', line 280

def once(event, tag: nil, &block)
  subscription = nil
  wrapper = lambda do |*args|
    subscription&.unsubscribe
    block.call(*args)
  end
  subscription = subscribe(event, tag: tag, &wrapper)
end

#repair_incomplete_tool_calls! ⇒ self

Removes incomplete tool call sequence if interrupted. Call this to repair Chat state after cancellation/exception.

Returns:

  • (self) —

    for chaining



567
568
569
570
571
572
573
574
575
576
577
578
579
 
# File 'lib/ruby_llm/chat.rb', line 567

def repair_incomplete_tool_calls! # rubocop:disable Metrics/PerceivedComplexity
  return self if tool_results_complete?

  @messages_mutex.synchronize do
    # Remove partial tool results
    @messages.pop while @messages.last&.role == :tool

    # Remove the incomplete assistant message with tool_calls
    last = @messages.last
    @messages.pop if last&.role == :assistant && last.respond_to?(:tool_calls) && last.tool_calls&.any?
  end
  self
end

#reset_messages!(preserve_system_prompt: true) ⇒ self

Clears messages from the conversation history. Thread-safe: uses mutex to protect message array.

Parameters:

  • preserve_system_prompt (Boolean) (defaults to: true) —

    if true (default), keeps system messages

Returns:

  • (self) —

    for chaining



511
512
513
514
515
516
517
518
519
520
 
# File 'lib/ruby_llm/chat.rb', line 511

def reset_messages!(preserve_system_prompt: true)
  @messages_mutex.synchronize do
    if preserve_system_prompt
      @messages.select! { |m| m.role == :system }
    else
      @messages.clear
    end
  end
  self
end

#responses_api_enabled? ⇒ Boolean

Checks if the Responses API is currently enabled for this chat.

Returns:

  • (Boolean) —

    true if using OpenAI Responses API



243
244
245
 
# File 'lib/ruby_llm/chat.rb', line 243

def responses_api_enabled?
  @provider.is_a?(Providers::OpenAIResponses)
end

#restore_messages(snapshot) ⇒ self

Restores messages from a previously taken snapshot.

Parameters:

  • snapshot (Array<Message>) —

    Previously saved message snapshot

Returns:

  • (self) —

    for chaining



502
503
504
 
# File 'lib/ruby_llm/chat.rb', line 502

def restore_messages(snapshot)
  set_messages(snapshot)
end

#restore_responses_session(session_data) ⇒ self

Restores a Responses API session from previously saved state. Used for persisting sessions across requests (e.g., Rails).

Parameters:

  • session_data (Hash) —

    Session data from ResponsesSession#to_h

Returns:

  • (self) —

    for chaining



228
229
230
231
232
233
234
235
236
237
238
 
# File 'lib/ruby_llm/chat.rb', line 228

def restore_responses_session(session_data)
  @responses_session = ResponsesSession.from_h(session_data)

  # Update provider session if already using Responses API
  if @provider.is_a?(Providers::OpenAIResponses)
    @provider = Providers::OpenAIResponses.new(@config, @responses_session, @responses_api_config || {})
    @connection = @provider.connection
  end

  self
end

#set_messages(new_messages) ⇒ self

Replaces the entire message history with new messages. Thread-safe: uses mutex to protect message array.

Parameters:

  • new_messages (Array<Message, Hash>) —

    New messages to set

Returns:

  • (self) —

    for chaining



480
481
482
483
484
485
486
487
488
 
# File 'lib/ruby_llm/chat.rb', line 480

def set_messages(new_messages) # rubocop:disable Naming/AccessorMethodName
  @messages_mutex.synchronize do
    @messages.clear
    new_messages.each do |msg|
      @messages << (msg.is_a?(Message) ? msg : Message.new(msg))
    end
  end
  self
end

#snapshot_messages ⇒ Array<Message>

Creates a snapshot of the current message history for checkpointing. Thread-safe: uses mutex to protect message array.

Returns:

  • (Array<Message>) —

    Duplicated messages for restoration later



494
495
496
 
# File 'lib/ruby_llm/chat.rb', line 494

def snapshot_messages
  @messages_mutex.synchronize { @messages.map(&:dup) }
end

#subscribe(event, tag: nil) { ... } ⇒ Subscription

Subscribes to an event with the given block. Returns a Subscription that can be used to unsubscribe.

Examples:

sub = chat.subscribe(:tool_call, tag: "metrics") { |tc| track(tc) }
# ... later
sub.unsubscribe

Parameters:

  • event (Symbol) —

    The event to subscribe to (:new_message, :end_message, :tool_call, :tool_result)

  • tag (String, nil) (defaults to: nil) —

    Optional tag for debugging/identification

Yields:

  • The block to call when the event fires

Returns:

  • (Subscription) —

    An object that can be used to unsubscribe

Raises:

  • (ArgumentError) —

    if event is not recognized



260
261
262
263
264
265
266
267
268
269
 
# File 'lib/ruby_llm/chat.rb', line 260

def subscribe(event, tag: nil, &block)
  @callback_monitor.synchronize do
    unless @callbacks.key?(event)
      raise ArgumentError, "Unknown event: #{event}. Valid events: #{@callbacks.keys.join(', ')}"
    end

    @callbacks[event] << block
    Subscription.new(@callbacks[event], block, monitor: @callback_monitor, tag: tag)
  end
end

#tool_results_complete? ⇒ Boolean

Checks if the last tool call has all corresponding results. Useful for diagnosing incomplete Chat state after interruptions.

Returns:

  • (Boolean) —

    true if all tool calls have results



549
550
551
552
553
554
555
556
557
558
559
560
561
 
# File 'lib/ruby_llm/chat.rb', line 549

def tool_results_complete? # rubocop:disable Metrics/PerceivedComplexity
  return true unless messages.any?

  last_assistant = messages.reverse.find do |m|
    m.role == :assistant && m.respond_to?(:tool_calls) && m.tool_calls&.any?
  end
  return true unless last_assistant

  expected_ids = last_assistant.tool_calls.keys.to_set
  actual_ids = messages.select { |m| m.role == :tool }.filter_map(&:tool_call_id).to_set

  expected_ids.subset?(actual_ids)
end

#with_context(context) ⇒ Object 



130
131
132
133
134
135
 
# File 'lib/ruby_llm/chat.rb', line 130

def with_context(context)
  @context = context
  @config = context.config
  with_model(@model.id, provider: @provider.slug, assume_exists: true)
  self
end

#with_headers(**headers) ⇒ Object 



142
143
144
145
 
# File 'lib/ruby_llm/chat.rb', line 142

def with_headers(**headers)
  @headers = headers
  self
end

#with_instructions(instructions, replace: false) ⇒ Object 



100
101
102
103
104
105
 
# File 'lib/ruby_llm/chat.rb', line 100

def with_instructions(instructions, replace: false)
  @messages = @messages.reject { |msg| msg.role == :system } if replace

  add_message role: :system, content: instructions
  self
end

#with_message_transaction { ... } ⇒ Object

Wraps operations in a transaction for rollback on failure. If an exception is raised, all messages added since the transaction started are removed. This ensures Chat state remains valid even on cancellation or errors.

Uses O(1) memory (just tracks index, no array duplication).

Yields:

  • Block to execute within the transaction

Returns:

  • (Object) —

    Result of the block

Raises:

  • Re-raises any exception after rolling back



531
532
533
534
535
536
537
538
539
540
541
542
543
 
# File 'lib/ruby_llm/chat.rb', line 531

def with_message_transaction
  start_index = @messages_mutex.synchronize { @messages.size }

  begin
    yield
  rescue StandardError => e
    # Truncate back to where we started (O(1) operation)
    @messages_mutex.synchronize do
      @messages.slice!(start_index..-1)
    end
    raise e
  end
end

#with_model(model_id, provider: nil, assume_exists: false) ⇒ Object 



119
120
121
122
123
 
# File 'lib/ruby_llm/chat.rb', line 119

def with_model(model_id, provider: nil, assume_exists: false)
  @model, @provider = Models.resolve(model_id, provider:, assume_exists:, config: @config)
  @connection = @provider.connection
  self
end

#with_params(**params) ⇒ Object 



137
138
139
140
 
# File 'lib/ruby_llm/chat.rb', line 137

def with_params(**params)
  @params = params
  self
end

#with_responses_api(stateful: false, store: true, truncation: :disabled, include: [], **options) ⇒ self

Enables OpenAI Responses API for this chat. Switches from chat/completions to the v1/responses endpoint.

Examples:

Basic usage

chat.with_responses_api.ask("Hello")

Stateful mode for token efficiency

chat.with_responses_api(stateful: true).ask("Hello")

With custom configuration

chat.with_responses_api(
  stateful: true,
  truncation: :auto,
  include: [:reasoning_encrypted_content]
).ask("Complex reasoning task")

Parameters:

  • stateful (Boolean) (defaults to: false) —

    Use previous_response_id for efficient multi-turn (default: false)

  • store (Boolean) (defaults to: true) —

    Store responses on OpenAI server (default: true)

  • truncation (Symbol) (defaults to: :disabled) —

    Truncation strategy (default: :disabled)

  • include (Array<Symbol>) (defaults to: []) —

    Additional data to include (e.g., [:reasoning_encrypted_content])

Returns:

  • (self) —

    for chaining



197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
 
# File 'lib/ruby_llm/chat.rb', line 197

def with_responses_api(stateful: false, store: true, truncation: :disabled, include: [], **options)
  unless @provider.is_a?(Providers::OpenAI)
    raise ArgumentError, 'with_responses_api is only supported for OpenAI providers'
  end

  @responses_api_config = {
    stateful: stateful,
    store: store,
    truncation: truncation,
    include: include,
    service_tier: options[:service_tier],
    max_tool_calls: options[:max_tool_calls]
  }

  # Initialize session if not already present
  @responses_session ||= ResponsesSession.new

  # Switch to OpenAIResponses provider if currently using standard OpenAI
  unless @provider.is_a?(Providers::OpenAIResponses)
    @provider = Providers::OpenAIResponses.new(@config, @responses_session, @responses_api_config)
    @connection = @provider.connection
  end

  self
end

#with_schema(schema) ⇒ Object 



147
148
149
150
151
152
153
154
155
156
157
158
 
# File 'lib/ruby_llm/chat.rb', line 147

def with_schema(schema)
  schema_instance = schema.is_a?(Class) ? schema.new : schema

  # Accept both RubyLLM::Schema instances and plain JSON schemas
  @schema = if schema_instance.respond_to?(:to_json_schema)
              schema_instance.to_json_schema[:schema]
            else
              schema_instance
            end

  self
end

#with_temperature(temperature) ⇒ Object 



125
126
127
128
 
# File 'lib/ruby_llm/chat.rb', line 125

def with_temperature(temperature)
  @temperature = temperature
  self
end

#with_tool(tool) ⇒ Object 



107
108
109
110
111
 
# File 'lib/ruby_llm/chat.rb', line 107

def with_tool(tool)
  tool_instance = tool.is_a?(Class) ? tool.new : tool
  @tools[tool_instance.name.to_sym] = tool_instance
  self
end

#with_tool_concurrency(mode = nil, max: nil) ⇒ self

Configures concurrent tool execution for this chat.

Examples:

chat.with_tool_concurrency(:async, max: 5)
     .with_tools(Weather, Stock, Currency)
     .ask("Get weather, stock price, and currency rate")

Parameters:

  • mode (Symbol, nil) (defaults to: nil) —

    Concurrency mode (:async, :threads, or nil for sequential)

  • max (Integer, nil) (defaults to: nil) —

    Maximum number of concurrent tool executions

Returns:

  • (self) —

    for chaining



170
171
172
173
174
 
# File 'lib/ruby_llm/chat.rb', line 170

def with_tool_concurrency(mode = nil, max: nil)
  @tool_concurrency = mode unless mode.nil?
  @max_concurrency = max if max
  self
end

#with_tools(*tools, replace: false) ⇒ Object 



113
114
115
116
117
 
# File 'lib/ruby_llm/chat.rb', line 113

def with_tools(*tools, replace: false)
  @tools.clear if replace
  tools.compact.each { |tool| with_tool tool }
  self
end