Class: RubyLLM::Provider

Inherits:
Object
  • Object
show all
Includes:
Support::Inspectable
Defined in:
lib/ruby_llm/provider.rb

Overview

A Provider connects RubyLLM to one AI service. It knows where to talk (host, authentication headers, configuration) and which protocol to speak for a given model and request. The wire formats themselves live under RubyLLM::Protocols.

Subclass Provider to support a new service, then make it available with ::register:

class Acme < RubyLLM::Provider
protocol :chat_completions, RubyLLM::Protocols::ChatCompletions

def self.configuration_options
  %i[acme_api_key]
end

def api_base
  'https://api.acme.ai/v1'
end

def headers
  { 'Authorization' => "Bearer #{@config.acme_api_key}" }
end
end

RubyLLM::Provider.register :acme, Acme

See the custom providers guide for the full walkthrough.

Constant Summary

Constants included from Support::Inspectable

Support::Inspectable::TRUNCATE_AT

Class Attribute Summary collapse

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Support::Inspectable

#full_inspect, #inspect, #pretty_print

Constructor Details

#initialize(config) ⇒ Provider

:nodoc:



51
52
53
54
55
 
# File 'lib/ruby_llm/provider.rb', line 51

def initialize(config) # :nodoc:
  @config = config
  ensure_configured!
  @connection = Transport::Connection.new(self, @config)
end

Class Attribute Details

.default_protocol ⇒ Object (readonly)

:nodoc:



447
448
449
 
# File 'lib/ruby_llm/provider.rb', line 447

def default_protocol
  @default_protocol
end

.slug ⇒ Object

Returns the provider slug, a short lowercase string that identifies the provider and prefixes its configuration keys. Set by ::register, or derived from the class name.



453
454
455
 
# File 'lib/ruby_llm/provider.rb', line 453

def slug
  @slug ||= to_s.split('::').last.downcase
end

Instance Attribute Details

#config ⇒ Object (readonly)

The Configuration the provider was built with.



47
48
49
 
# File 'lib/ruby_llm/provider.rb', line 47

def config
  @config
end

#connection ⇒ Object (readonly)

:nodoc:



49
50
51
 
# File 'lib/ruby_llm/provider.rb', line 49

def connection
  @connection
end

Class Method Details

.assume_models_exist? ⇒ Boolean

Returns whether the provider accepts model ids missing from the model registry. The base implementation returns false.

Returns:

  • (Boolean) 


516
517
518
 
# File 'lib/ruby_llm/provider.rb', line 516

def assume_models_exist?
  false
end

.capabilities ⇒ Object

Returns the provider's narrow model capability augmenter, or nil when models.dev and the provider listing are sufficient.



465
466
467
 
# File 'lib/ruby_llm/provider.rb', line 465

def capabilities
  nil
end

.configuration_options ⇒ Object

Returns every configuration key the provider contributes. ::register defines a Configuration accessor for each one. The base implementation returns an empty array.

def self.configuration_options
%i[acme_api_key acme_api_base]
end


499
500
501
 
# File 'lib/ruby_llm/provider.rb', line 499

def configuration_options
  []
end

.configuration_requirements ⇒ Object

Returns the configuration keys that must be set before the provider is usable. The base implementation returns an empty array.

def self.configuration_requirements
%i[acme_api_key]
end


487
488
489
 
# File 'lib/ruby_llm/provider.rb', line 487

def configuration_requirements
  []
end

.configured?(config) ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


526
527
528
 
# File 'lib/ruby_llm/provider.rb', line 526

def configured?(config) # :nodoc:
  configuration_requirements.all? { |req| config.send(req) }
end

.configured_providers(config) ⇒ Object

:nodoc:



597
598
599
600
601
 
# File 'lib/ruby_llm/provider.rb', line 597

def configured_providers(config) # :nodoc:
  providers.select do |_slug, provider_class|
    provider_class.configured?(config)
  end.values
end

.configured_remote_providers(config) ⇒ Object

:nodoc:



603
604
605
606
607
 
# File 'lib/ruby_llm/provider.rb', line 603

def configured_remote_providers(config) # :nodoc:
  providers.select do |_slug, provider_class|
    provider_class.remote? && provider_class.configured?(config)
  end.values
end

.display_name ⇒ Object

Returns the human-readable provider name, derived from the class name. Override for custom branding.



459
460
461
 
# File 'lib/ruby_llm/provider.rb', line 459

def display_name
  to_s.split('::').last
end

.local? ⇒ Boolean

Returns whether the provider talks to a locally hosted service. The base implementation returns false. Local providers such as Ollama return true.

Returns:

  • (Boolean) 


506
507
508
 
# File 'lib/ruby_llm/provider.rb', line 506

def local?
  false
end

.local_providers ⇒ Object

:nodoc:



589
590
591
 
# File 'lib/ruby_llm/provider.rb', line 589

def local_providers # :nodoc:
  providers.select { |_slug, provider_class| provider_class.local? }
end

.model_registry_files ⇒ Object

:nodoc:



585
586
587
 
# File 'lib/ruby_llm/provider.rb', line 585

def model_registry_files # :nodoc:
  @model_registry_files ||= {}
end

.model_required? ⇒ Boolean

Returns whether operation requires an inference model. Override for endpoints that operate on an explicitly configured resource.

Returns:

  • (Boolean) 


522
523
524
 
# File 'lib/ruby_llm/provider.rb', line 522

def model_required?(**)
  true
end

.models_dev_alias(_model_id, _models_dev_by_key, _provider_model = nil) ⇒ Object

:nodoc:



469
470
471
 
# File 'lib/ruby_llm/provider.rb', line 469

def models_dev_alias(_model_id, _models_dev_by_key, _provider_model = nil) # :nodoc:
  nil
end

.models_dev_model_id(id) ⇒ Object

The id RubyLLM registers for a models.dev entry. Providers whose catalog spells ids differently from models.dev override this.



475
476
477
 
# File 'lib/ruby_llm/provider.rb', line 475

def models_dev_model_id(id) # :nodoc:
  id
end

.protocol(name, protocol_class, batches: nil) ⇒ Object

Registers protocol_class under name. The first registered protocol becomes the provider's default. Pass batches: to compose batch operations into the registered protocol.

protocol :chat_completions, ChatCompletions
protocol :responses, Protocols::Responses, batches: Protocols::Responses::Batches


537
538
539
540
 
# File 'lib/ruby_llm/provider.rb', line 537

def protocol(name, protocol_class, batches: nil)
  @default_protocol = name.to_sym if protocols.empty?
  protocols[name.to_sym] = batches ? Class.new(protocol_class) { include batches } : protocol_class
end

.protocols ⇒ Object

:nodoc:



542
543
544
 
# File 'lib/ruby_llm/provider.rb', line 542

def protocols # :nodoc:
  @protocols ||= {}
end

.providers ⇒ Object

Returns the global registry of providers, a hash mapping slug symbols to provider classes.



581
582
583
 
# File 'lib/ruby_llm/provider.rb', line 581

def providers
  @providers ||= {}
end

.register(name, provider_class, models: nil) ⇒ Object

Registers provider_class under the slug name, making it available to RubyLLM.chat and the other top-level helpers. Stamps the class's slug, adds it to ::providers, and defines a Configuration accessor for each of its configuration options. A provider gem may pass the path to its bundled model catalog.

RubyLLM::Provider.register :acme, RubyLLM::Providers::Acme
RubyLLM::Provider.register :acme, RubyLLM::Providers::Acme,
                         models: File.expand_path('../../../models.json', __dir__)


556
557
558
559
560
561
 
# File 'lib/ruby_llm/provider.rb', line 556

def register(name, provider_class, models: nil)
  provider_class.slug = name.to_s
  providers[name.to_sym] = provider_class
  models ? model_registry_files[name.to_sym] = models : model_registry_files.delete(name.to_sym)
  RubyLLM::Configuration.register_provider_options(provider_class.configuration_options + [:"#{name}_protocol"])
end

.remote? ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


510
511
512
 
# File 'lib/ruby_llm/provider.rb', line 510

def remote? # :nodoc:
  !local?
end

.remote_providers ⇒ Object

:nodoc:



593
594
595
 
# File 'lib/ruby_llm/provider.rb', line 593

def remote_providers # :nodoc:
  providers.select { |_slug, provider_class| provider_class.remote? }
end

.resolve(name) ⇒ Object

:nodoc:



563
564
565
 
# File 'lib/ruby_llm/provider.rb', line 563

def resolve(name) # :nodoc:
  providers[name.to_sym]
end

.resolve!(name) ⇒ Object

:nodoc:



567
568
569
570
 
# File 'lib/ruby_llm/provider.rb', line 567

def resolve!(name) # :nodoc:
  providers[name.to_sym] ||
    raise(Error, "Unknown provider: #{name.inspect}. Available providers: #{providers.keys.join(', ')}")
end

.resolve_registry_id(model_id, _models, _config = nil) ⇒ Object

Resolves model_id to the id the registry stores it under for this provider. Defaults to the id unchanged; providers whose catalog ids differ from their request ids (Bedrock's region prefixes) override it.



575
576
577
 
# File 'lib/ruby_llm/provider.rb', line 575

def resolve_registry_id(model_id, _models, _config = nil)
  model_id
end

Instance Method Details

#animate_later(prompt, model:, with: nil, extend: nil, provider_options: {}) ⇒ Object

:nodoc:



334
335
336
337
338
339
 
# File 'lib/ruby_llm/provider.rb', line 334

def animate_later(prompt, model:, with: nil, extend: nil, provider_options: {}) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :animate)
  protocol.new(self, model).animate_later(
    prompt, model: model_id_for(model), with:, extend:, provider_options:
  )
end

#api_base ⇒ Object

Returns the base URL that relative endpoint paths resolve against. The base implementation raises NotImplementedError, so every subclass must define it.

def api_base
@config.acme_api_base || 'https://api.acme.ai/v1'
end

Raises:

  • (NotImplementedError) 


65
66
67
 
# File 'lib/ruby_llm/provider.rb', line 65

def api_base
  raise NotImplementedError
end

#assume_models_exist? ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


427
428
429
 
# File 'lib/ruby_llm/provider.rb', line 427

def assume_models_exist? # :nodoc:
  self.class.assume_models_exist?
end

#batch_cost(tokens, model:, category: :text_tokens) ⇒ Object

:nodoc:



251
252
253
254
255
256
257
258
259
260
 
# File 'lib/ruby_llm/provider.rb', line 251

def batch_cost(tokens, model:, category: :text_tokens) # :nodoc:
  standard = Cost.new(tokens:, model:, category:)
  return standard if tokens.reported_cost

  pricing = model.pricing.public_send(category)
  batch_tier = pricing.batch unless long_context_pricing?(pricing, tokens)
  batch = Cost.new(tokens:, model:, category:, tier: :batch) if batch_tier
  amounts = batch_cost_amounts(standard:, batch:, batch_tier:, model:)
  Cost.from_h(amounts, tokens:)
end

#batch_cost_amounts(standard:, batch:, batch_tier:, model:) ⇒ Object

:nodoc:



262
263
264
265
266
267
268
269
270
271
272
273
 
# File 'lib/ruby_llm/provider.rb', line 262

def batch_cost_amounts(standard:, batch:, batch_tier:, model:) # :nodoc:
  BATCH_RATE_BY_COMPONENT.to_h do |component, rate|
    amount = if batch_tier&.public_send(rate)
               batch.public_send(component)
             else
               value = standard.public_send(component)
               multiplier = batch_cost_multiplier(model:, component:)
               value * multiplier if value && multiplier
             end
    [component, amount]
  end
end

#batch_cost_multiplier ⇒ Object

:nodoc:



275
 
# File 'lib/ruby_llm/provider.rb', line 275

def batch_cost_multiplier(**) = nil # :nodoc:

#batch_protocol_name(protocol) ⇒ Object

:nodoc:



283
284
285
 
# File 'lib/ruby_llm/provider.rb', line 283

def batch_protocol_name(protocol) # :nodoc:
  protocols.key(protocol)&.to_s
end

#batch_results(id, batch_protocol: nil) ⇒ Object

:nodoc:



236
237
238
239
240
 
# File 'lib/ruby_llm/provider.rb', line 236

def batch_results(id, batch_protocol: nil) # :nodoc:
  protocol = resolve_batch_protocol(batch_protocol) || self.batch_protocol
  ensure_batches_supported!(protocol)
  protocol.new(self).batch_results(id)
end

#batch_status(raw_status, completed:, batch_protocol: nil) ⇒ Object

:nodoc:



242
243
244
245
246
247
248
249
 
# File 'lib/ruby_llm/provider.rb', line 242

def batch_status(raw_status, completed:, batch_protocol: nil) # :nodoc:
  protocol = resolve_batch_protocol(batch_protocol) || self.batch_protocol
  ensure_batches_supported!(protocol)
  parser = protocol.new(self)
  return parser.send(:parse_batch_status, raw_status, completed:) if parser.respond_to?(:parse_batch_status, true)

  completed ? :succeeded : :pending
end

#batches? ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


216
217
218
 
# File 'lib/ruby_llm/provider.rb', line 216

def batches? # :nodoc:
  batch_protocol.public_method_defined?(:create_batch)
end

#cache_content(content, model:, ttl: nil, instructions: nil, with: nil) ⇒ Object

:nodoc:



385
386
387
388
389
390
 
# File 'lib/ruby_llm/provider.rb', line 385

def cache_content(content, model:, ttl: nil, instructions: nil, with: nil) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :cache)
  protocol.new(self, model).cache_content(
    content, model: model_id_for(model), ttl:, instructions:, with:
  )
end

#cancel_batch(id) ⇒ Object

:nodoc:



231
232
233
234
 
# File 'lib/ruby_llm/provider.rb', line 231

def cancel_batch(id) # :nodoc:
  ensure_batches_supported!
  batch_protocol.new(self).cancel_batch(id)
end

#capabilities ⇒ Object

:nodoc:



105
106
107
 
# File 'lib/ruby_llm/provider.rb', line 105

def capabilities # :nodoc:
  self.class.capabilities
end

#compact(messages, model:, protocol: nil, headers: {}, before_request: [], usage_recorder: nil) ⇒ Object

:nodoc:



161
162
163
164
165
 
# File 'lib/ruby_llm/provider.rb', line 161

def compact(messages, model:, protocol: nil, headers: {}, before_request: [], usage_recorder: nil) # :nodoc:
  resolve_protocol(protocol, model).new(self, model).compact(
    messages, headers:, before_request:, usage_recorder:
  )
end

#complete(messages, tools:, temperature:, model:, provider_options: {}, headers: {}, schema: nil, max_output_tokens: nil, thinking: nil, citations: false, caching: nil, tool_prefs: nil, protocol: nil, before_request: [], usage_recorder: nil, provider_tools: [], compaction: nil, end_user: nil) ⇒ Object

:nodoc:



131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/ruby_llm/provider.rb', line 131

def complete(messages, tools:, temperature:, model:, provider_options: {}, headers: {}, schema: nil, # :nodoc:
             max_output_tokens: nil, thinking: nil, citations: false, caching: nil, tool_prefs: nil,
             protocol: nil, before_request: [], usage_recorder: nil, provider_tools: [],
             compaction: nil, end_user: nil, &)
  protocol_class = resolve_protocol(protocol, model, tools:, schema:, thinking:, tool_prefs:, citations:)
  protocol_class.new(self, model).complete(
    messages,
    tools: tools,
    provider_tools: provider_tools,
    tool_prefs: tool_prefs,
    temperature: temperature,
    max_output_tokens: max_output_tokens,
    provider_options: provider_options,
    headers: headers,
    schema: schema,
    thinking: thinking,
    citations: citations,
    caching: caching,
    compaction: compaction,
    end_user: end_user,
    before_request: before_request,
    usage_recorder: usage_recorder,
    &
  )
end

#configuration_requirements ⇒ Object

:nodoc:



109
110
111
 
# File 'lib/ruby_llm/provider.rb', line 109

def configuration_requirements # :nodoc:
  self.class.configuration_requirements
end

#configured? ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


419
420
421
 
# File 'lib/ruby_llm/provider.rb', line 419

def configured? # :nodoc:
  self.class.configured?(@config)
end

#count_tokens(messages, model:, tools: {}, tool_prefs: nil, thinking: nil, schema: nil, citations: false, caching: nil, protocol: nil) ⇒ Object

:nodoc:



189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/ruby_llm/provider.rb', line 189

def count_tokens(messages, model:, tools: {}, tool_prefs: nil, thinking: nil, schema: nil, # :nodoc:
                 citations: false, caching: nil, protocol: nil)
  protocol_class = resolve_protocol(protocol, model, tools:, schema:, thinking:, tool_prefs:, citations:)
  protocol_class.new(self, model).count_tokens(
    messages,
    tools: tools,
    tool_prefs: tool_prefs,
    thinking: thinking,
    schema: schema,
    citations: citations,
    caching: caching
  )
end

#create_batch(requests) ⇒ Object

:nodoc:



220
221
222
223
224
 
# File 'lib/ruby_llm/provider.rb', line 220

def create_batch(requests) # :nodoc:
  protocol = batch_protocol_for(requests)
  ensure_batches_supported!(protocol)
  protocol.new(self).create_batch(requests).merge(batch_protocol: protocol)
end

#delete_cache(name) ⇒ Object

:nodoc:



396
397
398
 
# File 'lib/ruby_llm/provider.rb', line 396

def delete_cache(name) # :nodoc:
  default_protocol.new(self).delete_cache(name)
end

#download_file(file_id) ⇒ Object

:nodoc:



409
410
411
412
 
# File 'lib/ruby_llm/provider.rb', line 409

def download_file(file_id) # :nodoc:
  ensure_files_supported!
  protocols.fetch(:files).new(self).download(file_id)
end

#embed(text, model:, dimensions:, task_type: nil, title: nil, with: nil, provider_options: {}) ⇒ Object

:nodoc:



300
301
302
303
304
305
 
# File 'lib/ruby_llm/provider.rb', line 300

def embed(text, model:, dimensions:, task_type: nil, title: nil, with: nil, provider_options: {}) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :embed)
  protocol.new(self, model).embed(
    text, model: model_id_for(model), dimensions:, task_type:, title:, with:, provider_options:
  )
end

#extend_cache(name, ttl:) ⇒ Object

:nodoc:



400
401
402
 
# File 'lib/ruby_llm/provider.rb', line 400

def extend_cache(name, ttl:) # :nodoc:
  default_protocol.new(self).extend_cache(name, ttl:)
end

#files? ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


287
288
289
 
# File 'lib/ruby_llm/provider.rb', line 287

def files? # :nodoc:
  protocols.key?(:files)
end

#find_batch(id) ⇒ Object

:nodoc:



226
227
228
229
 
# File 'lib/ruby_llm/provider.rb', line 226

def find_batch(id) # :nodoc:
  ensure_batches_supported!
  batch_protocol.new(self).find_batch(id)
end

#find_cache(name) ⇒ Object

:nodoc:



392
393
394
 
# File 'lib/ruby_llm/provider.rb', line 392

def find_cache(name) # :nodoc:
  default_protocol.new(self).find_cache(name)
end

#find_file(file_id) ⇒ Object

:nodoc:



404
405
406
407
 
# File 'lib/ruby_llm/provider.rb', line 404

def find_file(file_id) # :nodoc:
  ensure_files_supported!
  protocols.fetch(:files).new(self).find(file_id)
end

#find_research_job(id) ⇒ Object

:nodoc:



323
324
325
 
# File 'lib/ruby_llm/provider.rb', line 323

def find_research_job(id) # :nodoc:
  fetch_protocol(:research).new(self).find_research_job(id)
end

#headers ⇒ Object

Returns the headers merged into every request. The default is an empty hash. Override to supply authentication.

def headers
{ 'Authorization' => "Bearer #{@config.acme_api_key}" }
end


76
77
78
 
# File 'lib/ruby_llm/provider.rb', line 76

def headers
  {}
end

#list_file_uris(uri) ⇒ Object

:nodoc:



414
415
416
417
 
# File 'lib/ruby_llm/provider.rb', line 414

def list_file_uris(uri) # :nodoc:
  ensure_files_supported!
  protocols.fetch(:files).new(self).list_uris(uri)
end

#list_models ⇒ Object

:nodoc:



291
292
293
 
# File 'lib/ruby_llm/provider.rb', line 291

def list_models # :nodoc:
  listing_protocol.new(self).list_models
end

#local? ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


423
424
425
 
# File 'lib/ruby_llm/provider.rb', line 423

def local? # :nodoc:
  self.class.local?
end

#long_context_pricing?(pricing, tokens) ⇒ Boolean

:nodoc:

Returns:

  • (Boolean) 


277
278
279
280
281
 
# File 'lib/ruby_llm/provider.rb', line 277

def long_context_pricing?(pricing, tokens) # :nodoc:
  pricing.long_context &&
    pricing.long_context_threshold &&
    tokens.input.to_i + tokens.cache_read.to_i + tokens.cache_write.to_i > pricing.long_context_threshold
end

#moderate(input, model:, with: [], provider_options: {}) ⇒ Object

:nodoc:



312
313
314
315
316
317
 
# File 'lib/ruby_llm/provider.rb', line 312

def moderate(input, model:, with: [], provider_options: {}) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :moderate)
  protocol.new(self, model).moderate(
    input, model: model_id_for(model), with:, provider_options:
  )
end

#name ⇒ Object

Returns the human-readable provider name, delegating to ::display_name.



101
102
103
 
# File 'lib/ruby_llm/provider.rb', line 101

def name # :nodoc:
  self.class.display_name
end

#ocr(file, model:, pages: nil, provider_options: {}) ⇒ Object

:nodoc:



367
368
369
370
 
# File 'lib/ruby_llm/provider.rb', line 367

def ocr(file, model:, pages: nil, provider_options: {}) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :ocr)
  protocol.new(self, model).ocr(file, model: model_id_for(model), pages:, provider_options:)
end

#paint(prompt, model:, size:, count: nil, with: nil, mask: nil, provider_options: {}) ⇒ Object

:nodoc:



327
328
329
330
331
332
 
# File 'lib/ruby_llm/provider.rb', line 327

def paint(prompt, model:, size:, count: nil, with: nil, mask: nil, provider_options: {}) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :paint)
  protocol.new(self, model).paint(
    prompt, model: model_id_for(model), size:, count:, with:, mask:, provider_options:
  )
end

#parse_error(response) ⇒ Object

:nodoc:



431
432
433
434
435
436
437
438
439
440
441
442
443
444
 
# File 'lib/ruby_llm/provider.rb', line 431

def parse_error(response) # :nodoc:
  body = parse_error_body(response)
  return unless body

  case body
  when Hash
    error_part_message(body)
  when Array
    messages = body.filter_map { |part| error_part_message(part) }.reject(&:empty?)
    messages.join('. ') unless messages.empty?
  else
    body
  end
end

#preprocess_message(message, model:, protocol: nil) ⇒ Object

:nodoc:



203
204
205
206
207
208
209
210
211
212
213
214
 
# File 'lib/ruby_llm/provider.rb', line 203

def preprocess_message(message, model:, protocol: nil) # :nodoc:
  protocol_class = resolve_protocol(
    protocol,
    model,
    tools: {},
    schema: nil,
    thinking: nil,
    tool_prefs: nil,
    citations: false
  )
  protocol_class.new(self, model).preprocess_message(message)
end

#protocol_for(_model) ⇒ Object

Returns the protocol class to use for model. Override to route between registered protocols per model or request operation. An explicit protocol: override on the chat or the provider's _protocol configuration option takes precedence over this hook.

def protocol_for(model, **)
model.id.match?(/audio|realtime/) ? protocols[:chat_completions] : super
end


127
128
129
 
# File 'lib/ruby_llm/provider.rb', line 127

def protocol_for(_model, **)
  default_protocol
end

#protocols ⇒ Object

:nodoc:



113
114
115
 
# File 'lib/ruby_llm/provider.rb', line 113

def protocols # :nodoc:
  self.class.protocols
end

#render(messages, tools:, temperature:, model:, provider_options: {}, schema: nil, thinking: nil, max_output_tokens: nil, citations: false, caching: nil, tool_prefs: nil, protocol: nil, before_request: [], provider_tools: [], compaction: nil, end_user: nil) ⇒ Object

:nodoc:



167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
 
# File 'lib/ruby_llm/provider.rb', line 167

def render(messages, tools:, temperature:, model:, provider_options: {}, schema: nil, thinking: nil, # :nodoc:
           max_output_tokens: nil, citations: false, caching: nil, tool_prefs: nil, protocol: nil,
           before_request: [], provider_tools: [], compaction: nil, end_user: nil)
  protocol_class = resolve_protocol(protocol, model, tools:, schema:, thinking:, tool_prefs:, citations:)
  protocol_class.new(self, model).render(
    messages,
    tools: tools,
    provider_tools: provider_tools,
    tool_prefs: tool_prefs,
    temperature: temperature,
    max_output_tokens: max_output_tokens,
    provider_options: provider_options,
    schema: schema,
    thinking: thinking,
    citations: citations,
    caching: caching,
    compaction: compaction,
    end_user: end_user,
    before_request: before_request
  )
end

#render_embedding(text, model:, dimensions: nil) ⇒ Object

:nodoc:



307
308
309
310
 
# File 'lib/ruby_llm/provider.rb', line 307

def render_embedding(text, model:, dimensions: nil) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :embed)
  protocol.new(self, model).render_embedding(text, model: model_id_for(model), dimensions:)
end

#rerank(query, documents, model:, top_n: nil, provider_options: {}) ⇒ Object

:nodoc:



372
373
374
375
 
# File 'lib/ruby_llm/provider.rb', line 372

def rerank(query, documents, model:, top_n: nil, provider_options: {}) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :rerank)
  protocol.new(self, model).rerank(query, documents, model: model_id_for(model), top_n:, provider_options:)
end

#research_later(prompt, **options) ⇒ Object

:nodoc:



319
320
321
 
# File 'lib/ruby_llm/provider.rb', line 319

def research_later(prompt, **options) # :nodoc:
  fetch_protocol(:research).new(self).create_research_job(prompt, **options)
end

#retry_delay(_response) ⇒ Object

Returns how many seconds the service asked us to wait before retrying a rate-limited request, or nil when the response carries no timing information. The retry middleware already honors the standard Retry-After header; override this to read provider-specific rate-limit headers.

def retry_delay(response)
response.response_headers['x-acme-ratelimit-reset']&.to_f
end


90
91
92
 
# File 'lib/ruby_llm/provider.rb', line 90

def retry_delay(_response)
  nil
end

#slug ⇒ Object

Returns the provider slug, delegating to ::slug.



95
96
97
 
# File 'lib/ruby_llm/provider.rb', line 95

def slug
  self.class.slug
end

#speak(input, model:, voice:, format:, provider_options: {}) ⇒ Object

:nodoc:



341
342
343
344
345
346
 
# File 'lib/ruby_llm/provider.rb', line 341

def speak(input, model:, voice:, format:, provider_options: {}, &) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :speak)
  protocol.new(self, model).speak(
    input, model: model_id_for(model), voice:, format:, provider_options:, &
  )
end

#tokenize(text, model:) ⇒ Object

:nodoc:



295
296
297
298
 
# File 'lib/ruby_llm/provider.rb', line 295

def tokenize(text, model:) # :nodoc:
  protocol = resolve_protocol(nil, model, operation: :tokenize)
  protocol.new(self, model).tokenize(text, model: model_id_for(model))
end

#tool_approval_response(tool_call, approved:, model:, protocol: nil) ⇒ Object

:nodoc:



157
158
159
 
# File 'lib/ruby_llm/provider.rb', line 157

def tool_approval_response(tool_call, approved:, model:, protocol: nil) # :nodoc:
  resolve_protocol(protocol, model).new(self, model).tool_approval_response(tool_call, approved:)
end

#transcribe(audio_file, model:, language:, format: nil, timestamps: nil, speaker_names: nil, speaker_references: nil, provider_options: {}, prompt: nil, temperature: nil) ⇒ Object

:nodoc:



348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
 
# File 'lib/ruby_llm/provider.rb', line 348

def transcribe(audio_file, model:, language:, format: nil, timestamps: nil, speaker_names: nil, # :nodoc:
               speaker_references: nil, provider_options: {}, prompt: nil, temperature: nil, &)
  protocol = resolve_protocol(nil, model, operation: :transcribe).new(self, model)
  options = protocol.render_transcription_options(timestamps:, format:, streaming: block_given?)
  provider_options = Support::Utils.deep_merge(options, provider_options)
  protocol.transcribe(
    audio_file,
    model: model_id_for(model),
    language:,
    format:,
    speaker_names:,
    speaker_references:,
    provider_options:,
    prompt:,
    temperature:,
    &
  )
end

#upload_file(file, filename: nil, purpose: nil, expires_in: nil, uri: nil, content_type: nil, provider_options: {}) ⇒ Object

:nodoc:



377
378
379
380
381
382
383
 
# File 'lib/ruby_llm/provider.rb', line 377

def upload_file(file, filename: nil, purpose: nil, expires_in: nil, uri: nil, content_type: nil, # :nodoc:
                provider_options: {})
  ensure_files_supported!
  options = { filename:, purpose:, expires_in:, uri:, content_type:, provider_options: }.compact

  protocols.fetch(:files).new(self).upload(file, **options)
end