Class: RubyLLM::CachedContent

Inherits:
Object
  • Object
show all
Includes:
Support::Inspectable
Defined in:
lib/ruby_llm/cached_content.rb

Overview

A CachedContent is a provider-side prompt cache resource. Create one with ::create from a long, stable prompt prefix, then attach it to a chat so later requests read the cached tokens instead of resending them. This lifecycle requires a provider with managed cache resources; automatic prompt caching is configured with Chat#with_caching.

cache = RubyLLM.cache(big_document, model: 'gemini-3.7-flash', ttl: 3600)
chat = RubyLLM.chat(model: 'gemini-3.7-flash').with_caching(id: cache)
chat.ask "What does the document conclude?"
cache.delete

Cache names are provider-owned. Persist #provider alongside #name and pass it back when finding the cache later.

Constant Summary

Constants included from Support::Inspectable

Support::Inspectable::TRUNCATE_AT

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Support::Inspectable

#full_inspect, #inspect, #pretty_print

Constructor Details

#initialize(name:, **attributes) ⇒ CachedContent

:nodoc:



46
47
48
49
50
51
52
53
54
55
 
# File 'lib/ruby_llm/cached_content.rb', line 46

def initialize(name:, **attributes) # :nodoc:
  @name = name
  @model = attributes[:model]
  @provider_instance = attributes[:provider_instance]
  @provider = attributes[:provider] || @provider_instance&.slug
  @created_at = attributes[:created_at]
  @expires_at = attributes[:expires_at]
  @tokens = attributes[:tokens]
  @metadata = attributes[:metadata] || {}
end

Instance Attribute Details

#created_at ⇒ Object (readonly)

The Time the provider stored the cache.



35
36
37
 
# File 'lib/ruby_llm/cached_content.rb', line 35

def created_at
  @created_at
end

#expires_at ⇒ Object (readonly)

The Time the provider will delete the cache.



38
39
40
 
# File 'lib/ruby_llm/cached_content.rb', line 38

def expires_at
  @expires_at
end

#metadata ⇒ Object (readonly)

The raw provider response data for the cache, as a Hash.



44
45
46
 
# File 'lib/ruby_llm/cached_content.rb', line 44

def metadata
  @metadata
end

#model ⇒ Object (readonly)

The model the cache was created for.



29
30
31
 
# File 'lib/ruby_llm/cached_content.rb', line 29

def model
  @model
end

#name ⇒ Object (readonly)

The provider-assigned resource name, such as "cachedContents/abc123".



26
27
28
 
# File 'lib/ruby_llm/cached_content.rb', line 26

def name
  @name
end

#provider ⇒ Object (readonly)

The slug of the provider that stores the cache.



32
33
34
 
# File 'lib/ruby_llm/cached_content.rb', line 32

def provider
  @provider
end

#tokens ⇒ Object (readonly)

The number of tokens stored in the cache.



41
42
43
 
# File 'lib/ruby_llm/cached_content.rb', line 41

def tokens
  @tokens
end

Class Method Details

.create(content, model:, ttl: nil, instructions: nil, provider: nil, context: nil, with: nil) ⇒ Object

Creates a provider-side prompt cache from content and returns a CachedContent. Also available as RubyLLM.cache.

RubyLLM::CachedContent.create(big_document, model: 'gemini-3.7-flash')

content is the text to cache; pass file attachments with with: the way Chat#ask accepts them. instructions: caches a system prompt alongside the content, and ttl: sets the cache lifetime in seconds (Integer) or as a provider duration string such as "300s". When provider: is omitted, the model's default provider is used. The content must exceed the model's minimum cacheable token count.



88
89
90
91
92
93
 
# File 'lib/ruby_llm/cached_content.rb', line 88

def self.create(content, model:, ttl: nil, instructions: nil, provider: nil, context: nil, with: nil)
  config = context&.config || RubyLLM.config
  model_instance, provider_instance = Models.resolve(model, provider: provider, config: config)

  provider_instance.cache_content(content, model: model_instance, ttl:, instructions:, with:)
end

.find(name, provider: nil, context: nil) ⇒ Object

Fetches an existing cache resource by name and returns a CachedContent. When provider: is omitted, the provider of the configured default model is used.

cache = RubyLLM::CachedContent.find("cachedContents/abc123", provider: :gemini)


101
102
103
104
105
106
107
108
109
110
 
# File 'lib/ruby_llm/cached_content.rb', line 101

def self.find(name, provider: nil, context: nil)
  config = context&.config || RubyLLM.config

  provider_instance = if provider
                        Provider.resolve!(provider).new(config)
                      else
                        Models.resolve(config.default_model, config:).last
                      end
  provider_instance.find_cache(name)
end

Instance Method Details

#delete ⇒ Object

Deletes the cache resource from the provider. Returns self.



58
59
60
61
 
# File 'lib/ruby_llm/cached_content.rb', line 58

def delete
  @provider_instance.delete_cache(name)
  self
end

#inspect_attributes ⇒ Object

:nodoc:



20
21
22
 
# File 'lib/ruby_llm/cached_content.rb', line 20

def inspect_attributes # :nodoc:
  { name: name, model: model, provider: provider, expires_at: expires_at }
end

#renew(ttl:) ⇒ Object

Extends the cache's lifetime to ttl: seconds from now, given as an Integer or a provider duration string such as "600s". Updates #expires_at and returns self.

cache.renew(ttl: 3600)


69
70
71
72
73
74
 
# File 'lib/ruby_llm/cached_content.rb', line 69

def renew(ttl:)
  refreshed = @provider_instance.extend_cache(name, ttl: ttl)
  @expires_at = refreshed.expires_at
  @metadata = refreshed.metadata
  self
end