Class: SemanticCache::ClientWrapper

Inherits:
Object
  • Object
show all
Defined in:
lib/semantic_cache/client_wrapper.rb

Overview

Wraps an OpenAI client (or compatible) to automatically cache chat calls.

client = OpenAI::Client.new
cached = SemanticCache::ClientWrapper.new(client)

# All chat calls are automatically cached
cached.chat(parameters: { model: "gpt-4o", messages: [...] })

Or use the shorthand:

cached = SemanticCache.wrap(client)

Instance Method Summary collapse

Constructor Details

#initialize(client, cache: nil, **cache_options) ⇒ ClientWrapper

Returns a new instance of ClientWrapper.



17
18
19
20
 
# File 'lib/semantic_cache/client_wrapper.rb', line 17

def initialize(client, cache: nil, **cache_options)
  @client = client
  @cache = cache || Cache.new(**cache_options)
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(method) ⇒ Object

Delegate everything else to the wrapped client



40
41
42
43
44
45
46
 
# File 'lib/semantic_cache/client_wrapper.rb', line 40

def method_missing(method, ...)
  if @client.respond_to?(method)
    @client.send(method, ...)
  else
    super
  end
end

Instance Method Details

#chat(parameters: {}, **kwargs) ⇒ Object 



22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/semantic_cache/client_wrapper.rb', line 22

def chat(parameters: {}, **kwargs)
  messages = parameters[:messages] || parameters["messages"] || []
  model = parameters[:model] || parameters["model"]

  # Use the last user message as the cache key
  user_message = messages.reverse.find { |m| m[:role] == "user" || m["role"] == "user" }
  query = user_message && (user_message[:content] || user_message["content"])

  if query
    @cache.fetch(query, model: model) do
      @client.chat(parameters: parameters, **kwargs)
    end
  else
    @client.chat(parameters: parameters, **kwargs)
  end
end

#respond_to_missing?(method, include_private = false) ⇒ Boolean

Returns:

  • (Boolean) 


48
49
50
 
# File 'lib/semantic_cache/client_wrapper.rb', line 48

def respond_to_missing?(method, include_private = false)
  @client.respond_to?(method, include_private) || super
end

#semantic_cache ⇒ Object

Access the underlying cache for stats, invalidation, etc.



53
54
55
 
# File 'lib/semantic_cache/client_wrapper.rb', line 53

def semantic_cache
  @cache
end