Class: Fbe::Middleware::SqliteStore

Inherits:
Object
  • Object
show all
Defined in:
lib/fbe/middleware/sqlite_store.rb

Overview

Persisted SQLite store for Faraday::HttpCache

This class provides a persistent cache store backed by SQLite for use with Faraday::HttpCache middleware. It’s designed to cache HTTP responses from GitHub API calls to reduce API rate limit consumption and improve performance.

Key features:

  • Automatic version management to invalidate cache on version changes

  • Size-based cache eviction (configurable, defaults to 10MB)

  • Thread-safe SQLite transactions

  • JSON serialization for cached values

  • Filtering of non-cacheable requests (non-GET, URLs with query parameters)

Usage example:

store = Fbe::Middleware::SqliteStore.new(
  '/path/to/cache.db',
  '1.0.0',
  loog: logger,
  maxsize: '50Mb'
)
# Use with Faraday
Faraday.new do |builder|
  builder.use Faraday::HttpCache, store: store
end

The store automatically manages the SQLite database schema and handles cleanup operations when the database grows too large. Old entries are deleted based on their last access time to maintain the configured size limit.

Author

Yegor Bugayenko ([email protected])

Copyright

Copyright © 2024-2025 Zerocracy

License

MIT

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(path, version, loog: Loog::NULL, maxsize: '10Mb', maxvsize: '10Kb', ttl: nil, cache_min_age: nil) ⇒ SqliteStore

Initialize the SQLite store. or ttl is not nil or not Integer or not positive

Parameters:

  • path (String) —

    Path to the SQLite database file

  • version (String) —

    Version identifier for cache compatibility

  • loog (Loog) (defaults to: Loog::NULL) —

    Logger instance (optional, defaults to Loog::NULL)

  • maxsize (Integer) (defaults to: '10Mb') —

    Maximum database size in bytes (optional, defaults to 10MB)

  • maxvsize (Integer) (defaults to: '10Kb') —

    Maximum size in bytes of a single value (optional, defaults to 10Kb)

  • ttl (Integer, nil) (defaults to: nil) —

    lifetime of keys in hours

  • cache_min_age (Integer, nil) (defaults to: nil) —

    age which will could be overwritten in cache-control header

Raises:

  • (ArgumentError) —

    If path is nil/empty, directory doesn’t exist, version is nil/empty,



60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'lib/fbe/middleware/sqlite_store.rb', line 60

def initialize(path, version, loog: Loog::NULL, maxsize: '10Mb', maxvsize: '10Kb', ttl: nil, cache_min_age: nil)
  raise ArgumentError, 'Database path cannot be nil or empty' if path.nil? || path.empty?
  dir = File.dirname(path)
  raise ArgumentError, "Directory #{dir} does not exist" unless File.directory?(dir)
  raise ArgumentError, 'Version cannot be nil or empty' if version.nil? || version.empty?
  @path = File.absolute_path(path)
  @version = version
  @loog = loog
  @maxsize = Filesize.from(maxsize.to_s).to_i
  @maxvsize = Filesize.from(maxvsize.to_s).to_i
  raise ArgumentError, 'TTL can be nil or Integer > 0' if !ttl.nil? && !(ttl.is_a?(Integer) && ttl.positive?)
  @ttl = ttl
  if !cache_min_age.nil? && !(cache_min_age.is_a?(Integer) && cache_min_age.positive?)
    raise ArgumentError, 'Cache min age can be nil or Integer > 0'
  end
  @cache_min_age = cache_min_age
end

Instance Attribute Details

#path ⇒ Object (readonly)

Returns the value of attribute path.



48
49
50
 
# File 'lib/fbe/middleware/sqlite_store.rb', line 48

def path
  @path
end

Instance Method Details

#all ⇒ Array<Array>

Get all entries from the cache.

Returns:

  • (Array<Array>) —

    Array of [key, value] pairs



157
158
159
 
# File 'lib/fbe/middleware/sqlite_store.rb', line 157

def all
  perform { _1.execute('SELECT key, value FROM cache') }
end

#clear ⇒ void

This method returns an undefined value.

Clear all entries from the cache.



147
148
149
150
151
152
153
 
# File 'lib/fbe/middleware/sqlite_store.rb', line 147

def clear
  perform do |t|
    t.execute 'DELETE FROM cache;'
    t.execute "UPDATE meta SET value = ? WHERE key = 'version';", [@version]
  end
  @db.execute 'VACUUM;'
end

#delete(key) ⇒ nil

Delete a key from the cache.

Parameters:

  • key (String) —

    The cache key to delete

Returns:

  • (nil) 


98
99
100
101
 
# File 'lib/fbe/middleware/sqlite_store.rb', line 98

def delete(key)
  perform { _1.execute('DELETE FROM cache WHERE key = ?', [key]) }
  nil
end

#read(key) ⇒ Object?

Read a value from the cache.

Parameters:

  • key (String) —

    The cache key to read

Returns:

  • (Object, nil) —

    The cached value parsed from JSON, or nil if not found



81
82
83
84
85
86
87
88
89
90
91
92
93
 
# File 'lib/fbe/middleware/sqlite_store.rb', line 81

def read(key)
  value = perform do |t|
    t.execute('UPDATE cache SET touched_at = ?2 WHERE key = ?1;', [key, Time.now.utc.iso8601])
    t.execute('SELECT value FROM cache WHERE key = ? LIMIT 1;', [key])
  end.dig(0, 0)
  return unless value
  begin
    JSON.parse(Zlib::Inflate.inflate(value))
  rescue Zlib::Error => e
    @loog.info("Failed to decompress cached value for key: #{key}, error: #{e.message}, the key will be deleted")
    delete(key)
  end
end

#write(key, value) ⇒ nil

Note:

Values larger than 10KB are not cached

Note:

Non-GET requests and URLs with query parameters are not cached

Write a value to the cache.

Parameters:

  • key (String) —

    The cache key to write

  • value (Object) —

    The value to cache (will be JSON encoded)

Returns:

  • (nil) 


109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
 
# File 'lib/fbe/middleware/sqlite_store.rb', line 109

def write(key, value)
  return if value.is_a?(Array) && value.any? do |vv|
    req = JSON.parse(vv[0])
    req['method'] != 'get'
  end
  if @cache_min_age && value.is_a?(Array) && value[0].is_a?(Array) && value[0].size > 1
    begin
      resp = JSON.parse(value[0][1])
    rescue TypeError, JSON::ParserError => e
      @loog.info("Failed to parse response to rewrite the cache age: #{e.message}")
      resp = nil
    end
    cache_control = resp.dig('response_headers', 'cache-control') if resp.is_a?(Hash)
    if cache_control && !cache_control.empty?
      %w[max-age s-maxage].each do |key|
        age = cache_control.scan(/#{key}=(\d+)/i).first&.first&.to_i
        if age
          age = [age, @cache_min_age].max
          cache_control = cache_control.sub(/#{key}=(\d+)/, "#{key}=#{age}")
        end
      end
      resp['response_headers']['cache-control'] = cache_control
      value[0][1] = JSON.dump(resp)
    end
  end
  value = Zlib::Deflate.deflate(JSON.dump(value))
  return if value.bytesize > @maxvsize
  perform do |t|
    t.execute(<<~SQL, [key, value, Time.now.utc.iso8601])
      INSERT INTO cache(key, value, touched_at, created_at) VALUES(?1, ?2, ?3, ?3)
      ON CONFLICT(key) DO UPDATE SET value = ?2, touched_at = ?3, created_at = ?3
    SQL
  end
  nil
end