Class: Hoodoo::TransientStore

Inherits:
Object
  • Object
show all
Defined in:
lib/hoodoo/transient_store/transient_store.rb,
lib/hoodoo/transient_store/mocks/redis.rb,
lib/hoodoo/transient_store/mocks/dalli_client.rb,
lib/hoodoo/transient_store/transient_store/base.rb,
lib/hoodoo/transient_store/transient_store/redis.rb,
lib/hoodoo/transient_store/transient_store/memcached.rb,
lib/hoodoo/transient_store/transient_store/memcached_redis_mirror.rb

Overview

A simple abstraction over transient storage engines such as Memcached or Redis, making it it easier for client code to switch engines with very few changes. If the storage engine chosen when creating instances of this object is defined in application-wide configuration data, all you would need to do is change the configuration for all new TransientStore instances to use the new engine.

Defined Under Namespace

Classes: Base, Memcached, MemcachedRedisMirror, Mocks, Redis

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(storage_engine:, storage_host_uri:, default_maximum_lifespan: 604800, default_namespace: 'nz_co_loyalty_hoodoo_transient_store_') ⇒ TransientStore

Instantiate a new Transient storage object through which temporary data can be stored or retrieved.

The TransientStore abstraction is a high level and simple abstraction over heterogenous data storage engines. It does not expose the many subtle configuration settings usually available in such. If you need to take advantage of those at an item storage level, you’ll need to use a lower level interface and thus lock your code to the engine of choice.

Engine plug-ins are recommended to attempt to gain and test a connection to the storage engine when this object is constructed, so if building a TransientStore instance, ensure your chosen storage engine is running first. Exceptions may be raised by storage engines, so you will probably want to catch those with more civilised error handling code.

Named parameters are:

storage_engine

An entry from ::supported_storage_engines.

storage_host_uri

The engine-dependent connection URI. Consult documentation for your chosen engine to find out its connection URI requirements, along with the documentation for the constructor method of the plug-in in use, since in some cases requirements may be unusual (e.g. in Hoodoo::TransientStore::MemcachedRedisMirror).

default_maximum_lifespan

The default time-to-live for data items, in, seconds; can be overridden per item; default is 604800 seconds or 7 days.

default_namespace

Storage engine keys are namespaced with nz_co_loyalty_hoodoo_transient_store_ by default, though this can be overridden here. Pass a String or Symbol.



143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 143

def initialize(
  storage_engine:,
  storage_host_uri:,
  default_maximum_lifespan: 604800,
  default_namespace:        'nz_co_loyalty_hoodoo_transient_store_'
)

  unless self.class.supported_storage_engines().include?( storage_engine )

    # Be kind and use 'inspect' to indicate that we expect Symbols here
    # in the exception, because of the arising leading ':' in the output.
    #
    engines = self.class.supported_storage_engines().map { | symbol | "'#{ symbol.inspect }'" }
    allowed = engines.join( ', ' )

    raise "Hoodoo::TransientStore: Unrecognised storage engine '#{ storage_engine.inspect }' requested; allowed values: #{ allowed }"
  end

  @default_maximum_lifespan = default_maximum_lifespan
  @default_namespace        = ( default_namespace || '' ).to_s()
  @storage_engine           = storage_engine
  @storage_engine_instance  = @@supported_storage_engines[ storage_engine ].new(
    storage_host_uri: storage_host_uri,
    namespace:        @default_namespace
  )

end

Instance Attribute Details

#default_maximum_lifespan ⇒ Object (readonly)

Read this instance’s default item maximum lifespan, in seconds. See also ::new.



100
101
102
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 100

def default_maximum_lifespan
  @default_maximum_lifespan
end

#default_namespace ⇒ Object (readonly)

Read this instance’s default storage namespace, as a String. See also ::new.



105
106
107
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 105

def default_namespace
  @default_namespace
end

#storage_engine ⇒ Object (readonly)

Read this instance’s storage engine; see ::supported_storage_engines and ::new.



88
89
90
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 88

def storage_engine
  @storage_engine
end

#storage_engine_instance ⇒ Object (readonly)

Read the storage engine instance for the #storage_engine - this allows engine-specific configuration to be set where available, though this is strongly discouraged as it couples client code to the engine in use, defeating the main rationale behind the TransientStore abstraction.



95
96
97
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 95

def storage_engine_instance
  @storage_engine_instance
end

Class Method Details

.deregister(as:) ⇒ Object

Remove a storage engine plugin class from the supported collection. Any existing Hoodoo::TransientStore instances using the removed class will not be affected, but new instances cannot be made.

Named parameters are:

as

The value given to #register in the corresponding as parameter.



69
70
71
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 69

def self.deregister( as: )
  @@supported_storage_engines.delete( as )
end

.register(as:, using:) ⇒ Object

Register a new storage engine plugin class. It MUST inherit from and thus follow the template laid out in Hoodoo::TransientStore::Base.

Named parameters are:

as

The name, as a Symbol, for the ::new storage_engine input parameter, to select this plugin.

using

The class reference of the Hoodoo::TransientStore::Base subclass to be associated with the name given in as.

Example:

Hoodoo::TransientStore.register(
  as:    :memcached,
  using: Hoodoo::TransientStore::Memcached
)



45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 45

def self.register( as:, using: )
  as = as.to_sym

  @@supported_storage_engines = {} unless defined?( @@supported_storage_engines )

  unless using < Hoodoo::TransientStore::Base
    raise "Hoodoo::TransientStore.register requires a Hoodoo::TransientStore::Base subclass - got '#{ using.to_s }'"
  end

  if @@supported_storage_engines.has_key?( as )
    raise "Hoodoo::TransientStore.register: A storage engine called '#{ as }' is already registered"
  end

  @@supported_storage_engines[ as ] = using
end

.supported_storage_engines ⇒ Object

Return an array of the names of all supported storage engine names known to the Hoodoo::TransientStore class. Any one of those names can be used with the ::new storage_engine parameter.



77
78
79
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 77

def self.supported_storage_engines
  @@supported_storage_engines.keys()
end

Instance Method Details

#close ⇒ Object

If you aren’t going to use this instance again, it is good manners to immediately close its connection(s) to any storage engines by calling here.

No useful return value is generated and exceptions are ignored.



323
324
325
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 323

def close
  @storage_engine_instance.close() rescue nil
end

#delete(key:) ⇒ Object

Delete data previously stored with #set.

Named parameters are:

key

Key previously given to #set.

Returns:

  • true if deletion was successful, if the item has already expired or if the key is simply not recognised so there is no more work to do.

  • false if deletion failed but the reason is unknown.

  • An Exception instance if deletion failed and the storage engine raised an exception describing the problem.

Only non-empty String or Symbol keys are permitted, else an exception will be raised.



299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 299

def delete( key: )
  key = normalise_key( key, 'delete' )

  begin
    result = @storage_engine_instance.delete( key: key )

    if result != true && result != false
      raise "Hoodoo::TransientStore\#delete: Engine '#{ @storage_engine }' returned an invalid response"
    end

  rescue => e
    result = e

  end

  return result
end

#get(key:, allow_throw: false) ⇒ Object

Retrieve data previously stored with #set.

Named parameters are:

key

Key previously given to #set.

allow_throw

If true, exceptions raised by the underlying storage engine are thrown, else ignored and nil is returned. Optional; default is false.

Returns nil if the item is not found - either the key is wrong, the stored data has expired or the stored data has been evicted early from the storage engine’s pool.

Only non-empty String or Symbol keys are permitted, else an exception will be raised.



272
273
274
275
276
277
278
279
280
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 272

def get( key:, allow_throw: false )
  key = normalise_key( key, 'get' )

  begin
    @storage_engine_instance.get( key: key )
  rescue
    raise if allow_throw
  end
end

#set(key:, payload:, maximum_lifespan: nil) ⇒ Object

Set (write) a given payload into the storage engine with the given payload and maximum lifespan.

Payloads must only contain simple types such as Hash, Array, String and Integer. Complex types like Symbol, Date, Float, BigDecimal or custom objects are unlikely to serialise properly but since this depends upon the storage engine in use, errors may or may not be raised for misuse.

Storage engines usually have a maximum payload size limit; consult your engine administrator for information. For example, the default - but reconfigurable - maximum payload size for Memcached is 1MB.

For maximum possible compatibility:

  • Use only Hash payloads with String key/value paids and no nesting. You may choose to marshal the data into a String manually for unusual data requirements, manually converting back when reading stored data.

  • Keep the payload size as small as possible - large objects belong in bulk storage engines such as Amazon S3.

These are only guidelines though - heterogenous storage engine support and the ability of system administrators to arbitrarily configure those storage engines makes it impossible to be more precise.

Returns:

  • true if storage was successful

  • false if storage failed but the reason is unknown

  • An Exception instance if storage failed and the storage engine raised an exception describing the problem.

Named parameters are:

key

Storage key to use in the engine, which is then used in subsequent calls to #get and possibly eventually to #delete. Only non-empty Strings or Symbols are permitted, else an exception will be raised.

payload

Payload data to store under the given key. A flat Hash is recommended rather than simple types such as String (unless marshalling a complex type into such) in order to make potential additions to stored data easier to implement. Note that nil is prohibited.

maximum_lifespan

Optional maximum lifespan, seconds. Storage engines may chooset to evict payloads sooner than this; it is a maximum time, not a guarantee. Omit to use this TransientStore instance’s default value - see ::new. If you know you no longer need a piece of data at a particular point in the execution flow of your code, explicitly delete it via #delete rather than leaving it to expire. This maximises the storage engine’s pool free space and so minimises the chance of early item eviction.



227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
 
# File 'lib/hoodoo/transient_store/transient_store.rb', line 227

def set( key:, payload:, maximum_lifespan: nil )
  key = normalise_key( key, 'set' )

  if payload.nil?
    raise "Hoodoo::TransientStore\#set: Payloads of 'nil' are prohibited"
  end

  maximum_lifespan ||= @default_maximum_lifespan

  begin
    result = @storage_engine_instance.set(
      key:              key,
      payload:          payload,
      maximum_lifespan: maximum_lifespan
    )

    if result != true && result != false
      raise "Hoodoo::TransientStore\#set: Engine '#{ @storage_engine }' returned an invalid response"
    end

  rescue => e
    result = e

  end

  return result
end