Class: Dis::Storage

Inherits:
Object
  • Object
show all
Defined in:
lib/dis/storage.rb

Overview

Dis Storage

This is the interface for interacting with the storage layers.

All queries are scoped by object type, which will default to the table name of the model. Take care to use your own scope if you interact with the store directly, as models will purge expired content when they change.

Files are stored with a SHA1 digest of the file contents as the key. This ensures data is deduplicated per scope. Hash collisions will be silently ignored.

Layers should be added to Dis::Storage.layers. At least one writeable, non-delayed layer must exist.

Class Method Summary collapse

Class Method Details

.delayed_delete(type, hash) ⇒ Object

Deletes content from all delayed layers.

Dis::Storage.delayed_delete("things", hash)



122
123
124
125
126
 
# File 'lib/dis/storage.rb', line 122

def delayed_delete(type, hash)
  layers.delayed.writeable.each do |layer|
    layer.delete(type, hash)
  end
end

.delayed_store(type, hash) ⇒ Object

Transfers files from immediate layers to all delayed layers.

Dis::Storage.delayed_store("things", hash)



58
59
60
61
62
63
 
# File 'lib/dis/storage.rb', line 58

def delayed_store(type, hash)
  file = get(type, hash)
  layers.delayed.writeable.each do |layer|
    layer.store(type, hash, file)
  end
end

.delete(type, hash) ⇒ Object

Deletes a file from all layers. Kicks off a Dis::Jobs::Delete job if any delayed layers are defined. Returns true if the file existed in any immediate layers, or false if not.

Dis::Storage.delete("things", hash)
# => true
Dis::Storage.delete("things", hash)
# => false



107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/dis/storage.rb', line 107

def delete(type, hash)
  require_writeable_layers!
  deleted = false
  layers.immediate.writeable.each do |layer|
    deleted = true if layer.delete(type, hash)
  end
  if layers.delayed.writeable.any?
    Dis::Jobs::Delete.perform_later(type, hash)
  end
  deleted
end

.exists?(type, hash) ⇒ Boolean

Returns true if the file exists in any layer.

Dis::Storage.exists?("things", hash) # => true

Returns:

  • (Boolean) 


68
69
70
71
72
73
74
 
# File 'lib/dis/storage.rb', line 68

def exists?(type, hash)
  require_layers!
  layers.each do |layer|
    return true if layer.exists?(type, hash)
  end
  false
end

.file_digest(file) {|hash| ... } ⇒ Object

Returns a hex digest for a given binary. Accepts files, strings and Fog models.

Yields:

  • (hash) 


22
23
24
25
26
27
28
29
30
31
32
33
 
# File 'lib/dis/storage.rb', line 22

def file_digest(file, &block)
  hash = case file
  when Fog::Model
    digest.hexdigest(file.body)
  when String
    digest.hexdigest(file)
  else
    digest.file(file.path).hexdigest
  end
  yield hash if block_given?
  hash
end

.get(type, hash) ⇒ Object

Retrieves a file from the store.

stuff = Dis::Storage.get("things", hash)

If any misses are detected, it will try to fetch the file from the first available layer, then store it in all immediate layer.

Returns an instance of Fog::Model.

Raises:



84
85
86
87
88
89
90
91
92
93
94
95
96
 
# File 'lib/dis/storage.rb', line 84

def get(type, hash)
  require_layers!
  miss = false
  layers.each do |layer|
    if result = layer.get(type, hash)
      store_immediately!(type, result) if miss
      return result
    else
      miss = true
    end
  end
  raise Dis::Errors::NotFoundError
end

.layers ⇒ Object

Exposes the layer set, which is an instance of Dis::Layers.



37
38
39
 
# File 'lib/dis/storage.rb', line 37

def layers
  @layers ||= Dis::Layers.new
end

.store(type, file) ⇒ Object

Stores a file and returns a digest. Kicks off a Dis::Jobs::Store job if any delayed layers are defined.

hash = Dis::Storage.store("things", File.open('foo.bin'))
# => "8843d7f92416211de9ebb963ff4ce28125932878"



46
47
48
49
50
51
52
53
 
# File 'lib/dis/storage.rb', line 46

def store(type, file)
  require_writeable_layers!
  hash = store_immediately!(type, file)
  if layers.delayed.writeable.any?
    Dis::Jobs::Store.perform_later(type, hash)
  end
  hash
end