Class: AzureBlob::Client

Inherits:
Object
  • Object
show all
Defined in:
lib/azure_blob/client.rb

Overview

AzureBlob Client class. You interact with the Azure Blob api through an instance of this class.

Instance Method Summary collapse

Constructor Details

#initialize(account_name:, access_key: nil, principal_id: nil, container:, host: nil, **options) ⇒ Client

Returns a new instance of Client.



19
20
21
22
23
24
25
26
27
28
 
# File 'lib/azure_blob/client.rb', line 19

def initialize(account_name:, access_key: nil, principal_id: nil, container:, host: nil, **options)
  @account_name = 
  @container = container
  @host = host
  @cloud_regions = options[:cloud_regions]&.to_sym || :global
  @access_key = access_key
  @principal_id = principal_id
  @use_managed_identities = options[:use_managed_identities]
  signer unless options[:lazy]
end

Instance Method Details

#append_blob_block(key, content, options = {}) ⇒ Object

Append a block to an Append Blob

Calls to Append Block

Options:

:content_md5

Will ensure integrity of the upload. The checksum must be a base64 digest. Can be produced with OpenSSL::Digest::MD5.base64digest. The checksum must be the checksum of the block not the blob.



321
322
323
324
325
326
327
328
329
330
331
332
333
334
 
# File 'lib/azure_blob/client.rb', line 321

def append_blob_block(key, content, options = {})
  uri = generate_uri("#{container}/#{key}")
  query = { comp: "appendblock" }
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)

  headers = {
    "Content-Length": content_size(content),
    "Content-Type": options[:content_type],
    "Content-MD5": options[:content_md5],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).put(content)
end

#blob_exist?(key, options = {}) ⇒ Boolean

Returns a boolean indicating if the blob exists.

Calls to Get Blob Properties

Returns:

  • (Boolean) 


189
190
191
192
193
 
# File 'lib/azure_blob/client.rb', line 189

def blob_exist?(key, options = {})
  get_blob_properties(key, options).present?
rescue AzureBlob::Http::FileNotFoundError
  false
end

#commit_blob_blocks(key, block_ids, options = {}) ⇒ Object

Commits the list of blocks to a blob.

Calls to Put Block List

Takes a key (path) and an array of block ids

Options:

:content_md5

This is the checksum for the whole blob. The checksum is saved on the blob, but it is not validated! Add a checksum for each block if you want Azure to validate integrity.



375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
 
# File 'lib/azure_blob/client.rb', line 375

def commit_blob_blocks(key, block_ids, options = {})
  block_list = BlockList.new(block_ids)
  content = block_list.to_s
  uri = generate_uri("#{container}/#{key}")
  query = { comp: "blocklist" }
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)

  headers = {
    "Content-Length": content_size(content),
    "Content-Type": options[:content_type],
    "x-ms-blob-content-md5": options[:content_md5],
    "x-ms-blob-content-disposition": options[:content_disposition],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put(content)
end

#container_exist?(options = {}) ⇒ Boolean

Returns a boolean indicating if the container exists.

Calls to Get Container Properties

Returns:

  • (Boolean) 


230
231
232
 
# File 'lib/azure_blob/client.rb', line 230

def container_exist?(options = {})
  get_container_properties(options = {}).present?
end

#copy_blob(key, source_key, options = {}) ⇒ Object

Copy a blob between containers or within the same container

Calls to Copy Blob From URL

Parameters:

  • key: destination blob path

  • source_key: source blob path

  • options: additional options

    • source_client: AzureBlob::Client instance for the source container (optional) If not provided, copies from within the same container



93
94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/azure_blob/client.rb', line 93

def copy_blob(key, source_key, options = {})
  source_client = options.delete(:source_client) || self
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(timeout: options[:timeout]) if options[:timeout]

  source_uri = source_client.signed_uri(source_key, permissions: "r", expiry: Time.at(Time.now.to_i + 300).utc.iso8601)

  headers = {
    "x-ms-copy-source": source_uri.to_s,
    "x-ms-requires-sync": "true",
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put
end

#create_append_blob(key, options = {}) ⇒ Object

Creates a Blob of type append.

Calls to Put Blob

You are expected to append blocks to the blob with append_blob_block after creating the blob. Options:

:content_type

Will be saved on the blob in Azure.

:content_disposition

Will be saved on the blob in Azure.



297
298
299
300
301
302
303
304
305
306
307
308
309
310
 
# File 'lib/azure_blob/client.rb', line 297

def create_append_blob(key, options = {})
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(timeout: options[:timeout]) if options[:timeout]

  headers = {
    "x-ms-blob-type": "AppendBlob",
    "Content-Length": 0,
    "Content-Type": options[:content_type],
    "Content-MD5": options[:content_md5],
    "x-ms-blob-content-disposition": options[:content_disposition],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put(nil)
end

#create_block_blob(key, content, options = {}) ⇒ Object

Create a blob of type block. Will automatically split the the blob in multiple block and send the blob in pieces (blocks) if the blob is too big.

When the blob is small enough this method will send the blob through Put Blob

If the blob is too big, the blob is split in blocks sent through a series of Put Block requests followed by a Put Block List to commit the block list.

Takes a key (path), the content (String or IO object), and options.

Options:

:content_type

Will be saved on the blob in Azure.

:content_disposition

Will be saved on the blob in Azure.

:content_md5

Will ensure integrity of the upload. The checksum must be a base64 digest. Can be produced with OpenSSL::Digest::MD5.base64digest. The checksum is only checked on a single upload! To verify checksum when uploading multiple blocks, call directly put_blob_block with a checksum for each block, then commit the blocks with commit_blob_blocks.

:block_size

Block size in bytes, can be used to force the method to split the upload in smaller chunk. Defaults to AzureBlob::DEFAULT_BLOCK_SIZE and cannot be bigger than AzureBlob::MAX_UPLOAD_SIZE



51
52
53
54
55
56
57
 
# File 'lib/azure_blob/client.rb', line 51

def create_block_blob(key, content, options = {})
  if content_size(content) > (options[:block_size] || DEFAULT_BLOCK_SIZE)
    put_blob_multiple(key, content, **options)
  else
    put_blob_single(key, content, **options)
  end
end

#create_container(options = {}) ⇒ Object

Create the container

Calls to Create Container



237
238
239
240
241
242
243
244
245
246
247
248
 
# File 'lib/azure_blob/client.rb', line 237

def create_container(options = {})
  uri = generate_uri(container)
  headers = {}
  headers[:"x-ms-blob-public-access"] = "blob" if options[:public_access]
  headers[:"x-ms-blob-public-access"] = options[:public_access] if [ "container", "blob" ].include?(options[:public_access])
  headers.merge!(additional_headers(options))

  query = { restype: "container" }
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)
  response = Http.new(uri, headers, signer:).put
end

#delete_blob(key, options = {}) ⇒ Object

Delete a blob

Calls to Delete Blob

Takes a key (path) and options.

Options:

:delete_snapshots

Sets the value of the x-ms-delete-snapshots header. Default to include



117
118
119
120
121
122
123
124
125
126
 
# File 'lib/azure_blob/client.rb', line 117

def delete_blob(key, options = {})
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(timeout: options[:timeout]) if options[:timeout]

  headers = {
    "x-ms-delete-snapshots": options[:delete_snapshots] || "include",
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).delete
end

#delete_container(options = {}) ⇒ Object

Delete the container

Calls to Delete Container



253
254
255
256
257
258
259
 
# File 'lib/azure_blob/client.rb', line 253

def delete_container(options = {})
  uri = generate_uri(container)
  query = { restype: "container" }
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)
  response = Http.new(uri, additional_headers(options), signer:).delete
end

#delete_prefix(prefix, options = {}) ⇒ Object

Delete all blobs prefixed by the given prefix.

Calls to List blobs followed to a series of calls to Delete Blob

Takes a prefix and options

Look delete_blob for the list of options.



136
137
138
139
 
# File 'lib/azure_blob/client.rb', line 136

def delete_prefix(prefix, options = {})
  results = list_blobs(prefix:)
  results.each { |key| delete_blob(key) }
end

#generate_uri(path) ⇒ Object

Return a URI object to a resource in the container. Takes a path.

Example: generate_uri(“#{container}/#{key}”)



264
265
266
267
268
269
270
271
 
# File 'lib/azure_blob/client.rb', line 264

def generate_uri(path)
  # https://github.com/Azure/azure-storage-ruby/blob/master/common/lib/azure/storage/common/service/storage_service.rb#L191-L201
  encoded_path = CGI.escape(path.encode("UTF-8"))
  encoded_path = encoded_path.gsub(/%2F/, "/")
  encoded_path = encoded_path.gsub(/%5C/, "/")
  encoded_path = encoded_path.gsub(/\+/, "%20")
  URI.parse(File.join(host, encoded_path))
end

#get_blob(key, options = {}) ⇒ Object

Returns the full or partial content of the blob

Calls to the Get Blob endpoint.

Takes a key (path) and options.

Options:

:start

Starting point in bytes

:end

Ending point in bytes



71
72
73
74
75
76
77
78
79
80
 
# File 'lib/azure_blob/client.rb', line 71

def get_blob(key, options = {})
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(timeout: options[:timeout]) if options[:timeout]

  headers = {
    "x-ms-range": options[:start] && "bytes=#{options[:start]}-#{options[:end]}",
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).get
end

#get_blob_properties(key, options = {}) ⇒ Object

Returns a Blob object without the content.

Calls to Get Blob Properties

This can be used to obtain metadata such as content type, disposition, checksum or Azure custom metadata. To check for blob presence, look for blob_exist? as get_blob_properties raises on missing blob.



177
178
179
180
181
182
183
184
 
# File 'lib/azure_blob/client.rb', line 177

def get_blob_properties(key, options = {})
  uri = generate_uri("#{container}/#{key}")
  uri.query = URI.encode_www_form(timeout: options[:timeout]) if options[:timeout]

  response = Http.new(uri, additional_headers(options), signer:).head

  Blob.new(response)
end

#get_blob_tags(key, options = {}) ⇒ Object

Returns the tags associated with a blob

Calls to the Get Blob Tags endpoint.

Takes a key (path) of the blob.

Returns a hash of the blob’s tags.



202
203
204
205
206
207
208
209
210
 
# File 'lib/azure_blob/client.rb', line 202

def get_blob_tags(key, options = {})
  uri = generate_uri("#{container}/#{key}")
  query = { comp: "tags" }
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)
  response = Http.new(uri, additional_headers(options), signer:).get

  Tags.from_response(response).to_h
end

#get_container_properties(options = {}) ⇒ Object

Returns a Container object.

Calls to Get Container Properties

This can be used to see if the container exist or obtain metadata.



217
218
219
220
221
222
223
224
225
 
# File 'lib/azure_blob/client.rb', line 217

def get_container_properties(options = {})
  uri = generate_uri(container)
  query = { restype: "container" }
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)
  response = Http.new(uri, additional_headers(options), signer:, raise_on_error: false).head

  Container.new(response)
end

#list_blobs(options = {}) ⇒ Object

Returns a BlobList containing a list of keys (paths)

Calls to List blobs

Options:

:prefix

Prefix of the blobs to be listed. Defaults to listing everything in the container.

:max_results

Maximum number of results to return per page.



150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/azure_blob/client.rb', line 150

def list_blobs(options = {})
  uri = generate_uri(container)
  query = {
    comp: "list",
    restype: "container",
    prefix: options[:prefix].to_s.gsub(/\\/, "/"),
  }
  query[:maxresults] = options[:max_results] if options[:max_results]
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)

  fetcher = ->(marker) do
    query[:marker] = marker
    query.reject! { |key, value| value.to_s.empty? }
    uri.query = URI.encode_www_form(**query)
    response = Http.new(uri, additional_headers(options), signer:).get
  end

  BlobList.new(fetcher)
end

#put_blob_block(key, index, content, options = {}) ⇒ Object

Uploads a block to a blob.

Calls to Put Block

Returns the id of the block. Required to commit the list of blocks to a blob.

Options:

:content_md5

Must be the checksum for the block not the blob. The checksum must be a base64 digest. Can be produced with OpenSSL::Digest::MD5.base64digest.



346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
 
# File 'lib/azure_blob/client.rb', line 346

def put_blob_block(key, index, content, options = {})
  block_id = generate_block_id(index)
  uri = generate_uri("#{container}/#{key}")
  query = { comp: "block", blockid: block_id }
  query[:timeout] = options[:timeout] if options[:timeout]
  uri.query = URI.encode_www_form(**query)

  headers = {
    "Content-Length": content_size(content),
    "Content-Type": options[:content_type],
    "Content-MD5": options[:content_md5],
  }.merge(additional_headers(options))

  Http.new(uri, headers, signer:).put(content)

  block_id
end

#signed_uri(key, permissions:, expiry:, **options) ⇒ Object

Returns an SAS signed URI

Takes a

  • key (path)

  • A permission string (+“r”+, “rw”)

  • expiry as a UTC iso8601 time string

  • options



280
281
282
283
284
 
# File 'lib/azure_blob/client.rb', line 280

def signed_uri(key, permissions:, expiry:, **options)
  uri = generate_uri("#{container}/#{key}")
  uri.query = signer.sas_token(uri, permissions:, expiry:, **options)
  uri
end