Class: Gcloud::Storage::Bucket

Inherits:
Object
  • Object
show all
Defined in:
lib/gcloud/storage/bucket.rb,
lib/gcloud/storage/bucket/acl.rb,
lib/gcloud/storage/bucket/cors.rb,
lib/gcloud/storage/bucket/list.rb

Overview

# Bucket

Represents a Storage bucket. Belongs to a Project and has many Files.

Examples:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
file = bucket.file "path/to/my-file.ext"

Defined Under Namespace

Classes: Acl, Cors, DefaultAcl, List, Updater

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Bucket

Returns a new instance of Bucket.



49
50
51
52
 
# File 'lib/gcloud/storage/bucket.rb', line 49

def initialize
  @connection = nil
  @gapi = {}
end

Instance Attribute Details

#connection ⇒ Object 



41
42
43
 
# File 'lib/gcloud/storage/bucket.rb', line 41

def connection
  @connection
end

#gapi ⇒ Object 



45
46
47
 
# File 'lib/gcloud/storage/bucket.rb', line 45

def gapi
  @gapi
end

Class Method Details

.from_gapi(gapi, conn) ⇒ Object 



775
776
777
778
779
780
 
# File 'lib/gcloud/storage/bucket.rb', line 775

def self.from_gapi gapi, conn
  new.tap do |f|
    f.gapi = gapi
    f.connection = conn
  end
end

Instance Method Details

#acl ⇒ Object

The Bucket::Acl instance used to control access to the bucket.

A bucket has owners, writers, and readers. Permissions can be granted to an individual user’s email address, a group’s email address, as well as many predefined lists.

Examples:

Grant access to a user by pre-pending ‘“user-”` to an email:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.acl.add_reader "user-#{email}"

Grant access to a group by pre-pending ‘“group-”` to an email:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.acl.add_reader "group-#{email}"

Or, grant access via a predefined permissions list:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

bucket.acl.public!

See Also:



709
710
711
 
# File 'lib/gcloud/storage/bucket.rb', line 709

def acl
  @acl ||= Bucket::Acl.new self
end

#api_url ⇒ Object

A URL that can be used to access the bucket using the REST API.



75
76
77
 
# File 'lib/gcloud/storage/bucket.rb', line 75

def api_url
  @gapi["selfLink"]
end

#cors {|cors| ... } ⇒ Object

Returns the current CORS configuration for a static website served from the bucket.

The return value is a frozen (unmodifiable) array of hashes containing the attributes specified for the Bucket resource field [cors](cloud.google.com/storage/docs/json_api/v1/buckets#cors).

This method also accepts a block for updating the bucket’s CORS rules. See Cors for details.

Examples:

Retrieving the bucket’s CORS rules.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"
bucket.cors #=> [{"origin"=>["http://example.org"],
            #     "method"=>["GET","POST","DELETE"],
            #     "responseHeader"=>["X-My-Custom-Header"],
            #     "maxAgeSeconds"=>3600}]

Updating the bucket’s CORS rules inside a block.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage
bucket = storage.bucket "my-todo-app"

bucket.update do |b|
  b.cors do |c|
    c.add_rule ["http://example.org", "https://example.org"],
               "*",
               response_headers: ["X-My-Custom-Header"],
               max_age: 3600
  end
end

Yields:

  • (cors) —

    a block for setting CORS rules

Yield Parameters:

See Also:



130
131
132
133
134
135
136
137
 
# File 'lib/gcloud/storage/bucket.rb', line 130

def cors
  if block_given?
    cors_builder = Bucket::Cors.new @gapi["cors"]
    yield cors_builder
    self.cors = cors_builder if cors_builder.changed?
  end
  deep_dup_and_freeze @gapi["cors"]
end

#cors=(new_cors) ⇒ Object

Updates the CORS configuration for a static website served from the bucket.

Accepts an array of hashes containing the attributes specified for the [resource description of cors](cloud.google.com/storage/docs/json_api/v1/buckets#cors).

See Also:



149
150
151
 
# File 'lib/gcloud/storage/bucket.rb', line 149

def cors= new_cors
  patch_gapi! cors: new_cors
end

#create_file(file, path = nil, acl: nil, cache_control: nil, content_disposition: nil, content_encoding: nil, content_language: nil, content_type: nil, chunk_size: nil, crc32c: nil, md5: nil, metadata: nil, encryption_key: nil, encryption_key_sha256: nil) ⇒ Gcloud::Storage::File Also known as: upload_file, new_file

Creates a new File object by providing a path to a local file to upload and the path to store it with in the bucket.

A chunk_size value can be provided in the options to be used in resumable uploads. This value is the number of bytes per chunk and must be divisible by 256KB. If it is not divisible by 256KB then it will be lowered to the nearest acceptable value.

#### Customer-supplied encryption keys

By default, Google Cloud Storage manages server-side encryption keys on your behalf. However, a [customer-supplied encryption key](cloud.google.com/storage/docs/encryption#customer-supplied) can be provided with the encryption_key and encryption_key_sha256 options. If given, the same key and SHA256 hash also must be provided to subsequently download or copy the file. If you use customer-supplied encryption keys, you must securely manage your keys and ensure that they are not lost. Also, please note that file metadata is not encrypted, with the exception of the CRC32C checksum and MD5 hash. The names of files and buckets are also not encrypted, and you can read or update the metadata of an encrypted file without providing the encryption key.

#### Troubleshooting large uploads

You may encounter errors while attempting to upload large files. Below are a couple of common cases and their solutions.

##### Handling memory errors

If you encounter a memory error such as NoMemoryError, try performing a resumable upload and setting the chunk_size option to a value that works for your environment, as explained in the final example above.

##### Handling broken pipe errors

To avoid broken pipe (Errno::EPIPE) errors when uploading, add the [httpclient](rubygems.org/gems/httpclient) gem to your project, and the configuration shown below. These lines must execute after you require gcloud but before you make your first gcloud connection. The first statement configures [Faraday](rubygems.org/gems/faraday) to use httpclient. The second statement, which should only be added if you are using a version of Faraday at or above 0.9.2, is a workaround for [this gzip issue](github.com/GoogleCloudPlatform/gcloud-ruby/issues/367).

Examples:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

bucket.create_file "path/to/local.file.ext"

Specifying a destination path:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

bucket.create_file "path/to/local.file.ext",
                   "destination/path/file.ext"

Specifying the chunk size as a number of bytes:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

bucket.create_file "path/to/local.file.ext",
                   "destination/path/file.ext",
                   chunk_size: 1024*1024 # 1 MB chunk

Providing a customer-supplied encryption key:

require "gcloud"
require "digest/sha2"

gcloud = Gcloud.new
storage = gcloud.storage
bucket = storage.bucket "my-bucket"

# Key generation shown for example purposes only. Write your own.
cipher = OpenSSL::Cipher.new "aes-256-cfb"
cipher.encrypt
key = cipher.random_key
key_hash = Digest::SHA256.digest key

bucket.create_file "path/to/local.file.ext",
                   "destination/path/file.ext",
                   encryption_key: key,
                   encryption_key_sha256: key_hash

# Store your key and hash securely for later use.
file = bucket.file "destination/path/file.ext",
                   encryption_key: key,
                   encryption_key_sha256: key_hash

Avoiding broken pipe errors with large uploads:

require "gcloud"

# Use httpclient to avoid broken pipe errors with large uploads
Faraday.default_adapter = :httpclient

# Only add the following statement if using Faraday >= 0.9.2
# Override gzip middleware with no-op for httpclient
Faraday::Response.register_middleware :gzip =>
                                        Faraday::Response::Middleware

gcloud = Gcloud.new
storage = gcloud.storage

Parameters:

  • file (String) —

    Path of the file on the filesystem to upload.

  • path (String) (defaults to: nil) —

    Path to store the file in Google Cloud Storage.

  • acl (String) (defaults to: nil) —

    A predefined set of access controls to apply to this file.

    Acceptable values are:

    • auth, auth_read, authenticated, authenticated_read, authenticatedRead - File owner gets OWNER access, and allAuthenticatedUsers get READER access.

    • owner_full, bucketOwnerFullControl - File owner gets OWNER access, and project team owners get OWNER access.

    • owner_read, bucketOwnerRead - File owner gets OWNER access, and project team owners get READER access.

    • private - File owner gets OWNER access.

    • project_private, projectPrivate - File owner gets OWNER access, and project team members get access according to their roles.

    • public, public_read, publicRead - File owner gets OWNER access, and allUsers get READER access.

  • cache_control (String) (defaults to: nil) —

    The [Cache-Control](tools.ietf.org/html/rfc7234#section-5.2) response header to be returned when the file is downloaded.

  • content_disposition (String) (defaults to: nil) —

    The [Content-Disposition](tools.ietf.org/html/rfc6266) response header to be returned when the file is downloaded.

  • content_encoding (String) (defaults to: nil) —

    The [Content-Encoding ](tools.ietf.org/html/rfc7231#section-3.1.2.2) response header to be returned when the file is downloaded.

  • content_language (String) (defaults to: nil) —

    The [Content-Language](tools.ietf.org/html/bcp47) response header to be returned when the file is downloaded.

  • content_type (String) (defaults to: nil) —

    The [Content-Type](tools.ietf.org/html/rfc2616#section-14.17) response header to be returned when the file is downloaded.

  • chunk_size (Integer) (defaults to: nil) —

    The number of bytes per chunk in a resumable upload. Must be divisible by 256KB. If it is not divisible by 265KB then it will be lowered to the nearest acceptable value. If no value is provided it will use Upload.default_chunk_size. Optional.

  • crc32c (String) (defaults to: nil) —

    The CRC32c checksum of the file data, as described in [RFC 4960, Appendix B](tools.ietf.org/html/rfc4960#appendix-B). If provided, Cloud Storage will only create the file if the value matches the value calculated by the service. See [Validation](cloud.google.com/storage/docs/hashes-etags) for more information.

  • md5 (String) (defaults to: nil) —

    The MD5 hash of the file data. If provided, Cloud Storage will only create the file if the value matches the value calculated by the service. See [Validation](cloud.google.com/storage/docs/hashes-etags) for more information.

  • metadata (Hash) (defaults to: nil) —

    A hash of custom, user-provided web-safe keys and arbitrary string values that will returned with requests for the file as “x-goog-meta-” response headers.

  • encryption_key (String) (defaults to: nil) —

    Optional. A customer-supplied, AES-256 encryption key that will be used to encrypt the file. Must be provided if encryption_key_sha256 is provided.

  • encryption_key_sha256 (String) (defaults to: nil) —

    Optional. The SHA256 hash of the customer-supplied, AES-256 encryption key that will be used to encrypt the file. Must be provided if encryption_key is provided.

Returns:



644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
 
# File 'lib/gcloud/storage/bucket.rb', line 644

def create_file file, path = nil, acl: nil, cache_control: nil,
                content_disposition: nil, content_encoding: nil,
                content_language: nil, content_type: nil, chunk_size: nil,
                crc32c: nil, md5: nil, metadata: nil, encryption_key: nil,
                encryption_key_sha256: nil
  ensure_connection!
  options = { acl: File::Acl.predefined_rule_for(acl), md5: md5,
              cache_control: cache_control, content_type: content_type,
              content_disposition: content_disposition, crc32c: crc32c,
              content_encoding: content_encoding, chunk_size: chunk_size,
              content_language: content_language, metadata: metadata,
              encryption_key: encryption_key,
              encryption_key_sha256: encryption_key_sha256 }
  ensure_file_exists! file
  resumable = resumable_upload?(file)
  resp = @connection.upload_file resumable, name, file, path, options

  return File.from_gapi(resp.data, connection) if resp.success?
  fail ApiError.from_response(resp)
end

#created_at ⇒ Object

Creation time of the bucket.



81
82
83
 
# File 'lib/gcloud/storage/bucket.rb', line 81

def created_at
  @gapi["timeCreated"]
end

#default_acl ⇒ Object

The Bucket::DefaultAcl instance used to control access to the bucket’s files.

A bucket’s files have owners, writers, and readers. Permissions can be granted to an individual user’s email address, a group’s email address, as well as many predefined lists.

Examples:

Grant access to a user by pre-pending ‘“user-”` to an email:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.default_acl.add_reader "user-#{email}"

Grant access to a group by pre-pending ‘“group-”` to an email

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.default_acl.add_reader "group-#{email}"

Or, grant access via a predefined permissions list:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

bucket.default_acl.public!

See Also:



756
757
758
 
# File 'lib/gcloud/storage/bucket.rb', line 756

def default_acl
  @default_acl ||= Bucket::DefaultAcl.new self
end

#delete ⇒ Boolean

Permanently deletes the bucket. The bucket must be empty before it can be deleted.

The API call to delete the bucket may be retried under certain conditions. See Backoff to control this behavior.

Examples:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
bucket.delete

Returns:

  • (Boolean) —

    Returns true if the bucket was deleted.



342
343
344
345
346
347
348
349
350
 
# File 'lib/gcloud/storage/bucket.rb', line 342

def delete
  ensure_connection!
  resp = connection.delete_bucket name
  if resp.success?
    true
  else
    fail ApiError.from_response(resp)
  end
end

#file(path, generation: nil, encryption_key: nil, encryption_key_sha256: nil) ⇒ Gcloud::Storage::File? Also known as: find_file

Retrieves a file matching the path.

If a [customer-supplied encryption key](cloud.google.com/storage/docs/encryption#customer-supplied) was used with #create_file, the encryption_key and encryption_key_sha256 options must be provided or else the file’s CRC32C checksum and MD5 hash will not be returned.

Examples:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

file = bucket.file "path/to/my-file.ext"
puts file.name

Parameters:

  • path (String) —

    Name (path) of the file.

  • generation (Integer) (defaults to: nil) —

    When present, selects a specific revision of this object. Default is the latest version.

  • encryption_key (String) (defaults to: nil) —

    Optional. The customer-supplied, AES-256 encryption key used to encrypt the file, if one was provided to #create_file. Must be provided if encryption_key_sha256 is provided.

  • encryption_key_sha256 (String) (defaults to: nil) —

    Optional. The SHA256 hash of the customer-supplied, AES-256 encryption key used to encrypt the file, if one was provided to #create_file. Must be provided if encryption_key is provided.

Returns:



450
451
452
453
454
455
456
457
458
459
460
461
 
# File 'lib/gcloud/storage/bucket.rb', line 450

def file path, generation: nil, encryption_key: nil,
         encryption_key_sha256: nil
  ensure_connection!
  options = { generation: generation, encryption_key: encryption_key,
              encryption_key_sha256: encryption_key_sha256 }
  resp = connection.get_file name, path, options
  if resp.success?
    File.from_gapi resp.data, connection
  else
    fail ApiError.from_response(resp)
  end
end

#files(prefix: nil, delimiter: nil, token: nil, max: nil, versions: nil) ⇒ Array<Gcloud::Storage::File> Also known as: find_files

Retrieves a list of files matching the criteria.

Examples:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
files = bucket.files
files.each do |file|
  puts file.name
end

Retrieve all files: (See File::List#all)

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
files = bucket.files
files.all do |file|
  puts file.name
end

Parameters:

  • prefix (String) (defaults to: nil) —

    Filter results to files whose names begin with this prefix.

  • delimiter (String) (defaults to: nil) —

    Returns results in a directory-like mode. items will contain only objects whose names, aside from the prefix, do not contain delimiter. Objects whose names, aside from the prefix, contain delimiter will have their name, truncated after the delimiter, returned in prefixes. Duplicate prefixes are omitted.

  • token (String) (defaults to: nil) —

    A previously-returned page token representing part of the larger set of results to view.

  • max (Integer) (defaults to: nil) —

    Maximum number of items plus prefixes to return. As duplicate prefixes are omitted, fewer total results may be returned than requested. The default value of this parameter is 1,000 items.

  • versions (Boolean) (defaults to: nil) —

    If true, lists all versions of an object as distinct results. The default is false. For more information, see [Object Versioning ](cloud.google.com/storage/docs/object-versioning).

Returns:



400
401
402
403
404
405
406
407
408
409
410
411
412
413
 
# File 'lib/gcloud/storage/bucket.rb', line 400

def files prefix: nil, delimiter: nil, token: nil, max: nil, versions: nil
  ensure_connection!
  options = {
    prefix:    prefix,
    delimiter: delimiter,
    token:     token,
    max:       max,
    versions:  versions
  }
  resp = connection.list_files name, options
  fail ApiError.from_response(resp) unless resp.success?
  File::List.from_response resp, connection, name, prefix, delimiter, max,
                           versions
end

#id ⇒ Object

The ID of the bucket.



63
64
65
 
# File 'lib/gcloud/storage/bucket.rb', line 63

def id
  @gapi["id"]
end

#kind ⇒ Object

The kind of item this is. For buckets, this is always ‘storage#bucket`.



57
58
59
 
# File 'lib/gcloud/storage/bucket.rb', line 57

def kind
  @gapi["kind"]
end

#location ⇒ Object

The location of the bucket. Object data for objects in the bucket resides in physical storage within this region. Defaults to US. See the developer’s guide for the authoritative list.

See Also:



160
161
162
 
# File 'lib/gcloud/storage/bucket.rb', line 160

def location
  @gapi["location"]
end

#logging_bucket ⇒ Object

The destination bucket name for the bucket’s logs.

See Also:



169
170
171
 
# File 'lib/gcloud/storage/bucket.rb', line 169

def logging_bucket
  @gapi["logging"]["logBucket"] if @gapi["logging"]
end

#logging_bucket=(logging_bucket) ⇒ Object

Updates the destination bucket for the bucket’s logs.

Parameters:

  • logging_bucket (String) —

    The bucket to hold the logging output

See Also:



180
181
182
 
# File 'lib/gcloud/storage/bucket.rb', line 180

def logging_bucket= logging_bucket
  patch_gapi! logging_bucket: logging_bucket
end

#logging_prefix ⇒ Object

The logging object prefix for the bucket’s logs. For more information,

See Also:



189
190
191
 
# File 'lib/gcloud/storage/bucket.rb', line 189

def logging_prefix
  @gapi["logging"]["logObjectPrefix"] if @gapi["logging"]
end

#logging_prefix=(logging_prefix) ⇒ Object

Updates the logging object prefix. This prefix will be used to create log object names for the bucket. It can be at most 900 characters and must be a [valid object name](cloud.google.com/storage/docs/bucket-naming#objectnames). By default, the object prefix is the name of the bucket for which the logs are enabled.

See Also:



203
204
205
 
# File 'lib/gcloud/storage/bucket.rb', line 203

def logging_prefix= logging_prefix
  patch_gapi! logging_prefix: logging_prefix
end

#name ⇒ Object

The name of the bucket.



69
70
71
 
# File 'lib/gcloud/storage/bucket.rb', line 69

def name
  @gapi["name"]
end

#reload! ⇒ Object Also known as: refresh!

Reloads the bucket with current data from the Storage service.



762
763
764
765
766
767
768
769
770
 
# File 'lib/gcloud/storage/bucket.rb', line 762

def reload!
  ensure_connection!
  resp = connection.get_bucket name
  if resp.success?
    @gapi = resp.data
  else
    fail ApiError.from_response(resp)
  end
end

#storage_class ⇒ Object

The bucket’s storage class. This defines how objects in the bucket are stored and determines the SLA and the cost of storage. Values include STANDARD, NEARLINE, and DURABLE_REDUCED_AVAILABILITY.



211
212
213
 
# File 'lib/gcloud/storage/bucket.rb', line 211

def storage_class
  @gapi["storageClass"]
end

#update {|bucket| ... } ⇒ Object

Updates the bucket with changes made in the given block in a single PATCH request. The following attributes may be set: #cors=, #logging_bucket=, #logging_prefix=, #versioning=, #website_main=, and #website_404=. In addition, the #cors configuration accessible in the block is completely mutable and will be included in the request. (See Cors)

Examples:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
bucket.update do |b|
  b.website_main = "index.html"
  b.website_404 = "not_found.html"
  b.cors[0]["method"] = ["GET","POST","DELETE"]
  b.cors[1]["responseHeader"] << "X-Another-Custom-Header"
end

New CORS rules can also be added in a nested block:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage
bucket = storage.bucket "my-todo-app"

bucket.update do |b|
  b.cors do |c|
    c.add_rule ["http://example.org", "https://example.org"],
               "*",
               response_headers: ["X-My-Custom-Header"],
               max_age: 300
  end
end

Yields:

  • (bucket) —

    a block yielding a delegate object for updating the file



318
319
320
321
322
 
# File 'lib/gcloud/storage/bucket.rb', line 318

def update
  updater = Updater.new @gapi["cors"]
  yield updater
  patch_gapi! updater.updates unless updater.updates.empty?
end

#versioning=(new_versioning) ⇒ Boolean

Updates whether [Object Versioning](cloud.google.com/storage/docs/object-versioning) is enabled for the bucket.

Returns:

  • (Boolean) 


230
231
232
 
# File 'lib/gcloud/storage/bucket.rb', line 230

def versioning= new_versioning
  patch_gapi! versioning: new_versioning
end

#versioning? ⇒ Boolean

Whether [Object Versioning](cloud.google.com/storage/docs/object-versioning) is enabled for the bucket.

Returns:

  • (Boolean) 


219
220
221
 
# File 'lib/gcloud/storage/bucket.rb', line 219

def versioning?
  !@gapi["versioning"].nil? && @gapi["versioning"]["enabled"]
end

#website_404 ⇒ Object

The page returned from a static website served from the bucket when a site visitor requests a resource that does not exist.

See Also:



263
264
265
 
# File 'lib/gcloud/storage/bucket.rb', line 263

def website_404
  @gapi["website"]["notFoundPage"] if @gapi["website"]
end

#website_404=(website_404) ⇒ Object

Updates the page returned from a static website served from the bucket when a site visitor requests a resource that does not exist.

See Also:



274
275
276
 
# File 'lib/gcloud/storage/bucket.rb', line 274

def website_404= website_404
  patch_gapi! website_404: website_404
end

#website_main ⇒ Object

The index page returned from a static website served from the bucket when a site visitor requests the top level directory.

See Also:



241
242
243
 
# File 'lib/gcloud/storage/bucket.rb', line 241

def website_main
  @gapi["website"]["mainPageSuffix"] if @gapi["website"]
end

#website_main=(website_main) ⇒ Object

Updates the index page returned from a static website served from the bucket when a site visitor requests the top level directory.

See Also:



252
253
254
 
# File 'lib/gcloud/storage/bucket.rb', line 252

def website_main= website_main
  patch_gapi! website_main: website_main
end