Class: Gcloud::Storage::Bucket

Inherits:
Object
  • Object
show all
Defined in:
lib/gcloud/storage/bucket.rb,
lib/gcloud/storage/bucket/acl.rb,
lib/gcloud/storage/bucket/cors.rb,
lib/gcloud/storage/bucket/list.rb

Overview

Bucket

Represents a Storage bucket. Belongs to a Project and has many Files.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
file = bucket.file "path/to/my-file.ext"

Defined Under Namespace

Classes: Acl, Cors, DefaultAcl, List, Updater

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Bucket

Create an empty Bucket object.



48
49
50
51
 
# File 'lib/gcloud/storage/bucket.rb', line 48

def initialize #:nodoc:
  @connection = nil
  @gapi = {}
end

Instance Attribute Details

#connection ⇒ Object

The Connection object.



40
41
42
 
# File 'lib/gcloud/storage/bucket.rb', line 40

def connection
  @connection
end

#gapi ⇒ Object

The Google API Client object.



44
45
46
 
# File 'lib/gcloud/storage/bucket.rb', line 44

def gapi
  @gapi
end

Class Method Details

.from_gapi(gapi, conn) ⇒ Object

New Bucket from a Google API Client object.



760
761
762
763
764
765
 
# File 'lib/gcloud/storage/bucket.rb', line 760

def self.from_gapi gapi, conn #:nodoc:
  new.tap do |f|
    f.gapi = gapi
    f.connection = conn
  end
end

Instance Method Details

#acl ⇒ Object

The Bucket::Acl instance used to control access to the bucket.

A bucket has owners, writers, and readers. Permissions can be granted to an individual user’s email address, a group’s email address, as well as many predefined lists. See the Access Control guide for more.

Examples

Access to a bucket can be granted to a user by appending “user-” to the email address:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.acl.add_reader "user-#{email}"

Access to a bucket can be granted to a group by appending “group-” to the email address:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.acl.add_reader "group-#{email}"

Access to a bucket can also be granted to a predefined list of permissions:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

bucket.acl.public!



687
688
689
 
# File 'lib/gcloud/storage/bucket.rb', line 687

def acl
  @acl ||= Bucket::Acl.new self
end

#api_url ⇒ Object

A URL that can be used to access the bucket using the REST API.



74
75
76
 
# File 'lib/gcloud/storage/bucket.rb', line 74

def api_url
  @gapi["selfLink"]
end

#cors ⇒ Object

Returns the current CORS configuration for a static website served from the bucket. For more information, see Cross-Origin Resource Sharing (CORS). The return value is a frozen (unmodifiable) array of hashes containing the attributes specified for the Bucket resource field cors.

This method also accepts a block for updating the bucket’s CORS rules. See Bucket::Cors for details.

Examples

Retrieving the bucket’s CORS rules.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"
bucket.cors #=> [{"origin"=>["http://example.org"],
            #     "method"=>["GET","POST","DELETE"],
            #     "responseHeader"=>["X-My-Custom-Header"],
            #     "maxAgeSeconds"=>3600}]

Updating the bucket’s CORS rules inside a block.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage
bucket = storage.bucket "my-todo-app"

bucket.update do |b|
  b.cors do |c|
    c.add_rule ["http://example.org", "https://example.org"],
               "*",
               response_headers: ["X-My-Custom-Header"],
               max_age: 3600
  end
end



127
128
129
130
131
132
133
134
 
# File 'lib/gcloud/storage/bucket.rb', line 127

def cors
  if block_given?
    cors_builder = Bucket::Cors.new @gapi["cors"]
    yield cors_builder
    self.cors = cors_builder if cors_builder.changed?
  end
  deep_dup_and_freeze @gapi["cors"]
end

#cors=(new_cors) ⇒ Object

Updates the CORS configuration for a static website served from the bucket. For more information, see Cross-Origin Resource Sharing (CORS). Accepts an array of hashes containing the attributes specified for the resource description of cors.



143
144
145
 
# File 'lib/gcloud/storage/bucket.rb', line 143

def cors= new_cors
  patch_gapi! cors: new_cors
end

#create_file(file, path = nil, acl: nil, cache_control: nil, content_disposition: nil, content_encoding: nil, content_language: nil, content_type: nil, chunk_size: nil, crc32c: nil, md5: nil, metadata: nil) ⇒ Object Also known as: upload_file, new_file

Create a new File object by providing a path to a local file to upload and the path to store it with in the bucket.

Parameters

file

Path of the file on the filesystem to upload. (String)

path

Path to store the file in Google Cloud Storage. (String)

acl

A predefined set of access controls to apply to this file. (String)

Acceptable values are:

  • auth, auth_read, authenticated, authenticated_read, authenticatedRead - File owner gets OWNER access, and allAuthenticatedUsers get READER access.

  • owner_full, bucketOwnerFullControl - File owner gets OWNER access, and project team owners get OWNER access.

  • owner_read, bucketOwnerRead - File owner gets OWNER access, and project team owners get READER access.

  • private - File owner gets OWNER access.

  • project_private, projectPrivate - File owner gets OWNER access, and project team members get access according to their roles.

  • public, public_read, publicRead - File owner gets OWNER access, and allUsers get READER access.

cache_control

The Cache-Control response header to be returned when the file is downloaded. (String)

content_disposition

The Content-Disposition response header to be returned when the file is downloaded. (String)

content_encoding

The Content-Encoding response header to be returned when the file is downloaded. (String)

content_language

The Content-Language response header to be returned when the file is downloaded. (String)

content_type

The Content-Type response header to be returned when the file is downloaded. (String)

chunk_size

The number of bytes per chunk in a resumable upload. Must be divisible by 256KB. If it is not divisible by 265KB then it will be lowered to the nearest acceptable value. (Integer)

crc32c

The CRC32c checksum of the file data, as described in RFC 4960, Appendix B. If provided, Cloud Storage will only create the file if the value matches the value calculated by the service. See Validation for more information. (String)

md5

The MD5 hash of the file data. If provided, Cloud Storage will only create the file if the value matches the value calculated by the service. See Validation for more information. (String)

metadata

A hash of custom, user-provided web-safe keys and arbitrary string values that will returned with requests for the file as “x-goog-meta-” response headers. (Hash)

Returns

Gcloud::Storage::File

Examples

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

bucket.create_file "path/to/local.file.ext"

Additionally, a destination path can be specified.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

bucket.create_file "path/to/local.file.ext",
                   "destination/path/file.ext"

A chunk_size value can be provided in the options to be used in resumable uploads. This value is the number of bytes per chunk and must be divisible by 256KB. If it is not divisible by 265KB then it will be lowered to the nearest acceptable value.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

bucket.create_file "path/to/local.file.ext",
                   "destination/path/file.ext",
                   chunk_size: 1024*1024 # 1 MB chunk

Troubleshooting large uploads

You may encounter errors while attempting to upload large files. Below are a couple of common cases and their solutions.

Handling memory errors

If you encounter a memory error such as NoMemoryError, try performing a resumable upload and setting the chunk_size option to a value that works for your environment, as explained in the final example above.

Handling broken pipe errors

To avoid broken pipe (Errno::EPIPE) errors when uploading, add the httpclient gem to your project, and the configuration shown below. These lines must execute after you require gcloud but before you make your first gcloud connection. The first statement configures Faraday to use httpclient. The second statement, which should only be added if you are using a version of Faraday at or above 0.9.2, is a workaround for this gzip issue.

require "gcloud"

# Use httpclient to avoid broken pipe errors with large uploads
Faraday.default_adapter = :httpclient

# Only add the following statement if using Faraday >= 0.9.2
# Override gzip middleware with no-op for httpclient
Faraday::Response.register_middleware :gzip =>
                                        Faraday::Response::Middleware

gcloud = Gcloud.new
storage = gcloud.storage



617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
 
# File 'lib/gcloud/storage/bucket.rb', line 617

def create_file file, path = nil, acl: nil, cache_control: nil,
                content_disposition: nil, content_encoding: nil,
                content_language: nil, content_type: nil, chunk_size: nil,
                crc32c: nil, md5: nil, metadata: nil
  ensure_connection!
  options = { acl: File::Acl.predefined_rule_for(acl), md5: md5,
              cache_control: cache_control, content_type: content_type,
              content_disposition: content_disposition, crc32c: crc32c,
              content_encoding: content_encoding, chunk_size: chunk_size,
              content_language: content_language, metadata: metadata }
  ensure_file_exists! file
  resumable = resumable_upload?(file)
  resp = @connection.upload_file resumable, name, file, path, options

  return File.from_gapi(resp.data, connection) if resp.success?
  fail ApiError.from_response(resp)
end

#created_at ⇒ Object

Creation time of the bucket.



80
81
82
 
# File 'lib/gcloud/storage/bucket.rb', line 80

def created_at
  @gapi["timeCreated"]
end

#default_acl ⇒ Object

The Bucket::DefaultAcl instance used to control access to the bucket’s files.

A bucket’s files have owners, writers, and readers. Permissions can be granted to an individual user’s email address, a group’s email address, as well as many predefined lists. See the Access Control guide for more.

Examples

Access to a bucket’s files can be granted to a user by appending “user-” to the email address:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.default_acl.add_reader "user-#{email}"

Access to a bucket’s files can be granted to a group by appending “group-” to the email address:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

email = "[email protected]"
bucket.default_acl.add_reader "group-#{email}"

Access to a bucket’s files can also be granted to a predefined list of permissions:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-todo-app"

bucket.default_acl.public!



741
742
743
 
# File 'lib/gcloud/storage/bucket.rb', line 741

def default_acl
  @default_acl ||= Bucket::DefaultAcl.new self
end

#delete(retries: nil) ⇒ Object

Permanently deletes the bucket. The bucket must be empty before it can be deleted.

Parameters

retries

The number of times the API call should be retried. Default is Gcloud::Backoff.retries. (Integer)

Returns

true if the bucket was deleted.

Examples

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
bucket.delete

The API call to delete the bucket may be retried under certain conditions. See Gcloud::Backoff to control this behavior, or specify the wanted behavior in the call:

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
bucket.delete retries: 5



337
338
339
340
341
342
343
344
345
346
 
# File 'lib/gcloud/storage/bucket.rb', line 337

def delete retries: nil
  ensure_connection!
  options = { retries: retries }
  resp = connection.delete_bucket name, options
  if resp.success?
    true
  else
    fail ApiError.from_response(resp)
  end
end

#file(path, generation: nil) ⇒ Object Also known as: find_file

Retrieves a file matching the path.

Parameters

path

Name (path) of the file. (String)

generation

When present, selects a specific revision of this object. Default is the latest version. (Integer)

Returns

Gcloud::Storage::File or nil if file does not exist

Example

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

file = bucket.file "path/to/my-file.ext"
puts file.name



460
461
462
463
464
465
466
467
468
469
 
# File 'lib/gcloud/storage/bucket.rb', line 460

def file path, generation: nil
  ensure_connection!
  options = { generation: generation }
  resp = connection.get_file name, path, options
  if resp.success?
    File.from_gapi resp.data, connection
  else
    fail ApiError.from_response(resp)
  end
end

#files(prefix: nil, delimiter: nil, token: nil, max: nil, versions: nil) ⇒ Object Also known as: find_files

Retrieves a list of files matching the criteria.

Parameters

prefix

Filter results to files whose names begin with this prefix. (String)

delimiter

Returns results in a directory-like mode. items will contain only objects whose names, aside from the prefix, do not contain delimiter. Objects whose names, aside from the prefix, contain delimiter will have their name, truncated after the delimiter, returned in prefixes. Duplicate prefixes are omitted.

token

A previously-returned page token representing part of the larger set of results to view. (String)

max

Maximum number of items plus prefixes to return. As duplicate prefixes are omitted, fewer total results may be returned than requested. The default value of this parameter is 1,000 items. (Integer)

versions

If true, lists all versions of an object as distinct results. The default is false. For more information, see Object Versioning . (Boolean)

Returns

Array of Gcloud::Storage::File (See Gcloud::Storage::File::List)

Examples

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
files = bucket.files
files.each do |file|
  puts file.name
end

If you have a significant number of files, you may need to paginate through them: (See File::List#token)

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"

all_files = []
tmp_files = bucket.files
while tmp_files.any? do
  tmp_files.each do |file|
    all_files << file
  end
  # break loop if no more buckets available
  break if tmp_files.token.nil?
  # get the next group of files
  tmp_files = bucket.files token: tmp_files.token
end



415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
 
# File 'lib/gcloud/storage/bucket.rb', line 415

def files prefix: nil, delimiter: nil, token: nil, max: nil, versions: nil
  ensure_connection!
  options = {
    prefix:    prefix,
    delimiter: delimiter,
    token:     token,
    max:       max,
    versions:  versions
  }
  resp = connection.list_files name, options
  if resp.success?
    File::List.from_response resp, connection
  else
    fail ApiError.from_response(resp)
  end
end

#id ⇒ Object

The ID of the bucket.



62
63
64
 
# File 'lib/gcloud/storage/bucket.rb', line 62

def id
  @gapi["id"]
end

#kind ⇒ Object

The kind of item this is. For buckets, this is always storage#bucket.



56
57
58
 
# File 'lib/gcloud/storage/bucket.rb', line 56

def kind
  @gapi["kind"]
end

#location ⇒ Object

The location of the bucket. Object data for objects in the bucket resides in physical storage within this region. Defaults to US. See the developer’s guide for the authoritative list.

cloud.google.com/storage/docs/concepts-techniques



154
155
156
 
# File 'lib/gcloud/storage/bucket.rb', line 154

def location
  @gapi["location"]
end

#logging_bucket ⇒ Object

The destination bucket name for the bucket’s logs. For more information, see Access Logs.



161
162
163
 
# File 'lib/gcloud/storage/bucket.rb', line 161

def logging_bucket
  @gapi["logging"]["logBucket"] if @gapi["logging"]
end

#logging_bucket=(logging_bucket) ⇒ Object

Updates the destination bucket for the bucket’s logs. For more information, see Access Logs. (String)



169
170
171
 
# File 'lib/gcloud/storage/bucket.rb', line 169

def logging_bucket= logging_bucket
  patch_gapi! logging_bucket: logging_bucket
end

#logging_prefix ⇒ Object

The logging object prefix for the bucket’s logs. For more information, see Access Logs.



176
177
178
 
# File 'lib/gcloud/storage/bucket.rb', line 176

def logging_prefix
  @gapi["logging"]["logObjectPrefix"] if @gapi["logging"]
end

#logging_prefix=(logging_prefix) ⇒ Object

Updates the logging object prefix. This prefix will be used to create log object names for the bucket. It can be at most 900 characters and must be a valid object name. By default, the object prefix is the name of the bucket for which the logs are enabled. For more information, see Access Logs. (String)



189
190
191
 
# File 'lib/gcloud/storage/bucket.rb', line 189

def logging_prefix= logging_prefix
  patch_gapi! logging_prefix: logging_prefix
end

#name ⇒ Object

The name of the bucket.



68
69
70
 
# File 'lib/gcloud/storage/bucket.rb', line 68

def name
  @gapi["name"]
end

#reload! ⇒ Object Also known as: refresh!

Reloads the bucket with current data from the Storage service.



747
748
749
750
751
752
753
754
755
 
# File 'lib/gcloud/storage/bucket.rb', line 747

def reload!
  ensure_connection!
  resp = connection.get_bucket name
  if resp.success?
    @gapi = resp.data
  else
    fail ApiError.from_response(resp)
  end
end

#storage_class ⇒ Object

The bucket’s storage class. This defines how objects in the bucket are stored and determines the SLA and the cost of storage. Values include STANDARD, NEARLINE, and DURABLE_REDUCED_AVAILABILITY.



197
198
199
 
# File 'lib/gcloud/storage/bucket.rb', line 197

def storage_class
  @gapi["storageClass"]
end

#update {|updater| ... } ⇒ Object

Updates the bucket with changes made in the given block in a single PATCH request. The following attributes may be set: #cors=, #logging_bucket=, #logging_prefix=, #versioning=, #website_main=, and #website_404=. In addition, the #cors configuration accessible in the block is completely mutable and will be included in the request.

Examples

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage

bucket = storage.bucket "my-bucket"
bucket.update do |b|
  b.website_main = "index.html"
  b.website_404 = "not_found.html"
  b.cors[0]["method"] = ["GET","POST","DELETE"]
  b.cors[1]["responseHeader"] << "X-Another-Custom-Header"
end

New CORS rules can also be added in a nested block. See Bucket::Cors for details.

require "gcloud"

gcloud = Gcloud.new
storage = gcloud.storage
bucket = storage.bucket "my-todo-app"

bucket.update do |b|
  b.cors do |c|
    c.add_rule ["http://example.org", "https://example.org"],
               "*",
               response_headers: ["X-My-Custom-Header"],
               max_age: 300
  end
end

Yields:

  • (updater) 


295
296
297
298
299
 
# File 'lib/gcloud/storage/bucket.rb', line 295

def update
  updater = Updater.new @gapi["cors"]
  yield updater
  patch_gapi! updater.updates unless updater.updates.empty?
end

#versioning=(new_versioning) ⇒ Object

Updates whether Object Versioning is enabled for the bucket. (Boolean)



213
214
215
 
# File 'lib/gcloud/storage/bucket.rb', line 213

def versioning= new_versioning
  patch_gapi! versioning: new_versioning
end

#versioning? ⇒ Boolean

Whether Object Versioning is enabled for the bucket.

Returns:

  • (Boolean) 


205
206
207
 
# File 'lib/gcloud/storage/bucket.rb', line 205

def versioning?
  !@gapi["versioning"].nil? && @gapi["versioning"]["enabled"]
end

#website_404 ⇒ Object

The page returned from a static website served from the bucket when a site visitor requests a resource that does not exist. For more information, see How to Host a Static Website .



241
242
243
 
# File 'lib/gcloud/storage/bucket.rb', line 241

def website_404
  @gapi["website"]["notFoundPage"] if @gapi["website"]
end

#website_404=(website_404) ⇒ Object

Updates the page returned from a static website served from the bucket when a site visitor requests a resource that does not exist. For more information, see How to Host a Static Website . (String)



251
252
253
 
# File 'lib/gcloud/storage/bucket.rb', line 251

def website_404= website_404
  patch_gapi! website_404: website_404
end

#website_main ⇒ Object

The index page returned from a static website served from the bucket when a site visitor requests the top level directory. For more information, see How to Host a Static Website .



222
223
224
 
# File 'lib/gcloud/storage/bucket.rb', line 222

def website_main
  @gapi["website"]["mainPageSuffix"] if @gapi["website"]
end

#website_main=(website_main) ⇒ Object

Updates the index page returned from a static website served from the bucket when a site visitor requests the top level directory. For more information, see How to Host a Static Website . (String)



232
233
234
 
# File 'lib/gcloud/storage/bucket.rb', line 232

def website_main= website_main
  patch_gapi! website_main: website_main
end