Class: Fog::Storage::OpenStack::Real

Inherits:
Object
  • Object
show all
Includes:
OpenStack::Core
Defined in:
lib/fog/storage/openstack.rb,
lib/fog/storage/openstack/requests/get_object.rb,
lib/fog/storage/openstack/requests/put_object.rb,
lib/fog/storage/openstack/requests/public_url.rb,
lib/fog/storage/openstack/requests/copy_object.rb,
lib/fog/storage/openstack/requests/head_object.rb,
lib/fog/storage/openstack/requests/get_container.rb,
lib/fog/storage/openstack/requests/delete_object.rb,
lib/fog/storage/openstack/requests/put_container.rb,
lib/fog/storage/openstack/requests/get_containers.rb,
lib/fog/storage/openstack/requests/head_container.rb,
lib/fog/storage/openstack/requests/head_containers.rb,
lib/fog/storage/openstack/requests/delete_container.rb,
lib/fog/storage/openstack/requests/put_object_manifest.rb,
lib/fog/storage/openstack/requests/get_object_http_url.rb,
lib/fog/storage/openstack/requests/get_object_https_url.rb,
lib/fog/storage/openstack/requests/put_static_obj_manifest.rb,
lib/fog/storage/openstack/requests/delete_multiple_objects.rb,
lib/fog/storage/openstack/requests/put_dynamic_obj_manifest.rb,
lib/fog/storage/openstack/requests/delete_static_large_object.rb,
lib/fog/storage/openstack/requests/post_set_meta_temp_url_key.rb

Instance Attribute Summary

Attributes included from OpenStack::Core

#auth_token, #auth_token_expiration, #current_tenant, #current_user, #current_user_id, #openstack_cache_ttl, #openstack_domain_id, #openstack_domain_name, #openstack_identity_prefix, #openstack_project_domain, #openstack_project_domain_id, #openstack_project_id, #openstack_user_domain, #openstack_user_domain_id, #unscoped_token

Class Method Summary collapse

Instance Method Summary collapse

Methods included from OpenStack::Core

#credentials, #initialize_identity, #reload

Constructor Details

#initialize(options = {}) ⇒ Real


89
90
91
92
93
94
95
96
97
98
99
100
# File 'lib/fog/storage/openstack.rb', line 89

def initialize(options = {})
  initialize_identity options

  @openstack_service_type           = options[:openstack_service_type] || ['object-store']
  @openstack_service_name           = options[:openstack_service_name]

  @connection_options               = options[:connection_options] || {}

  authenticate
  @persistent = options[:persistent] || false
  @connection = Fog::Core::Connection.new("#{@scheme}://#{@host}:#{@port}", @persistent, @connection_options)
end

Class Method Details

.not_found_classObject


85
86
87
# File 'lib/fog/storage/openstack.rb', line 85

def self.not_found_class
  Fog::Storage::OpenStack::NotFound
end

Instance Method Details

#change_account(account) ⇒ Object

Change the current account while re-using the auth token.

This is usefull when you have an admin role and you're able to HEAD other user accounts, set quotas, list files, etc.

For example:

# List current user account details
service = Fog::Storage[:openstack]
service.request :method => 'HEAD'

Would return something like:

:                      AUTH_1234
Date:                         Tue, 05 Mar 2013 16:50:52 GMT
X--Bytes-Used:         0 (0.00 Bytes)
X--Container-Count:    0
X--Object-Count:       0

Now let's change the account

service.('AUTH_3333')
service.request :method => 'HEAD'

Would return something like:

:                      AUTH_3333
Date:                         Tue, 05 Mar 2013 16:51:53 GMT
X--Bytes-Used:         23423433
X--Container-Count:    2
X--Object-Count:       10

If we wan't to go back to our original admin account:

service.

138
139
140
141
142
# File 'lib/fog/storage/openstack.rb', line 138

def ()
  @original_path ||= @path
  version_string = @path.split('/')[1]
  @path = "/#{version_string}/#{account}"
end

#copy_object(source_container_name, source_object_name, target_container_name, target_object_name, options = {}) ⇒ Object

Copy object

Parameters

  • source_container_name<~String> - Name of source bucket

  • source_object_name<~String> - Name of source object

  • target_container_name<~String> - Name of bucket to create copy in

  • target_object_name<~String> - Name for new copy of object

  • options<~Hash> - Additional headers


13
14
15
16
17
18
19
# File 'lib/fog/storage/openstack/requests/copy_object.rb', line 13

def copy_object(source_container_name, source_object_name, target_container_name, target_object_name, options = {})
  headers = {'X-Copy-From' => "/#{source_container_name}/#{source_object_name}"}.merge(options)
  request(:expects => 201,
          :headers => headers,
          :method  => 'PUT',
          :path    => "#{Fog::OpenStack.escape(target_container_name)}/#{Fog::OpenStack.escape(target_object_name)}")
end

#create_temp_url(container, object, expires, method, options = {}) ⇒ Object

creates a temporary url

Parameters

  • container<~String> - Name of container containing object

  • object<~String> - Name of object to get expiring url for

  • expires<~Time> - An expiry time for this url

  • method<~String> - The method to use for accessing the object (GET, PUT, HEAD)

  • options<~Hash> - An optional options hash

    • 'scheme'<~String> - The scheme to use (http, https)

    • 'host'<~String> - The host to use

    • 'port'<~Integer> - The port to use

Returns

  • response<~Excon::Response>:

    • body<~String> - url for object

See Also

docs.rackspace.com/files/api/v1/cf-devguide/content/Create_TempURL-d1a444.html

Raises:

  • (ArgumentError)

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
# File 'lib/fog/storage/openstack/requests/get_object_https_url.rb', line 37

def create_temp_url(container, object, expires, method, options = {})
  raise ArgumentError, "Insufficient parameters specified." unless container && object && expires && method
  raise ArgumentError, "Storage must be instantiated with the :openstack_temp_url_key option" if @openstack_temp_url_key.nil?

  scheme = options[:scheme] || @scheme
  host = options[:host] || @host
  port = options[:port] || @port

  # POST not allowed
  allowed_methods = %w(GET PUT HEAD)
  unless allowed_methods.include?(method)
    raise ArgumentError, "Invalid method '#{method}' specified. Valid methods are: #{allowed_methods.join(', ')}"
  end

  expires = expires.to_i
  object_path_escaped   = "#{@path}/#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object, "/")}"
  object_path_unescaped = "#{@path}/#{Fog::OpenStack.escape(container)}/#{object}"
  string_to_sign = "#{method}\n#{expires}\n#{object_path_unescaped}"

  hmac = Fog::HMAC.new('sha1', @openstack_temp_url_key)
  sig  = sig_to_hex(hmac.sign(string_to_sign))

  temp_url_options = {
    :scheme => scheme,
    :host   => host,
    :port   => port,
    :path   => object_path_escaped,
    :query  => "temp_url_sig=#{sig}&temp_url_expires=#{expires}"
  }
  URI::Generic.build(temp_url_options).to_s
end

#delete_container(name) ⇒ Object

Delete an existing container

Parameters

  • name<~String> - Name of container to delete


10
11
12
13
14
15
16
# File 'lib/fog/storage/openstack/requests/delete_container.rb', line 10

def delete_container(name)
  request(
    :expects => 204,
    :method  => 'DELETE',
    :path    => Fog::OpenStack.escape(name)
  )
end

#delete_multiple_objects(container, object_names, options = {}) ⇒ Excon::Response

Deletes multiple objects or containers with a single request.

To delete objects from a single container, container may be provided and object_names should be an Array of object names within the container.

To delete objects from multiple containers or delete containers, container should be nil and all object_names should be prefixed with a container name.

Containers must be empty when deleted. object_names are processed in the order given, so objects within a container should be listed first to empty the container.

Up to 10,000 objects may be deleted in a single request. The server will respond with 200 OK for all requests. response.body must be inspected for actual results.

Examples:

Delete objects from a container

object_names = ['object', 'another/object']
conn.delete_multiple_objects('my_container', object_names)

Delete objects from multiple containers

object_names = ['container_a/object', 'container_b/object']
conn.delete_multiple_objects(nil, object_names)

Delete a container and all it's objects

object_names = ['my_container/object_a', 'my_container/object_b', 'my_container']
conn.delete_multiple_objects(nil, object_names)

45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
# File 'lib/fog/storage/openstack/requests/delete_multiple_objects.rb', line 45

def delete_multiple_objects(container, object_names, options = {})
  body = object_names.map do |name|
    object_name = container ? "#{container}/#{name}" : name
    URI.encode(object_name)
  end.join("\n")

  response = request({
                       :expects => 200,
                       :method  => 'DELETE',
                       :headers => options.merge('Content-Type' => 'text/plain',
                                                 'Accept'       => 'application/json'),
                       :body    => body,
                       :query   => {'bulk-delete' => true}
                     }, false)
  response.body = Fog::JSON.decode(response.body)
  response
end

#delete_object(container, object) ⇒ Object

Delete an existing object

Parameters

  • container<~String> - Name of container to delete

  • object<~String> - Name of object to delete


11
12
13
14
15
16
17
# File 'lib/fog/storage/openstack/requests/delete_object.rb', line 11

def delete_object(container, object)
  request(
    :expects => 204,
    :method  => 'DELETE',
    :path    => "#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object)}"
  )
end

#delete_static_large_object(container, object, options = {}) ⇒ Excon::Response

Delete a static large object.

Deletes the SLO manifest object and all segments that it references. The server will respond with 200 OK for all requests. response.body must be inspected for actual results.


26
27
28
29
30
31
32
33
34
35
36
37
# File 'lib/fog/storage/openstack/requests/delete_static_large_object.rb', line 26

def delete_static_large_object(container, object, options = {})
  response = request({
                       :expects => 200,
                       :method  => 'DELETE',
                       :headers => options.merge('Content-Type' => 'text/plain',
                                                 'Accept'       => 'application/json'),
                       :path    => "#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object)}",
                       :query   => {'multipart-manifest' => 'delete'}
                     }, false)
  response.body = Fog::JSON.decode(response.body)
  response
end

#get_container(container, options = {}) ⇒ Object

Get details for container and total bytes stored

Parameters

  • container<~String> - Name of container to retrieve info for

  • options<~String>:

    • 'limit'<~String> - Maximum number of objects to return

    • 'marker'<~String> - Only return objects whose name is greater than marker

    • 'prefix'<~String> - Limits results to those starting with prefix

    • 'path'<~String> - Return objects nested in the pseudo path

Returns

  • response<~Excon::Response>:

    • headers<~Hash>:

      • 'X-Account-Container-Count'<~String> - Count of containers

      • 'X-Account-Bytes-Used'<~String> - Bytes used

    • body<~Array>:

      • 'bytes'<~Integer> - Number of bytes used by container

      • 'count'<~Integer> - Number of items in container

      • 'name'<~String> - Name of container

      • item<~Hash>:

        • 'bytes'<~String> - Size of object

        • 'content_type'<~String> Content-Type of object

        • 'hash'<~String> - Hash of object (etag?)

        • 'last_modified'<~String> - Last modified timestamp

        • 'name'<~String> - Name of object


30
31
32
33
34
35
36
37
38
# File 'lib/fog/storage/openstack/requests/get_container.rb', line 30

def get_container(container, options = {})
  options = options.reject { |_key, value| value.nil? }
  request(
    :expects => 200,
    :method  => 'GET',
    :path    => Fog::OpenStack.escape(container),
    :query   => {'format' => 'json'}.merge!(options)
  )
end

#get_containers(options = {}) ⇒ Object

List existing storage containers

Parameters

  • options<~Hash>:

    • 'limit'<~Integer> - Upper limit to number of results returned

    • 'marker'<~String> - Only return objects with name greater than this value

Returns

  • response<~Excon::Response>:

    • body<~Array>:

      • container<~Hash>:

        • 'bytes'<~Integer>: - Number of bytes used by container

        • 'count'<~Integer>: - Number of items in container

        • 'name'<~String>: - Name of container


19
20
21
22
23
24
25
26
27
# File 'lib/fog/storage/openstack/requests/get_containers.rb', line 19

def get_containers(options = {})
  options = options.reject { |_key, value| value.nil? }
  request(
    :expects => [200, 204],
    :method  => 'GET',
    :path    => '',
    :query   => {'format' => 'json'}.merge!(options)
  )
end

#get_object(container, object, &block) ⇒ Object

Get details for object

Parameters

  • container<~String> - Name of container to look in

  • object<~String> - Name of object to look for


11
12
13
14
15
16
17
18
19
20
21
22
23
# File 'lib/fog/storage/openstack/requests/get_object.rb', line 11

def get_object(container, object, &block)
  params = {
    :expects => 200,
    :method  => 'GET',
    :path    => "#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object)}"
  }

  if block_given?
    params[:response_block] = block
  end

  request(params, false)
end

#get_object_http_url(container, object, expires, options = {}) ⇒ Object

Get an expiring object http url

Parameters

  • container<~String> - Name of container containing object

  • object<~String> - Name of object to get expiring url for

  • expires<~Time> - An expiry time for this url

Returns

  • response<~Excon::Response>:

    • body<~String> - url for object


15
16
17
# File 'lib/fog/storage/openstack/requests/get_object_http_url.rb', line 15

def get_object_http_url(container, object, expires, options = {})
  create_temp_url(container, object, expires, "GET", {:port => 80}.merge(options).merge(:scheme => "http"))
end

#get_object_https_url(container, object, expires, options = {}) ⇒ Object

Get an expiring object https url from Cloud Files

Parameters

  • container<~String> - Name of container containing object

  • object<~String> - Name of object to get expiring url for

  • expires<~Time> - An expiry time for this url

Returns

  • response<~Excon::Response>:

    • body<~String> - url for object


15
16
17
# File 'lib/fog/storage/openstack/requests/get_object_https_url.rb', line 15

def get_object_https_url(container, object, expires, options = {})
  create_temp_url(container, object, expires, "GET", {:port => 443}.merge(options).merge(:scheme => "https"))
end

#head_container(container) ⇒ Object

List number of objects and total bytes stored

Parameters

  • container<~String> - Name of container to retrieve info for

Returns

  • response<~Excon::Response>:

    • headers<~Hash>:

      • 'X-Container-Object-Count'<~String> - Count of containers

      • 'X-Container-Bytes-Used'<~String> - Bytes used


15
16
17
18
19
20
21
22
# File 'lib/fog/storage/openstack/requests/head_container.rb', line 15

def head_container(container)
  request(
    :expects => 204,
    :method  => 'HEAD',
    :path    => Fog::OpenStack.escape(container),
    :query   => {'format' => 'json'}
  )
end

#head_containersObject

List number of containers and total bytes stored

Returns

  • response<~Excon::Response>:

    • headers<~Hash>:

      • 'X-Account-Container-Count'<~String> - Count of containers

      • 'X-Account-Bytes-Used'<~String> - Bytes used


12
13
14
15
16
17
18
19
# File 'lib/fog/storage/openstack/requests/head_containers.rb', line 12

def head_containers
  request(
    :expects => 200..299,
    :method  => 'HEAD',
    :path    => '',
    :query   => {'format' => 'json'}
  )
end

#head_object(container, object) ⇒ Object

Get headers for object

Parameters

  • container<~String> - Name of container to look in

  • object<~String> - Name of object to look for


11
12
13
14
15
16
17
# File 'lib/fog/storage/openstack/requests/head_object.rb', line 11

def head_object(container, object)
  request({
            :expects => 200,
            :method  => 'HEAD',
            :path    => "#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object)}"
          }, false)
end

#post_set_meta_temp_url_key(key) ⇒ Object

Set the account wide Temp URL Key. This is a secret key that's used to generate signed expiring URLs.

Once the key has been set with this request you should create new Storage objects with the :openstack_temp_url_key option then use the get_object_https_url method to generate expiring URLs.

*** CAUTION *** changing this secret key will invalidate any expiring URLS generated with old keys.

Parameters

  • key<~String> - The new Temp URL Key

Returns

  • response<~Excon::Response>

See Also

docs.rackspace.com/files/api/v1/cf-devguide/content/Set_Account_Metadata-d1a4460.html


23
24
25
26
27
28
29
# File 'lib/fog/storage/openstack/requests/post_set_meta_temp_url_key.rb', line 23

def post_set_meta_temp_url_key(key)
  request(
    :expects => [201, 202, 204],
    :method  => 'POST',
    :headers => {'X-Account-Meta-Temp-Url-Key' => key}
  )
end

#public_url(container = nil, object = nil) ⇒ Object

Get public_url for an object

Parameters

  • container<~String> - Name of container to look in

  • object<~String> - Name of object to look for


11
12
13
14
15
16
# File 'lib/fog/storage/openstack/requests/public_url.rb', line 11

def public_url(container = nil, object = nil)
  return nil if container.nil?
  u = "#{url}/#{Fog::OpenStack.escape(container)}"
  u << "/#{Fog::OpenStack.escape(object)}" unless object.nil?
  u
end

#put_container(name, options = {}) ⇒ Object

Create a new container

Parameters

  • name<~String> - Name for container, should be < 256 bytes and must not contain '/'


10
11
12
13
14
15
16
17
18
19
20
# File 'lib/fog/storage/openstack/requests/put_container.rb', line 10

def put_container(name, options = {})
  headers = options[:headers] || {}
  headers['X-Container-Read'] = '.r:*' if options[:public]
  headers['X-Remove-Container-Read'] = 'x' if options[:public] == false
  request(
    :expects => [201, 202],
    :method  => 'PUT',
    :path    => Fog::OpenStack.escape(name),
    :headers => headers
  )
end

#put_dynamic_obj_manifest(container, object, options = {}) ⇒ Object

Create a new dynamic large object manifest

Creates an object with a X-Object-Manifest header that specifies the common prefix (“<container>/<prefix>”) for all uploaded segments. Retrieving the manifest object streams all segments matching this prefix. Segments must sort in the order they should be concatenated. Note that any future objects stored in the container along with the segments that match the prefix will be included when retrieving the manifest object.

All segments must be stored in the same container, but may be in a different container than the manifest object. The default X-Object-Manifest header is set to “container/object”, but may be overridden in options to specify the prefix and/or the container where segments were stored. If overridden, names should be CGI escaped (excluding spaces) if needed (see OpenStack.escape).

Options Hash (options):

  • 'X-Object-Manifest' (String) — default: "container/object"

    “<container>/<prefix>” for segment objects.

Raises:

  • (Fog::Storage::OpenStack::NotFound)

    HTTP 404

  • (Excon::Errors::BadRequest)

    HTTP 400

  • (Excon::Errors::Unauthorized)

    HTTP 401

  • (Excon::Errors::HTTPStatusError)

See Also:


28
29
30
31
32
33
34
35
36
37
# File 'lib/fog/storage/openstack/requests/put_dynamic_obj_manifest.rb', line 28

def put_dynamic_obj_manifest(container, object, options = {})
  path = "#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object)}"
  headers = {'X-Object-Manifest' => path}.merge(options)
  request(
    :expects => 201,
    :headers => headers,
    :method  => 'PUT',
    :path    => path
  )
end

#put_object(container, object, data, options = {}, &block) ⇒ Object

Create a new object

When passed a block, it will make a chunked request, calling the block for chunks until it returns an empty string. In this case the data parameter is ignored.

Parameters

  • container<~String> - Name for container, should be < 256 bytes and must not contain '/'

  • object<~String> - Name for object

  • data<~String|File> - data to upload

  • options<~Hash> - config headers for object. Defaults to {}.

  • block<~Proc> - chunker


18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# File 'lib/fog/storage/openstack/requests/put_object.rb', line 18

def put_object(container, object, data, options = {}, &block)
  if block_given?
    params = {:request_block => block}
    headers = options
  else
    data = Fog::Storage.parse_data(data)
    headers = data[:headers].merge!(options)
    params = {:body => data[:body]}
  end

  params.merge!(
    :expects    => 201,
    :idempotent => !params[:request_block],
    :headers    => headers,
    :method     => 'PUT',
    :path       => "#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object)}"
  )

  request(params)
end

#put_object_manifest(container, object, options = {}) ⇒ Object

Create a new dynamic large object manifest

This is an alias for #put_dynamic_obj_manifest for backward compatibility.


8
9
10
# File 'lib/fog/storage/openstack/requests/put_object_manifest.rb', line 8

def put_object_manifest(container, object, options = {})
  put_dynamic_obj_manifest(container, object, options)
end

#put_static_obj_manifest(container, object, segments, options = {}) ⇒ Object

Create a new static large object manifest.

A static large object is similar to a dynamic large object. Whereas a GET for a dynamic large object manifest will stream segments based on the manifest's X-Object-Manifest object name prefix, a static large object manifest streams segments which are defined by the user within the manifest. Information about each segment is provided in segments as an Array of Hash objects, ordered in the sequence which the segments should be streamed.

When the SLO manifest is received, each segment's etag and size_bytes will be verified. The etag for each segment is returned in the response to #put_object, but may also be calculated. e.g. Digest::MD5.hexdigest(segment_data)

The maximum number of segments for a static large object is 1000, and all segments (except the last) must be at least 1 MiB in size. Unlike a dynamic large object, segments are not required to be in the same container.

Examples:

segments = [
  { :path => 'segments_container/first_segment',
    :etag => 'md5 for first_segment',
    :size_bytes => 'byte size of first_segment' },
  { :path => 'segments_container/second_segment',
    :etag => 'md5 for second_segment',
    :size_bytes => 'byte size of second_segment' }
]
put_static_obj_manifest('my_container', 'my_large_object', segments)

Raises:

  • (Fog::Storage::OpenStack::NotFound)

    HTTP 404

  • (Excon::Errors::BadRequest)

    HTTP 400

  • (Excon::Errors::Unauthorized)

    HTTP 401

  • (Excon::Errors::HTTPStatusError)

See Also:


42
43
44
45
46
47
48
49
50
51
# File 'lib/fog/storage/openstack/requests/put_static_obj_manifest.rb', line 42

def put_static_obj_manifest(container, object, segments, options = {})
  request(
    :expects => 201,
    :method  => 'PUT',
    :headers => options,
    :body    => Fog::JSON.encode(segments),
    :path    => "#{Fog::OpenStack.escape(container)}/#{Fog::OpenStack.escape(object)}",
    :query   => {'multipart-manifest' => 'put'}
  )
end

#reset_account_nameObject


144
145
146
# File 'lib/fog/storage/openstack.rb', line 144

def 
  @path = @original_path
end