Class: R2::Storage

Inherits:
Object
  • Object
show all
Defined in:
lib/r2/storage.rb

Overview

Storage layer responsible for communicating with Cloudflare R2.

Uses the aws-sdk-s3 gem with the S3-compatible endpoint provided by the application configuration.

Transient network failures are retried with exponential backoff (see R2::Retry), so brief instabilities do not fail the operation.

Constant Summary collapse

RETRYABLE_ERRORS =

Error classes considered transient and therefore retried.

[Seahorse::Client::NetworkingError].freeze

Instance Method Summary collapse

Constructor Details

#initialize(config, logger: nil, retry_policy: nil, sleeper: nil) ⇒ Storage

Initializes the storage with the configured credentials and bucket.

Parameters:

  • config (Configuration) —

    application configuration

  • logger (#debug, nil) (defaults to: nil) —

    logger used for diagnostics

  • retry_policy (Retry::Policy, nil) (defaults to: nil) —

    retry settings, defaults to the project policy

  • sleeper (#call, nil) (defaults to: nil) —

    waiting strategy used between attempts



27
28
29
30
31
32
33
34
35
36
37
38
39
 
# File 'lib/r2/storage.rb', line 27

def initialize(config, logger: nil, retry_policy: nil, sleeper: nil)
    @bucket = config.bucket
    @logger = logger || R2::Logging::NullLogger.new
    @retry_policy = retry_policy || Retry::Policy.new
    @sleeper = sleeper
    @s3 = Aws::S3::Client.new(
        region: config.region,
        access_key_id: config.access_key_id,
        secret_access_key: config.secret_access_key,
        endpoint: config.endpoint,
        force_path_style: true
    )
end

Instance Method Details

#delete(key:) ⇒ Object

Deletes an object from the configured bucket.

The operation is considered successful when Cloudflare R2 completes the request without raising an error.

Parameters:

  • key (String) —

    object key in the bucket

Raises:



82
83
84
85
86
87
88
89
90
91
92
93
94
 
# File 'lib/r2/storage.rb', line 82

def delete(key:)
    @logger.debug("Deleting object #{key.inspect} from bucket #{@bucket.inspect}.")
    with_retries(operation: "delete", key: key) do
        @s3.delete_object(
            bucket: @bucket,
            key: key
        )
    end
    @logger.debug("Deletion of object #{key.inspect} completed.")
    nil
rescue StandardError => e
    raise_storage_error(e, operation: "delete", key: key)
end

#download(key:, destination:) ⇒ String

Downloads an object from the configured bucket.

The content is streamed directly to the destination file, so large objects do not need to be fully loaded into memory. Every attempt writes the file from the beginning, avoiding a partially written content when a retry is needed.

Parameters:

  • key (String) —

    object key in the bucket

  • destination (String) —

    local path where the content is written

Returns:

  • (String) —

    destination path

Raises:



133
134
135
136
137
138
139
140
141
142
143
144
 
# File 'lib/r2/storage.rb', line 133

def download(key:, destination:)
    @logger.debug("Downloading object #{key.inspect} to #{destination.inspect}.")
    with_retries(operation: "download", key: key) do
        File.open(destination, "wb") do |file|
            @s3.get_object(bucket: @bucket, key: key, response_target: file)
        end
    end
    @logger.debug("Download of object #{key.inspect} completed.")
    destination
rescue StandardError => e
    raise_storage_error(e, operation: "download", key: key, destination: destination)
end

#exists?(key:) ⇒ Boolean

Checks whether an object exists in the configured bucket.

The object metadata is requested instead of the content, so the check is cheap regardless of the object size.

Parameters:

  • key (String) —

    object key in the bucket

Returns:

  • (Boolean) —

    true when the object exists

Raises:



154
155
156
157
158
159
160
161
162
163
164
165
166
 
# File 'lib/r2/storage.rb', line 154

def exists?(key:)
    @logger.debug("Checking whether object #{key.inspect} exists in bucket #{@bucket.inspect}.")
    with_retries(operation: "exists", key: key) do
        @s3.head_object(bucket: @bucket, key: key)
    end
    @logger.debug("Object #{key.inspect} was found in bucket #{@bucket.inspect}.")
    true
rescue Aws::S3::Errors::NoSuchKey, Aws::S3::Errors::NotFound
    @logger.debug("Object #{key.inspect} was not found in bucket #{@bucket.inspect}.")
    false
rescue StandardError => e
    raise_storage_error(e, operation: "exists", key: key)
end

#list(prefix: nil) ⇒ Array<String>

Lists the objects stored in the configured bucket.

Every page of the listing is requested, so all the stored objects are returned regardless of the amount of keys in the bucket.

Parameters:

  • prefix (String, nil) (defaults to: nil) —

    lists only the objects whose keys start with the prefix

Returns:

  • (Array<String>) —

    keys of the stored objects

Raises:



104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/r2/storage.rb', line 104

def list(prefix: nil)
    @logger.debug("Listing objects in bucket #{@bucket.inspect} with prefix #{prefix.inspect}.")
    keys = []
    token = nil

    loop do
        response = list_page(token, prefix)
        keys.concat(response.contents.map(&:key))
        token = response.next_continuation_token
        break if token.nil? || token.empty?
    end

    @logger.debug("Found #{keys.size} object(s) in bucket #{@bucket.inspect}.")
    keys
rescue StandardError => e
    raise_storage_error(e, operation: "list")
end

#upload(key:, body:, content_type: nil) ⇒ Object

Uploads an object to the configured bucket.

Receives content already prepared by the layer that uses the storage and delivers it to Cloudflare R2.

The content type is determined from the object key, unless an explicit value is given, so the stored object is served with the correct type instead of the generic binary type.

When the content comes from a stream, it is rewound before every attempt, since a retry must read the content from the beginning.

Parameters:

  • key (String) —

    object key in the bucket

  • body (IO, String) —

    content of the object to upload

  • content_type (String, nil) (defaults to: nil) —

    content type stored in the object metadata

Raises:



57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/r2/storage.rb', line 57

def upload(key:, body:, content_type: nil)
    type = content_type || ContentType.for(key)
    @logger.debug("Uploading object #{key.inspect} as #{type.inspect} to bucket #{@bucket.inspect}.")
    with_retries(operation: "upload", key: key) do
        body.rewind if body.respond_to?(:rewind)
        @s3.put_object(
            bucket: @bucket,
            key: key,
            body: body,
            content_type: type
        )
    end
    @logger.debug("Upload of object #{key.inspect} completed.")
    nil
rescue StandardError => e
    raise_storage_error(e, operation: "upload", key: key)
end