Class: Technoweenie::AttachmentFu::Backends::S3Backend

Inherits:
BackendDelegator
  • Object
show all
Defined in:
lib/technoweenie/attachment_fu/backends/s3_backend.rb

Overview

AWS::S3 Storage Backend

Enables use of Amazon’s Simple Storage Service as a storage mechanism

Requirements

Requires the AWS::S3 Library for S3 by Marcel Molina Jr. installed either as a gem or a as a Rails plugin.

Configuration

Configuration is done via RAILS_ROOT/config/amazon_s3.yml and is loaded according to the RAILS_ENV. The minimum connection options that you must specify are a bucket name, your access key id and your secret access key. If you don’t already have your access keys, all you need to sign up for the S3 service is an account at Amazon. You can sign up for S3 and get access keys by visiting aws.amazon.com/s3.

If you wish to use Amazon CloudFront to serve the files, you can also specify a distibution domain for the bucket. To read more about CloudFront, visit aws.amazon.com/cloudfront

Example configuration (RAILS_ROOT/config/amazon_s3.yml)

development:
  bucket_name: appname_development
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

test:
  bucket_name: appname_test
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

production:
  bucket_name: appname
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

You can change the location of the config path by passing a full path to the :s3_config_path option.

has_attachment :storage => :s3, :s3_config_path => (RAILS_ROOT + '/config/s3.yml')

Required configuration parameters

  • :access_key_id - The access key id for your S3 account. Provided by Amazon.

  • :secret_access_key - The secret access key for your S3 account. Provided by Amazon.

  • :bucket_name - A unique bucket name (think of the bucket_name as being like a database name).

If any of these required arguments is missing, a MissingAccessKey exception will be raised from AWS::S3.

About bucket names

Bucket names have to be globaly unique across the S3 system. And you can only have up to 100 of them, so it’s a good idea to think of a bucket as being like a database, hence the correspondance in this implementation to the development, test, and production environments.

The number of objects you can store in a bucket is, for all intents and purposes, unlimited.

Optional configuration parameters

  • :server - The server to make requests to. Defaults to s3.amazonaws.com.

  • :port - The port to the requests should be made on. Defaults to 80 or 443 if :use_ssl is set.

  • :use_ssl - If set to true, :port will be implicitly set to 443, unless specified otherwise. Defaults to false.

  • :distribution_domain - The CloudFront distribution domain for the bucket. This can either be the assigned

    distribution domain (ie. XXX.cloudfront.net) or a chosen domain using a CNAME. See CloudFront for more details.

Usage

To specify S3 as the storage mechanism for a model, set the acts_as_attachment :storage option to :s3.

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3
end

Customizing the path

By default, files are prefixed using a pseudo hierarchy in the form of :table_name/:id, which results in S3 urls that look like: http(s)://:server/:bucket_name/:table_name/:id/:filename with :table_name representing the customizable portion of the path. You can customize this prefix using the :path_prefix option:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :path_prefix => 'my/custom/path'
end

Which would result in URLs like http(s)://:server/:bucket_name/my/custom/path/:id/:filename.

Using different bucket names on different models

By default the bucket name that the file will be stored to is the one specified by the :bucket_name key in the amazon_s3.yml file. You can use the :bucket_key option to overide this behavior on a per model basis. For instance if you want a bucket that will hold only Photos you can do this:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :bucket_key => :photo_bucket_name
end

And then your amazon_s3.yml file needs to look like this.

 development:
   bucket_name: appname_development
   access_key_id: <your key>
   secret_access_key: <your key>

 test:
   bucket_name: appname_test
   access_key_id: <your key>
   secret_access_key: <your key>

 production:
   bucket_name: appname
   photo_bucket_name: appname_photos
   access_key_id: <your key>
   secret_access_key: <your key>

If the bucket_key you specify is not there in a certain environment then attachment_fu will
default to the <tt>bucket_name</tt> key.  This way you only have to create special buckets
this can be helpful if you only need special buckets in certain environments.

Permissions

By default, files are stored on S3 with public access permissions. You can customize this using the :s3_access option to has_attachment. Available values are :private, :public_read_write, and :authenticated_read.

Other options

Of course, all the usual configuration options apply, such as content_type and thumbnails:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :content_type => ['application/pdf', :image], :resize_to => 'x50'
  has_attachment :storage => :s3, :thumbnails => { :thumb => [50, 50], :geometry => 'x50' }
end

Accessing S3 URLs

You can get an object’s URL using the s3_url accessor. For example, assuming that for your postcard app you had a bucket name like ‘postcard_world_development’, and an attachment model called Photo:

@postcard.s3_url # => http(s)://s3.amazonaws.com/postcard_world_development/photos/1/mexico.jpg

The resulting url is in the form: http(s)://:server/:bucket_name/:table_name/:id/:file. The optional thumbnail argument will output the thumbnail’s filename (if any).

Additionally, you can get an object’s base path relative to the bucket root using base_path:

@photo.file_base_path # => photos/1

And the full path (including the filename) using full_filename:

@photo.full_filename # => photos/

Niether base_path or full_filename include the bucket name as part of the path. You can retrieve the bucket name using the bucket_name method.

Accessing CloudFront URLs

You can get an object’s CloudFront URL using the cloudfront_url accessor. Using the example from above: The resulting url is in the form: :distribution_domain/:table_name/:id/:file

If you set :cloudfront to true in your model, the public_url will be the CloudFront URL, not the S3 URL.

Defined Under Namespace

Classes: ConfigFileNotFoundError, RequiredLibraryNotFoundError

Constant Summary collapse

@@s3_config_path = 
nil

Instance Attribute Summary collapse

Attributes inherited from BackendDelegator

#attachment_options

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from BackendDelegator

#__getobj__

Constructor Details

#initialize(obj, opts) ⇒ S3Backend

Returns a new instance of S3Backend.



180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 180

def initialize(obj, opts)
  # zendesk classic rails usage note
  # all the options from our attachments.yml come in through the opts argument here.
  super(obj, opts)

  self.s3_config = if @@s3_config_path
    # config from file.
    YAML.load(ERB.new(File.read(@@s3_config_path)).result).fetch(ENV['RAILS_ENV']).symbolize_keys
  else
    # config entirely through initializer

    # a few options need renaming.
    opts[:access_key_id] = opts[:s3_access_key] if opts.has_key? :s3_access_key
    opts[:secret_access_key] = opts[:s3_secret_key] if opts.has_key? :s3_secret_key

    opts
  end

  # :use_ssl defaults to true now in AWS::SDK
  # the rest of our code relies on checking for this value in s3_config
  s3_config[:use_ssl] = true unless s3_config.has_key?(:use_ssl)

  @bucket_name = self.s3_config[:bucket_name]

end

Instance Attribute Details

#bucket_name ⇒ Object (readonly)

Returns the value of attribute bucket_name.



176
177
178
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 176

def bucket_name
  @bucket_name
end

#s3_config ⇒ Object

Returns the value of attribute s3_config.



175
176
177
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 175

def s3_config
  @s3_config
end

Class Method Details

.included_in_base(base) ⇒ Object

:nodoc:



206
207
208
209
210
211
212
213
214
215
216
217
218
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 206

def self.included_in_base(base) #:nodoc:

  begin
    require 'aws-sdk-v1'
  rescue LoadError
    raise RequiredLibraryNotFoundError.new('AWS::SDK could not be loaded')
  end

  if base.attachment_options[:s3_config_path]
    @@s3_config_path = base.attachment_options[:s3_config_path] || Rails.root.join('config/amazon_s3.yml').to_s
  end

end

Instance Method Details

#attachment_path_id ⇒ Object

The attachment ID used in the full path of a file



261
262
263
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 261

def attachment_path_id
  ((respond_to?(:parent_id) && parent_id) || @obj.id).to_s
end

#authenticated_s3_url(*args) ⇒ Object

All private objects are accessible via an authenticated GET request to the S3 servers. You can generate an authenticated url for an object like this:

@photo.authenticated_s3_url

By default authenticated urls expire 5 minutes after they were generated.

Expiration options can be specified either with an absolute time using the :expires option, or with a number of seconds relative to now with the :expires_in option:

# Absolute expiration date (October 13th, 2025)
@photo.authenticated_s3_url(:expires => Time.mktime(2025,10,13).to_i)

# Expiration in five hours from now
@photo.authenticated_s3_url(:expires_in => 5.hours)

You can specify whether the url should go over SSL with the :use_ssl option. By default, the ssl settings for the current connection will be used:

@photo.authenticated_s3_url(:use_ssl => true)

Finally, the optional thumbnail argument will output the thumbnail’s filename (if any):

@photo.authenticated_s3_url('thumbnail', :expires_in => 5.hours, :use_ssl => true)



342
343
344
345
346
347
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 342

def authenticated_s3_url(*args)
  options   = args.extract_options!
  options[:expires_in] = options[:expires_in].to_i if options[:expires_in]
  thumbnail = args.shift
  self.bucket.objects[full_filename(thumbnail)].url_for(:get,options).to_s
end

#base_path ⇒ Object

The pseudo hierarchy containing the file relative to the bucket name Example: :table_name/:id



267
268
269
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 267

def base_path
  File.join(attachment_options[:path_prefix], attachment_path_id)
end

#bucket ⇒ Object 



349
350
351
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 349

def bucket
  @bucket_c ||= connection.buckets[bucket_name]
end

#cloudfront_distribution_domain ⇒ Object 



251
252
253
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 251

def cloudfront_distribution_domain
  distribution_domain
end

#cloudfront_url(thumbnail = nil) ⇒ Object

All public objects are accessible via a GET request to CloudFront. You can generate a url for an object using the cloudfront_url method.

@photo.cloudfront_url

The resulting url is in the form: http://:distribution_domain/:table_name/:id/:file using the :distribution_domain variable set in the configuration parameters in RAILS_ROOT/config/amazon_s3.yml.

The optional thumbnail argument will output the thumbnail’s filename (if any).



306
307
308
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 306

def cloudfront_url(thumbnail = nil)
  "http://" + s3_config[:distribution_domain] + "/" + full_filename(thumbnail)
end

#connection ⇒ Object 



220
221
222
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 220

def connection
  @s3 ||= AWS::S3.new(s3_config)
end

#current_data ⇒ Object 



353
354
355
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 353

def current_data
  bucket.objects[full_filename].read
end

#destroy_file ⇒ Object

Called in the after_destroy callback



358
359
360
361
362
363
364
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 358

def destroy_file
  bucket.objects[full_filename].delete
rescue Errno::ECONNRESET
  retries ||= 0
  retries += 1
  retry if retries <= 1
end

#distribution_domain ⇒ Object 



236
237
238
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 236

def distribution_domain
  @distribution_domain = s3_config[:distribution_domain]
end

#full_filename(thumbnail = nil) ⇒ Object

The full path to the file relative to the bucket name Example: :table_name/:id/:filename



273
274
275
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 273

def full_filename(thumbnail = nil)
  File.join(base_path, thumbnail_name_for(thumbnail))
end

#hostname ⇒ Object 



228
229
230
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 228

def hostname
  connection.client.endpoint
end

#notify_rename ⇒ Object

called by the ActiveRecord class from filename=



256
257
258
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 256

def notify_rename
  @old_filename = filename unless filename.nil? || @old_filename
end

#port_string ⇒ Object 



232
233
234
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 232

def port_string
  connection.client.port.to_s
end

#protocol ⇒ Object 



224
225
226
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 224

def protocol
  @protocol ||= s3_config[:use_ssl] ? 'https://' : 'http://'
end

#public_url(*args) ⇒ Object 



310
311
312
313
314
315
316
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 310

def public_url(*args)
  if attachment_options[:cloudfront]
    cloudfront_url(args)
  else
    s3_url(args)
  end
end

#rename_file ⇒ Object 



366
367
368
369
370
371
372
373
374
375
376
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 366

def rename_file
  return unless @old_filename && @old_filename != filename

  old_full_filename = File.join(base_path, @old_filename)

  o = bucket.objects[old_full_filename]
  o.rename_to(full_filename)

  @old_filename = nil
  true
end

#s3_hostname ⇒ Object 



243
244
245
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 243

def s3_hostname
  hostname
end

#s3_port_string ⇒ Object 



247
248
249
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 247

def s3_port_string
  port_string
end

#s3_protocol ⇒ Object 



239
240
241
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 239

def s3_protocol
  protocol
end

#s3_url(thumbnail = nil) ⇒ Object

All public objects are accessible via a GET request to the S3 servers. You can generate a url for an object using the s3_url method.

@photo.s3_url

The resulting url is in the form: http(s)://:server/:bucket_name/:table_name/:id/:file where the :server variable defaults to AWS::S3 URL::DEFAULT_HOST (s3.amazonaws.com) and can be set using the configuration parameters in RAILS_ROOT/config/amazon_s3.yml.

The optional thumbnail argument will output the thumbnail’s filename (if any).



287
288
289
290
291
292
293
294
295
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 287

def s3_url(thumbnail = nil)
  # leave out the port if redundant
  if ( s3_config[:use_ssl] && s3_port_string.to_s == '443' ) || ( ! s3_config[:use_ssl] && s3_port_string.to_s == '80' )
    port_string = ''
  else
    port_string = ':' + s3_port_string
  end
  File.join(s3_protocol + bucket_name + '.' + s3_hostname + port_string, full_filename(thumbnail))
end

#save_to_storage ⇒ Object 



378
379
380
381
382
383
384
385
386
387
388
389
390
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 378

def save_to_storage
  if save_attachment?
    obj = bucket.objects[full_filename]
    if temp_path
      obj.write(:file => temp_path, :content_type => content_type, :server_side_encryption => :aes256)
    else
      obj.write(temp_data, :content_type => content_type, :server_side_encryption => :aes256)
    end
  end

  @old_filename = nil
  true
end