Module: Technoweenie::AttachmentFu::Backends::S3Backend

Defined in:
lib/technoweenie/attachment_fu/backends/s3_backend.rb

Overview

AWS::S3 Storage Backend

Enables use of Amazon’s Simple Storage Service as a storage mechanism

Requirements

Requires the AWS::S3 Library for S3 by Marcel Molina Jr. installed either as a gem or a as a Rails plugin.

Configuration

Configuration is done via #{Rails.root}/config/amazon_s3.yml and is loaded according to the #{Rails.env}. The minimum connection options that you must specify are a bucket name, your access key id and your secret access key. If you don’t already have your access keys, all you need to sign up for the S3 service is an account at Amazon. You can sign up for S3 and get access keys by visiting aws.amazon.com/s3.

If you wish to use Amazon CloudFront to serve the files, you can also specify a distibution domain for the bucket. To read more about CloudFront, visit aws.amazon.com/cloudfront

Example configuration (#Rails.root/config/amazon_s3.yml)

development:
  bucket_name: appname_development
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

test:
  bucket_name: appname_test
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

production:
  bucket_name: appname
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

You can change the location of the config path by passing a full path to the :s3_config_path option.

has_attachment :storage => :s3, :s3_config_path => (#{Rails.root} + '/config/s3.yml')

Required configuration parameters

  • :access_key_id - The access key id for your S3 account. Provided by Amazon.

  • :secret_access_key - The secret access key for your S3 account. Provided by Amazon.

  • :bucket_name - A unique bucket name (think of the bucket_name as being like a database name).

If any of these required arguments is missing, a MissingAccessKey exception will be raised from AWS::S3.

About bucket names

Bucket names have to be globaly unique across the S3 system. And you can only have up to 100 of them, so it’s a good idea to think of a bucket as being like a database, hence the correspondance in this implementation to the development, test, and production environments.

The number of objects you can store in a bucket is, for all intents and purposes, unlimited.

Optional configuration parameters

  • :server - The server to make requests to. Defaults to s3.amazonaws.com.

  • :port - The port to the requests should be made on. Defaults to 80 or 443 if :use_ssl is set.

  • :use_ssl - If set to true, :port will be implicitly set to 443, unless specified otherwise. Defaults to false.

  • :distribution_domain - The CloudFront distribution domain for the bucket. This can either be the assigned

    distribution domain (ie. XXX.cloudfront.net) or a chosen domain using a CNAME. See CloudFront for more details.

Usage

To specify S3 as the storage mechanism for a model, set the acts_as_attachment :storage option to :s3.

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3
end

Customizing the path

By default, files are prefixed using a pseudo hierarchy in the form of :table_name/:id, which results in S3 urls that look like: http(s)://:server/:bucket_name/:table_name/:id/:filename with :table_name representing the customizable portion of the path. You can customize this prefix using the :path_prefix option:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :path_prefix => 'my/custom/path'
end

Which would result in URLs like http(s)://:server/:bucket_name/my/custom/path/:id/:filename.

Using different bucket names on different models

By default the bucket name that the file will be stored to is the one specified by the :bucket_name key in the amazon_s3.yml file. You can use the :bucket_key option to overide this behavior on a per model basis. For instance if you want a bucket that will hold only Photos you can do this:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :bucket_key => :photo_bucket_name
end

And then your amazon_s3.yml file needs to look like this.

 development:
   bucket_name: appname_development
   access_key_id: <your key>
   secret_access_key: <your key>

 test:
   bucket_name: appname_test
   access_key_id: <your key>
   secret_access_key: <your key>

 production:
   bucket_name: appname
   photo_bucket_name: appname_photos
   access_key_id: <your key>
   secret_access_key: <your key>

If the bucket_key you specify is not there in a certain environment then attachment_fu will
default to the <tt>bucket_name</tt> key.  This way you only have to create special buckets
this can be helpful if you only need special buckets in certain environments.

Permissions

By default, files are stored on S3 with public access permissions. You can customize this using the :s3_access option to has_attachment. Available values are :private, :public_read_write, and :authenticated_read.

Other options

Of course, all the usual configuration options apply, such as content_type and thumbnails:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :content_type => ['application/pdf', :image], :resize_to => 'x50'
  has_attachment :storage => :s3, :thumbnails => { :thumb => [50, 50], :geometry => 'x50' }
end

Accessing S3 URLs

You can get an object’s URL using the s3_url accessor. For example, assuming that for your postcard app you had a bucket name like ‘postcard_world_development’, and an attachment model called Photo:

@postcard.s3_url # => http(s)://s3.amazonaws.com/postcard_world_development/photos/1/mexico.jpg

The resulting url is in the form: http(s)://:server/:bucket_name/:table_name/:id/:file. The optional thumbnail argument will output the thumbnail’s filename (if any).

Additionally, you can get an object’s base path relative to the bucket root using base_path:

@photo.file_base_path # => photos/1

And the full path (including the filename) using full_filename:

@photo.full_filename # => photos/

Niether base_path or full_filename include the bucket name as part of the path. You can retrieve the bucket name using the bucket_name method.

Accessing CloudFront URLs

You can get an object’s CloudFront URL using the cloudfront_url accessor. Using the example from above: The resulting url is in the form: :distribution_domain/:table_name/:id/:file

If you set :cloudfront to true in your model, the public_filename will be the CloudFront URL, not the S3 URL.

Defined Under Namespace

Modules: ClassMethods Classes: ConfigFileNotFoundError, RequiredLibraryNotFoundError

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.distribution_domain ⇒ Object 



220
221
222
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 220

def self.distribution_domain
  @distribution_domain = s3_config[:distribution_domain]
end

.hostname ⇒ Object 



212
213
214
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 212

def self.hostname
  @hostname ||= s3_config[:server] || AWS::S3::DEFAULT_HOST
end

.included(base) ⇒ Object

:nodoc:



175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 175

def self.included(base) #:nodoc:
  mattr_reader :bucket_name, :s3_config

  begin
    require 'aws/s3'
    include AWS::S3
  rescue LoadError
    raise RequiredLibraryNotFoundError.new('AWS::S3 could not be loaded')
  end

  begin
    @@s3_config_path = base.attachment_options[:s3_config_path] || File.join(Rails.root, 'config', 'amazon_s3.yml')
    @@s3_config = @@s3_config = YAML.load(ERB.new(File.read(@@s3_config_path)).result)[Rails.env].symbolize_keys
  #rescue
  #  raise ConfigFileNotFoundError.new('File %s not found' % @@s3_config_path)
  end

  bucket_key = base.attachment_options[:bucket_key]

  if bucket_key and s3_config[bucket_key.to_sym]
    eval_string = "def bucket_name()\n  \"#{s3_config[bucket_key.to_sym]}\"\nend"
  else
    eval_string = "def bucket_name()\n  \"#{s3_config[:bucket_name]}\"\nend"
  end
  base.class_eval(eval_string, __FILE__, __LINE__)

  Base.establish_connection!(s3_config.slice(:access_key_id, :secret_access_key, :server, :port, :use_ssl, :persistent, :proxy))

  # Bucket.create(@@bucket_name)

  base.before_update :rename_file
end

.port_string ⇒ Object 



216
217
218
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 216

def self.port_string
  @port_string ||= (s3_config[:port].nil? || s3_config[:port] == (s3_config[:use_ssl] ? 443 : 80)) ? '' : ":#{s3_config[:port]}"
end

.protocol ⇒ Object 



208
209
210
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 208

def self.protocol
  @protocol ||= s3_config[:use_ssl] ? 'https://' : 'http://'
end

Instance Method Details

#attachment_path_id ⇒ Object

The attachment ID used in the full path of a file



249
250
251
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 249

def attachment_path_id
  ((respond_to?(:parent_id) && parent_id) || id).to_s
end

#authenticated_s3_url(*args) ⇒ Object

All private objects are accessible via an authenticated GET request to the S3 servers. You can generate an authenticated url for an object like this:

@photo.authenticated_s3_url

By default authenticated urls expire 5 minutes after they were generated.

Expiration options can be specified either with an absolute time using the :expires option, or with a number of seconds relative to now with the :expires_in option:

# Absolute expiration date (October 13th, 2025)
@photo.authenticated_s3_url(:expires => Time.mktime(2025,10,13).to_i)

# Expiration in five hours from now
@photo.authenticated_s3_url(:expires_in => 5.hours)

You can specify whether the url should go over SSL with the :use_ssl option. By default, the ssl settings for the current connection will be used:

@photo.authenticated_s3_url(:use_ssl => true)

Finally, the optional thumbnail argument will output the thumbnail’s filename (if any):

@photo.authenticated_s3_url('thumbnail', :expires_in => 5.hours, :use_ssl => true)



325
326
327
328
329
330
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 325

def authenticated_s3_url(*args)
  options   = args.extract_options!
  options[:expires_in] = options[:expires_in].to_i if options[:expires_in]
  thumbnail = args.shift
  S3Object.url_for(full_filename(thumbnail), bucket_name, options)
end

#base_path(thumbnail = nil) ⇒ Object

The pseudo hierarchy containing the file relative to the bucket name Example: :table_name/:id



255
256
257
258
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 255

def base_path(thumbnail = nil)
  file_system_path = (thumbnail ? thumbnail_class : self).attachment_options[:path_prefix]
  File.join(file_system_path, attachment_path_id)
end

#cloudfront_distribution_domain ⇒ Object 



356
357
358
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 356

def cloudfront_distribution_domain
  Technoweenie::AttachmentFu::Backends::S3Backend.distribution_domain
end

#cloudfront_url(thumbnail = nil) ⇒ Object

All public objects are accessible via a GET request to CloudFront. You can generate a url for an object using the cloudfront_url method.

@photo.cloudfront_url

The resulting url is in the form: http://:distribution_domain/:table_name/:id/:file using the :distribution_domain variable set in the configuration parameters in #{Rails.root}/config/amazon_s3.yml.

The optional thumbnail argument will output the thumbnail’s filename (if any).



289
290
291
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 289

def cloudfront_url(thumbnail = nil)
  s3_protocol + cloudfront_distribution_domain + "/" + full_filename(thumbnail)
end

#create_temp_file ⇒ Object 



332
333
334
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 332

def create_temp_file
  write_to_temp_file current_data
end

#current_data ⇒ Object 



336
337
338
339
340
341
342
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 336

def current_data
  if attachment_options[:encrypted_storage] && self.respond_to?(:encryption_key) && self.encryption_key != nil
    EncryptedData.decrypt_data(S3Object.value(full_filename, bucket_name), self.encryption_key)
  else
    S3Object.value full_filename, bucket_name
  end
end

#filename=(value) ⇒ Object

Overwrites the base filename writer in order to store the old filename



243
244
245
246
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 243

def filename=(value)
  @old_filename = filename unless filename.nil? || @old_filename
  write_attribute :filename, sanitize_filename(value)
end

#full_filename(thumbnail = nil) ⇒ Object

The full path to the file relative to the bucket name Example: :table_name/:id/:filename



262
263
264
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 262

def full_filename(thumbnail = nil)
  File.join(base_path(thumbnail), thumbnail_name_for(thumbnail))
end

#public_filename(*args) ⇒ Object 



293
294
295
296
297
298
299
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 293

def public_filename(*args)
  if attachment_options[:cloudfront]
    cloudfront_url(*args)
  else
    s3_url(*args)
  end
end

#s3_hostname ⇒ Object 



348
349
350
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 348

def s3_hostname
  Technoweenie::AttachmentFu::Backends::S3Backend.hostname
end

#s3_port_string ⇒ Object 



352
353
354
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 352

def s3_port_string
  Technoweenie::AttachmentFu::Backends::S3Backend.port_string
end

#s3_protocol ⇒ Object 



344
345
346
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 344

def s3_protocol
  Technoweenie::AttachmentFu::Backends::S3Backend.protocol
end

#s3_url(thumbnail = nil) ⇒ Object

All public objects are accessible via a GET request to the S3 servers. You can generate a url for an object using the s3_url method.

@photo.s3_url

The resulting url is in the form: http(s)://:server/:bucket_name/:table_name/:id/:file where the :server variable defaults to AWS::S3 URL::DEFAULT_HOST (s3.amazonaws.com) and can be set using the configuration parameters in #{Rails.root}/config/amazon_s3.yml.

The optional thumbnail argument will output the thumbnail’s filename (if any).



276
277
278
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 276

def s3_url(thumbnail = nil)
  File.join(s3_protocol + s3_hostname + s3_port_string, bucket_name, full_filename(thumbnail))
end