Module: IMW::Schemes::S3

Defined in:
lib/imw/schemes/s3.rb

Overview

Defines methods for reading and writing data to Amazon S3 buckets.

IMW.open('s3://my_bucket/path/to/some/file.csv')

Learn more about Amazon Web Services.

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.copy(source, destination) ⇒ IMW::Resource

Copy S3 resource source to destination.

Parameters:

Returns:

Raises:



106
107
108
109
110
111
112
113
 
# File 'lib/imw/schemes/s3.rb', line 106

def self.copy source, destination
  source      = IMW.open(source)
  destination = IMW.open(destination)
  raise IMW::PathError.new("Bucket names must be non-blank and match to 'copy'") unless source.bucket.present? && destination.bucket.present? && source.bucket == destination.bucket
  make_connection!
  AWS::S3::Object.copy(source.raw_path, destination.raw_path, destination.bucket)
  destination
end

.get(source, destination) ⇒ IMW::Resource

Download source from S3 into destination.

Parameters:

Returns:

Raises:



89
90
91
92
93
94
95
96
97
98
99
 
# File 'lib/imw/schemes/s3.rb', line 89

def self.get source, destination
  source      = IMW.open(source)
  destination = IMW.open!(destination)
  raise IMW::ArgumentError.new("source must be on S3 -- #{source} given") unless source.on_s3?
  make_connection!
  AWS::S3::S3Object.stream(source.raw_path, source.bucket) do |chunk|
    destination.write(chunk)
  end
  destination.close
  destination.reopen
end

.put(source, destination) ⇒ IMW::Resource

Store source into destination.

Parameters:

Returns:

Raises:



75
76
77
78
79
80
81
82
 
# File 'lib/imw/schemes/s3.rb', line 75

def self.put source, destination
  source       = IMW.open(source)
  destintation = IMW.open(destination)
  raise IMW::ArgumentError.new("destination must be on S3 -- #{destination} given") unless destination.on_s3?
  make_connection!
  AWS::S3::S3Object.store(destination.raw_path, source.io, destination.bucket)
  destination
end

Instance Method Details

#bucket ⇒ String

For an S3 resource, the bucket is just the hostname.

Returns:



15
16
17
 
# File 'lib/imw/schemes/s3.rb', line 15

def bucket
  host
end

#cp(new_uri) ⇒ IMW::Resource

Copy this resource to the new_uri.

Parameters:

Returns:



31
32
33
34
 
# File 'lib/imw/schemes/s3.rb', line 31

def cp new_uri
  #IMW::Tools::Transferer.new(:cp, self, new_uri).transfer!
  IMW::Schemes::S3.get(self, new_uri)
end

#exist? ⇒ true, false Also known as: exists?

Does this resource exist on S3?

Returns:

  • (true, false) 


39
40
41
 
# File 'lib/imw/schemes/s3.rb', line 39

def exist?
  AWS::S3::S3Object.exists?(raw_path, bucket)
end

#join(*paths) ⇒ IMW::Resource

Return the resource at the base path of this resource joined to path.

IMW.open('s3:://bucket/path/to/dir').join('subdir')
#=> IMW::Resource at 's3://bucket/path/to/dir/subdir'

Parameters:

Returns:



123
124
125
 
# File 'lib/imw/schemes/s3.rb', line 123

def join *paths
  IMW.open(File.join(stripped_uri.to_s, *paths))
end

#on_s3? ⇒ true, false Also known as: is_s3?

Is this resource an S3 resource?

Returns:

  • (true, false) 


22
23
24
 
# File 'lib/imw/schemes/s3.rb', line 22

def on_s3?
  true
end

#read ⇒ String

Return the contents of this S3 object.

Returns:



66
67
68
 
# File 'lib/imw/schemes/s3.rb', line 66

def read
  AWS::S3::S3Object.value(raw_path, bucket)
end

#rm ⇒ IMW::Resource Also known as: rm!

Remove this resource from S3.

Returns:



47
48
49
 
# File 'lib/imw/schemes/s3.rb', line 47

def rm
  AWS::S3::S3Object.delete(raw_path, bucket)
end

#s3n_url ⇒ String

Return the S3N URL for this S3 object

resource = IMW.open('s3://my_bucket/path/to/some/obj')
resource.s3n_url
=> 's3n://my_bucket/path/to/some/obj'

Returns:



59
60
61
 
# File 'lib/imw/schemes/s3.rb', line 59

def s3n_url
  uri.to_s.gsub(/^s3:/, 's3n:')
end