Class: Mamiya::Storages::S3Proxy

Inherits:
S3 show all
Defined in:
lib/mamiya/storages/s3_proxy.rb

Overview

Because there’s no S3 endpoint in Amazon VPC, fetching from instances with no public IP may consume your NAT instances’ bandwidth. This basically uses Amazon S3 but use s3_proxy.gem for fetching from specific host to avoid heavy bandwidth load.

Note: s3_proxy is a simple Rack app that proxies HTTP GET requests to Amazon S3 GetObject. You can use it with puma + nginx proxy_cache to avoid heavy bandwidth load.

Instance Attribute Summary

Attributes inherited from Abstract

#application, #config

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from S3

find, initiate_s3_with_config, #packages, #push, #remove

Methods inherited from Abstract

find, #initialize, #packages, #prune, #push, #remove

Constructor Details

This class inherits a constructor from Mamiya::Storages::Abstract

Class Method Details

.s3_config(base) ⇒ Object

:nodoc:



51
52
53
54
55
56
 
# File 'lib/mamiya/storages/s3_proxy.rb', line 51

def self.s3_config(base) # :nodoc:
  superclass.s3_config(base).tap do |c|
    c.delete(:proxy_host)
    c.delete(:proxy_ssl_verify_none)
  end
end

Instance Method Details

#fetch(package_name, dir) ⇒ Object 



15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
 
# File 'lib/mamiya/storages/s3_proxy.rb', line 15

def fetch(package_name, dir)
  package_key, meta_key = package_and_meta_key_for(package_name)

  package_path = File.join(dir, File.basename(package_key))
  meta_path = File.join(dir, File.basename(meta_key))

  if File.exists?(package_path) && File.exists?(meta_path)
    raise AlreadyFetched
  end

  open(package_path, 'wb+') do |io|
    proxy_get(package_key, io)
  end
  open(meta_path, 'wb+') do |io|
    proxy_get(meta_key, io)
  end

  return Mamiya::Package.new(package_path)

rescue NotFound, AlreadyFetched => e
  raise e

rescue Exception => e
  File.unlink package_path if package_path && File.exists?(package_path)
  File.unlink meta_path if meta_path && File.exists?(meta_path)

  raise e
end

#meta(package_name) ⇒ Object 



44
45
46
47
48
49
 
# File 'lib/mamiya/storages/s3_proxy.rb', line 44

def meta(package_name)
  _, meta_key = package_and_meta_key_for(package_name)
  JSON.parse(proxy_get(meta_key, nil, &:body))
rescue NotFound
  return nil
end