Class: Ferto::Client

Inherits:
Object
  • Object
show all
Defined in:
lib/ferto/client.rb

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(opts = {}) ⇒ Client

Returns a new instance of Client.

Parameters:

  • opts (Hash{Symbol => String, Fixnum}) (defaults to: {})

Options Hash (opts):

  • :scheme (String)
  • :host (String)
  • :path (String)
  • :port (Fixnum)
  • :connect_timeout (Fixnum)
  • :timeout (Fixnum)
  • :aggr_limit (Fixnum) 


37
38
39
40
41
42
43
44
45
46
 
# File 'lib/ferto/client.rb', line 37

def initialize(opts = {})
  opts = DEFAULT_CONFIG.merge(opts)
  @scheme = opts[:scheme]
  @host = opts[:host]
  @path = opts[:path]
  @port = opts[:port]
  @connect_timeout = opts[:connect_timeout]
  @timeout = opts[:timeout]
  @aggr_limit = opts[:aggr_limit]
end

Instance Attribute Details

#aggr_limit ⇒ Fixnum (readonly)

Returns The maximum concurrent download requests that you allow the service to make.

Returns:

  • (Fixnum) —

    The maximum concurrent download requests that you allow the service to make.



27
28
29
 
# File 'lib/ferto/client.rb', line 27

def aggr_limit
  @aggr_limit
end

#connect_timeout ⇒ Fixnum (readonly)

Returns:

  • (Fixnum) 


19
20
21
 
# File 'lib/ferto/client.rb', line 19

def connect_timeout
  @connect_timeout
end

#host ⇒ String (readonly)

Returns:

  • (String) 


10
11
12
 
# File 'lib/ferto/client.rb', line 10

def host
  @host
end

#path ⇒ String (readonly)

Returns The Downloader service path for enqueueing new downloads.

Returns:

  • (String) —

    The Downloader service path for enqueueing new downloads.



13
14
15
 
# File 'lib/ferto/client.rb', line 13

def path
  @path
end

#port ⇒ String (readonly)

Returns:

  • (String) 


16
17
18
 
# File 'lib/ferto/client.rb', line 16

def port
  @port
end

#scheme ⇒ String (readonly)

Returns:

  • (String) 


7
8
9
 
# File 'lib/ferto/client.rb', line 7

def scheme
  @scheme
end

#timeout ⇒ Fixnum (readonly)

Returns The maximum time in seconds that you allow the ‘libcurl` transfer operation to take.

Returns:

  • (Fixnum) —

    The maximum time in seconds that you allow the ‘libcurl` transfer operation to take.



23
24
25
 
# File 'lib/ferto/client.rb', line 23

def timeout
  @timeout
end

Instance Method Details

#download(aggr_id:, aggr_limit: @aggr_limit, url:, aggr_proxy: nil, download_timeout: nil, user_agent: nil, callback_url: "", callback_dst: "", callback_type: "", callback_error_type: "", callback_error_dst: "", mime_type: "", extra: {}, request_headers: {}, s3_bucket: nil, s3_region: nil, subpath: nil) ⇒ Ferto::Response

Sends a request to Downloader and returns its reply.

Examples:

client.download(
  url: 'http://foo.bar/a.jpg',
  callback_type: 'http',
  callback_dst: 'http://myapp.com/handle-download',
  aggr_id: 'foo', aggr_limit: 3,
  download_timeout: 120,
  aggr_proxy: 'http://myproxy.com/',
  user_agent: 'my-useragent',
  mime_type: "image/jpeg",
  request_headers: { "Accept" => "image/*,*/*;q=0.8" },
  extra: { something: 'someone' }
)

Parameters:

  • url (String) —

    the resource to be downloaded

  • callback_type (String) (defaults to: "")
  • callback_dst (String) (defaults to: "") —

    the callback destination

  • callback_error_type (String) (defaults to: "")
  • callback_error_dst (String) (defaults to: "") —

    the callback destination in case the job fails

  • mime_type (String) (defaults to: "") —

    (default: “”) accepted MIME types for the resource

  • aggr_id (String) —

    aggregation identifier

  • aggr_limit (Integer) (defaults to: @aggr_limit) —

    aggregation concurrency limit

  • aggr_proxy (String) (defaults to: nil) —

    the HTTP proxy to use for downloading the resource, by default no proxy is used. The proxy is set up on aggregation level and it cannot be updated for an existing aggregation.

  • download_timeout (Integer) (defaults to: nil) —

    the maximum time to wait for the resource to be downloaded in seconds, by default there is no timeout

  • user_agent (String) (defaults to: nil) —

    the User-Agent string to use for downloading the resource, by default it uses the User-Agent string set in the downloader’s configuration

  • request_headers (Hash) (defaults to: {}) —

    the request headers that will be used in downloader when performing the actual request in order to fetch the desired resource

  • subpath (String) (defaults to: nil) —

    the subfolder(s) that the jobs will be stored under the top level directory of storage backend

Returns:

Raises:

  • (Ferto::ConnectionError) —

    if there was an error scheduling the job to downloader with respect to the fact that a Curl ConnectionFailedError occured

  • (Ferto::ResponseError) —

    if a response code of 40X or 50X is received

See Also:



94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
 
# File 'lib/ferto/client.rb', line 94

def download(aggr_id:, aggr_limit: @aggr_limit, url:,
             aggr_proxy: nil, download_timeout: nil, user_agent: nil,
             callback_url: "", callback_dst: "", callback_type: "",
             callback_error_type: "", callback_error_dst: "",
             mime_type: "", extra: {},
             request_headers: {},
             s3_bucket: nil, s3_region: nil, subpath: nil)
  uri = URI::HTTP.build(
    scheme: scheme, host: host, port: port, path: path
  )
  body = build_body(
    aggr_id, aggr_limit, url,
    callback_url, callback_type, callback_dst,
    callback_error_type, callback_error_dst,
    aggr_proxy, download_timeout, user_agent,
    mime_type, extra, request_headers,
    s3_bucket, s3_region, subpath
  )
  # Curl.post reuses the same handler
  begin
    res = Curl.post(uri.to_s, body.to_json) do |handle|
      handle.headers = build_header(aggr_id)
      handle.connect_timeout = connect_timeout
      handle.timeout = timeout
    end

    case res.response_code
    when 400..599
      error_msg = ("An error occured during the download call. "  \
        "Received a #{res.response_code} response code and body " \
        "#{res.body_str}")
      raise Ferto::ResponseError.new(error_msg, res)
    end
  rescue Curl::Err::ConnectionFailedError => e
    raise Ferto::ConnectionError.new(e)
  end

  Ferto::Response.new res
end