Ferto

A Ruby client for skroutz/downloader.

Installation

Add this line to your application's Gemfile:

gem 'ferto'

And then execute:

$ bundle

Or install it yourself as:

$ gem install ferto

Usage

Creating a client

client = Ferto::Client.new(
  scheme: 'http',
  host: 'localhost',
  port: 8000
)

Downloading a file

downloader supports multiple notification backends for the download results. Currently, there are two supported options: an HTTP and a Kafka backend. Let's see how to issue a download request in each case:

HTTP-based notification backend

dl_resp = client.download(aggr_id: 'bucket1',
                          aggr_limit: 3,
                          url: 'http://example.com/',
                          mime_type: 'text/html',
                          callback_type: 'http',
                          callback_dst: 'http://myservice.com/downloader_callback',
                          extra: { some_extra_info: 'info' })

In order for a service to consume downloader's result, it must accept the HTTP callback in the endpoint denoted by callback_dst.

Kafka-based notification backend

dl_resp = client.download(aggr_id: 'bucket1',
                          aggr_limit: 3,
                          url: 'http://example.com/',
                          mime_type: 'text/html',
                          callback_type: 'kafka',
                          callback_dst: 'my-kafka-topic',
                          extra: { some_extra_info: 'info' })

To consume the downloader's result, you can use your favorite Kafka library and consume the callback message from my-kafka-topic (passed in callback_dst).

If the connection with the downloader API was successful, the aforementioned dl_resp is a Ferto::Response object. If the client failed to connect, a Ferto::ConnectionError exception is raised.

To handle the actual callback message, e.g. from inside a Rails controller:

class DownloaderController < ApplicationConroller
  def callback
    cb = Ferto::Callback.new(callback_params)

    if cb.download_successful?
      # Download cb.download_url
    else
      # Log failure
    end
  end

  def callback_params
    params.permit!.to_h
  end
end

For the detailed semantics of each option and the format of the callback payload, please, refer to the official downloader's documentation (download parameters, callback payload).

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/skroutz/ferto.