Shrine::Plugins::Transloadit
Provides Transloadit integration for Shrine.
Transloadit offers advanced file processing for all sorts of media, including images, videos, audio, and documents, along with importing from and exporting to various file storage services.
Setup
While Transloadit is able to export processed files to many storage services, this plugin currently supports only Amazon S3 (just because there are no Shrine integrations written for other services on that list yet).
gem "shrine-transloadit"
gem "aws-sdk"
require "shrine"
require "shrine/storage/s3"
Shrine.plugin :transloadit,
auth_key: "your transloadit key",
auth_secret: "your transloadit secret"
Shrine.storages[:store] = Shrine::Storage::S3.new(
access_key_id: "xyz",
secret_access_key: "abc",
region: "my-region",
bucket: "my-app",
)
post "/webhooks/transloadit" do
Shrine::Attacher.transloadit_save(params)
end
This setup assumes you want to do direct uploads to
Transloadit. This is completely optional, the plugin will work equally well
with whatever your :cache
storage is.
How it works
Transloadit works in a way that you create an "assembly", which contains all information about how the file(s) should be processed, from import to export. Processing itself happens asynchronously, and you can give Transloadit a URL which it will POST results to when processing finishes.
This plugin allows you to easily implement this webhook flow. You can intercept promoting, and submit a Transloadit assembly using the cached file, along with a URL to the route in your app where you'd like Transloadit to POST the results of processing. Then you can call the plugin again in the route to save the results to your attachment column.
The demo app shows a complete implementation of this flow, and can serve as a good baseline for your own implementation.
Usage
Transloadit assemblies are built inside #transloadit_process
method in your
uploader, and you can use some convenient helper methods which the plugin
provides.
class MyUploader < Shrine
def transloadit_process(io, context)
resized = transloadit_file(io)
.add_step("resize", "/image/resize", width: 800)
transloadit_assembly(resized, context: context)
end
end
These helper methods just provide a higher-level interface over the transloadit gem, which you might want look at to get a better understanding of how building assemblies works.
In short, in Transloadit every action, be it import, processing, or export, is a "step". Each step is defined by its robot and arguments, and needs to have a unique name. Transloadit allows you to define the entire processing flow (which can result in multiple files) as a collection of steps, which is called an "assembly". Once the assembly is built it can be submitted to Transloadit.
Versions
With Transloadit you can create multiple files in a single assembly, and this plugin allows you to leverage that in form of a hash of versions.
class MyUploader < Shrine
plugin :versions
def transloadit_process(io, context)
original = transloadit_file(io)
medium = original.add_step("resize_500", "/image/resize", width: 500)
small = original.add_step("resize_300", "/image/resize", width: 300)
files = {original: original, medium: medium, small: small}
transloadit_assembly(files, context: context)
end
end
Direct uploads
Transloadit supports direct uploads, allowing you to do additional processing on upload, along with a jQuery plugin for easy integration. Generally you only want to do some light processing on direct uploads, and without any exporting, so that you have better control over your Transloadit bandwidth.
When direct upload finishes, Transloadit returns information about the uploaded
file(s). The default :cache
storage allows you to store Transloadit's
temporary file URL as an attachment. This means that you can use the following
JavaScript to convert Transloadit's data hash into Shrine's format:
{
id: data['url'], // we save the URL
storage: 'cache',
metadata: {
size: data['size'],
filename: data['name'],
mime_type: data['mime'],
width: data['meta'] && data['meta']['width'],
height: data['meta'] && data['meta']['height'],
transloadit: data['meta'],
}
}
See the demo app for a complete implementation of direct uploads.
Templates
Transloadit recommends using templates, since they allow you to replay failed assemblies, and also allow you not to expose credentials in your HTML.
Here is an example where the whole processing is defined inside a template, and we just set the location of the imported file.
# Your saved template
{
steps: {
import: {
robot: "/http/import",
url: "..."
},
resize: {
robot: "/image/resize",
use: "import",
width: 800
}
}
}
class MyUploader < Shrine
def transloadit_process(io, context)
transloadit_assembly("my_template", steps: {import: {url: io.url}})
end
end
Webhooks
Transloadit performs its processing asynchronously, and you can provide a URL where you want Transloadit to POST results of processing once it's finished.
class MyUploader < Shrine
def transloadit_process(io, context)
# ...
transloadit_assembly(files, notify_url: "http://myapp.com/webhooks/transloadit")
end
end
Then in your POST /webhooks/transloadit
route you can call the plugin to
automatically save the results to the attachment column in Shrine's format.
post "/webhooks/transloadit" do
Shrine::Attacher.transloadit_save(params)
end
Backgrounding
Even though submitting a Transloadit assembly doesn't require any uploading, it still does two HTTP requests, so you might want to put it into a background job. This plugin naturally hooks onto Shrine's backgrounding plugin:
Shrine::Attacher.promote { |data| TransloaditJob.perform_async(data) }
class TransloaditJob
include Sidekiq::Worker
def perform(data)
Shrine::Attacher.transloadit_process(data)
end
end
Tracking progress
When an assembly is submitted, Transloadit returns a lot of useful information about the status of that assembly, which the plugin saves to the cached attachment's metadata.
response = photo.image.transloadit_response
response.body #=>
# {
# "ok" => "ASSEMBLY_EXECUTING",
# "message" => "The assembly is currently being executed.",
# "assembly_id" => "83d07d10414011e68cc8c5df79919836",
# "assembly_url" => "http://api2.janani.transloadit.com/assemblies/83d07d10414011e68cc8c5df79919836",
# "execution_start" => "2016/07/03 17:06:42 GMT",
# "execution_duration" => 2.113,
# "params" => "{\"steps\":{...}}",
# ...
# }
At an point during the execution of the assembly you can refresh this information:
response.finished? #=> false
response.reload!
response.finished? #=> true
Metadata
For each processed file Transloadit also extracts a great deal of useful metadata. When the Transloadit processing is finished and the results are saved as a Shrine attachment, this metadata will be automatically used to populate the attachment's metadata.
Additionally the Transloadit's metadata hash will be saved in an additional metadata key, so that you can access any other values:
photo = Photo.create(image: image_file)
photo.image.metadata["transloadit"] #=>
# {
# "date_recorded" => "2013/09/04 08:03:39",
# "date_file_created" => "2013/09/04 12:03:39 GMT",
# "date_file_modified" => "2016/07/11 02:27:11 GMT",
# "aspect_ratio" => "1.504",
# "city" => "Decatur",
# "state" => "Georgia",
# "country" => "United States",
# "latitude" => 33.77519301,
# "longitude" => -84.295608,
# "orientation" => "Horizontal (normal)",
# "colorspace" => "RGB",
# "average_color" => "#8b8688",
# ...
# }
Import & Export
Every TransloaditFile
needs to have an import and an export step. This plugin
automatically generates those steps for you:
transloadit_file(io)
# is equivalent to
file = transloadit_file
file.add_step(transloadit_import_step("import", io))
transloadit_assembly({original: original, thumb: thumb})
# is equivalent to
transloadit_assembly({
original: original.add_step(transloadit_export_step("export_original")),
thumb: thumb.add_step(transloadit_export_step("export_thumb")),
})
If you want/need to generate these steps yourself, you can just use the expanded forms.
Transloadit gem
If you want to have complete control over how steps are generated, you can just
use the transloadit gem directly. This plugin doesn't care how you generate
your steps, it only requires you to return an instance of
Transloadit::Assembly
.
class MyUploader < Shrine
def transloadit_process(io, context)
# build options
transloadit #=> #<Transloadit>
transloadit.assembly(options)
end
end
The import/export helper methods simply generate a Transloadit::Step
object,
and you can pass additional options:
class MyUploader < Shrinee
def transloadit_process(io, context)
transloadit_import_step("import", io) #=> #<Transloadit::Step>
transloadit_export_step("export", path: "mypath") #=> #<Transloadit::Step>
end
end
The #add_step
method for TransloaditFile
is just a convenient way to add
steps where :use
is automatically set to previous step.
Testing
In development or test environment you cannot use webhooks, because Transloadit as an external service cannot access your localhost. In this case you can just do polling:
class MyUploader < Shrine
def transloadit_process(io, context)
# ...
if ENV["RACK_ENV"] == "production"
notify_url = "https://myapp.com/webhooks/transloadit"
else
# In development we cannot receive webhooks, because Transloadit as an
# external service cannot reach our localhost.
end
transloadit_assembly(files, context: context, notify_url: notify_url)
end
end
class TransloaditJob
include Sidekiq::Worker
def perform(data)
attacher = Shrine::Attacher.transloadit_process(data)
# Webhooks won't work in development, so we can just use polling.
unless ENV["RACK_ENV"] == "production"
response = attacher.get.transloadit_response
until response.finished?
sleep 1
response.reload!
end
attacher.transloadit_save(response.body)
end
end
end
Contributing
Before you can run tests, you need to first create an .env
file in the
project root containing your Transloadit and Amazon S3 credentials:
# .env
TRANSLOADIT_AUTH_KEY="..."
TRANSLOADIT_AUTH_SECRET="..."
S3_BUCKET="..."
S3_REGION="..."
S3_ACCESS_KEY_ID="..."
S3_SECRET_ACCESS_KEY="..."
Afterwards you can run the tests:
$ bundle exec rake test