Barbeque Build Status

Job queue system to run job with Docker

Project Status

Barbeque is under development but already used on production at Cookpad.
Documentation is work in progress.

What's Barbeque?

Barbeque is a job queue system that consists of:

  • Web console to manage jobs
  • Web API to queue a job
  • Worker to execute a job

A job for Barbeque is a command you configured on web console. A message serialized by JSON and a job name are given to the command when performed. In Barbeque worker, they are done on Docker container.

Why Barbeque?

  • You can achieve job-level auto scaling using tools like Amazon ECS EC2 Auto Scaling group
    • For Amazon ECS, Barbeque has Hako runner
  • You don't have to manage infrastructure for each application like Resque or Sidekiq

For details, see Scalable Job Queue System Built with Docker // Speaker Deck.

Deployment

Web API & console

Install barbeque.gem to an empty Rails app and mount Barbeque::Engine. And deploy it as you like.

You also need to prepare MySQL, Amazon SQS and Amazon S3.

Worker

$ rake barbeque:worker BARBEQUE_QUEUE=default

Usage

Web API documentation is available at doc/toc.md.

Ruby

barbeque_client.gem has API client and ActiveJob integration.

License

The gem is available as open source under the terms of the MIT License.