Jobs Autoscaling

This inst-jobs plugin enables autoscaling of job worker pools using AWS Autoscaling Groups.

Installation

Add this line to your application's Gemfile:

gem 'inst-jobs-autoscaling'

Usage

Before using the plugin you'll need to configure your jobs workers to run as an AWS ASG. This typically means creating an AMI containing your application's code, and then setting scaleup/scaledown actions based on either CPU or job queue length.

Using this plugin requires configuring inst-jobs to use the ParentProcess WorkQueue implementation, which is not currently the default. Add to your delayed_jobs.yml file:

production:
  work_queue: parent_process
  workers: ...

If jobs aren't configured to use parent_process this plugin will still run, but do nothing. This allows easily switching between work queue implementations while parent_process is still experimental.

To enable the plugin you'll need to provide the ASG name to the app via an ENV var or other means, and add a line in an application initializer:

require 'jobs_autoscaling'
action = JobsAutoscaling::AwsAction.new(asg_name: "my_asg_name")
autoscaler = JobsAutoscaling::Monitor.new(action: action)
autoscaler.activate!

To just log what actions would be taken, rather than make actual ASG API calls, use the LoggerAction:

action = JobsAutoscaling::LoggerAction.new

License

The gem is available as open source under the terms of the MIT License.