Jobs Autoscaling
This inst-jobs plugin enables autoscaling of job worker pools using AWS Autoscaling Groups.
Installation
Add this line to your application's Gemfile:
gem 'inst-jobs-autoscaling'
Usage
Before using the plugin you'll need to configure your jobs workers to run as an AWS ASG. This typically means creating an AMI containing your application's code, and then setting scaleup/scaledown actions based on either CPU or job queue length.
Using this plugin requires configuring inst-jobs to use the ParentProcess
WorkQueue implementation, which is not currently the default. Add to your
delayed_jobs.yml
file:
production:
work_queue: parent_process
workers: ...
If jobs aren't configured to use parent_process
this plugin will still run,
but do nothing. This allows easily switching between work queue implementations
while parent_process
is still experimental.
To enable the plugin you'll need to provide the ASG name to the app via an ENV var or other means, and add a line in an application initializer:
require 'jobs_autoscaling'
action = JobsAutoscaling::AwsAction.new(asg_name: "my_asg_name")
autoscaler = JobsAutoscaling::Monitor.new(action: action)
autoscaler.activate!
To just log what actions would be taken, rather than make actual ASG API calls,
use the LoggerAction
:
action = JobsAutoscaling::LoggerAction.new
License
The gem is available as open source under the terms of the MIT License.