Rocket Job
Ruby's missing batch system
Checkout https://rocketjob.io/
Documentation
Support
- Questions? Join the chat room on Gitter for rocketjob support
- Report bugs
Rocket Job 4
Rocket Job Pro is now open sourced and included within Rocket Job.
The RocketJob::Batch
plugin now adds batch processing capabilites to break up a single task into many
concurrent workers processing slices of the entire job at the same time.
Example:
class MyJob < RocketJob::Job
include RocketJob::Batch
self.description = "Reverse names"
self.destroy_on_complete = false
self.collect_output = true
# Method to call by all available workers at the same time.
# Reverse the characters for each line:
def perform(line)
line.reverse
end
end
Upload a file for processing, for example names.csv
which could contain:
jack
jane
bill
john
blake
chris
dave
marc
To queue the above job for processing:
job = MyJob.new
job.upload('names.csv')
job.save!
Once the job has completed, download the results into a file:
job.download('names_reversed.csv')
Contributing to the documentation
To contribute to the documentation it is as easy as forking the repository and then editing the markdown pages directly via the github web interface.
For more complex documentation changes checkout the source code locally.
Local checkout
- Fork the repository in github.
- Checkout your fork of the source code locally.
- Install Jekyll ~~~ cd docs bundle update ~~~
- Run Jekyll web server: ~~~ jekyll serve ~~~
- Open a web browser to view the local documentation: http://127.0.0.1:4000
- Edit the files in the
/docs
folder. - Refresh the page to see the changes.
Once the changes are complete, submit a github pull request.
Upgrading to V3
V3 replaces MongoMapper with Mongoid which supports the latest MongoDB Ruby client driver.
Upgrading Mongo Config file
Replace mongo.yml
with mongoid.yml
.
Start with the sample mongoid.yml.
For more information on the new Mongoid config file.
Note: The rocketjob
and rocketjob_slices
clients in the above mongoid.yml
file are required.
Other changes
Arguments are no longer supported, use fields for defining all named arguments for a job.
Replace usages of
rocket_job do
to set default values:
rocket_job do |job|
job.priority = 25
end
With:
self.priority = 25
- Replace
key
withfield
when adding attributes to a job:
key :inquiry_defaults, Hash
With:
field :inquiry_defaults, type: Hash, default: {}
- Replace usage of
public_rocket_job_properties
with theuser_editable
option:
field :priority, type: Integer, default: 50, user_editable: true
Ruby Support
Rocket Job is tested and supported on the following Ruby platforms:
- Ruby 2.1, 2.2, 2.3, 2.4, and above
- JRuby 9.0.5 and above
Dependencies
- MongoDB
- Persists job information.
- Version 2.7 or greater.
- Semantic Logger
- Highly concurrent scalable logging.
Versioning
This project uses Semantic Versioning.