Parallel batch distributes your batches across servers.

It is based on ActiveRecord and relies on the database to ensure records are processed only once.

To get started include Parallel batch into your Gemfile and run bundle install.

gem 'parallel_batch'

Then run the following migration to create the parallel_batches table.

class CreateParallelBatches < ActiveRecord::Migration
  def self.up
    create_table :parallel_batches do |t|
      t.string :type
      t.string :offset # It works either with integer or string IDs.
      t.timestamps
    end

    add_index :parallel_batches, :type, :unique: true
  end

  def self.down
    drop_table :parallel_batches
  end
end

Your parallel batch must be a sub class of ParallelBatch. It must implement scope and perform methods. Perform is called for each record selected by your defined scope.

# File: app/models/track_batch.rb
class TrackBatch < ParallelBatch
  def scope
    Track.not_encoded
  end

  def perform(track)
    track.encode_mp3
  end
end

For starting your batches SSH into your servers and run the following command.

TrackBatch.start(5) # Create 5 workers in their own process.

To Contribute:

1. Fork it

2. Create your feature branch (‘git checkout -b my-new-feature`)

3. Commit your changes (‘git commit -am ’Added some feature’‘)

4. Push to the branch (‘git push origin my-new-feature`)

5. Create new Pull Request