Outboxer

Background

Typically in event driven Ruby on Rails applications:

1. a domain event model is created in an SQL database table
2. a sidekiq worker is queued to handle the domain event asynchronously

Problem

As these two operations span multiple database types (SQL and redis), they can not be combined into a single atomic operation using a transaction. If either step fails, inconsistencies can occur.

Solution

Outboxer is a simple Ruby on Rails implementation of the transactional outbox pattern: a well established solution to this problem. It ensures both operations succeed eventually, or both fail.

Getting started

Installation

1. Add the Outboxer gem to your application's Gemfile:

gem 'outboxer'

1. Install the Outboxer gem:

bundle install

1. Generate the migration and publisher files

bin/rails generate outboxer:install

Usage

1. Include Outboxer::Outboxable into your existing Message model

class Message < ApplicationRecord
  include Outboxer::Outboxable
end

2. Update the generated bin/publish block e.g.

Outboxer::Publisher.publish do |message:, logger:|
  logger.info("[#{message.id}] publishing")

  HardJob.perform_async({ "message_id" => message.id })

  logger.info("[#{message.id}] published")
end

3. Migrate the database

bin/rake db:migrate

4. Run the publisher

bin/publisher

Implementation

1. when an ActiveRecord model that includes Outbox::Outboxable is created, an unpublished Outboxer::Message is automatically created in the same transaction, with Outboxer::Message#message polymorphically assigned to the original model

2. When the publisher finds a new unpublished Outboxer::Message, it yields to a user-supplied block and then:

   • removes it if the task completes successfully
   • marks it as failed and records the error if there's a problem

To see all the parts working together in a single place, check out the publisher_spec.rb

Motivation

Outboxer was created to help high growth SAAS companies transition to event driven architecture quickly.

Specifically this means:

1. fast integration into existing Ruby on Rails applications (< 1 hour)
2. comprehensive documentation
3. high reliability in production environments
4. forever free to use in commerical applications (MIT licence)

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/fast-programmer/outboxer. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the Outboxer project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.