StagedEvent
StagedEvent is a Ruby implementation of the transactional outbox architectural pattern.
This gem is relevant for any process that wants to publish events to a set of subscribers. It automatically saves outgoing events as records in a database table, from which a separate background process reads, publishes, and (once publishing is confirmed) deletes them. StagedEvent also provides a listener to subscribe to and receive these events from other processes.
Staging events this way allows them to be persisted within the same database transaction as associated domain-specific records, guaranteeing at-least-once delivery and thus eventual consistency with the receiving system(s).
For now, StagedEvent makes the following assumptions:
- Events are defined as protobufs (and you have generated Ruby classes for them)
- Google Pub/Sub is the underyling messaging infrastructure
- You're using ActiveRecord (with a Postgres database >= version 9.5, due to the usage of the SQL
SKIP LOCKED
clause).
Installation
Add this line to your application's Gemfile:
gem "staged_event"
Then, in your application directory:
$ bundle install
$ rails generate staged_event:install
This will create an initializer at config/initializers/staged_event.rb
, which you should modify with settings specific to your application.
The installer will also generate a migration to create the database table for staged events waiting to be published. Remember to run it using:
rails db:migrate
Publishing Events
In order for events to actually be published, StagedEvent provides a rake task that should be kept running alongside your application server(s):
rake staged_event:publisher
Receiving Events
In order to receive events from publishers, StagedEvent provides a rake task that should be kept running alongside your application server(s):
rake staged_event:subscriber
In order to process incoming events, you define a callback in the staged_event initializer file.
Regenerating Ruby from protobufs
StagedEvent uses a one-off protobuf definition to serialize and deserialize events that are defined as protobufs. In case it becomes necessary to recreate the auto-generated ruby, the command for that (from the repository root) is:
protoc --ruby_out=./ "./lib/staged_event/event_envelope.proto"