rmb-rails: RESTful Message Beans for Ruby on Rails

Introduction

During a lengthy career in software development, and later as an enterprise architect, I saw a lot of message broker deployments in many large corporations. These systems were used primarily to knit together disparate computing ‘silos’ into a single fabric, as primarily an integration tool. The implementations varied significantly, some were well engineered and managed, others were basket cases of poor planning. One significant problem that was common in many of these deployments was the requirement to add broker subscriber behavior inside existing applications. Many of these applications were single threaded (often written in COBOL) and had no real ability to construct a thread to wait on some subscriber.receive method. This issue was resolved by a variety of means, many of which involved polling the subscriber queue periodically, picking up some or all of the messages waiting, and processing them through the application. Polling does not scale well. If the application had many worker instances, they all tended to compete with one another and together, consumed a lot of CPU time in polling loops. If you slowed down the polling frequency, then you got latency problems where messages sat in the queue when the readers were sleeping. The real solution was to construct some form of subscriber adapter that could inject messages into the applications’ standard incoming data stream. The Enterprise Java Message Bean is a good example of this kind of adapter. Message arrival on a designated queue or topic caused the activation of a Message Bean within the EJB container, allowing full participation in transactions within the container.

There is a real parallel between a Java EJB application and a Ruby-on-Rails application, and the Rails environment needs a method to get incoming message broker subscriptions injected directly into the main Rails data stream. So, how can we convert an MQ message into an http request? The rubygem rmb-rails addresses this issue.

Theory of Operation

As stated above, experience shows that polling a queue to pick up messages waiting there does not scale well. I decided to create an infrastructure where daemon processes, called listeners, can connect to a specified message broker, receive messages from that broker, and then forward them on to a standard Rails controller using a standard Rails protocol. In other words, the daemon listener processes act as browsers doing a form of file upload. The receiving rails controller is built on standard REST principles, and fits easily into any Rails application.

Component classes

module RMB

The rmb-rails gem is packaged within a single module, called RMB. The component classes are described below. See the rdoc for each class for a full description.

class RMB::Properties

The primary interface between the gem and the calling rails app consists of a multilevel hash. The hash is partially complete, but requires that several key value pairs be supplied.

class RMB::ListenerClient

This class implements the interface between the calling application and the daemon listeners.

class RMB::ListenerMain

This class, along with concrete Subscriber and Submitter subclasses, makes up the daemon listener process.

class RMB::Subscriber

This is an abstract superclass for all Subscriber classes. Every daemon has an instance of a Subscriber class acting as the ‘front end’ of the daemon, listening and blocking on the broker.

class RMB::Submitter

This is an abstract superclass for all Submitter classes. Every daemon has an instance of a Submitter class acting as the forwarding agent, taking messages received by the subscriber, packaging the message as an http request, and posts it to the specified controller/action pair.

class RMB::StompSubscriber

This is a concrete subclass of Subscriber, which uses the Stomp protocol to listen to an ActiveMQ message broker.

class RMB::MechanizeSubmitter

This is a concrete subclass of Submitter, which uses the WWW:Mechanize gem to package the message as a document and submit it to the controller.

Installation

You will need an ActiveMQ message broker to use this gem. See activemq.apache.org/ for download and configuration instructions.

$ gem sources -a gems.github.com (you only have to do this once) $ sudo gem install explainer-rmb-rails

or

$ sudo gem install rmb-rails (to get the same gem from rubyforge)

Usage

Add the following require statement to your code to get access to this gem:

require ‘rmb-rails’

Create an instance of RMB:ListenerClient to start/stop each listener daemon used. Get the RESTful-Message-Beans(github.com/explainer/RESTful-Message-Beans/tree/master) rails app, which provides a complete demo rails application using this gem.

Credits

ActiveMQ You can find the ActiveMQ message broker here.

Enterprise Recipes with Ruby and Rails A special thanks to Maik Schmidt for this useful book.

Jeweler gem Thanks to Josh Nichols for this useful tool.