File: README — Documentation for ap4r (0.3.0)

What is AP4R?

AP4R, Asynchronous Processing for Ruby, is the implementation of reliable asynchronous message processing. It provides message queuing, and message dispatching. Using asynchronous processing, we can cut down turn-around-time of web applications by queuing, or can utilize more machine power by load-balancing. Also AP4R nicely ties with your Ruby on Rails applications. See Hello World sample application from rubyforge.

rubyforge.org/projects/ap4r/

Features

Business logics can be implemented as simple Web applications, or ruby code, whether it’s called asynchronously or synchronously.
Asynchronous messaging is reliable by RDBMS persistence (now MySQL only) or file persistence, under the favor of reliable-msg.
Load balancing over multiple AP4R processes on single/multiple servers is supported.
Asynchronous logics are called via various protocols, such as XML-RPC, SOAP, HTTP PUT, and more.
Using store and forward function, at-least-omce QoS level is provided.

Typical process flow

A client(e.g. a web browser) makes a request to a web server (Apache, Lighttpd, etc…).
A rails application (a synchronous logic) is executed on mongrel via mod_proxy or something.
At the last of the synchronous logic, message(s) are put to AP4R (AP4R provides a helper).
Once the synchronous logic is done, the clients receives a response immediately.
AP4R queues the message, and requests it to the web server asynchronously.
An asynchronous logic, implemented as usual rails action, is executed.

Installation

Use RubyGems command.

$ sudo gem install ap4r --include-dependencies

Working directory

Create your working directory (ex. my_work) wherever you want.

$ ap4r_setup my_work
$ cd my_work

Its structure is as follows.

my_work
 +-- config
 +-- log
 +-- script
 +-- tmp

Message Persistence

AP4R uses reliable-msg for message persistence. It enables disk or RDBMS persistence. See references for details of reliable-msg. Install MySQL if you choose MySQL persistence. We recommend to install MySQL/Ruby library to connect to MySQL. For convinience, Ruby/MySQL bundled in the ActiveRecord gem can be used. Create a table in your database. (If you use topics via reliable-msg, create another slightly different table.)

CREATE TABLE `reliable_msg_queues` (
  `id` varchar(255) NOT NULL default '',
  `queue` varchar(255) NOT NULL default '',
  `headers` text NOT NULL,
  `object` blob NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=binary;

Configuration file

A command ap4r_setup have created a template configuration file (my_work/config/queues.cfg).

--- 
store: 
  type: mysql
  host: localhost
  database: test
  username: test
  password: 
drb: 
  host: 
  port: 6438
  acl: allow 127.0.0.1
dispatchers:
  -
    targets: queue.*
    threads: 1
#carriers:
#  - 
#    source_uri: druby://another.host.local:6438
#    threads: 1

queues.cfg has four parts.

persistent role (store: )
- Set database or file information.
message listener (drb:)
- Set IP, and port number which are used by clients.
- acl = access control list, exmaple below.
  
  allow 127.0.0.1 allow 10.0.0.0/8 deny 192.168.0.1
asynchronous process invokers (dispatchers:)
- Dispatchers handle messages in queues specified by targets,
- with as many threads as specified by threads.
- Each thread waits (is blocked) until the invocation returns.
message routing (carriers:) # EXPERIMENTAL
- Carriers get messages from an AP4R server specified by source_uri,
- with as many threads as specified by threads,
- and put messages to a local AP4R server.

Monitoring

Future plan: connections with monitoring tools such as Cacti and ZABBIX.

Cacti
- www.cacti.net/
ZABBIX
- www.zabbix.org/

Sample - HelloWorld -

There is an asynchronous application sample with file persistence, where a synchronous logic outputs “Hello” to a file, and an asynchronous logic appends “World”. Once the synchronous logic has done, a client (a web browser) displays response, and there is only “Hello” in a file. Since the asynchronous logic sleeps ten seconds and appends to the file, wait briefly and you can see whole “HelloWorld” in the file

Install Ruby, Rails and AP4R and configure AP4R in reference above sections
- No need for MySQL.
Make sample application available
- If you have an SVN client,
```
$ svn co svn://rubyforge.org/var/ap4r/tags/ap4r-0.2.0/sample [some directory]
```
- Unless
  1. download the sample from RubyForge
```
http://rubyforge.org/projects/ap4r/
filename: HelloWorld.tar.gz
```
  2. and extract to an appropricate directory(ex. HelloWorld ). There are app/, components/,… directories under HelloWorld.
Start WEBRick.
- In a command line
```
$ cd HelloWorld
$ ruby script\server
```
- If you see Welcome screen at localhost:3000, it’s ok.
Start AP4R.
- In another command line,
- start AP4R with specifying the configuraion file.
```
$ cd my_work
$ ruby script/start -c config/queues_disk.cfg
```
Execute a synchronous logic.
- localhost:3000/sync_hello/execute
- A file HelloWorld.txt is creted under HelloWorld/ which contains a word “Hello”.
- execute_via_soap, execute_via_http and execute_with_saf actions are also available.
Confirm the execution of an asynchronous logic.
- Wait ten seconds,
- and look into the file HelloWorld.txt again, there must be “HelloWorld”.
Think of application to your real application
- This mechanism can work in general applications.
  - An order acceptance logic instead of printing “Hello”.
  - Asynchronous logic of auto shipping order or accounting instead of appending “World”.

Future Plan

only-once QoS
DLQ recovery
flow volume control, load balancing
7x24 support (e.g. rolling database tables)
monitoring (e.g. thread status, web frontend)
Coordination with Ruby on Rails, such as development/testing lifesycle support.