File: README — Documentation for ffi-rzmq (0.5.1)

ffi-rzmq

by Chuck Remes
http://www.zeromq.org/bindings:ruby-ffi

DESCRIPTION:

This gem wraps the ZeroMQ networking library using the ruby FFI (foreign function interface). It’s a pure ruby wrapper so this gem can be loaded and run by any ruby runtime that supports FFI. Right now that means MRI 1.9.x and JRuby.

The impetus behind this library was to provide support for ZeroMQ in JRuby which has native threads. Unlike MRI, MacRuby, IronRuby and Rubinius which all have a GIL, JRuby allows for threaded access to ruby code from outside extensions. ZeroMQ is heavily threaded, so until the other runtimes remove their GIL, JRuby will likely be the best environment to run this library.

PERFORMANCE

Using FFI introduces some minimal overhead. In my latest benchmarks, I was unable to detect any measurable performance drop due to FFI regardless of which ruby runtime was tested. JRuby had the best overall performance (with –server) once it warmed up. MRI behaved quite well too and has a much lower memory footprint than JRuby (use the trunk version of the FFI bindings to fix several threading issues affecting MRI).

Due to odd interactions between 0mq threads and the GIL (Giant Interpreter Lock), I recommend using JRuby 1.5.1 or later. JRuby has no GIL.

The hope is that in a multi-threaded environment that JRuby’s native threads and lack of GIL will provide the best ZeroMQ performance using the ruby language.

Unfortunately, there is really no reasonable way to support zero-copy using Ruby. Any time data needs to be accessible by the Ruby runtime, it must be copied out of native memory to the Ruby heap. The same is true for the reverse. I am investigating ways to “pin” primitive arrays in memory for Rubinius and JRuby to achieve zero-copy, but that is a ways off.

FEATURES/PROBLEMS:

This gem is brand new and has minimal tests. I’m certain there are a ton of bugs, so please open issues for them here or fork this project, fix them, and send me a pull request.

Running this gem with MRI 1.8.x will likely exhibit a LOT of problems in the form of hangs, segmentation faults and bus errors. The threading support in MRI 1.8.x is irrevocably broken (according to the brains behind the Ruby FFI project) so tread carefully.

Running this gem with MRI 1.9.x may also exhibit some hangs particularly at shutdown. This is due to a threading interaction between the 0mq library and MRI’s threads. Apparently MRI does not “adopt” external threads when they make a call into the MRI runtime so some deadlocks may occur due to the GIL not getting released. Fixes for this should be sent to ruby-core.

All features are implemented with the exception of the 0mq devices (forwarder, queue, streamer). For implementations of these devices, see the 0mq Ruby reactor project:

http://github.com/chuckremes/zmqmachine

SYNOPSIS:

Client code:

require 'rubygems'
require 'ffi-rzmq'

if ARGV.length < 3
  puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
  exit
end

bind_to = ARGV[0]
message_size = ARGV[1].to_i
roundtrip_count = ARGV[2].to_i

ctx = ZMQ::Context.new 1
s = ctx.socket ZMQ::REP
s.setsockopt(ZMQ::HWM, 100)
s.bind(bind_to)

roundtrip_count.times do
  msg = s.recv_string 0
  raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
  s.send_string msg, 0
end

# give the lib time to flush any remaining messages
sleep 1

Server code:

require 'rubygems'
require 'ffi-rzmq'

if ARGV.length < 3
  puts "usage: remote_lat <connect-to> <message-size> <roundtrip-count>"
  exit
end

connect_to = ARGV[0]
message_size = ARGV[1].to_i
roundtrip_count = ARGV[2].to_i

ctx = ZMQ::Context.new 1
s = ctx.socket ZMQ::REQ
s.connect(connect_to)

msg = "#{ '3' * message_size }"

start_time = Time.now

roundtrip_count.times do
  s.send_string msg, 0
  msg = s.recv_string 0
  raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
end

REQUIREMENTS:

* 0mq 2.0.7 or 2.0.8

The ZeroMQ library must be installed on your system in a well-known location like /usr/local/lib. This is the default for new ZeroMQ installs.

Future releases may include the library as a C extension built at time of installation.

* ffi (> 0.6.3)

Install the current master version of FFI that fixes several threading problems under MRI 1.9.x. Unfortunately, MRI 1.8.x is irretrievably broken so it isn’t recommended if you plan to use threads or callbacks.

Code and installation instructions can be found on github:

http://github.com/ffi/ffi

INSTALL:

A full gem has been released to rubygems.org as of release 0.5.0. Make sure the ZeroMQ library is already installed on your system.

% gem install ffi-rzmq # should grab the latest release

To build from git master:

% git clone git://github.com/chuckremes/ffi-rzmq
% cd ffi-rzmq
% gem build ffi-rzmq.gemspec
% gem install ffi-rzmq-*.gem

LICENSE:

(The MIT License)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ‘Software’), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.