CZTop

CZTop is a CZMQ binding for Ruby. It is based on czmq-ffi-gen, the generated low-level FFI binding of CZMQ and has a focus on being easy to use for Rubyists (POLS) and providing first class support for security mechanisms (like CURVE).

You might wanna check out cztop-patterns. It's still very new, but will contain some reusable patterns described in the Zguide.

Build Status on Travis CI Code Climate Inline docs Dependency Status Coverage Status ISC License

Goals

Here are some some of the goals I have/had in mind for this library:

  • [x] as easy as possible, Ruby-esque API
  • [x] first class support for security (CURVE mechanism)
    • [x] including handling of certificates
  • [x] support MRI, Rubinius, and JRuby
  • [x] high-quality API documentation
  • [x] 100% test coverage
  • [x] provide a portable Z85 implementation
  • [x] use it to replace the Celluloid::ZMQ part of Celluloid
  • [ ] implement some of the missing (CZMQ based) Ruby examples in the ZMQ Guide

Overview

Class Hierarchy

Here's an overview of the core classes:

More information in the API documentation.

Features

  • Ruby-like API
    • method names
    • sending a message via a socket is done with Socket#<<
      • socket << "simple message"
      • socket << ["multi", "frame", "message"]
    • #x= methods instead of #set_x (e.g. socket options)
    • #[] where it makes sense (e.g. on a Message, Config, or Certificate)
    • no manual error checking needed
    • if there's an error, an appropriate exception is raised
    • of course, no manual dealing with the ZMQ context
  • easy security
    • use Socket#CURVE_server!(cert) on the server
    • and Socket#CURVE_client!(client_cert, server_cert) on the client
  • socket types as Ruby classes
    • no need to manually pass some constant
    • but you can: CZTop::Socket.new_by_type(:REP)
    • e.g. #subscribe only exists on CZTop::Socket::SUB
  • DRAFT API ready
    • CLIENT/SERVER/RADIO/DISH/SCATTER/GATHER and other DRAFT methods are supported if the libraries (ZMQ/CZMQ) have been compiled with DRAFT APIs enabled (--enable-drafts)
    • there is #routing_id and #routing_id= on the following classes:
    • CZTop::Message
    • CZTop::Frame
    • there is #group and #group= on CZTop::Frame
  • ZMTP 3.1 heartbeat ready
    • socket.options.heartbeat_ivl = 2000
    • socket.options.heartbeat_timeout = 8000

Requirements

You'll need:

  • CZMQ >= 4.0.0
  • ZMQ >= 4.2.0

For security mechanisms like CURVE, it's recommended to use Libsodium. However, ZMQ can be compiled with tweetnacl enabled.

To install on OSX using homebrew, run:

$ brew install libsodium
$ brew install zmq  --HEAD --with-libsodium
$ brew install czmq --HEAD

If you're running Linux, go check this page to get more help. Make sure to install CZMQ, not only ZMQ.

Note: Currently (as of May 2016), when compiling ZMQ from master, it may be required to pass --enable-drafts to ./configure to make sure all the zmq_poller_*() functions are available. However, this doesn't seem to be the case on all systems.

Supported Ruby versions

See .travis.yml for a list of Ruby versions against which CZTop is tested.

At the time of writing, these include:

  • MRI (2.3, 2.2)
  • Rubinius (HEAD)
  • JRuby 9000 (HEAD)

Installation

To use this gem, add this line to your application's Gemfile:

gem 'cztop'

And then execute:

$ bundle

Or install it yourself as:

$ gem install cztop

Documentation

The API should be fairly straight-forward to anyone who is familiar with CZMQ and Ruby. The following API documentation is currently available:

Feel free to start a wiki page.

Performance

Performance should be pretty okay since this is based on czmq-ffi-gen, which is reasonably thin. CZTop is basically only a convenience layer on top, with some nice error checking. But hey, it's Ruby. Don't expect 5M messages per second with a latency of 3us.

The measured latency on my laptop ranges from ~20us to ~60us per message for 1kb messages, depending on whether transport is inproc, IPC, or TCP/IP.

Make sure you check out the perf directory for latency and throughput measurement scripts.

Usage

See the examples directory for some examples. Here's a very simple one:

rep.rb:

#!/usr/bin/env ruby
require 'cztop'

# create and bind socket
socket = CZTop::Socket::REP.new("ipc:///tmp/req_rep_example")
puts "<<< Socket bound to #{socket.last_endpoint.inspect}"

# Simply echo every message, with every frame String#upcase'd.
while msg = socket.receive
  puts "<<< #{msg.to_a.inspect}"
  socket << msg.to_a.map(&:upcase)
end

req.rb:

#!/usr/bin/env ruby
require 'cztop'

# connect
socket = CZTop::Socket::REQ.new("ipc:///tmp/req_rep_example")
puts ">>> Socket connected."

# simple string
socket << "foobar"
msg = socket.receive
puts ">>> #{msg.to_a.inspect}"

# multi frame message as array
socket << %w[foo bar baz]
msg = socket.receive
puts ">>> #{msg.to_a.inspect}"

# manually instantiating a Message
msg = CZTop::Message.new("bla")
msg << "another frame" # append a frame
socket << msg
msg = socket.receive
puts ">>> #{msg.to_a.inspect}"

##
# This will send 20 additional messages:
#
#   $ ./req.rb 20
#
if ARGV.first
  ARGV.first.to_i.times do
    socket << ["fooooooooo", "baaaaaar"]
    puts ">>> " + socket.receive.to_a.inspect
  end
end

Running it

$ ./rep.rb & ./req.rb 3
[3] 35321
>>> Socket connected.
<<< Socket bound to "ipc:///tmp/req_rep_example"
<<< ["foobar"]
>>> ["FOOBAR"]
<<< ["foo", "bar", "baz"]
>>> ["FOO", "BAR", "BAZ"]
<<< ["bla", "another frame"]
>>> ["BLA", "ANOTHER FRAME"]
<<< ["fooooooooo", "baaaaaar"]
>>> ["FOOOOOOOOO", "BAAAAAAR"]
<<< ["fooooooooo", "baaaaaar"]
>>> ["FOOOOOOOOO", "BAAAAAAR"]
<<< ["fooooooooo", "baaaaaar"]
>>> ["FOOOOOOOOO", "BAAAAAAR"]
$

TODO

  • [x] pack generated code into its own gem (czmq-ffi-gen)
  • think of a neat Ruby API, including:
    • [x] Actor
    • [x] Beacon
    • [x] Certificate
    • [x] Socket
    • [50%] access to all socket options
    • [x] Security mechanisms
    • [x] Message
    • [x] Frame
    • [x] enumerable Frames
    • [x] Authenticator
    • [x] Loop
    • [x] Monitor
    • [x] Poller
    • [x] Proxy
    • [x] Config
    • [x] Z85
  • write the missing XML API files in CZMQ
    • [x] zarmour.xml
    • [x] zconfig.xml
    • [x] zsock_option.xml
    • [x] zcert.xml
    • [x] zcertstore.xml
  • [x] check availability of libsodium within CZTop
  • [x] read error strings for exceptions where appropriate (zmq_strerror)
  • [x] add support for ZMTP 3.1 heartbeats in CZMQ
  • [x] add padded variant of Z85
  • add more examples
    • [x] simple REQ/REP
    • [x] Taxy System with CURVE security and heartbeating
    • [x] change from ROUTER/DEALER to SERVER/CLIENT
    • [x] Actor with Ruby block
    • [ ] PUSH/PULL
    • [ ] PUB/SUB
  • [ ] add performance benchmarks
    • [x] inproc latency
    • [x] inproc throughput
    • [x] local/remote latency
    • [ ] local/remote throughput
    • see perf directory
  • [x] support older versions of ZMQ
    • [x] ZMQ HEAD
    • [x] tested on CI
    • [x] ZMQ 4.1
    • [x] tested on CI
    • as of April 2016, this isn't the case anymore
    • [x] ZMQ 4.0
    • [x] tested on CI
    • as of March 2016, this isn't the case anymore
    • [ ] ZMQ 3.2
    • too big a pain (d5172ab)
  • [x] support multiple versions of CZMQ
    • [x] CZMQ HEAD
    • [x] test on CI
    • [x] CZMQ 3.0.2 (current stable)
    • no zcert_meta_unset() (zeromq/czmq#1246)
      • [x] adapt czmq-ffi-gen so it doesn't raise while attach_function
    • no zproc(especially no zproc_has_curve())
      • [x] adapt czmq-ffi-gen so it doesn't raise while attach_function, attach zsys_has_curve() instead (under same name)
    • [x] adapt test suite to skip affected test examples
    • [x] test on CI
    • as of March, 2016, this isn't the case anymore
  • [x] port Poller to zmq_poll()
    • backwards compatible (#add_reader, #add_writer, #wait behave the same)
    • but in addition, it has #readables and #writables which return arrays of sockets
    • could then be used in Celluloid::ZMQ
  • [ ] add Message#to_s
    • return frame as string, if there's only one frame
    • raise if there are multiple frames
    • only safe to use on messages from SERVER/CLIENT sockets
    • single-part messages are the future
    • not sure yet
  • [x] get rid of Loop
    • cannot handle CLIENT/SERVER sockets
    • there good timer gems for Ruby
    • Poller can be used to embed in an existing event loop (Celluloid), or make your own trivial one
  • [x] provide z85encode and z85decode utilities
    • can be used in a pipeline (limited memory usage)
    • reusable interface: Z85::Pipe

Reasons

Why another CZMQ Ruby binding? Here is a list of existing projects I found and the issues with them, from my point of view:

Furthermore, I knew about the generated low-level Ruby FFI binding in the zeromq/czmq repository. I wanted to make use of it because I love that it's generated (and thus, most likely correct and up-to-date). Unfortunately, it was in pretty bad shape and missing a few CZMQ classes.

So I decided to improve the quality and usability of the binding and add the missing classes. The result is czmq-ffi-gen which provides a solid foundation for CZTop.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/paddor/cztop.

To run the tests before/after you made any changes to the source and have created a test case for it, use rake spec.

License

The gem is available as open source under the terms of the ISC License. See the LICENSE file.