CZTop
CZTop is a CZMQ binding for Ruby. It is based on czmq-ffi-gen, the generated low-level FFI binding of CZMQ and has a focus on being easy to use for Rubyists (POLS) and providing first class support for security mechanisms (like CURVE).
You might wanna check out cztop-patterns. It's still very new, but will contain some reusable patterns described in the Zguide.
Goals
Here are some some of the goals I have/had in mind for this library:
- [x] as easy as possible, Ruby-esque API
- [x] first class support for security (CURVE mechanism)
- [x] including handling of certificates
- [x] support MRI, Rubinius, and JRuby
- [x] high-quality API documentation
- [x] 100% test coverage
- [x] provide a portable Z85 implementation
- unlike fpesce/z85, which is a C extension
- [x] use it to replace the Celluloid::ZMQ part of Celluloid
- [ ] implement some of the missing (CZMQ based) Ruby examples in the ZMQ Guide
Overview
Class Hierarchy
Here's an overview of the core classes:
- CZTop
- Actor
- Authentiator
- Beacon
- Certificate
- CertStore
- Config
- Frame
- Message
- Monitor
- Metadata
- Proxy
- Poller (based on
zmq_poller_*()
functions) - Aggregated
- ZPoller
- Socket
- REQ < Socket
- REP < Socket
- ROUTER < Socket
- DEALER < Socket
- PUSH < Socket
- PULL < Socket
- PUB < Socket
- SUB < Socket
- XPUB < Socket
- XSUB < Socket
- PAIR < Socket
- STREAM < Socket
- CLIENT < Socket
- SERVER < Socket
- RADIO < Socket
- DISH < Socket
- SCATTER < Socket
- GATHER < Socket
- Z85
- Padded < Z85
- Pipe
- ZAP
More information in the API documentation.
Features
- Ruby-like API
- method names
- sending a message via a socket is done with
Socket#<<
socket << "simple message"
socket << ["multi", "frame", "message"]
#x=
methods instead of#set_x
(e.g. socket options)#[]
where it makes sense (e.g. on a Message, Config, or Certificate)- no manual error checking needed
- if there's an error, an appropriate exception is raised
- of course, no manual dealing with the ZMQ context
- easy security
- use
Socket#CURVE_server!(cert)
on the server - and
Socket#CURVE_client!(client_cert, server_cert)
on the client
- use
- socket types as Ruby classes
- no need to manually pass some constant
- but you can:
CZTop::Socket.new_by_type(:REP)
- e.g.
#subscribe
only exists on CZTop::Socket::SUB
- DRAFT API ready
- CLIENT/SERVER/RADIO/DISH/SCATTER/GATHER and other DRAFT methods are supported if the libraries (ZMQ/CZMQ) have been compiled with DRAFT APIs enabled (
--enable-drafts
) - there is
#routing_id
and#routing_id=
on the following classes: - CZTop::Message
- CZTop::Frame
- there is
#group
and#group=
on CZTop::Frame
- CLIENT/SERVER/RADIO/DISH/SCATTER/GATHER and other DRAFT methods are supported if the libraries (ZMQ/CZMQ) have been compiled with DRAFT APIs enabled (
- ZMTP 3.1 heartbeat ready
socket.options.heartbeat_ivl = 2000
socket.options.heartbeat_timeout = 8000
Requirements
You'll need:
- CZMQ >= 4.0.0
- ZMQ >= 4.2.0
For security mechanisms like CURVE, it's recommended to use Libsodium. However, ZMQ can be compiled with tweetnacl enabled.
To install on OSX using homebrew, run:
$ brew install libsodium
$ brew install zmq --HEAD --with-libsodium
$ brew install czmq --HEAD
If you're running Linux, go check this page to get more help. Make sure to install CZMQ, not only ZMQ.
Note: Currently (as of May 2016), when compiling ZMQ from master, it may
be required to pass --enable-drafts
to ./configure
to make sure all the
zmq_poller_*()
functions are available. However, this doesn't seem to be the
case on all systems.
Supported Ruby versions
See .travis.yml for a list of Ruby versions against which CZTop is tested.
At the time of writing, these include:
- MRI (2.3, 2.2)
- Rubinius (HEAD)
- JRuby 9000 (HEAD)
Installation
To use this gem, add this line to your application's Gemfile:
gem 'cztop'
And then execute:
$ bundle
Or install it yourself as:
$ gem install cztop
Documentation
The API should be fairly straight-forward to anyone who is familiar with CZMQ and Ruby. The following API documentation is currently available:
- YARD API documentation (HEAD)
- YARD API documentation (release)
Feel free to start a wiki page.
Performance
Performance should be pretty okay since this is based on czmq-ffi-gen, which is reasonably thin. CZTop is basically only a convenience layer on top, with some nice error checking. But hey, it's Ruby. Don't expect 5M messages per second with a latency of 3us.
The measured latency on my laptop ranges from ~20us to ~60us per message for 1kb messages, depending on whether transport is inproc, IPC, or TCP/IP.
Make sure you check out the perf directory for latency and throughput measurement scripts.
Usage
See the examples directory for some examples. Here's a very simple one:
rep.rb:
#!/usr/bin/env ruby
require 'cztop'
# create and bind socket
socket = CZTop::Socket::REP.new("ipc:///tmp/req_rep_example")
puts "<<< Socket bound to #{socket.last_endpoint.inspect}"
# Simply echo every message, with every frame String#upcase'd.
while msg = socket.receive
puts "<<< #{msg.to_a.inspect}"
socket << msg.to_a.map(&:upcase)
end
req.rb:
#!/usr/bin/env ruby
require 'cztop'
# connect
socket = CZTop::Socket::REQ.new("ipc:///tmp/req_rep_example")
puts ">>> Socket connected."
# simple string
socket << "foobar"
msg = socket.receive
puts ">>> #{msg.to_a.inspect}"
# multi frame message as array
socket << %w[foo bar baz]
msg = socket.receive
puts ">>> #{msg.to_a.inspect}"
# manually instantiating a Message
msg = CZTop::Message.new("bla")
msg << "another frame" # append a frame
socket << msg
msg = socket.receive
puts ">>> #{msg.to_a.inspect}"
##
# This will send 20 additional messages:
#
# $ ./req.rb 20
#
if ARGV.first
ARGV.first.to_i.times do
socket << ["fooooooooo", "baaaaaar"]
puts ">>> " + socket.receive.to_a.inspect
end
end
Running it
$ ./rep.rb & ./req.rb 3
[3] 35321
>>> Socket connected.
<<< Socket bound to "ipc:///tmp/req_rep_example"
<<< ["foobar"]
>>> ["FOOBAR"]
<<< ["foo", "bar", "baz"]
>>> ["FOO", "BAR", "BAZ"]
<<< ["bla", "another frame"]
>>> ["BLA", "ANOTHER FRAME"]
<<< ["fooooooooo", "baaaaaar"]
>>> ["FOOOOOOOOO", "BAAAAAAR"]
<<< ["fooooooooo", "baaaaaar"]
>>> ["FOOOOOOOOO", "BAAAAAAR"]
<<< ["fooooooooo", "baaaaaar"]
>>> ["FOOOOOOOOO", "BAAAAAAR"]
$
TODO
- [x] pack generated code into its own gem (czmq-ffi-gen)
- think of a neat Ruby API, including:
- [x] Actor
- [x] Beacon
- [x] Certificate
- [x] Socket
- [50%] access to all socket options
- [x] Security mechanisms
- [x] Message
- [x] Frame
- [x] enumerable Frames
- [x] Authenticator
- [x] Loop
- [x] Monitor
- [x] Poller
- [x] Proxy
- [x] Config
- [x] Z85
- write the missing XML API files in CZMQ
- [x] zarmour.xml
- [x] zconfig.xml
- [x] zsock_option.xml
- [x] zcert.xml
- [x] zcertstore.xml
- [x] check availability of libsodium within CZTop
- [x] read error strings for exceptions where appropriate (zmq_strerror)
- [x] add support for ZMTP 3.1 heartbeats in CZMQ
- [x] add padded variant of Z85
- add more examples
- [x] simple REQ/REP
- [x] Taxy System with CURVE security and heartbeating
- [x] change from ROUTER/DEALER to SERVER/CLIENT
- [x] Actor with Ruby block
- [ ] PUSH/PULL
- [ ] PUB/SUB
- [ ] add performance benchmarks
- [x] inproc latency
- [x] inproc throughput
- [x] local/remote latency
- [ ] local/remote throughput
- see perf directory
- [x] support older versions of ZMQ
- [x] ZMQ HEAD
- [x] tested on CI
- [x] ZMQ 4.1
- [x] tested on CI
- as of April 2016, this isn't the case anymore
- [x] ZMQ 4.0
- [x] tested on CI
- as of March 2016, this isn't the case anymore
- [ ] ZMQ 3.2
- too big a pain (d5172ab)
- [x] support multiple versions of CZMQ
- [x] CZMQ HEAD
- [x] test on CI
- [x] CZMQ 3.0.2 (current stable)
- no
zcert_meta_unset()
(zeromq/czmq#1246)- [x] adapt czmq-ffi-gen so it doesn't raise while
attach_function
- [x] adapt czmq-ffi-gen so it doesn't raise while
- no
zproc
(especially nozproc_has_curve()
)- [x] adapt czmq-ffi-gen so it doesn't raise while
attach_function
, attachzsys_has_curve()
instead (under same name)
- [x] adapt czmq-ffi-gen so it doesn't raise while
- [x] adapt test suite to skip affected test examples
- [x] test on CI
- as of March, 2016, this isn't the case anymore
- [x] port Poller to
zmq_poll()
- backwards compatible (
#add_reader
,#add_writer
,#wait
behave the same) - but in addition, it has
#readables
and#writables
which return arrays of sockets - could then be used in Celluloid::ZMQ
- backwards compatible (
- [ ] add
Message#to_s
- return frame as string, if there's only one frame
- raise if there are multiple frames
- only safe to use on messages from SERVER/CLIENT sockets
- single-part messages are the future
- not sure yet
- [x] get rid of Loop
- cannot handle CLIENT/SERVER sockets
- there good timer gems for Ruby
- Poller can be used to embed in an existing event loop (Celluloid), or make your own trivial one
- [x] provide
z85encode
andz85decode
utilities- can be used in a pipeline (limited memory usage)
- reusable interface:
Z85::Pipe
Reasons
Why another CZMQ Ruby binding? Here is a list of existing projects I found and the issues with them, from my point of view:
- Asmod4n/ruby-ffi-czmq
- outdated
- according to its author, it's an "abomination"
- methodmissing/rbczmq
- no support for security features (see methodmissing/rbczmq#28)
- no JRuby support (see methodmissing/rbczmq#48)
- doesn't feel like Ruby
- mtortonesi/ruby-czmq
- no tests
- no documentation
- outdated
- doesn't feel like Ruby
- chuckremes/ffi-rzmq
- support discontinued
- low level ZMQ gem, not CZMQ
Furthermore, I knew about the generated low-level Ruby FFI binding in the zeromq/czmq repository. I wanted to make use of it because I love that it's generated (and thus, most likely correct and up-to-date). Unfortunately, it was in pretty bad shape and missing a few CZMQ classes.
So I decided to improve the quality and usability of the binding and add the missing classes. The result is czmq-ffi-gen which provides a solid foundation for CZTop.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/paddor/cztop.
To run the tests before/after you made any changes to the source and have
created a test case for it, use rake spec
.
License
The gem is available as open source under the terms of the ISC License. See the LICENSE file.