ZeroMQ

Ahwulf sent me a link to an interesting little message-passing library, ZeroMQ (0MQ).  Lean, mean, frugal.

There are a host of language wrappers (C, C++, Lua, Perl, Python, Ruby, .NET etc) which makes it pretty handy for interop. A quick C++ example:

#include <zmq.hpp>

using namespace zmq ;

int main(int argc, const char* const argv[])
{
  // Create a ZMQ "Context" with one I/O thread.
  context_t ctx(1) ;

  // ZMQ_DOWNSTREAM creates a socket solely for sending messages to.
  socket_t outsock(ZMQ_DOWNSTREAM) ;
  // ZMQ_UPSTREAM creates a socekt solely for receiving messages.
  socket_t insock(ZMQ_UPSTREAM) ;

  static const char* const InprocessSocketName = "inproc://my-inproc-sock" ;
  outsock.bind(InprocessSocketName) ;
  insock.connect(InprocessSocketName) ;

  // Send a message to the down-stream thread.
  {
    // Allocate a message with 64 bytes of storage space.
    message_t outMessage(64) ;
    sprintf((char*)outMessage.data(), "Hello world") ;
    outsock.send(outMessage) ;
  }

  // Receive any messages from the up-stream thread.
  {
    message_t inMessage ;
    insock.recv(&inMessage) ;
    printf("Received [%s]\n", (const char*)inMessage.data()) ;
  }

  return 0 ;
}

At first glance, it looks like yet-another-socket-wrapper. But the simplicity of the API belies an elegant – but still fairly simple – underbelly.

0MQ provides a number of transports, the above demonstrates the in-process (inter-thread) communication. There are also an inter-process mechanism, tcp, udp and multicast transports.

Where it gets elegant is that 0MQ handles multiplexing for you. That is, N threads can listen to the same socket and receive messages, and 0MQ will handle fair distribution and return routing.

The socket types are a little confusing at first.

For tightly-coupled request/response protocols, they provide ZMQ_REQ (send -> recv flow), and ZMQ_REP (recv -> send flow).

You can also do “publish/subscribe”, with ZMQ_PUB and ZMQ_SUB. Subscribers can limit which messages they receive.

zmq::socket_t inSock(ZMQ_SUB) ; // Subscriber socket.
inSock.connect("tcp://127.0.0.1:23456") ; // Connect to the publisher
inSock.setsockopt(ZMQ_SUBSCRIBE, "prefix", 6) ; // Only receive messages starting with 'prefix'.

The ZMQ_UPSTREAM and ZMQ_DOWNSTREAM demonstrated in the top example are intended for data that is pipelined between stages of parallel processing.

There is also a dedicated one-to-one socket pair, ZMQ_PAIR which lets you arbitrarily send/receive.

ZMQ_REQ and ZMQ_REP are out: if you’re going to sit and wait for a reply, there’s not a lot of point in offloading the work.

OTOH, sometimes I do want a reply, which seemed to eliminate ZMQ_PUB/SUB and the STREAMs.

But, actually, the stream model works fine. You send offloads to the workers via a DOWN->UP pair, and then any responses you send back via another DOWN->UP pair.