Thursday, June 6, 2013

ZeroMQ, HWM, and INPROC

I have been banging my head pretty hard for the past 2 days using ZeroMQ with a combination of inproc transports and HWM. In my scenario I have an inproc ZMQ_PUSH socket pushing and an inproc ZMQ_PULL reading from the pipe. The client (pusher) blocked somewhere between 1 and 2k messages and no matter what I set it's ZMQ_HWM to it just kept blocking. As the project I'm working on requires something like a dynamic HWM I wrote a custom implementation of HWM and deal with them in the application and just wanted to disable the built-in version. Before filing a bug report I decided to take a glance into the github repo.. and that's where it all made sense, here's a snippet from this source:
        // The total HWM for an inproc connection should be the sum of
        // the binder's HWM and the connector's HWM.
        int sndhwm = 0;
        if (options.sndhwm != 0 && peer.options.rcvhwm != 0)
            sndhwm = options.sndhwm + peer.options.rcvhwm;
        int rcvhwm = 0;
        if (options.rcvhwm != 0 && peer.options.sndhwm != 0)
            rcvhwm = options.rcvhwm + peer.options.sndhwm;
And it's even clearly stated in the comments just above, I need to set the HWM to 0 on both the sender AND the receiver :) Moral of the story is: if you need to set HWM limits on inproc sockets, you have to set the limits on both sides!