A little ZMQ Gotcha. Probably my fault.
Posted on July 23rd, 2016
I'm working on a project where I need to quickly send some data from a Linux machine to a windows machine. Enter ZMQ.
Writing an appication in Python 3.5 and pyzmq 15.2.0 on a CentOS 7 server. I'm building a dictionary of objects which I use to serialize with jsonpickle into a JSON object to push over the network.
So I make my pickle:
and then send it over the wire:
and then afterwards disconnect
Now in some cases this worked perfectly fine the data hit the other side without issue, in some cases it got list.
Now the dictionary I am generating got quite large full of bits of processed data and I though intitally that I may be having an encoding issue so I set about debugging and removing part of my dictionary one by one to see if the system started working. I eventually narrowed it down to point key in the dictionary that when removed it would send the data without any issue what so ever. This reinforced my thought it was some weird encoding issue. I should have walked away from the computer at this point.
I got further confused when I logged the entire JSON output to console the application would always send. This output was obviously before s.close() was called.
Anyway, after lunch and my 4th espresso I finally realised the problem. I forgot that ZMQ is essentially fire and forget. What was happening was that calling send_string/close() and if the data was small enough (I.E without the key I thought was broken) it would send between the time it took send_string to finish and close() to start working. If the data was large enough that the data was still sending while close() got to work, the data wouldn't arrive.
I simply verified this with a sleep before s.close() and the data arrived every time without issue.
There is probably some note in the pyzmq and ZMQ docs that I have missed, but this got me for a few hours.