Yesterday I pushed a change set for review that fixes an odd corner case for the Guam IMAP proxy/filter tool that was uncovered thanks to the Kolab Now Beta program which allows people to try out new exciting things before we inflict them upon the world at large. So first let me thank those who are using Kolab Now Beta and giving us great and valuable feedback before turning to the details of this neat little bug.
So the report was that IMAP proxying was breaking with iOS devices. But, and here's the intriguing bit, only when connecting using implicit TLS; connecting to the IMAP server normally and upgrading with STARTTLS worked fine. What gives?
In IMAP, commands sent to the server are expected to start with a string of characters which becomes the identifying tag for that transaction, usually taking the form of "TAG COMMAND ARGS". The tag can be whatever the client wants, though many clients just use a number that increases monotonically (1, 2, 3, 4, ...). The server will use that tag to prefix the success/failure response in the case of multi-line responses, or tag the response itself in the case of simpler one-line responses. This allows the client to match up the server response with the request and know when the server is indeed finished spewing bytes at it.
We looked at the network traffic and in that specific case iOS devices fragment the IMAP client call into one packet with the tag and one packet with the command. No other client does this, and even iOS devices do not do this when using the STARTTLS upgrade mechanism. As a small performance hack, I had allowed the assumption that the "TAG COMMAND" part of client messages would never be fragmented on the network. This prevented the need for buffering and other bookkeeping in the application within a specific critical code path. It was an assumption that was indeed not guaranteed, but the world appeared to be friendly and cooperating. After all, what application would send "4" in one network packet, and then "XLIST" in a completely separate one? Would it (the application, the socket implementation, ..) not compose this nicely into one little buffer and send it all at once? If so, what network topology would ever fragment a tiny packet of a few dozen bytes into one byte packets? Seemed safe enough, what could go wrong .. oh, those horrible words.
So thanks to one client in one particular configuration being particularly silly, if technically still within its rights, I had to introduce a bit of buffering when and where necessary. So I took the opportunity to do a little performance enhancement that was on my TODO while I was mucking about in there: tag/command parsing which is necessary and useful for rules to determine whether they care about the current state of the connection, is now both centralized and cached. So instead of happening twice for each incoming fragment of a command (in the common case), it now happens at most once per client command, and that will hold no matter how many rules are added to a ruleset.
So, one bug squashed, and one little performance enhancement, thanks to user feedback and the Kolab Now Beta program. As soon as the patch gets through code review, it should get pushed through packaging and deployed on Kolab Now Beta. Huzzah.a