YoDuh's Stupid Outlook Bug

yoduh@getaclue.org

I (obviously) don't use outlook, but a customer called in with a problem and I managed to narrow it down to this. The instance where I found this in the wild was with a couple of spam messages, so someone has a broken spam mailer (suprise).

Affected clients: All known versions of outlook (I specifically tested: Outlook Express, Outlook 97, Outlook 98, Outlook 2000 and I applied appropriate service packs with no change in behavior). Other mail clients seem to handle this more intelligently (Netscape, and Eudora light at least are unaffected). It is possible that some clients will handle this differently (ie: skipping messages, etc..), but that is unverified. If anyone finds any other clients that are affected let me know.

Updates: Apparently outlook with service pack 2 usually gives an error but continues the download (skipping the second message). Some people are reporting that they didn't get an error with outlook 2000.

Synopsis: MUA's use a messages uidl as part of a pop transaction to check whether they have received the message before (most include other checks also). Usually the field is calculated when the message is first read by the popper. A lot of poppers will store the uidl for that message in an X-UIDL: header to avoid having to recalculate it every time the client checks thier e-mail. However, mail messages may arrive into a mailbox with a predefined X-UIDL: header. Most popper daemons will use this header instead of calculating a new header. If two messages arrive with the same (duplicate) X-UIDL: header outlook will freak on the initial negotiation where is sends a uidl command to the pop server. Most of the time Outlook will simply stop all activity at that point and you won't be able to download anything, very rarely (one in about 20 tries) it will give you an error and then let you continue the download.

Updates: Note that a popper could legitimately generate a dupelicate UIDL if there were two identical messages in the mailbox. QPopper uses basically the full headers to calc this so the possibility of duplicating one is close to nill there - whether or not other poppers are simular I can't say.

Sample broken mail box (on sendmail/popper system). Notice the duplicate X-UIDL headers, the messages don't have to be contiguous (as shown here), and normally a mail file will have a lot more headers in it (duh).

Diffs to qpopper to ignore the stored UID's. This is sort of a broken way to handle it, properly qpopper should be smarter about checking for duplicates in the file, or comparing a calculated UID against the stored one (although that defeats the purpose of storing it in the first place). Applying this patch may result in an increased load on busy mail servers as they will have to recalculate the uidl for each message every time the client accesses it. I'm sure that there are a lot of other (better) ways to solve it.

If you can't patch your popper you can fix the offending mailbox by simply stripping X-UIDL: headers from the mailbox (something like "grep -v ^X-UIDL:" mailbox > mailbox.tmp; mv mailbox.tmp mailbox" should work in most cases).

Being unable (and unwilling even if I did have it) to hack M$ code, and to lazy to patch any other pop daemons than qpopper, thats all the fixes I have for now. If anyone actually writes something for any other pop daemons or better patches for the above let me know & I'll link them here (with credit if desired).