Saturday, October 16, 2010

Twisted is long way from practical use


I have been trying to get imap bodystructure working to please all the clients i could get my hands on. So far the twisted imap (10.1.0) performed badly esp when you need it to work with Outlook + Outlook Express + ThunderBird + IPhone + RoundCube.

Yeah, Imap as i mentioned many times even in Pycon Singapore, is a hell of protocol of email. Many flags like “recent” of “draft” are just outdate and should not even be used…

JP Calderone wrote in the source :

# XXX - This does not properly handle multipart messages
# BODYSTRUCTURE is obscenely complex and criminally under-documented.

I had to resort to write an imap unit test that will test the reply from 2 servers and compare them and make changes from there to fix the imap4 bodystructure response.

The unit test tool helped a lot, i had to tweak various replies, for eg handling application attachment when they appear in mime attributes.

@ I got it working correct and tested with all the clients mentioned above with all kinds of attachment and sizes.

Source: (a fixed imap4 reply)
('OK', [('3 (BODY[HEADER] {513}', 'User-Agent: Thunderbird (Windows/201
00228)\r\nTo: Undisclosed <>\r\nSubject: marcus\r\nRetu
rn-Path: <>\r\nReceived: from [] ([127.0.0
.1]) by (MailNow! 5); Sat, 16 Oct 2010 19:50:55 +080
0\r\nMime-Version: 1.0\r\nMessage-Id: <>\r\nF
rom: Marcus <>\r\nDate: Sat, 16 Oct 2010 16:09:38 +0800
\r\nContent-Type: multipart/mixed; boundary="------------03060008020007080806070
7"\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Type: text/plain; charset="ISO-88
59-1"; format="flowed"\r\nContent-Transfer-Encoding: 7bit\r\n\r\n'), (' BODY[2.M
IME] {200}', 'Content-Type: application/; name="04June2010 - Problem
&  Solution.xls"\r\nContent-Transfer-Encoding: base64\r\nContent-Disposition: i
nline; filename="04June2010 - Problem &  Solution.xls"\r\n\r\n'), ')'])

Dest: (reply from SmarterMail, a ‘correct’ server)
('OK', [('6 (BODY[HEADER] {769}', 'Return-Path: <>\r\nR
eceived: from [] by with
SMTP;\r\n   Sat, 16 Oct 2010 19:56:44 +0800\r\nReceived: from [] ([127
.0.0.1]) by (MailNow! 5);\r\n    Sat, 16 Oct 2010 16
:09:43 +0800\r\nMessage-ID: <>\r\nDate: Sat,
16 Oct 2010 16:09:38 +0800\r\nFrom: Marcus <>\r\nUser-A
gent: Thunderbird (Windows/20100228)\r\nMIME-Version: 1.0\r\nTo: Undisc
losed <>\r\nSubject: marcus\r\nContent-Type: multipart/
mixed;\r\n   boundary="------------030600080200070808060707"\r\nX-MN-SPAM-DTL: s
pam=0, scr=0, ver=\r\nX-Rcpt-To: <>\r\nX-SmarterMail-Sp
am: Bayesian Filtering, SPF_None\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Typ
e: text/plain; charset="ISO-8859-1"; format="flowed"\r\nContent-Transfer-Encodin
g: 7bit\r\n\r\n'), (' BODY[2.MIME] {208}', 'Content-Type: application/
el;\r\n   name="04June2010 - Problem &  Solution.xls"\r\nContent-Transfer-Encodi
ng: base64\r\nContent-Disposition: inline;\r\n   filename="04June2010 - Problem
&  Solution.xls"\r\n\r\n'), ')'])