Saturday, October 16, 2010

Twisted IMAP4.py is long way from practical use

 

I have been trying to get imap bodystructure working to please all the clients i could get my hands on. So far the twisted imap (10.1.0) performed badly esp when you need it to work with Outlook + Outlook Express + ThunderBird + IPhone + RoundCube.

Yeah, Imap as i mentioned many times even in Pycon Singapore, is a hell of protocol of email. Many flags like “recent” of “draft” are just outdate and should not even be used…

JP Calderone wrote in the source :

# XXX - This does not properly handle multipart messages
# BODYSTRUCTURE is obscenely complex and criminally under-documented.

I had to resort to write an imap unit test that will test the reply from 2 servers and compare them and make changes from there to fix the imap4 bodystructure response.

The unit test tool helped a lot, i had to tweak various replies, for eg handling application attachment when they appear in mime attributes.

@ I got it working correct and tested with all the clients mentioned above with all kinds of attachment and sizes.

Source: (a fixed imap4 reply)
('OK', [('3 (BODY[HEADER] {513}', 'User-Agent: Thunderbird 2.0.0.24 (Windows/201
00228)\r\nTo: Undisclosed <marcus@internetnow.com.my>\r\nSubject: marcus\r\nRetu
rn-Path: <marcus2@internetnow.com.my>\r\nReceived: from [192.168.17.1] ([127.0.0
.1]) by Tool-Box.internetnow.com.my (MailNow! 5); Sat, 16 Oct 2010 19:50:55 +080
0\r\nMime-Version: 1.0\r\nMessage-Id: <4CB95DC2.3040201@internetnow.com.my>\r\nF
rom: Marcus <marcus@internetnow.com.my>\r\nDate: Sat, 16 Oct 2010 16:09:38 +0800
\r\nContent-Type: multipart/mixed; boundary="------------03060008020007080806070
7"\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Type: text/plain; charset="ISO-88
59-1"; format="flowed"\r\nContent-Transfer-Encoding: 7bit\r\n\r\n'), (' BODY[2.M
IME] {200}', 'Content-Type: application/vnd.ms-excel; name="04June2010 - Problem
&  Solution.xls"\r\nContent-Transfer-Encoding: base64\r\nContent-Disposition: i
nline; filename="04June2010 - Problem &  Solution.xls"\r\n\r\n'), ')'])

Dest: (reply from SmarterMail, a ‘correct’ server)
('OK', [('6 (BODY[HEADER] {769}', 'Return-Path: <marcus@internetnow.com.my>\r\nR
eceived: from 132.115.in-addr.arpa [115.132.75.110] by hs4.thehosting2u.com with
SMTP;\r\n   Sat, 16 Oct 2010 19:56:44 +0800\r\nReceived: from [127.0.0.1] ([127
.0.0.1]) by Tool-Box.internetnow.com.my (MailNow! 5);\r\n    Sat, 16 Oct 2010 16
:09:43 +0800\r\nMessage-ID: <4CB95DC2.3040201@internetnow.com.my>\r\nDate: Sat,
16 Oct 2010 16:09:38 +0800\r\nFrom: Marcus <marcus@internetnow.com.my>\r\nUser-A
gent: Thunderbird 2.0.0.24 (Windows/20100228)\r\nMIME-Version: 1.0\r\nTo: Undisc
losed <marcus@internetnow.com.my>\r\nSubject: marcus\r\nContent-Type: multipart/
mixed;\r\n   boundary="------------030600080200070808060707"\r\nX-MN-SPAM-DTL: s
pam=0, scr=0, ver=\r\nX-Rcpt-To: <marcus@internetnowasp.net>\r\nX-SmarterMail-Sp
am: Bayesian Filtering, SPF_None\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Typ
e: text/plain; charset="ISO-8859-1"; format="flowed"\r\nContent-Transfer-Encodin
g: 7bit\r\n\r\n'), (' BODY[2.MIME] {208}', 'Content-Type: application/vnd.ms-exc
el;\r\n   name="04June2010 - Problem &  Solution.xls"\r\nContent-Transfer-Encodi
ng: base64\r\nContent-Disposition: inline;\r\n   filename="04June2010 - Problem
&  Solution.xls"\r\n\r\n'), ')'])