Friday, November 12, 2010

twisted.web : rabbit’s hole

Finally i surrender. I switched one of my project totally off Twisted web. Using Twisted web is way too much work and pitfalls.

The only way to use Twisted web is that you use it in the very first place.

If you are starting a project involving building a simple http server, just avoid twisted .

Tuesday, November 2, 2010

Once again…Tutorial for poor souls : C# Windows Service


Looking at some of the tutorials online, i am once again baffled on how ridiculous “google” results are on creating Windows Service for C#.

I wonder where these people get the ideas from posting such advice and tutorial that hardly works but is confusing. If you refer to my other rants on multi-language .NET and calling C dll you will see what i mean.

Anyway here is how u do it. If you find reading it confusing, take a glance at the snapshots below.

1. I assume you are using VS 2010 and C#

2. Start a new project, select windows Service.

3.You will see 2 class, click on the tab with the word “designer”, right click “properties”.  Select “Add Installer”

4. You will get ProjectInstaller.cs. Click on the tab with “ProjectInstaller.cs [Design]” and click on the “serviceinstaller1” and on the lower right “Solution Explorer” you will see the properties. Set the “ServiceName” to any name u want to be used as the service name (the Service Name that appears when u right-click on any services in the services control panel properties). The service name here is the one that will be used not the one you see on the other service class that was created when u created this project (in 1st screen shot, the properties u see there, that service name is not the one that will appear or use in the actual services)

5. Set the Display Name to the name you WANT to see shown in the list of services in the Services control panel (services.msc)

6. Click on “ServiceProcessInstaller1” and Set the “Account” to the LocalSystem. If you don’t do this, you will get some stupid pop-up asking you to key in the username and password to use for running this service as that user.

7.Now all you need to do is to fill up your actual Service1.cs’s “OnStart” and “OnStop” with whatever routines you want to run and stop as a service.

8. Copy Installutil.exe from c:\windows\\framework\v2.0xxxxx to c:\windows

9. Run the command prompt / cmd as “Administrator” and goto where your .EXE is built and run “installutil xxxxx.exe” to install it.







Saturday, October 16, 2010

Twisted is long way from practical use


I have been trying to get imap bodystructure working to please all the clients i could get my hands on. So far the twisted imap (10.1.0) performed badly esp when you need it to work with Outlook + Outlook Express + ThunderBird + IPhone + RoundCube.

Yeah, Imap as i mentioned many times even in Pycon Singapore, is a hell of protocol of email. Many flags like “recent” of “draft” are just outdate and should not even be used…

JP Calderone wrote in the source :

# XXX - This does not properly handle multipart messages
# BODYSTRUCTURE is obscenely complex and criminally under-documented.

I had to resort to write an imap unit test that will test the reply from 2 servers and compare them and make changes from there to fix the imap4 bodystructure response.

The unit test tool helped a lot, i had to tweak various replies, for eg handling application attachment when they appear in mime attributes.

@ I got it working correct and tested with all the clients mentioned above with all kinds of attachment and sizes.

Source: (a fixed imap4 reply)
('OK', [('3 (BODY[HEADER] {513}', 'User-Agent: Thunderbird (Windows/201
00228)\r\nTo: Undisclosed <>\r\nSubject: marcus\r\nRetu
rn-Path: <>\r\nReceived: from [] ([127.0.0
.1]) by (MailNow! 5); Sat, 16 Oct 2010 19:50:55 +080
0\r\nMime-Version: 1.0\r\nMessage-Id: <>\r\nF
rom: Marcus <>\r\nDate: Sat, 16 Oct 2010 16:09:38 +0800
\r\nContent-Type: multipart/mixed; boundary="------------03060008020007080806070
7"\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Type: text/plain; charset="ISO-88
59-1"; format="flowed"\r\nContent-Transfer-Encoding: 7bit\r\n\r\n'), (' BODY[2.M
IME] {200}', 'Content-Type: application/; name="04June2010 - Problem
&  Solution.xls"\r\nContent-Transfer-Encoding: base64\r\nContent-Disposition: i
nline; filename="04June2010 - Problem &  Solution.xls"\r\n\r\n'), ')'])

Dest: (reply from SmarterMail, a ‘correct’ server)
('OK', [('6 (BODY[HEADER] {769}', 'Return-Path: <>\r\nR
eceived: from [] by with
SMTP;\r\n   Sat, 16 Oct 2010 19:56:44 +0800\r\nReceived: from [] ([127
.0.0.1]) by (MailNow! 5);\r\n    Sat, 16 Oct 2010 16
:09:43 +0800\r\nMessage-ID: <>\r\nDate: Sat,
16 Oct 2010 16:09:38 +0800\r\nFrom: Marcus <>\r\nUser-A
gent: Thunderbird (Windows/20100228)\r\nMIME-Version: 1.0\r\nTo: Undisc
losed <>\r\nSubject: marcus\r\nContent-Type: multipart/
mixed;\r\n   boundary="------------030600080200070808060707"\r\nX-MN-SPAM-DTL: s
pam=0, scr=0, ver=\r\nX-Rcpt-To: <>\r\nX-SmarterMail-Sp
am: Bayesian Filtering, SPF_None\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Typ
e: text/plain; charset="ISO-8859-1"; format="flowed"\r\nContent-Transfer-Encodin
g: 7bit\r\n\r\n'), (' BODY[2.MIME] {208}', 'Content-Type: application/
el;\r\n   name="04June2010 - Problem &  Solution.xls"\r\nContent-Transfer-Encodi
ng: base64\r\nContent-Disposition: inline;\r\n   filename="04June2010 - Problem
&  Solution.xls"\r\n\r\n'), ')'])

Monday, September 6, 2010

Windows Service : Mystery of the Slow startup

I have this .NET service application that is always reported in EventViewer to be “not responding” or takes too long to start.

The problem occurs only when a reboot is done. If the service was stop/start after reboot the speed is normal but if it starts automatically after reboot it takes around 90 seconds just to get it started.

The app main task is just start a webserver and connects to it.

So here is the condition :

Starting the service manually after the reboot = fine

First time auto starting after reboot = 90 seconds delay and reported to be “not responding” or hang, but runs fine .

I have nailed the culprit to WebClient!  Thats right, using WebClient in your . NET code as a service will result in it starting up slowly, this might have to do with the possibility that System.Net.WebClient uses the shared components  of IE and also its host of registry entries. This is SUPER SLOW when you reboot the first time.

Solution : i wrote my own TCP code to mimic WebClient and discard using WebClient in a Service app.

Case  close.

Monday, August 30, 2010

IronPython : the future…


As a developer who focuses more on development  for Windows platform but loved the power of Python, one can’t help but to keep a tab on the development  of IronPython.

I made myself a clear conditional “breakpoint” that when i do see IronPython officially supported in Visual Studio releases, i would use IronPython for my Windows development.

Its been some time  since i have waited with much hope to such a release only to be greeted by some semi-1.1 released of sample Visual studio integration and that obsolete VS isolated integration sample still hosted in codeplex.

The latest as of this writing is Python2.7 Alpha one that comes with some free tools to make the IDE 2010 works with IronPython.

I also came across this article :

And (one of the guys who wrote about using VIM with python as IDE comment)

Alain M. Lafon said...

I read this on the weekend just before I wanted to adopt IronPython into our company. It was supposed to make C# obsolete.
Well.. what can I say. It sucks to use any M$ tech. I should stick to OSX and Linux and leave my current employer due to strict use of M$ tech.
Sorry to say that man, but your story reads just like that to me.
I want to thank you for your great work, though. And I wish you all the best for your NY future(;

Now back to my Cpython on WingIDE….

Saturday, May 15, 2010

PythonNet and PythonService.exe

Well folks, just a week ago i was happy to discover PythonNet. Unfortunately it doesn't work
when its running under PythonService.exe which is the most natural way to run python as services as coded by Mark.

It runs fine in 32 bit Os, but the moment i tried it on Windows 2008 64 bit, it just terminated unexpectedly. The problem lies with the statement "import clr". i have exhausted all ways to get this working without writing my own pythonservice replacement and i conclude that its not usable
except for writing standalone tools that client need to manually click on or run by the scheduler.

Running the same program on a 64 bit OS manually works!, so the problem could lie in the
way python.runtime.dll is loaded by pythonservice.exe on "import clr".

Meanwhile, life goes on :-(, its back to .COM in .NET whenever we need Python to use certain calls to an existing .NET library.

Blogged with the Flock Browser

Tuesday, May 4, 2010

python and .net – unlikely symbiosis


If you love python and want to use all the libraries of python but you also need to work on .NET projects. You might be tempted to look at IronPython…but unless MS openly supports IronPython in VS and also all those libraries that makes Python such a kick ass language, you should skip it.

On the other hand, I found “pythonnet”, this is probably one of the best answer on using python and .net .

All you need to do is copy the clr.pyd file to ur python installation directory and also the python.runtime.dll file and you can do things  like …

import clr
import System.Windows.Forms as WinForms
from System.Drawing import Color, Size, Point
from System.Text import Encoding
from System.IO import File
import System

class Wordpad(WinForms.Form):
    """A simple example winforms application similar to wordpad."""


Cool like hell!.

Tuesday, April 13, 2010

IIS 6 & 7 running 32 bit .NET applications

Does this executable filename sounds familiar to you? Its unbelievable how ridiculous the error message given by IIS when you happened to be deploying 32 bit .NET webapps on a 64 bit machine.
It says something along the line of : An attempt to was made to load a program with an incorrect format.
Gee, one would have thought that error messages like "Unable to load 32 bit applications in 64 bit IIS with current settings" would make more sense and spare the poor developers some dead brain cells.

For IIS6, the solution was to run 2 command line that does 2 things :-
1. Enable32BitAppOnWin64
2. Register the 32bit version of aspnet_regiis -i -enable
( you can find the complete instruction by googling the keywords above)

For IIS7, the solution is to do step (1) but not with a command line, rather with
the IIS configuration UI where you can find the app pool settings.

I did write a python script that does that fix for II6, will be posting this later on this blog.

Wednesday, March 31, 2010

Death of Game writers : Stupid pirates

What got me hooked into programming was this idea i had when i was 13, to write a game. Anyway that never happen and i ended up writing commercial system applications.

Recently i visited Sg and got myself an original TitanQuest : Immortal Throne (expansion set), i played the Titan Quest few years back and wasn’t aware of the expansion set so at SGD 28 , it was a bargain.

I was curious about the developers of that game and was surprised it has folded. Here is a msg taken from one of the company’s top guy “Iron Lore” :



Originally Posted by Micheal Fitch
So, ILE shut down. This is tangentially related to that, not why they shut down, but part of why it was such a difficult freaking slog trying not to. It's a rough, rough world out there for independent studios who want to make big games, even worse if you're single-team and don't have a successful franchise to ride or a wealthy benefactor. Trying to make it on PC product is even tougher, and here's why.

Piracy. Yeah, that's right, I said it. No, I don't want to re-hash the endless "piracy spreads awareness", "I only pirate because there's no demo", "people who pirate wouldn't buy the game anyway" round-robin. Been there, done that. I do want to point to a couple of things, though.
One, there are other costs to piracy than just lost sales. For example, with TQ, the game was pirated and released on the nets before it hit stores. It was a fairly quick-and-dirty crack job, and in fact, it missed a lot of the copy-protection that was in the game. One of the copy-protection routines was keyed off the quest system, for example. You could start the game just fine, but when the quest triggered, it would do a security check, and dump you out if you had a pirated copy. There was another one in the streaming routine. So, it's a couple of days before release, and I start seeing people on the forums complaining about how buggy the game is, how it crashes all the time. A lot of people are talking about how it crashes right when you come out of the first cave. Yeah, that's right. There was a security check there.

So, before the game even comes out, we've got people bad-mouthing it because their pirated copies crash, even though a legitimate copy won't. We took a lot of **** on this, completely undeserved mind you. How many people decided to pick up the pirated version because it had this reputation and they didn't want to risk buying something that didn't work? Talk about your self-fulfilling prophecy.
One guy went so far as to say he'd bought the retail game and it was having the exact same crashes, so it must be the game itself. This was one of the most vocal detractors, and we got into it a little bit. He swore up and down that he'd done everything above-board, installed it on a clean machine, updated everything, still getting the same crashes. It was our fault, we were stupid, our programmers didn't know how to make games - some other guy asked "do they code with their feet?". About a week later, he realized that he'd forgotten to re-install his BIOS update after he wiped the machine. He fixed that, all his crashes went away. At least he was man enough to admit it.
So, for a game that doesn't have a Madden-sized advertising budget, word of mouth is your biggest hope, and here we are, before the game even releases, getting bashed to hell and gone by people who can't even be bothered to actually pay for the game. What was the ultimate impact of that? Hard to measure, but it did get mentioned in several reviews. Think about that the next time you read "we didn't have any problems running the game, but there are reports on the internet that people are having crashes."
Two, the numbers on piracy are really astonishing. The research I've seen pegs the piracy rate at between 70-85% on PC in the US, 90%+ in Europe, off the charts in Asia. I didn't believe it at first. It seemed way too high. Then I saw that Bioshock was selling 5 to 1 on console vs. PC. And Call of Duty 4 was selling 10 to 1. These are hardcore games, shooters, classic PC audience stuff. Given the difference in install base, I can't believe that there's that big of a difference in who played these games, but I guess there can be in who actually payed for them.

Let's dig a little deeper there. So, if 90% of your audience is stealing your game, even if you got a little bit more, say 10% of that audience to change their ways and pony up, what's the difference in income? Just about double. That's right, double. That's easily the difference between commercial failure and success. That's definitely the difference between doing okay and founding a lasting franchise. Even if you cut that down to 1% - 1 out of every hundred people who are pirating the game - who would actually buy the game, that's still a 10% increase in revenue. Again, that's big enough to make the difference between breaking even and making a profit.
Titan Quest did okay. We didn't lose money on it. But if even a tiny fraction of the people who pirated the game had actually spent some god-damn money for their 40+ hours of entertainment, things could have been very different today. You can bitch all you want about how piracy is your god-given right, and none of it matters anyway because you can't change how people behave... whatever. Some really good people made a seriously good game, and they might still be in business if piracy weren't so rampant on the PC. That's a fact.

Enough about piracy. Let's talk about hardware vendors. Trying to make a game for PC is a freaking nightmare, and these guys make it harder all the time. Integrated video chips; integrated audio. These were two of our biggest headaches. Not only does this crap make people think - and wrongly - that they have a gaming-capable PC when they don't, the drive to get the cheapest components inevitably means you've got hardware out there with little or no driver support, marginal adherence to standards, and sometimes bizarre conflicts with other hardware.
And it just keeps getting worse. CD/DVD drives with bad firmware, video cards that look like they should be a step-up from a previous generation, but actually aren't, drivers that need to be constantly updated, separate rendering paths for optimizing on different chips, oh my god. Put together consumers who want the cheapest equipment possible with the best performance, manufacturers who don't give a **** what happens to their equipment once they ship it, and assemblers who need to work their margins everywhere possible, and you get a lot of shitty hardware out there, in innumerable configurations that you can't possibly test against. But, it's always the game's fault when something doesn't work.
Even if you get over the hump on hardware compatibility - and god knows, the hardware vendors are constantly making it worse - if you can, you still need to deal with software conflicts. There are a lot of apps running on people's machines that they're not even aware of, or have become such a part of the computer they don't even think of them as being apps anymore. IM that's always on; peer-to-peer clients running in the background; not to mention the various adware and malware crap that people pick up doing things they really shouldn't. Trying to run a CPU and memory heavy app in that environment is a nightmare. But, again, it's always the game's fault if it doesn't work.

Which brings me to the audience. There's a lot of stupid people out there. Now, don't get me wrong, there's a lot of very savvy people out there, too, and there were some great folks in the TQ community who helped us out a lot. But, there's a lot of stupid people. Basic, basic stuff, like updating your drivers, or de-fragging your hard drive, or having antivirus so your machine isn't a teetering pile of rogue programs. PC folks want to have the freedom to do whatever the hell they want with their machines, and god help them they will do it; more power to them, really. But god forbid something that they've done - or failed to do - creates a problem with your game. There are few better examples of the "it can't possibly be my fault" culture in the west than gaming forums.

And while I'm at it, I don't want to spare the reviewers either. We had one reviewer - I won't name names, you can find it if you look hard enough - who missed the fact that you can teleport from wherever you are in TQ back to any of the major towns you've visited. So, this guy was hand-carting all of his stuff back to town every time his inventory was full. Through the entire game. Now, not only was this in the manual, and in the roll-over tooltips for the UI, but it was also in the tutorial, the very first time you walk past one of these giant pads that lights up like a beacon to the heavens. Nonetheless, he missed it, and he commented in his review how tedious this was and how much he missed being able to portal back to town. When we - and lots of our fans - pointed out that this was the reviewer's fault, not the game's, they amended the review. But, they didn't change the score. Do you honestly think that not having to run back to town all the time to sell your stuff wouldn't have made the game a better experience?

We had another reviewer who got crashes on both the original and the expansion pack. We worked with him to figure out what was going on; the first time, it was an obscure peripheral that was causing the crash, a classic hardware conflict for a type of hardware that very, very few people have. The second time, it was in a pre-release build that we had told him was pre-release. After identifying the problem, getting him around it, and verifying that the bug was a known issue and had been fixed in the interim, he still ran the story with a prominent mention of this bug. With friends like that...
Alright, I'm done. Making PC products is not all fun and games. It's an uphill slog, definitely. I'm a lifelong PC gamer, and hope to continue to work on PC games in the future, but man, they sure don't make it easy.
Best, Mike

Are you are software pirate user? Congrats, u just killed another great software company. Stealing software and stealing diamond is the result of the same criminal will. There is really no difference except its harder to catch the former.

Friday, February 19, 2010

Programmer’s Friend : Suunto T1

Its common knowledge that programmers tend to sit long hours in front of the computer and have next to zero amount of sports activity. Esp in countries like Malaysia or Thailand, jogging outside your house tantamount to suicide, given the kind emission rules around here.

Thus, its very important that programmers stays fit and indoor exercise is something that you don’t like but u need it gravely.

I found a real cheap way to exercise in house, get a Suunto T1 watch and that is basically your “profiler” so whatever “unit test” you do to your body, the T1 will profile and show you if what you did is indeed “enough” or your just fooling yourself.

With this watch, you can buy those much cheaper exercise equipment that does not comes with yet another heart rate monitor and i can tell you, the T1 and the belt gives the most accurate monitoring anyway.

Basically its a heart rate monitor that allows personalized settings like age, weight, height so that it can recommend a suitable heart rate that you should achieved and work on.

Recently after changing both the batteries on the watch and the strap i realize its no longer functioning. The website FAQ says “code not found” is bad news and i need to reset the belt and try again.

Well, here is something that the manual and website didn’t tell you.

You need to change the batteries in the BELT FIRST, before you change the batteries in the WATCH.  By doing this sequence, the watch will figure out that it needs to find the belt with a new code and will accept whatever code the belt is sending out.

Now, go on …run the unit test.

Monday, February 15, 2010

Vcds , dvds , blueray n bookstores

Things are sure looking bleak for video shops in malaysia . It takes just around 4 hours to download a movie via torrent .

Apart from piracy issues, here are some facts that seems to highlight the gloomy outlook of such businesses in Malaysia .

- dl movies have even better translation than the localized originals here . Look at Bleach for eg , the xxxbayo dls translations are better

- lousy quality on localized original. I buy original as much as I can . Its pretty disappointed that some cheap originals have utterly inferior quality . Take Twilight for eg , that local company that produce it is terrible . The show is lacking in contrast and has smudged colors . A friend of mine who decides to dl got a much better quality.

- Malaysian censorship . Dl movies are not censored , the local rebranded originals are. Nobody likes a tampered original .

I guess bookstores are finding it hard to survive too with all those ebooks illegal copies floating around and the crappy moral values of the new generation that believes anything u can dl for free is ok since u don't get caught aka stealing.

Its a repulsive sight to see how some would just take up magazines and read it all the way without buying . Some folks even mishandled mags n books while reading so much so that no one would buy it at that condition.


(Synced via BlackBerry->MailNow! 5)

Thursday, January 14, 2010

Inspiration + programmer

I admit sometimes i feel darn old and tired of programming. All these Malaysia boleh and after 11 years of starting my company i have not succeeded in getting my product out in a big way to the world. Yes it sold well in Malaysia and i exported some of my work in component to Europe, but the product as a whole by me and my team is not exporting.

I need inspiration and i need it badly.

Thank God…last week i stumble accidently on my answer. Its a watch. What has a watch gotto do with my line of work?Everything.

For one, you see all watches does more or less the same thing and yet each company behind those brands manages to export and have its own customers and loyal followings. In today’s world, you will have to strike lottery to be doing a software that someone else does not have or works similarly.

Second, watches is about craftsmanship, material, engineering and technology . Technology however is limited only to certain brand of watches . Needless to say watches that markets only on heritage and brand has zero inspiration to me.

If the watches companies could survive and yet make a good living out of it and export to the whole world, i reckon i could too. That is the first line of inspiration i found.

The best watch company that have the right ethos and philosophy is Casio G-Shock. Yeah sure its no prestigious bs and ego marketing, but hell the inspiration that G-shock provides is everything i needed to get my product in line.

If i could get our product in line with the Casio G-shock ( …see that red text on the right in that site….yes that is what i want to do .

Shock the World.

Right now as i am typing this, i am wearing a G-shock 1500bd, stainless steel black strap . Yes, i am fully inspired to continue my journey in programming and steer my company and products using this philosophy. Build it bloody solid, anti-fragile, works like clockwork and loads of technology. Heck this thing does 200 meters water resistance while something that cost 10x is doing less.