codemagnet

A dangerous Pitfall in using cPickle + import

2011-08-09T05:54:00.001-07:00

Assuming you have a module : myfunc.py

from defunc_single import *

And defunc_single have a class that is :

class Hello :

def __init__ (self) :

self.name = “hello”

Now if you access this Hello via myfunc.py like this :

o = myfunc.Hello()

cPickle.dumps(o)

and write it using cPickle, the module defunc_single is also pickled into the file.

So that means along the way, if you felt that you wanted a new module : lovefunc_single.py with a better Hello and you thought that you can just replace the import statement in myfunc.py to :

from lovefunc_single import *

The cPickle will fail to load the existing data unless you stick to defunc_single.py

Zodb : What you should know.

2011-07-29T00:20:00.001-07:00

A programmer recently was dismissed for failing to produce satisfactory results after working on a project for 4 months.

He told his employer that “zodb is just too hard”.

Now i have a similar project that i am working on and decided to use Zodb for my storage. As usual i started to google for some quick examples and documentation.

2 weeks into the project i realize that some of my data are not “persisting” as it should while it managed to fool the unit test . I wrote some small single modules to simulate the situation and realized that the same data are not persisting even though another variation of the same code does.

To cut the story short, if you are doing or intend to use Zodb as your storage, just follow the rules below, forget whatever “friendly” samples you have seen, they just don’t reflect the issues that can arise :

1. Any classes that you store in the Zodb, derive it from “Persistent”.

Eg :

from persistent import Persistent

class Whatever (Persistent) :

2. Any dictionary or List that you want to store in Zodb, don’t bet on your luck, use the following wrappers.

If you want to store dict, use PersistentMapping

If you want to store list, use PersistentList

from persistent.mapping import PersistentMapping

from persistent.list import PersistentList

root[secret_key] = PersistentMapping()

root[secret_key][“mykey”] = “yada”

3. The connection and root must be opened by the thread that wants to use it or only use 1 single thread

storage     = FileStorage(Path)

db          = DB(storage)

connection = db.open()
root        = connection.root()

while “storage” and “db” are shareable across threads, “connection” and “root” are not. Each new thread must do the “db.open()” and “connection.root()” call.

Alternatively you can assign one single thread to do all the above opening of handles and access the “root” and “connection” freely among multi-threads, but you must ensure that “single thread” you assigned will be the same one closing the database.

Irresponsible Sharing

2011-07-12T18:14:00.000-07:00

Its pretty frustrating sometimes when you want some quick scripts to do your job and some of the given scripts are not only mediocre but non functioning.
Today i had the urge to quickly backup a directory i have to a remote ftp, so i figured that surely someone must have written a python script on this! After 30 minutes of searching and looking around, i found :
http://code.google.com/p/ftpsync2d/downloads/detail?name=ftpsync.py

But guess what, the code don't work on Windows. The bugs you get from it are very typical of newbie who have never coded in dual platform. For eg, it expect the path to be in "/" where else when it ran on windows it automatically converts to "\" and it bombed out on assertions.

Dumb? No. Just typical and if i knew i gonna spend 30 minutes to do this i might as well fire-up my FTP client and do it.

Gee guys/gals, if you want to post some codes and scripts on the Internet for the community, please at least raise its quality to an acceptable level. We have enough junk codes shared, we don't need any more of it, or the least, label the project that it only works on Linux/MAC/Windows.

twisted.web : rabbit’s hole

2010-11-12T12:59:00.001-08:00

Finally i surrender. I switched one of my project totally off Twisted web. Using Twisted web is way too much work and pitfalls.

The only way to use Twisted web is that you use it in the very first place.

If you are starting a project involving building a simple http server, just avoid twisted .

Once again…Tutorial for poor souls : C# Windows Service

2010-11-02T08:18:00.001-07:00

Looking at some of the tutorials online, i am once again baffled on how ridiculous “google” results are on creating Windows Service for C#.

I wonder where these people get the ideas from posting such advice and tutorial that hardly works but is confusing. If you refer to my other rants on multi-language .NET and calling C dll you will see what i mean.

Anyway here is how u do it. If you find reading it confusing, take a glance at the snapshots below.

1. I assume you are using VS 2010 and C#

2. Start a new project, select windows Service.

3.You will see 2 class, click on the tab with the word “designer”, right click “properties”. Select “Add Installer”

4. You will get ProjectInstaller.cs. Click on the tab with “ProjectInstaller.cs [Design]” and click on the “serviceinstaller1” and on the lower right “Solution Explorer” you will see the properties. Set the “ServiceName” to any name u want to be used as the service name (the Service Name that appears when u right-click on any services in the services control panel properties). The service name here is the one that will be used not the one you see on the other service class that was created when u created this project (in 1st screen shot, the properties u see there, that service name is not the one that will appear or use in the actual services)

5. Set the Display Name to the name you WANT to see shown in the list of services in the Services control panel (services.msc)

6. Click on “ServiceProcessInstaller1” and Set the “Account” to the LocalSystem. If you don’t do this, you will get some stupid pop-up asking you to key in the username and password to use for running this service as that user.

7.Now all you need to do is to fill up your actual Service1.cs’s “OnStart” and “OnStop” with whatever routines you want to run and stop as a service.

8. Copy Installutil.exe from c:\windows\microsoft.net\framework\v2.0xxxxx to c:\windows

9. Run the command prompt / cmd as “Administrator” and goto where your .EXE is built and run “installutil xxxxx.exe” to install it.

Twisted IMAP4.py is long way from practical use

2010-10-16T06:02:00.001-07:00

I have been trying to get imap bodystructure working to please all the clients i could get my hands on. So far the twisted imap (10.1.0) performed badly esp when you need it to work with Outlook + Outlook Express + ThunderBird + IPhone + RoundCube.

Yeah, Imap as i mentioned many times even in Pycon Singapore, is a hell of protocol of email. Many flags like “recent” of “draft” are just outdate and should not even be used…

JP Calderone wrote in the source :

# XXX - This does not properly handle multipart messages
# BODYSTRUCTURE is obscenely complex and criminally under-documented.

I had to resort to write an imap unit test that will test the reply from 2 servers and compare them and make changes from there to fix the imap4 bodystructure response.

The unit test tool helped a lot, i had to tweak various replies, for eg handling application attachment when they appear in mime attributes.

@ I got it working correct and tested with all the clients mentioned above with all kinds of attachment and sizes.

Source: (a fixed imap4 reply)
('OK', [('3 (BODY[HEADER] {513}', 'User-Agent: Thunderbird 2.0.0.24 (Windows/201
00228)\r\nTo: Undisclosed <marcus@internetnow.com.my>\r\nSubject: marcus\r\nRetu
rn-Path: <marcus2@internetnow.com.my>\r\nReceived: from [192.168.17.1] ([127.0.0
.1]) by Tool-Box.internetnow.com.my (MailNow! 5); Sat, 16 Oct 2010 19:50:55 +080
0\r\nMime-Version: 1.0\r\nMessage-Id: <4CB95DC2.3040201@internetnow.com.my>\r\nF
rom: Marcus <marcus@internetnow.com.my>\r\nDate: Sat, 16 Oct 2010 16:09:38 +0800
\r\nContent-Type: multipart/mixed; boundary="------------03060008020007080806070
7"\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Type: text/plain; charset="ISO-88
59-1"; format="flowed"\r\nContent-Transfer-Encoding: 7bit\r\n\r\n'), (' BODY[2.M
IME] {200}', 'Content-Type: application/vnd.ms-excel; name="04June2010 - Problem
& Solution.xls"\r\nContent-Transfer-Encoding: base64\r\nContent-Disposition: i
nline; filename="04June2010 - Problem & Solution.xls"\r\n\r\n'), ')'])

Dest: (reply from SmarterMail, a ‘correct’ server)
('OK', [('6 (BODY[HEADER] {769}', 'Return-Path: <marcus@internetnow.com.my>\r\nR
eceived: from 132.115.in-addr.arpa [115.132.75.110] by hs4.thehosting2u.com with
SMTP;\r\n   Sat, 16 Oct 2010 19:56:44 +0800\r\nReceived: from [127.0.0.1] ([127
.0.0.1]) by Tool-Box.internetnow.com.my (MailNow! 5);\r\n    Sat, 16 Oct 2010 16
:09:43 +0800\r\nMessage-ID: <4CB95DC2.3040201@internetnow.com.my>\r\nDate: Sat,
16 Oct 2010 16:09:38 +0800\r\nFrom: Marcus <marcus@internetnow.com.my>\r\nUser-A
gent: Thunderbird 2.0.0.24 (Windows/20100228)\r\nMIME-Version: 1.0\r\nTo: Undisc
losed <marcus@internetnow.com.my>\r\nSubject: marcus\r\nContent-Type: multipart/
mixed;\r\n   boundary="------------030600080200070808060707"\r\nX-MN-SPAM-DTL: s
pam=0, scr=0, ver=\r\nX-Rcpt-To: <marcus@internetnowasp.net>\r\nX-SmarterMail-Sp
am: Bayesian Filtering, SPF_None\r\n\r\n'), (' BODY[1.MIME] {100}', 'Content-Typ
e: text/plain; charset="ISO-8859-1"; format="flowed"\r\nContent-Transfer-Encodin
g: 7bit\r\n\r\n'), (' BODY[2.MIME] {208}', 'Content-Type: application/vnd.ms-exc
el;\r\n   name="04June2010 - Problem & Solution.xls"\r\nContent-Transfer-Encodi
ng: base64\r\nContent-Disposition: inline;\r\n   filename="04June2010 - Problem
& Solution.xls"\r\n\r\n'), ')'])

Windows Service : Mystery of the Slow startup

2010-09-06T03:33:00.001-07:00

I have this .NET service application that is always reported in EventViewer to be “not responding” or takes too long to start.

The problem occurs only when a reboot is done. If the service was stop/start after reboot the speed is normal but if it starts automatically after reboot it takes around 90 seconds just to get it started.

The app main task is just start a webserver and connects to it.

So here is the condition :

Starting the service manually after the reboot = fine

First time auto starting after reboot = 90 seconds delay and reported to be “not responding” or hang, but runs fine .

I have nailed the culprit to WebClient! Thats right, using WebClient in your . NET code as a service will result in it starting up slowly, this might have to do with the possibility that System.Net.WebClient uses the shared components of IE and also its host of registry entries. This is SUPER SLOW when you reboot the first time.

Solution : i wrote my own TCP code to mimic WebClient and discard using WebClient in a Service app.

Case close.

IronPython : the future…

2010-08-30T00:47:00.001-07:00

As a developer who focuses more on development for Windows platform but loved the power of Python, one can’t help but to keep a tab on the development of IronPython.

I made myself a clear conditional “breakpoint” that when i do see IronPython officially supported in Visual Studio releases, i would use IronPython for my Windows development.

Its been some time since i have waited with much hope to such a release only to be greeted by some semi-1.1 released of sample Visual studio integration and that obsolete VS isolated integration sample still hosted in codeplex.

The latest as of this writing is Python2.7 Alpha one that comes with some free tools to make the IDE 2010 works with IronPython.

I also came across this article :

http://blog.jimmy.schementi.com/2010/08/start-spreading-news-future-of-jimmy.html

And (one of the guys who wrote about using VIM with python as IDE comment)

Alain M. Lafon said...

I read this on the weekend just before I wanted to adopt IronPython into our company. It was supposed to make C# obsolete.
Well.. what can I say. It sucks to use any M$ tech. I should stick to OSX and Linux and leave my current employer due to strict use of M$ tech.
Sorry to say that man, but your story reads just like that to me.
I want to thank you for your great work, though. And I wish you all the best for your NY future(;

Now back to my Cpython on WingIDE….

PythonNet and PythonService.exe

2010-05-15T03:23:00.001-07:00

Well folks, just a week ago i was happy to discover PythonNet. Unfortunately it doesn't work
when its running under PythonService.exe which is the most natural way to run python as services as coded by Mark.

It runs fine in 32 bit Os, but the moment i tried it on Windows 2008 64 bit, it just terminated unexpectedly. The problem lies with the statement "import clr". i have exhausted all ways to get this working without writing my own pythonservice replacement and i conclude that its not usable
except for writing standalone tools that client need to manually click on or run by the scheduler.

Running the same program on a 64 bit OS manually works!, so the problem could lie in the
way python.runtime.dll is loaded by pythonservice.exe on "import clr".

Meanwhile, life goes on :-(, its back to .COM in .NET whenever we need Python to use certain calls to an existing .NET library.

Blogged with the Flock Browser

python and .net – unlikely symbiosis

2010-05-04T10:06:00.001-07:00

If you love python and want to use all the libraries of python but you also need to work on .NET projects. You might be tempted to look at IronPython…but unless MS openly supports IronPython in VS and also all those libraries that makes Python such a kick ass language, you should skip it.

On the other hand, I found “pythonnet”, this is probably one of the best answer on using python and .net .

http://pythonnet.sourceforge.net/readme.html

All you need to do is copy the clr.pyd file to ur python installation directory and also the python.runtime.dll file and you can do things like …

import clr
import System.Windows.Forms as WinForms
from System.Drawing import Color, Size, Point
from System.Text import Encoding
from System.IO import File
import System

class Wordpad(WinForms.Form):
"""A simple example winforms application similar to wordpad."""

…

Cool like hell!.

IIS 6 & 7 running 32 bit .NET applications

2010-04-13T09:05:00.000-07:00

Corflags.exe

Does this executable filename sounds familiar to you? Its unbelievable how ridiculous the error message given by IIS when you happened to be deploying 32 bit .NET webapps on a 64 bit machine.

It says something along the line of : An attempt to was made to load a program with an incorrect format.

Gee, one would have thought that error messages like "Unable to load 32 bit applications in 64 bit IIS with current settings" would make more sense and spare the poor developers some dead brain cells.

For IIS6, the solution was to run 2 command line that does 2 things :-

1. Enable32BitAppOnWin64

2. Register the 32bit version of aspnet_regiis -i -enable

( you can find the complete instruction by googling the keywords above)

For IIS7, the solution is to do step (1) but not with a command line, rather with

the IIS configuration UI where you can find the app pool settings.

I did write a python script that does that fix for II6, will be posting this later on this blog.

Death of Game writers : Stupid pirates

2010-03-31T05:10:00.001-07:00

What got me hooked into programming was this idea i had when i was 13, to write a game. Anyway that never happen and i ended up writing commercial system applications.

Recently i visited Sg and got myself an original TitanQuest : Immortal Throne (expansion set), i played the Titan Quest few years back and wasn’t aware of the expansion set so at SGD 28 , it was a bargain.

I was curious about the developers of that game and was surprised it has folded. Here is a msg taken from one of the company’s top guy “Iron Lore” :

Quote:

Originally Posted by Micheal Fitch
Greetings:
So, ILE shut down. This is tangentially related to that, not why they shut down, but part of why it was such a difficult freaking slog trying not to. It's a rough, rough world out there for independent studios who want to make big games, even worse if you're single-team and don't have a successful franchise to ride or a wealthy benefactor. Trying to make it on PC product is even tougher, and here's why.

Piracy
Piracy. Yeah, that's right, I said it. No, I don't want to re-hash the endless "piracy spreads awareness", "I only pirate because there's no demo", "people who pirate wouldn't buy the game anyway" round-robin. Been there, done that. I do want to point to a couple of things, though.
One, there are other costs to piracy than just lost sales. For example, with TQ, the game was pirated and released on the nets before it hit stores. It was a fairly quick-and-dirty crack job, and in fact, it missed a lot of the copy-protection that was in the game. One of the copy-protection routines was keyed off the quest system, for example. You could start the game just fine, but when the quest triggered, it would do a security check, and dump you out if you had a pirated copy. There was another one in the streaming routine. So, it's a couple of days before release, and I start seeing people on the forums complaining about how buggy the game is, how it crashes all the time. A lot of people are talking about how it crashes right when you come out of the first cave. Yeah, that's right. There was a security check there.

So, before the game even comes out, we've got people bad-mouthing it because their pirated copies crash, even though a legitimate copy won't. We took a lot of **** on this, completely undeserved mind you. How many people decided to pick up the pirated version because it had this reputation and they didn't want to risk buying something that didn't work? Talk about your self-fulfilling prophecy.
One guy went so far as to say he'd bought the retail game and it was having the exact same crashes, so it must be the game itself. This was one of the most vocal detractors, and we got into it a little bit. He swore up and down that he'd done everything above-board, installed it on a clean machine, updated everything, still getting the same crashes. It was our fault, we were stupid, our programmers didn't know how to make games - some other guy asked "do they code with their feet?". About a week later, he realized that he'd forgotten to re-install his BIOS update after he wiped the machine. He fixed that, all his crashes went away. At least he was man enough to admit it.
So, for a game that doesn't have a Madden-sized advertising budget, word of mouth is your biggest hope, and here we are, before the game even releases, getting bashed to hell and gone by people who can't even be bothered to actually pay for the game. What was the ultimate impact of that? Hard to measure, but it did get mentioned in several reviews. Think about that the next time you read "we didn't have any problems running the game, but there are reports on the internet that people are having crashes."
Two, the numbers on piracy are really astonishing. The research I've seen pegs the piracy rate at between 70-85% on PC in the US, 90%+ in Europe, off the charts in Asia. I didn't believe it at first. It seemed way too high. Then I saw that Bioshock was selling 5 to 1 on console vs. PC. And Call of Duty 4 was selling 10 to 1. These are hardcore games, shooters, classic PC audience stuff. Given the difference in install base, I can't believe that there's that big of a difference in who played these games, but I guess there can be in who actually payed for them.

Let's dig a little deeper there. So, if 90% of your audience is stealing your game, even if you got a little bit more, say 10% of that audience to change their ways and pony up, what's the difference in income? Just about double. That's right, double. That's easily the difference between commercial failure and success. That's definitely the difference between doing okay and founding a lasting franchise. Even if you cut that down to 1% - 1 out of every hundred people who are pirating the game - who would actually buy the game, that's still a 10% increase in revenue. Again, that's big enough to make the difference between breaking even and making a profit.
Titan Quest did okay. We didn't lose money on it. But if even a tiny fraction of the people who pirated the game had actually spent some god-damn money for their 40+ hours of entertainment, things could have been very different today. You can bitch all you want about how piracy is your god-given right, and none of it matters anyway because you can't change how people behave... whatever. Some really good people made a seriously good game, and they might still be in business if piracy weren't so rampant on the PC. That's a fact.

Hardware
Enough about piracy. Let's talk about hardware vendors. Trying to make a game for PC is a freaking nightmare, and these guys make it harder all the time. Integrated video chips; integrated audio. These were two of our biggest headaches. Not only does this crap make people think - and wrongly - that they have a gaming-capable PC when they don't, the drive to get the cheapest components inevitably means you've got hardware out there with little or no driver support, marginal adherence to standards, and sometimes bizarre conflicts with other hardware.
And it just keeps getting worse. CD/DVD drives with bad firmware, video cards that look like they should be a step-up from a previous generation, but actually aren't, drivers that need to be constantly updated, separate rendering paths for optimizing on different chips, oh my god. Put together consumers who want the cheapest equipment possible with the best performance, manufacturers who don't give a **** what happens to their equipment once they ship it, and assemblers who need to work their margins everywhere possible, and you get a lot of shitty hardware out there, in innumerable configurations that you can't possibly test against. But, it's always the game's fault when something doesn't work.
Even if you get over the hump on hardware compatibility - and god knows, the hardware vendors are constantly making it worse - if you can, you still need to deal with software conflicts. There are a lot of apps running on people's machines that they're not even aware of, or have become such a part of the computer they don't even think of them as being apps anymore. IM that's always on; peer-to-peer clients running in the background; not to mention the various adware and malware crap that people pick up doing things they really shouldn't. Trying to run a CPU and memory heavy app in that environment is a nightmare. But, again, it's always the game's fault if it doesn't work.

Audience
Which brings me to the audience. There's a lot of stupid people out there. Now, don't get me wrong, there's a lot of very savvy people out there, too, and there were some great folks in the TQ community who helped us out a lot. But, there's a lot of stupid people. Basic, basic stuff, like updating your drivers, or de-fragging your hard drive, or having antivirus so your machine isn't a teetering pile of rogue programs. PC folks want to have the freedom to do whatever the hell they want with their machines, and god help them they will do it; more power to them, really. But god forbid something that they've done - or failed to do - creates a problem with your game. There are few better examples of the "it can't possibly be my fault" culture in the west than gaming forums.

Reviews
And while I'm at it, I don't want to spare the reviewers either. We had one reviewer - I won't name names, you can find it if you look hard enough - who missed the fact that you can teleport from wherever you are in TQ back to any of the major towns you've visited. So, this guy was hand-carting all of his stuff back to town every time his inventory was full. Through the entire game. Now, not only was this in the manual, and in the roll-over tooltips for the UI, but it was also in the tutorial, the very first time you walk past one of these giant pads that lights up like a beacon to the heavens. Nonetheless, he missed it, and he commented in his review how tedious this was and how much he missed being able to portal back to town. When we - and lots of our fans - pointed out that this was the reviewer's fault, not the game's, they amended the review. But, they didn't change the score. Do you honestly think that not having to run back to town all the time to sell your stuff wouldn't have made the game a better experience?

We had another reviewer who got crashes on both the original and the expansion pack. We worked with him to figure out what was going on; the first time, it was an obscure peripheral that was causing the crash, a classic hardware conflict for a type of hardware that very, very few people have. The second time, it was in a pre-release build that we had told him was pre-release. After identifying the problem, getting him around it, and verifying that the bug was a known issue and had been fixed in the interim, he still ran the story with a prominent mention of this bug. With friends like that...
Alright, I'm done. Making PC products is not all fun and games. It's an uphill slog, definitely. I'm a lifelong PC gamer, and hope to continue to work on PC games in the future, but man, they sure don't make it easy.
Best, Mike

Are you are software pirate user? Congrats, u just killed another great software company. Stealing software and stealing diamond is the result of the same criminal will. There is really no difference except its harder to catch the former.

Programmer’s Friend : Suunto T1

2010-02-19T09:41:00.001-08:00

Its common knowledge that programmers tend to sit long hours in front of the computer and have next to zero amount of sports activity. Esp in countries like Malaysia or Thailand, jogging outside your house tantamount to suicide, given the kind emission rules around here.

Thus, its very important that programmers stays fit and indoor exercise is something that you don’t like but u need it gravely.

I found a real cheap way to exercise in house, get a Suunto T1 watch and that is basically your “profiler” so whatever “unit test” you do to your body, the T1 will profile and show you if what you did is indeed “enough” or your just fooling yourself.

With this watch, you can buy those much cheaper exercise equipment that does not comes with yet another heart rate monitor and i can tell you, the T1 and the belt gives the most accurate monitoring anyway.

Basically its a heart rate monitor that allows personalized settings like age, weight, height so that it can recommend a suitable heart rate that you should achieved and work on.

Recently after changing both the batteries on the watch and the strap i realize its no longer functioning. The website FAQ says “code not found” is bad news and i need to reset the belt and try again.

Well, here is something that the manual and website didn’t tell you.

You need to change the batteries in the BELT FIRST, before you change the batteries in the WATCH. By doing this sequence, the watch will figure out that it needs to find the belt with a new code and will accept whatever code the belt is sending out.

Now, go on …run the unit test.

Vcds , dvds , blueray n bookstores

2010-02-15T12:57:00.001-08:00

Things are sure looking bleak for video shops in malaysia . It takes just around 4 hours to download a movie via torrent .

Apart from piracy issues, here are some facts that seems to highlight the gloomy outlook of such businesses in Malaysia .

- dl movies have even better translation than the localized originals here . Look at Bleach for eg , the xxxbayo dls translations are better

- lousy quality on localized original. I buy original as much as I can . Its pretty disappointed that some cheap originals have utterly inferior quality . Take Twilight for eg , that local company that produce it is terrible . The show is lacking in contrast and has smudged colors . A friend of mine who decides to dl got a much better quality.

- Malaysian censorship . Dl movies are not censored , the local rebranded originals are. Nobody likes a tampered original .

I guess bookstores are finding it hard to survive too with all those ebooks illegal copies floating around and the crappy moral values of the new generation that believes anything u can dl for free is ok since u don't get caught aka stealing.

Its a repulsive sight to see how some would just take up magazines and read it all the way without buying . Some folks even mishandled mags n books while reading so much so that no one would buy it at that condition.

Duh...

(Synced via BlackBerry->MailNow! 5)

Inspiration + programmer

2010-01-14T07:56:00.001-08:00

I admit sometimes i feel darn old and tired of programming. All these Malaysia boleh and after 11 years of starting my company i have not succeeded in getting my product out in a big way to the world. Yes it sold well in Malaysia and i exported some of my work in component to Europe, but the product as a whole by me and my team is not exporting.

I need inspiration and i need it badly.

Thank God…last week i stumble accidently on my answer. Its a watch. What has a watch gotto do with my line of work?Everything.

For one, you see all watches does more or less the same thing and yet each company behind those brands manages to export and have its own customers and loyal followings. In today’s world, you will have to strike lottery to be doing a software that someone else does not have or works similarly.

Second, watches is about craftsmanship, material, engineering and technology . Technology however is limited only to certain brand of watches . Needless to say watches that markets only on heritage and brand has zero inspiration to me.

If the watches companies could survive and yet make a good living out of it and export to the whole world, i reckon i could too. That is the first line of inspiration i found.

The best watch company that have the right ethos and philosophy is Casio G-Shock. Yeah sure its no prestigious bs and ego marketing, but hell the inspiration that G-shock provides is everything i needed to get my product in line.

If i could get our product in line with the Casio G-shock (g-shock.jp) …see that red text on the right in that site….yes that is what i want to do .

Shock the World.

Right now as i am typing this, i am wearing a G-shock 1500bd, stainless steel black strap . Yes, i am fully inspired to continue my journey in programming and steer my company and products using this philosophy. Build it bloody solid, anti-fragile, works like clockwork and loads of technology. Heck this thing does 200 meters water resistance while something that cost 10x is doing less.

.NET remoting via COM and Python

2009-11-13T05:31:00.001-08:00

I bumped into a problem when trying to talk to C# DLLS that are created by my team while i am using Python. Creating a COM C# DLL is the key to bridge the communication since creating COM interface via C# is the easiest thing to do compared to C++.

Now one of the C# DLLs is loaded as a singleton in Windows Service thus i need my C# COM to talk to the same class via remoting.

That is when i kept on encountering “return argument has invalid type” whenever the remoting part was done. Putting "[Serializable]” into the class declarating did not help.

It ran perfectly when the class in the C# dll is used as a local class but it balked whenever the same class was returned via remoting and to me that is just ONE line of change.

// MyClass myClass = new ManagerClass() <--- local class

// RemotingReturnMyClass(out myClass) <—remoting

I went on to combining the COM class code and the C# singleton DLL in the attempt to create a singleton COM so that the C# singleton class can be accessed via my python code to no avail.

After DAYS (yup days) of trying to solve it, i suddenly realize something that i missed. What if i put the C# DLL into c:\python25 since maybe the problem is all about the marshaller and proxy not being able to locate it…….

Bingo.

Plone, Drupal or What?

2009-11-03T00:11:00.001-08:00

A quick migration of a static website to a CMS version was needed. However one of the requirement is that it must be as ‘non-programmer’ as possible since an assigned web designer will be doing the project.

After toying around with Plone, its way too technical and the curve is just too much to even be considered. It is also pretty slow, around 1.3 times slower in a Pentium Celeron with 1GB memory compared to Drupal on a dummy blank site.

So Drupal was next in the list and in fact chosen for the job , however as 2 weeks gone by, it was obvious that what we know about Drupal is in fact largely OVERRATED. There are certain theme layouts that doesn’t appear correctly on different browsers, very limited customization without going to php and way too much work for a non-technical person to get it looking nice, not to mentioned that most sites on Drupal looks similar.

Finally we stumbled upon SiteFinity by SmarterTools, the condition however was that it had to run on IIS but in this case we had no issues with that since the project doesn’t mind whether its IIS or Apache (who the hell going to modify apache code in the name of open source?)

SiteFinity not only blows Drupal to stoneage, more importantly we saved the Web designer guy from committing suicide. For once she was able to get something done.

** there is a community version available free for use

Removing “achtbanen” trojan links

2009-10-19T10:00:00.001-07:00

Ok so someone got careless and somehow his website files have been replaced with links to a trojan. Each of the html/asp files have been injected with a script “<script src=http://achtbanen.org/images/blah blah blah></script>”

I was asked to write a script to clean it up, so here it is, a script injector remover, basically will work on any injected one liner script (modify the signature on “FindThis” in the source yourself), by default will remove this particular achtbanen line.

TrjLinkRemove.py

     # Copyright 2009, codemagnet.blogspot.com

    # Free to use ;-), just keep this credit comments.

    import sys

    import os

    import nowlog

    logfile = nowlog.SetLogConfig("removed.log")

    """<script src=http://achtbanen.org/images/b-one-default.php ></script>"""

    FindThis = "<script src=http://achtbanen.org/images"

    FindThisJS = "document.write('<script src=http://"

    MaximumIndexTail = 150 #  search 150 off the first found signature

    def dirwalk(dir):

        "walk a directory tree, using a generator"

        try :

            for f in os.listdir(dir):

                fullpath = os.path.join(dir,f)

                if os.path.isdir(fullpath) and not os.path.islink(fullpath):

                    for x in dirwalk(fullpath):  # recurse into subdir

                        yield x

                else:

                    yield fullpath            

        except :

            pass

    def CleanFile (filename) :

        global FindThis

        try :

            f = open(filename, "rb")

            fb = f.read()

            f.close()

            i = fb.find(FindThis) 

            if i == -1 :

                return

            # found

            o = fb.find("</script>", i+len(FindThis), i+len(FindThis) + MaximumIndexTail)

            if o == -1:

                logfile.error("partial found, ignored, %s", filename)

                return

            # create a new file

            newfile = filename + ".trj.rmv"

            p = open(filename + ".trj.rmv", "wb")

            p.write(fb[:i])

            p.write(fb[o+9:])

            p.close()

            os.rename(filename, filename + ".xxxx")

            os.rename(newfile, filename)

            os.unlink(filename + ".xxxx")

            logfile.info("Cleaning : %s", filename)        

        except :

            logfile.error("unable to open file %s", filename)

            return

    #-------------------------------------------------------------------------#

    def CleanFileJS (filename) :

        global FindThis

        try :

            f = open(filename, "rb")

            fb = f.read()

            f.close()

            i = fb.find(FindThisJS) 

            if i == -1 :

                return

            # found

            o = fb.find(".php", i+len(FindThisJS), i+len(FindThisJS) + MaximumIndexTail)

            if o == -1:

                logfile.error("partial found, ignored, %s", filename)

                return

            # create a new file

            newfile = filename + ".trj.rmv"

            p = open(filename + ".trj.rmv", "wb")

            p.write(fb[:i])

            p.close()

            os.rename(filename, filename + ".xxxx")

            os.rename(newfile, filename)

            os.unlink(filename + ".xxxx")

            logfile.info("Cleaning : %s", filename)        

        except :

            logfile.error("unable to open file %s", filename)

            return

    def cleanup (path) :

        for x in dirwalk(path) :

            o = x.split(".")

            if o[-1].lower() in ["asp","html","htm", "aspx"] :

                CleanFile(x)          

                continue

            if o[-1].lower() == "js" :

                CleanFileJS(x)        

    if __name__ == "__main__" :

        if len(sys.argv) < 2 :

            print "Removes achtbanen trojan links from all webfiles"

            print "TrjLinkRemove <path>"

            print "e.g TrjLinkRemove c:\\iisroot"

            sys.exit(1)

        if not os.path.exists(sys.argv[1]) :

            print "invalid path given"

            sys.exit(1)

        cleanup(sys.argv[1])

nowlog.py

    import logging, logging.handlers

    def SetLogConfig (namefile):

        logfile = logging.handlers.TimedRotatingFileHandler(namefile , 'midnight', 1, backupCount=14)

        logfile.setLevel(logging.INFO)    

        FORMAT = "%(asctime)-15s %(levelname)s:[%(thread)d]:%(message)s"           

        logfile.setFormatter(logging.Formatter(FORMAT))    

        ch = logging.StreamHandler()

        ch.setLevel(logging.INFO)

        Logger = logging.getLogger(namefile)

        Logger.addHandler(logfile)

        Logger.addHandler(ch)

        Logger.setLevel(logging.INFO)    

        return Logger

* sorry about the formatting –.- was using a code snippet plugin

World First Email Server with MSN Robot (Avatar)

2009-10-14T11:57:00.001-07:00

This is probably true as the team gets ready to launch MailNow!5

MailNow! 5 is a Windows Mail Server that is of the likes of SmarterMail, Mdaemon, Merak and Hmail. The difference would be the amount of innovation that the team built into the MailNow! 5 that would eliminate log searching, greps etc for mail tracing , anti domain queue hogging and features on accountability that is way off the conventional school.

One distinct one is the world first Email Server Robot or Avatar. Its is an MSN robot that would inform users of the mail server status, new mails alert + details, collect log files, send SMS alerts, compose emails, retrieve emails, provide Email server spam statistics, emails per hour statistics and provide an “eliza” chat for idle users.

Looks really cool when it was demoed today. I guess the folks attending the launch on 12 Nov, 2009 will get some sneak peek too. (www.internetnow.com.my)

If you see any other mail servers adding this feature in the future, you heard it here first. That Malaysian made, MailNow! 5 , the windows mail server…had the first ever robot.

The team won’t be patenting it, the team here is just too engrossed in technical development than to invest in any legal procedures to write such a patent. Chances is MS Exchange would build one super robot in the future and who’s gonna sue?

RoundCube on Windows

2009-09-29T00:38:00.001-07:00

While testing the high speed IMAP server that i developed, i wanted to see how it perform with some open source webmail, particularly those that works with IMAP.

One of the most popular one that i found is RoundCube.

1. First you need to install the WAMPSERVER. The php included should be version 5++

2. Open up php.ini and make sure “extension_dir” points to the right directory which is under …\php\ext. Put in the extension module for sql . e.g

extension_dir = "/usr/local/apache/php/ext"
extension=php_mysql.dll

Yeah i created an c:/usr even on my Windows so that i don’t have to much changes.

3. Open up httpd.conf and make sure u have :-

<IfModule dir_module>

DirectoryIndex index.html index.php

</IfModule>

Double check to see if the following is loaded :

LoadModule php5_module "/usr/local/apache/php/php5apache2_2.dll"

4. Install Mysql for Windows. (This can open up one whole can of worms if you get it wrong)

5. Open the INSTALL file found in RoundCube and follow the instruction to create the database for roundcube. I would advice you to stick to the names given.

5. Open the roundcube\conf\, rename the *.dist to *. (whithout dist extension). E.g db.inc.php.dist –> db.inc.php

Edit db.inc.php and set the :

config['db_dsnw'] = 'mysql://roundcube:pass@localhost/roundcubemail'

to the username password that is created in 5, if you followed the default it should be :

config['db_dsnw'] = 'mysql://roundcube:password@localhost/roundcubemail'

Edit the main.inc.php and set the this to where ur email server is :

$rcmail_config['smtp_server'] = '10.8.0.21';

6. Now copy this whole roundcube folder to a subfolder under htdocs (apache) . For e.g c:\usr\local\apachce\htdocs\roundcube

And you can try and access it via : http://x.x.x.x./roundcube

Good luck.

Preaching Python : The obstacles

2009-09-16T21:22:00.001-07:00

I am a big fan of Python. I am a Windows Developer. I’ve been doing C/C++ as long as i can remember before jumping to C# to leverage on the .NET.

I abandoned Java when it was 2 years old, it just never got me attracted. Maybe its because my field centralizes on system level and security and not enterprise business applications.

Python has a distinct place in development today. But try convincing the following crowd :-

1. A Windows C#/ASP.NET developer, been using VB since 1997.

2. Java Developer with over 7 years of exp

3. C++ /C hardcore

4. PHP developer

Here are the list of the most common obstacles :

1. Python is SLOW

2. Python has no strong Enterprise Framework

3. Python has weak data typing, its hard to create APIs for users to extend.

4. C# can do everything that Python can and it works better in Windows and..faster.

5. Python GUI development in Windows sucks.

6. see no.1

Python is slow. Compared to C. Which language is faster than C ? Assembly maybe but definitely not Java or C#. Does the program speed really matter so much when practically in most applications the speed differences might not even exist to be practical.

Lets say you are developing a Network application that accepts data from the Internet and do something with it. The internet bandwidth speed from your ISP to you is nothing compared to your LAN speed. Lately The CPU processing power has grown from exponential to straight upward beating Moore’s Law. In this case python would do perfectly well, the bottleneck is not the application speed but rather the internet speed.

One of the exceptions would be if you are doing scientific research and mathematical data of which nanoseconds per iteration differences might matter to you.

Python is slow? Use C then for the part that requires special computational algorithm . Recently a friend of mine showed me a game called Civilization, its the latest version and it uses Python. It runs fast and flawlessly on my Vista. I believe they used python just for the main framework of the game coding while the graphic rendering is back to C/C++ . Now that is smart.

Python is slow? Blink. Did u see the difference?

Windows Live, Twisted

2009-09-15T21:54:00.001-07:00

This is the first posting via the Windows Live Writer. Today my MSN Messenger pops up and literally forced me to upgrade my Msgr to the Windows Live version.

What surprises me was the list of other services that you get to choose to install apart from Messenger itself and one of them is this Windows Live Writer. It allows you to post to various blogs (the first one in the menu is MS own service naturally…)

The online office documents is the other apps. This put the power of the MS Office to the internet for laymen. It makes google docs looks like cavemen offering. The google doc is laden with incompatible glitches when uploading MS documents over, and MS answers to this is a solid one. Try uploading a powerpoint and you will get what i mean….not a big issue , minor displacement here and there but still…

So what has this gotta do with Twisted ? Nothing really, except that the solid feel you get when using components from MS compared to Twisted (open source component) sometimes is evident, at least to developers that are not just MS bashers. On the other hand, MS is super slow in adopting and promoting what is evidently internet driven standards and if you want to develop for Windows, chances is that MS already have some offering that is either free, “included in the OS/will be” or some bundling Server products. The development for Windows software is a grim future to many Windows developers (except for project based solutions) . Of course the gang of MS partners and those getting projects from MS as partners via MS solutions are making tons.

For independent product development, Windows leaves Anti-virus, some security related fields and libraries/components the only profitable playing field for Windows Developers looking at “off the shelf” development.

Now back to twisted, i was looking at imap4.py, the “search_” functions have a typo bug that will never work , a simple one…missing an “s” on 3 functions of which i have reported to twistedmatrix. If MS components have such bugs, it would be totally unacceptable and laughable. I was thinking to myself, is this a framework that has so much hoo-haa in Python and over 7 years of evolution? Don’t even want to start on its poor documentation. If its not because of its beautiful skeletal architecture for various protocols, i would say asynch chat in the standard Python library would be more productive. (noob ! noob!, yeah spare me the insults)

Python is the best thing i have found in the OSS world. Each day passed by and i still find it the most attractive programming language. I place Python on the same playing fields as C++, Java, C#, not Perl and other scripting languages. Its meant for serious development and deserved to be used in an IDE and adopted by full fledged teams.

Are you using C#? Try python, use the Wing-ide. You won’t regret! You won’t lose much of the VS pampering that you are so used to and you get to think …somewhat…differently.

Google And Microsoft : Twin Effect

2009-08-18T01:13:00.000-07:00

The evidence can be found everywhere. Headlines, headways and in some people's head.
The pattern is recognizable given that one have been around since DOS days.
Google, the company that was once just associated with "search" is no longer staying where its domain is. It has tentacles that reaches far and wide but with one clear difference. The community don't realize it or maybe they do but they sure don't hate it.

Perhaps MS's reputation and brand was so powerfully seared into the hearts and minds of computing folks that we could only identify MS as the only possible gigantic sofware company that we could hate. One that squashes its competitor mercilessly using various "partnership" tactics through the history of IT. Novell...Nescape....scandisk....C++ builder...Word Perfect...Quattro Pro....OS/2......Isn't scandisk just a program, yeah, so was Netscape browser. Sure some of these companies survived but became obscure tech names, a fraction of what they could have been.

Times are changing indeed. Windows Mobile is losing out. Microsoft Live Search is a joke. A victim of its own game in an ever changing world. Its rare to hear MS losing in an area that it plans to focus or profit from but the new kid in town is nothing like MS have seen. Previously everyone had to fight MS in MS own turf...and since that turf is the OS itself, it was rarely a fair fight. Its 90% of the World's desktops dear, get real. But what if you challenge MS in a very unlikely place...the cloud?

When MS IE7 could not open the Google Wave page and Chrome could use it perfectly (ok firefox and other browsers worked somewhat) , you get a very familiar feeling. This creepy Dejavu feeling just jumps out of the page into your lap.

Android phones? Google Doc? GMail for Corporates? GAE hosted domain.

China always had a long history of bad Emperors and those who attempt to topple them. Given such power and position, it was usual for Emperors to behave like morons and do what they want regardless of the citizens sentiments. Heck an Emperor gets to have 600++ wives!

But Chinese History also teaches us one thing, topple one Emperor, the one taking over his place is no better.

Google Wave Hackathon Malaysia

2009-08-15T06:58:00.000-07:00

Today i attended the Wave hackathon. Some lucky fellas here are supposed to be the first in Malaysia to get google wave accounts (for the sandbox) .

Nazrul (the founder of MYGTUG) gave an overview and everyone quickly surfed and googled for more info . Basically for most of us, its the first time we are exposed to this Wave thing. This whole idea of blurring the line between emails and chats is boggling but opens up a whole new way of using "emails".

Due to some problems faced by other participants, somehow my "robot" was the only one demo-able at the end of the day. I guess my decision to just use the robot for fun things turns out to be easier to implement, Java programmers also complained about some issues with the api and Wave's unstability in handling certain events.

I went in with Python and i was using WingIde Pro, that must have been a real time saver since the autocompletion and module search gave lots of revelation of how some of the Wave Api works and fixes most easy errors. The problem is that you can't do a local test, so all the code must be uploaded before you can test it , simple errors will be a waste of time.

Here are some "traps" to avoid in creating a "robot" :-

1. If you add a capability to the robot AFTER you already uploaded your first robot, you will need to bump the version in your "whateverrobot.py" under the _main_ so that Google Wave will recognize that you are supporting a new event handling.

Java programmers will need to modify the capabilities.xml file to bump the version.

2. Don't just use the Tutorial example in creating Blips, you will end up modifying the blip

that the user posted instead of creating a new one since that is what the Tutorial example does.

You will get something like "User and robot" in the blip which is probably not what you want, instead just : newblip = root_wavelet.CreateBlip().GetDocument()

You can also try blip.CreateChild() if want it to appear below the original blip.

You must use the Tutorial example to learn and do the first upload though! Just don't

copy everyline when u want to do your actual one.

3. Upload the tutorial example so make sure all your other setup is already working.

You will need to setup a Google Application Engine account and a choice of either

Python or Java. (for Python i was using ver 2.5.4). Setting up Google Application Engine requires a valid and real mobile phone (it will sms a validation code).

4. IE.7 is unsupported!! Yeah, i guess Google is doing to Microsoft what Microsoft used

to do to some companies. So make sure you just stick to Chrome when participating.

Overall, it was a pleasant and enjoyable event.

Here is the robot i wrote (for those who have google wave account) :

Id : 369robot@appspot.com

Have Fun!

Too much support?

2009-08-06T19:40:00.000-07:00

I got this screen this morning when searching for something in MS.

Did MS just redirected all support calls to online search? ;-)

C# calling C DLL routines (YET ANOTHER ONE)

2009-08-03T07:27:00.000-07:00

I am totally baffled on the examples given by sites on how to call C routines in DLL via C#.
Not only are many of them repeating one same example after another, you soon realize they are not very helpful when you want to do something that is common like passing a structure with all kinds of types inside.

So here is one example that serve as an answer to MANY of the pinvoke/calling C apis problem :- ( i wanted to say 90% but i can't justify or prove it ;-))

I got this example structure that has :

struct ToughCallDef
{
char * ptrName; // this will contain some return string values
int ptrNameLen; // normal integer
MEMBLOCK somestruct; // a structure
ANOTERBLOCK* structpointer; // structure pointer
int* ptrtoMyInt; // integer pointer
char * ptrID // this must be supplied
};

Just translate it this way :-

[StructLayout(LayoutKind.Sequential)]
public class ToughCallDef
{
IntPtr ptrName;
MEMBLOCK somestruct; // you need to define MEMBLOCK also
IntPtr structPointer;
IntPtr ptrtoMyInt;
IntPtr ptrID;
}

And if the call requires a reference to this "ToughCall" :

[DllImport("anotherc.dll")]
public static extern int Call_C_API(ref ptrToughCall);

define it this way instead :

[DllImport("anotherc.dll")]
public static extern int Call_C_API(IntPtr ptrToughCall);

Basically anything to do with Pointers, regardless of whether its integer *, char *, structure reference etc, just use IntPtr.

1 . If the member is a "char *" that you expect return values you point the IntPtr this way :

szReason = new StringBuilder(100);
zReason.Append(' ', 100); // empty string with spaces
ToughCall.PtrName = Marshal.StringToHGlobalAnsi(m_szReason.ToString());

2. If the member is a "char *" that you need to assign a value before passing in :

ToughCall.ptrID = Marshal.StringToHGlobalAnsi(StringVar);

3. If the member is a structure pointer, you assign it this way :

ANOTHERBLOCK o = new ANOTHERBLOCK();
ToughCall.structPointer = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(ANOTHERBLOCK)));
Marshal.StructureToPtr(o, ToughCall.structPointer, true);

4. After calling the C routine you will need to do the following to get the values :-

If its a string, use :
string s = Marshal.PtrToStringAnsi(ToughCall.PtrName)
If its a structure use :
Marshal.PtrToStructure(ToughCall.structPointer, o);
Free any allocated strings/struct pointers via :
Marshal.FreeHGlobal

That's it.

Twitter (python api) on Nokia S60

2009-07-22T01:18:00.000-07:00

I wanted a system whereby people can sms to a phone that will log it to twitter.
With this system an organization can keep track of what its support/sales staff are at any given point of time.
After tinkering around with the twitter python api and Nokia s60 python (1.9.6), i finally
got it working...(duh...that took me more than an hour...)

There is however ONE big catch (read end of the section)

Here is the code :

Gotchas :-

1 - Ah yes you also need to change that getusername routine in twitter, else it wont work

2- Pay attention to that pesky pyS60 Application Packager (READ the README file) , this is not py2exe. Basically, you will need to rename your main file to some fix default.py.

3- It will keep asking you for connection (access point) on each msg , unless your using some newer phones like nokia 5800 XM. If you are using N82, ALL the popular examples shown in the internet like these :

http://discussion.forum.nokia.com/forum/showthread.php?t=163939

http://snippets.dzone.com/tag/pys60

http://croozeus.com/blogs/?p=836

DOESN'T Work. The reason is due to some "compatibility problem" between socket and btsocket.

"Twitter" uses urllib2 not urllib, and these people probably have never tested it on that api. Someone from Silicon Valley once said that Nokia know nuts about promoting development, and thus he rejected their offer to develop for symbian and instead move the whole team to iphone.

In some ways, i agree....but i still love my nokias ;-) I reckon the same thing would have worked in Iphone much earlier and with less blood on my desk.

** Update :

Marcelo Barros from Croozeus :

Yes, that is the problem. I reported it at maemo some time ago.
I suggest you to use my twitter api. Extend it if necessary.
http://code.google.com/p/wordmobi/source/browse/trunk/wordmobi/src/s60twitter.py
It uses urllib and simplejson (I ported it to S60, code in the same dir).

Note : If you still want to use the twitterapi as it is and not the ported version, you will

need to hardcode the "getusername" and stick to "socket" only calls, avoid using btsocket.

C# calling Python scripts and processing output

2009-07-12T20:26:00.001-07:00

Occasionally there would be a situation where the C# needs to call a python script and work with its output. While this might sounds like a piece of cake, it does have its gotcha, that is when
you want to process each line one by one and you happened to have somekind of timing (time.sleep) or some other python codes that works correctly under console but unable to be processed by C#.

Below is how you do it : (read until the end, there is a gotcha)

private static void OutputHandler(object sendingProcess, DataReceivedEventArgs outLine)
{
if (outLine.Data != null)
Console.Out.WriteLine(outLine.Data.ToString());
}

static void Main(string[] args)
{
Process p = new Process();
p.StartInfo.FileName = "python.exe";
p.StartInfo.Arguments = "c:\\test\\test.py";
p.StartInfo.UseShellExecute = false;
p.StartInfo.CreateNoWindow = true;
p.StartInfo.RedirectStandardOutput = true;
p.OutputDataReceived += new DataReceivedEventHandler(OutputHandler);
p.Start();
p.BeginOutputReadLine();
p.WaitForExit();
p.Close();
}

Assuming ur python script is the following (test.py) :

import time
import sys

for x in range(100) :
print x
time.sleep(1)

You will be surprised that you dont get anything. For some reason, time.sleep(1) and possibly
some other python library calls would result in the output not being flushed. The solution
would be to "flush" it :

for x in range(100) :
print x
sys.stdout.flush()

Vmware and the Virtualization Gotcha's

2009-07-12T18:51:00.000-07:00

There is this lab that has 4 different Win2k servers, each running a different application.
Server A - Mailserver
Server B - Intranet application with MS SQL db
Server C - Vpn server and backup
Server D - Test server

Along the way, the admin decides to put all these dated servers into a VM and slot it in to a high end Server from Dell. The whole porting process took around 2 weeks and when it was over,
everyone was happy with the new setup. No more additional switching cables and multiple monitors lyring around and the perceived energy savings cost was a bonus.

However one day, the mailserver began to feel very slow, Procexp (sysinternals) itself was myteriously taking up 45% cpu and more and even with the mailserver service stopped the pc still feels awkwardly slow. Rebooting didnt that mail server VM doesn't help either.

When i came in to help out in this scenario, the first thing i went thru was the list of VMs running in the server. One particular VM is taking up 26% of the CPU of the main server, however that should not be the reason why it would affect the mail server VM. Upon closer inspection however, i notice some native apps running on the test machine VM (26%) that is using up the VM's tcp/ip port very quickly.
It then became logically clear that this was the problem, pausing that VM immediately restored the other server's performance and that was like a 100% improvement.

I suspect the problem is because the main server is still just an OS with the normal limitation of the 65535 ports on a single ip and single network card. Since all the VMs runs on this machine , that test application was just blasting away the network resource, this turns the allocation of the real main server NDIS packets resources into an ugly situation where each of the VMs are queueing up to get its allocation.

Another case solve. Another Hamster Huey and the GooiKablooi award for virtualization.

Linux Desktop And Windows 7

2009-06-04T10:33:00.000-07:00

I must admit, Ubuntu has reconfigured my view towards the Linux desktop distro. Yes, its not something new, but from version 8.1 onwards since last year, the refinement is just slightly short of a usable desktop for common users.

Most of the stuffs are similar to windows in that you have hardware detection app running in the background with an icon in the system tray. The wifi detector works its best in trying to minimise "geeky" configurations and in my test notebook DELL inspiron 1520, it works straight out without the need for me to configure it manually and tweaking the config files.

With the common host of free apps like openoffice and email clients (Evolution - the email client is still lacking in refinement compared to Outlook Express but Thunderbird isn't), heck, for most Janes and Johns who just want to have email, internet surfing and office documents, this would feel right at home. The system works faster and you even feel "safer" without having any anti-virus, anti-malware, anti-adware, malicious removal tools , Windows Defender and all those hooking apps that layers itself one on top of the other.

Nine months ago, when i tried OpenSuse on my other notebook , it was hell, i had to configure and download specific wifi patch just to make it work. If a common user would have to choose between OpenSuse and Windows OS, even Win 98 would be a better option (speaking strictly in terms of usability for users who want things to just work and not do any command lines, scavaging forums and pulling hairs)

Ubuntu, rightly one of the most popular distro now, deserves an applause from the Linux community. Instead of sticking to the unspoken rule of "we can't copy microsoft, if you can't configure some simple scripts you are lame", it went straight on to make Linux a much friendlier distro for desktop. By providing an option without the need to repartition the HD and installing it side by side with Windows XP/Vista, it became an attractive option for users to try it.

The introduction of Version 8.1 was nothing short of perfect timing too. Vista, being an infamous slow OS - thanks to the "dev team" that insists on working on the new approach and forsake previous compatibility.

The fact is, Vista is slow. There is just something wrong with its design.
Personally i do not use any anti-virus, instead i even ran defrag religiously and PerfectDisk periodically to make sure i have optimum speed for my HD files. Still it was slow, i made sure my services are configured correctly and disabled all that i can lay my hands on and it is still slow. The frighthening thing was the fact that my Vista ran 3 times faster than any other Vista around my office on similar hardware, yet i find the response unacceptable.

Ubuntu on the other hand, ran seemingly faster even when it was installed using the side by side installation. Thanks to Vista, I believe tons of users flock to try Linux distro and many who happened to try Ubuntu would be positively surprised. Just this 2 weeks, i notice that there are 2 more collegues who installed Ubuntu and was excited to show it around.

However, there are still many hurdles for Ubuntu to overcome before it can become a replacement for Windows. For e.g, try connecting it to a projector, and good luck.
Try getting Compiz to work, chances are that you won't get it working correctly in the first 30 minutes and oh, pls update your Nvidia drivers else be prepared to do some file editing.

To date, there are around 200 Linux distros, some says 450. This is actually good news to Microsoft. As long as these jokers cannot unite, Windows will continue to dominate. Every aspiring Linux techie with enough willpower will want to produce his/her own Distro with some lousy excuse for yet another distro.

I would bet that if the community would drop all the similar distros and say just concentrate on 3 :

1. CentOS or RedHat for Server
2. Ubuntu for Desktop / Workstation
3. Knoppix for small storage / misc applications

Or better :
Ubuntu Desktop, Ubuntu Server, Ubuntu Mini.

Then all the big hardware manufacturers and the gaming industry would ensure that they will provide support for this alternative OS. Ms.Window would have to taste its own medicine in pitting against FREE OS. Yup, taste its own medicine. The number of companies that MS brought down just by including the said product/feature FREE are calling out for judgement day.

Netscape, RIP.

The only other area that MS still have not managed to kill are the Anti-Virus companies. Free Anti-virus seems to be playing along MS agenda all the time and yet failed to be executed. Hmm, before this happens, i should seriously fold up my sleeves and start working on this project i have in mind for some time now, the Corporate-Anti-Malware-End-Of-Anti-Virus project. A solution that will guarantee a corporate environment to be clean of viruses using a filter driver and a concept that involves a model pc, central sandbox and whitelisting. No more antivirus scanning and signature subscriptions!

Yawn....signing off.

Virtual Desktops and Cubes on Vista : Compiz, Ubuntu

2009-05-27T20:08:00.000-07:00

Recently i been playing with Ubuntu 8.1 and quite fancy the desktop cube that was
available in the Compiz.

If you look at google, there are many Linux users that posted cool videos of this "virtual desktop" and its effects, the comments ranged anywhere from MS bashing to degrading insults to people who chooses to use Windows/Vista instead.

Well here are the good news.

You can download a cheap (Free) and fast virtual desktop manager for Vista here :
http://technet.microsoft.com/en-us/sysinternals/cc817881.aspx

It allows you 4 desktop that you can switched via ALT-Num, which is a good hotkey.
Its only 68k, that shows how little work is required to have such functionality in Windows.

"How about the cube! We want the cube!!!"

The cube is available here, you have two options :-

One that is not free :
http://www.otakusoftware.com/deskspace/index.html

One that is free :
http://www.docs.kr/entry/Download-Shock-4Way3D-en

Have fun!

In hardware we put our trust....

2009-05-13T20:11:00.000-07:00

This morning i got a call from a customer. Running a powerful custom VPN solution (ipsec) for Windows that i wrote with multi-branch and failure switchover. He was just upgraded to a new version just few days back and now he complains that it hangs the server.

The symptoms are :
- server unresponsive and hangs periodically
- it even hangs while trying to reboot

After we turn on the VNC and remote in, i couldn't find anything wrong with the system. The guy on the other side then exclaim "How did you manage to login? I can't even get the CTRL-ALT-DEL to work". It was then obvious that i could login by asking VNC to send the CTRL-ALT-DEL, so i queried him further and the problem was actually his Keyboard.

An erratic hardware failure on the keyboard (capslock, numlock not responding) gives you the impression that the Server has hung.

This event got me thinking. Why do people always associate failures with Software and not hardwares when a problem first surfaced? I believe the reason lies much to the way sofware are produced compared to hardware. Sure, there are good programmers and bad programmers, they are also good manufacturers and bad manufacturers. But the key lies in the fact that
"bad manufacturers" still have to adhere to certain quality and the fact that creating a hardware requires much resources and after thought before committing to one.

Hardware specifications are dead static and factories that produce it must have certain quality check. If you are planning to produce a cheap alternative graphic card to the market, you will still pay a lot of attention to some quality and testing cause its going to cause you a bomb to start a factory on it. The same goes for keyboard, mouse, webcams and the whole list of hardwares that we associate with the PC. Yea sure, Logitech wares that are made in china have lousy fittings and cheap plastics but it still works as advertised.
Electronic components are "reusable code" at its' highest form. Nobody builds a hardware from scratch, they select the components that are already widely available in the industry and just design its own mold and maybe a few specific components.
Take the Razer mouse for example, the optical component and the RAM are manufactured elsewhere i am sure, while the casing, the scroll wheel and buttons are Razer's design.

Software on the other hand, much of its true value are hidden from the naked eyes. How a particular function is done varies from each programmer and so are its reliability. Much of what users perceive are just GUI while the core engine could fail in a thousand ways.

This brings me back to the Demmings 14 points some believe are the reasons why Japanese goods are viewed as high quality and reliability, it should be practiced in Software manufacturing as well.

Demming's 14 point :-

1."Create constancy of purpose towards improvement". Replace short-term reaction with long-term planning.

2."Adopt the new philosophy". The implication is that management should actually adopt his philosophy, rather than merely expect the workforce to do so.

3."Cease dependence on inspection". If variation is reduced, there is no need to inspect manufactured items for defects, because there won't be any. - (Unit test?)

4."Move towards a single supplier for any one item." Multiple suppliers mean variation between feedstocks. -(Oh dear, not Microsoft pls...)

5."Improve constantly and forever". Constantly strive to reduce variation.

6."Institute training on the job". If people are inadequately trained, they will not all work the same way, and this will introduce variation.

7."Institute leadership". Deming makes a distinction between leadership and mere supervision. The latter is quota- and target-based.

8."Drive out fear". Deming sees management by fear as counter- productive in the long term, because it prevents workers from acting in the organisation's best interests.

9."Break down barriers between departments". Another idea central to TQM is the concept of the 'internal customer', that each department serves not the management, but the other departments that use its outputs.

10."Eliminate slogans". Another central TQM idea is that it's not people who make most mistakes - it's the process they are working within. Harassing the workforce without improving the processes they use is counter-productive.

11."Eliminate management by objectives". Deming saw production targets as encouraging the delivery of poor-quality goods.

12."Remove barriers to pride of workmanship". Many of the other problems outlined reduce worker satisfaction.

13."Institute education and self-improvement".

14."The transformation is everyone's job".

Ldaptor, Python List, Django

2009-04-13T20:34:00.000-07:00

LDAPtor - The king who never made it

I wanted to deploy ldap server on windows. Apache DS with a 200 simulated user environment went kaput and need frequents restarting/backup. Openldap on Windows via the berkeley db was prone to corruption as well. Nothing seems to work robustly under Windows for some of these open source projects, no matter how popular it was.

Then i found Ldaptor, written completely in python via the Twisted framework. Wasn't really a big fan of the Twisted framework, heck, the only book on Twisted i saw by Oreilly, had examples that are "badly" coded in my personal opinion, works though. One can't doubt the benefit of having such a large ready made framework.

After just 5 minutes trying out ldaptor, it bombs out, won't even run, turns out that some modules are referencing the previous Twisted "components.interface" and should be replaced with "zope.interface.Interface", then it runs. Now just have to figure out how to import the Ldif into it...documentation seems to be scarce on ldaptor..."Ldaptor will one day rise to world domination"

Thats the problem with a LOT of open source projects, abandoned or left dying prematurely when the Initiator found something more profitable to do...

> Switch Channel ....

I am really grateful for some of python default implementation and behaviour, especially on its container.

I wanted to get a list of files in a directory and process it according to its "creation time". i did a "glob.glog" on a directory, the return result was anything but sorted. The normal procedure would be something like this :-

1. get each of the files creation time

2. sort based on the creation time

3. process the file

In python that translates to :-

ActualFiles = [(os.stat(x).st_ctime,x) for x in GetFilesListing(path))]

ActualFiles.sort()

for x in ActualFiles :

# do something with already sorted x[1]

When i wanted to remove some items from a list and yet continue to process the list :

for x in MyList[:] :

if x.value == "whatever_i_am_looking_for" :

MyList.remove(x)

If this was translated to C++ it would be something like this (warning: it won't work!)

for (ptr = listcopy.begin(); ptr != listcopy.end(); ptr++)

{

if (ptr->value == "whatever_i_am_looking_for")

{

listorg.erase(ptr)

}

Thus, someone who starts on python would probably find it hard to move to C/C++ but

someone who starts from C/C++ would easily addicted to python and appreciate its fun way of working (lazy).

> Django

I wonder how does Python web building such as Django matches against premium platforms like ASP.Net. Just the other day some people working on PHP was amazed to see how building cool working websites was done in ASP.NET. All those ajax coding are so automated it makes similar offering in PHP looks primitive not to mention the state of the art Visual Studio IDE.

Frankly developing website is something that is out of my reach but it would be great to see how Python fare in this area. Better than PHP? Ruby on rails? Come on, it has to do better.

Programmer's Mind vs None Programmer

2009-03-28T02:59:00.000-07:00

There are basically two types of jokes among the elite programmers when it comes to people who are non technically inclined or non-programmers.

#1 :

Sometimes people in Malaysia here thinks that they can just pump in a few million, start a team and expect to get a project done. Little do they know of the various traps and software pitfalls that awaits such a team. Unless its a crony project that don't mind accepting substandard deliveries, most of these projects often end up in the drain. Sure there are some big companies here that hire top Chief architects whom never deliver any projects to a real competition, all inhouse or cronies companies deliveries. Little do these self deceived Architects are aware of how weak their skill and proficiency are in that area. When you are the Chief Architect of such big corporations, everyone wants to kiss ur ass, no matter what ur true capabilities are.

This phenomena is so common that over here, the local folks have a name for it. Its called "Jaguh Kampung". In english it means "Village Champion". What happens when a few village champions join an international competition? They learn the meaning of "Village".

Each year in local software Awards here we see various Sofwares being conferred the "Software of the year", often these Software are either developed for its own use or for its branches. If you look at the track record, almost all the winners have ran out of business or no longer using it.

#2 :

There are Sales and Business people who don't know how dumb they look when saying something to the technical team or when accepting a business proposal whom they know nothing of in programming terms. When the technical team rejected the proposal, they have the impression that their team is incapable or does not have business sense, where in actual fact, the

problem is in the mirror.

Someone i knew once asked me to write a local chat software, i asked him "what is the sales forcast?" He replied "well i am sure there are many, let me call up this ABC company and see if they want it". (This is a clear sign to "reject" the idea, no proper thoughts given to the project and just do a blind jab by asking some programmers to commit to it).

Anyway, to deter him i ask him further "what are the specifications?". He replied "Oh, its very simple just do exactly what Skype can do and make it work in LAN".

** If you are not laughing already, you should stop reading this post.

Another story i heard is about this lady, who has many connections and "cables" to VIP people. Often she would try to get some projects from these people and her technical partner would reject it. She would then be very upset and gave some lectures on "How important is business and why nothing is impossible to code". A very sad partnership that is destined to end up in the gutter. What most of these "business" people don't understand is that, What you see on paper and in request is VERY different when its translated to actual coding. It can be so different that its actually supernaturally unrealistic to take it up. For example would you commit a team to write a product in 6 months and due to economics of scale, you would need to pay USD 1000 for an appliance/embedded system while out there, there are similar products retailing at USD 400 that is doing exactly the same thing?

If you have such a boss or partner , you should use this statement to bring enlightenment to him/her by saying "Actually if we want to make money we should just get a team to write an OS what works exactly like Vista but with a budget of USD 100k." or "Lets write a Windows 2008 replacement in 6 months, that sounds easy too isn't it?" or "Lets write a replacement for MS-Exchange and make it cooloer in 6 months"

Windows Mobile : The evolution of O2 -> Dopod -> HTC (its HTC all along)

2009-03-08T20:18:00.000-07:00

The Windows mobile has always been manufactured by HTC. Many years ago, its called

O2, sleek looking Phone PDA that usually sells well on first launch and turns out to be very buggy devices . The cycle repeats when new versions are launced. This is because out there, there are many people who love Windows and used to the Windows Experience, the option of having a mobile that has outlook and other familiar apps working is a big motivation for the purchase.

However, O2 really took the bulk of the fall as people began to wake up and identified it as an unusable device. Guess what? After the demised of O2 brand in Malaysia here, Dopod came up, with some brand new units and exciting offerings . Again, same forgiveness and embracement by the community, good sales and popularity of Dopod began to soar....only to be greeted by the same negative feedback and buggy usage.

The problem really lies with Windows mobile and cheap plastics production that easily wear off and scratches for usage that was never designed to last. Windows mobile started off as Windows CE hacked and rush into a market that is dominated by Nokia and Symbian. When it comes to phone usage, it sucks and is buggy.

I remember the O2-XPhone, boy, i must say i can't find a dumber smartphone than that one and yet, it was sold presumtiously to the community with no sense of responsibility and accountability.

Now that dopod is dying due to the same reasons that O2 went down, HTC decides its time to use its own name and produce units and avoid mistakes that was bourned by its previous two brands.

Will i buy an HTC? No way, i pity those companies who are associated with the prev two brands and if history prove itself well, HTC units will go the same cycle.

Switching away from Streamyx...or use together with Maxis

2009-02-23T23:56:00.000-08:00

The last few weeks have been pretty frustrating for me.
I can't access public servers in an acceptable speed and some anti-virus websites and even my online banking was crawling to a halt. I notice the problem is affecting all streamyx users as i tried in office, in my house and confirmed with other collegues in areas like Damasara, Ampang and Puchong. Even trying out kids games like http://www.bigfishgames.com/ for my kid and http://www.standardchartered.com.my/ was slower than a 28.8 modem.

I did notice there are some "cheats", access to google.com and youtube.com is fast however try something else it drops dead. Seems like Tm.net is using somekind of packet shaper, one that is implemented in a very very bad way.

But there is this happy guy in my office that seems to be talking greek when we complain about Streamyx woes lately and he is using Maxis broadband. Heck, i thought give it a few more days maybe it will improved after all nothing beats experience and that is what Tm.net is, a veteran in Malaysia as an ISP.

While we oogled over his fast connection response in the office the inconvenience of terminating the Streamyx and trying out something new kept me unmoved.

Finally today, the last straw broke so to speak... I needed to prepare a demo and that particular site is slowing to a crawl when accessed by streamyx.

2 hours later and RM 100 poorer (for the wireless modem), i connected it to my exising wifi modem/router which i am using for streamyx, pulled off the phone line, some simple configuration and wahla! Everything works normally, my wife wont even know i switched when she comes home and use her other notebook.

^_^

There is ONE big catch though :

http://www.maxis.com.my/personal/broadband/termncon.asp
Excessive Usage
5.3.1
Each customer’s total usage per month shall NOT exceed 3GB of data volume transmitted (total upload and download usage). This is to ensure that no individual hogs the bandwidth at all times.
5.3.2
If a customer wishes to use more than the volume permitted,
a written request should be made to Maxis.
5.3.2
Maxis reserves the right to reject the said application or charge for any additional data volume transmitted from what is already allocated.

This is the only part that makes streamyx still the most popular choice. 3GB is nothing as you and i know for today's internet need. Heck even free emails has 10GB free volume.
Did i read this policy? NO, the promoter never talked on this problem nor show you the policy.
So I was right, Maxis is inexperience on this area, but for now, It will have to be or lets hope Maxis will have some sense and make a more realistic limitation.

**** Latest update ******
A technical guy from TM has informed me that there is a MAJOR problem on their international link that will affect many access outside of Malaysia. The problem is promised to be fixed by
5th March 2009.
Well, i guess i will keep both ...until a super isp emerge.

Since most servers in Malaysia is under streamyx and if you are in IT you will need to support these servers or remote in etc, i suggest you use the following "batch" file so that you can access out of malaysia servers via Maxis and local servers via Streamyx to alleviate the current problem :

Assuming your Streamyx router is 192.168.1.1 and your Maxis is 192.168.2.1, point ur gateway to 192.168.2.1 and put the following into a *.bat file which u need to execute whenever u want to connect to local malaysia servers.

route add 219.0.0.0 mask 255.0.0.0 192.168.1.1
route add 118.0.0.0 mask 255.0.0.0 192.168.1.1
route add 218.0.0.0 mask 255.0.0.0 192.168.1.1
route add 60.0.0.0 mask 255.0.0.0 192.168.1.1
route add 202.0.0.0 mask 255.0.0.0 192.168.1.1

Keeping and Switching Multiple Python Version on Windows

2009-01-05T20:32:00.000-08:00

Assuming you have python 2.5 and python 2.6 installed.

Here is what you need to do :-

1. Right click on My Computer, under advance, environment variables.
Change all c:\python25 or c:\python26 path that u can find here to C:\python2x.
That includes c:\python2x\scripts, c:\python2x\lib, c:\python2x

2. Rename your c:\python25 (or whatever it is) to c:\python25.bin
Rename your c:\python26 (whatever it is) to c:\python26

3. type this to your command prompt (admin rights) :
ftype Python.CompiledFile=C:\Python2x\python.exe "%1" %*
ftype Python.File=C:\Python2x\python.exe "%1" %*

4. Create 2 batch file :
batch file 1.
xcopy c:\python25 c:\python2x /s /I

batch file 2.
xcopy c:\python26 c:\python2x /s /I

5. Fixup all the scripts dependencies.
For eg, goto c:\python25.bin\scripts, open up easy_install-2.5-script.py
Change the first line to : #!C:\Python2x\python.exe

You can repeat step 5 for other modules like nosetests, coverage etc. Else you will get the dreadful message "Cannot find Python executable ....."

Now, if you want to use python 2.5, just run the batch file 1 in your command prompt.
If you want to use python 2.6, just run batch file 2 in your command prompt.

There you go....

Python 2.6 : ez_setup and setuptools, pylint on Windows

2008-12-16T22:13:00.000-08:00

After i installed 2.6 i notice there is no setuptools windows installer in the python.org site.

So to manually install it you need to :-

1. Download this egg file :

http://pypi.python.org/packages/2.6/s/setuptools/

2. Make sure u have download ez_setup.py from

http://peak.telecommunity.com/dist/

3. ez_setup setuptools-0.6c9-py2.6.egg

PJ Eby :

Actually, you can just download the source, unzip it, and do "setup.py bdist_wininst" to build your own .exe installer.

Now you need to download the pylint for checking /integrating with ur ide.

And again, u notice that u will fail to get it installed via "easy_install pylint", so here

is the manual way.

http://www.logilab.org/view?rql=Any%20X%20WHERE%20X%20eid%20848

http://www.logilab.org/view?rql=Any%20X%20WHERE%20X%20eid%20856

http://www.logilab.org/view?rql=Any%20X%20WHERE%20X%20eid%20857

(all the download links are at the upper right hand corner)

Download and unzip to all the folders, then goto each of them, common, astng, pylint and run :

python setup.py install

Case close.

.py association in Vista/Windows Missing

2008-12-16T20:29:00.000-08:00

I was playing with version 2.6 Python without uninstalling version 2.5.

And for some reason i realize that when i execute a module without specifying the python.exe, the module could not get the arguments that was passed in.

For eg.

test.py argument1 argument 2

(argument1 and argument2 is missing from the sys.argv)

But

python test.py argument1 argument2 : works

Initially i tried "assoc .py" followed by "ftype Python.File" and the association was CORRECT!

It shows : Python.File="C:\Python26\python.exe" "%1" %*

Later i found out it was because there is still a registry key that is mapping wrongly.

Changing that registry key solves the problem.

Refer to the picture.

Python Gotchas and Pitfalls for Windows Programmers

2008-11-02T23:07:00.000-08:00

The no.1 pitfall i face is the "\r\n" that windows uses for its text "open" file where else in linux it will be "\n".

Although this might seems obvious or easy to fix by using "open("xxxxx", mode="wb")" when writing a string to a file, its not as simple as it sounds.

This is because many of the functions that is supposed to be ported to Windows will failed to work, one example is the mbox and maildir, if you use the "import mailbox" and attempt to use maildir to store the files, even if you issue a "rb" on an eml, when you store it via :

dest_mbox = mailbox.Maildir(path + folder, create=True)

msgdata = email.message_from_string(emailtext)

new_key = dest_mbox.add(msgdata)

The stored eml message contains "\r\r\n". This will then be a problem when you try to parse it via the email module.

Who knows what other python provided modules works this way too...

Pylint On WingIDE Check on Save!

2008-08-11T09:49:00.001-07:00

Now lets face it, for a data typeless language like python, its easy to get errors that other languages IDE would have picked up easily. For example, if you were to point "ooo = hellu()" and hellu() is actually a typo, it should have been "hello()". You wont get any error on the IDE until you run it.

This becomes pretty irritating when u have a number of these erorrs occuring when you are running your applications and you wished such flimsy bugs would have been picked by the IDE before you ever ran it.

Solution : Use Pylint.
Install it and goto "tools"->pylint and you will see a window on the right handside with "pylint" as the tab caption. (pls refer to other documentation on where to get and install pylint)

And what you need now is to have Pylint autocheck everytime u save a module so that all modules gets pre-checked and those irritating typos i mentioned earlier and other bugs will get caught.

Here is what you do, open up "editor-extensions.py" found in ur WingIDE installed directory under "scripts" (make a backup of this file first to somewhere else). Add the following lines at the end :-


def _connect_to_savepoint(doc):

def _on_savepoint(val):

if val == 0 :
return

# Get editor and do action
ed = wingapi.gApplication.GetActiveDocument()
if ed == None :
return

wingapi.gApplication.ExecuteCommand('pylint-execute')

connect_id = doc.Connect('save-point', _on_savepoint)

def _savepointinit():
wingapi.gApplication.Connect('document-open', _connect_to_savepoint)
for doc in wingapi.gApplication.GetOpenDocuments():
_connect_to_savepoint(doc)

_savepointinit()

And there you have it! Pylint total integration. Notice that if you open new modules and save it sometimes u get the "tried to execute an unavailable command .internal.gui.save". Just ignore this, the autosave in Pylint makes the IDE notice that you dont need to save the file again.

Python : Working Directory Woes

2008-08-07T20:05:00.000-07:00

Python has unittest that allows you to have a framework for testing your modules.
There is also nosetests. If you are using Wing IDE, there is a built in Test panel.
All is beautiful.

The problem is, if your project uses calls to "locate directory of where your script ran" you're in for a ride. Unlike windows api where u have a definite api to get this, the various ways in python to get the value resides in os.sys.path[0], sys.argv[0] and os.getcwd().

ALL the values you have seen will not get you the values you want, in unix when its run by using python or using the crond or when you run it from another script located elsewhere or in windows when you run it with "unittest, nosetests and even the Wing's Test panel" these values are going to give you a different result.

And then there is the py2exe ....which will give you yet another surprise as the path return is "\dist\library.zip"

Solution :

Temporarily put some guards for tests enviroment and use the following :-

def GetCurrentWorkingDir () :

if "Wing" in sys.argv[0] : return os.sys.path[0] + os.sep
if "nosetests" in sys.argv[0] : return os.sys.path[0] + os.sep
return os.path.abspath(os.path.join(os.path.dirname(sys.argv[0]))) + os.sep

Py2exe and email.generator, email.blah blah errors

2008-07-31T23:35:00.001-07:00

Create this file :-

import email
msg = email.MIMEText.MIMEText("dsafdafdasfA")
print "ok"

Create setup.py like below :

from distutils.core import setup
import py2exe
setup(console=['testmime.py'])

Run python setup.py py2exe and you will get errors

The following modules appear to be missing['email.Generator', 'email.Iterators', 'email.Utils']

And your program will bombed out with errors like ""Import error : No module name text"

To solve this i just added " print self.__name__" under __init__.py of \email package.
And it seems the LazyImporter is confusing py2exe.

Solution ?

import email
import email.mime.text
import email.iterators
import email.generator
import email.utils

Python : POP3 SSL Gmail Proxy

2008-07-10T02:51:00.000-07:00

There was a friend of mine who went to Macau and his company uses Gmail for hosting corporate emails. He has a local mailserver that would want to download from this gmail and it doesnt support POP3 over SSL (port 995).

So once again Python comes to the rescue, 2 hours later, its done. A generic POP3SSL proxy that will allow any mailserver/client to collect mails from gmail via POP3SSL.

Just run : pop3ssl.exe host port (ur local ip and port that u want to use as proxy)
Example : pop3ssl 10.8.0.1 110 (or if ur mailserver is in the same machine, use another port)

Download it here : pop3ssl.zip

Note although its free for use, pls leave a comment here on how this program helped you.

Pythonist Rants

2008-07-09T20:33:00.000-07:00

Its interesting to see how Pythonist does some stuffs and then suddenly you find that doing that in some other language wasn't a bad idea either ....and then...suddenly (again) it dawns on you that many things could be done in Python that is IMPOSSIBLE to duplicate exactly .

Lets say you have a vector in C++ that you need to remove an item if it fits a certain scenario.

The layman's way is to do the following in C++ (not using algorightm stl ;-))

vector::iterator ptr;
for (ptr = abc.begin(); ptr != abc.end(); ptr++)
{
if (*ptr == "red")
{
abc.erase(ptr)
if (ptr == abc.end()) break;
ptr--
}
}

This is one of the way Pythonist does it :-

for x in abc[:]: # this means makes a copy of abc
if x == "red" :
acb.remove(x)

Yup, coming from C++, one's mentality one would wonder why we didnt use that similar method in Python in C++. However, this wouldn't work in C++ without futher modifications, this is because the "erase" is not taking a value but rather the iterator which means if you do this on the original vector but you passed in the iterator instance from the copy, it wouldn't work.

Then there is the story of smtpd.py. Its basically a smtp proxy/server skeleton that supports the basic commands. And how does one go and implement the "MAIL FROM", "RCPT TO", "DATA" method? The normal thinking would be to do a method/function for each of these commands and then there is the switch or if-else statement working on a read buffer.

How did smtpd.py do it?

method = getattr(self, 'smtp_' + command, None)
if not method:
self.push('502 Error: command "%s" not implemented' % command)
return
method(arg)

Introspection! It basically appends the command read and massaged from the buffer and append to the "smtp_" and invoke the method ....

Python fun project : MSN CHAT bot

2008-07-08T01:47:00.000-07:00

I downloaded MSN-lib 3.6 from sourceforge and soon the naughty idea of linking this with Eliza
the therapist (joe, jeff , jez version of Eliza in python) cropped up almost instantly in my head.
So using a skeleton code of using MSN-lib i picked up from some forum, i did 1+ 1 = 2.

To make it realisic, i provide the "first msg" and the victim input before the bot takes full auto drive.

(All credits due to the original author and contributor to msnlib, eliza and related sources)

I must say i had a great time laughing over the chat logs as my friends and peers communicated with my chatbot. I made the program to log to some hardcoded directory of c:\msnbot since some sample codes is logging to some unix paths.

Well, if u need a break and a laugh, here is the chatbot and continue reading to find out how to configure it :-

Open the file : marbot.py, edit the values below and enter ur hotmail login and password.

m.email = "xxxx"
m.pwd = "xxxx"

Also change the name and nick to ur own :-

m.change_nick("-M- The future")

Ok, done now you are ready to go :-

python marbot.py

or just :

marbot.py

After running for a while it will ask 2 lines of input , the first line is the email id of the "victim" you intend to start a conversation with, the 2nd line is the first message you want to send to this person.

From then on, just watch and laugh as your friend communicate with Eliza. The logs are found in c:\msnbot (create this directory or unzip the files below to this directory)

Download the fun bot here

Visual C++ BrainBench Certification

2008-06-23T19:09:00.000-07:00

I boldly went and took the Visual C++ BrainBench Certification without preparing myself in any refreshment task. I am very much caught in a surprise as i notice that the Visual C++ BrainBench actually means Windows programming in C++ , that includes OLE/COM, WndProc from scratch, device context and loads of other stuffs which frankly i have not been coding these natively but took the easy IDE way out.

It was a good test i must admit, it reveals to me how much i have taken for granted the advance tools and how much less i remembered of these subjects.

Here are the results anyway ( :-( scored within 3.x ...my python was 3.3, i was expecting a 4.x for a familiar language): BrainBench Visual C++ supervised :

Results for xxxxx

Test Takers Email: xxxxx

Date Taken: 23-Jun-2008
Test Event ID: ZC11178-CKQSJHX6

Overall Score: 3.56
Weights: 100% Visual C++
Elapsed time: 53 min 9 sec
Visual C++
Score 3.56
Percentile : Scored higher than 85% of previous examinees

Proficiency Level:
Advanced (Master)

Demonstrates a clear understanding of many advanced concepts within this topic. Appears capable of mentoring others on most projects in this area.

Strong Areas
General C++ Language
MFC
OLE/COM
General Windows Programming

Weak Areas
None noted

Python BrainBench Certification

2008-06-22T23:48:00.000-07:00

I have been playing periodically with Python for a few months, i decided to take the BrainBench Certification was carried out by Deltaknot (http://www.deltaknot.com/). Brainbench by Deltaknot however appoints a supervisory staff to monitor the test taken so as to avoid where "few people answering a question"and other forms of "cheat".

Anyway, the test clearly shows some weakness in my understanding of Python, which will be the article for this blog.

But first, here is the result :-

Python 2.4 Assessment Report

Report Information
----------------------------------------------------------------
Examinee Name: (my name)
Date of Report: 6/23/2008
Prepared For: Deltaknot International Sdn Bhd
Prepared By: Brainbench

Assessment Results
----------------------------------------------------------------
Assessment Taken: Python 2.4
Date of Assessment: 6/23/2008
Overall Score: 3.32 (1.0 - 5.0, where 5 = best)
Worldwide Percentile: 67 (99 = highest)

Additional Score Data
----------------------------------------------------------------

Demonstrates a solid understanding of core concepts within this topic. Appears capable of working on most projects in this area with moderate assistance. May require some initial assistance with advanced concepts, however.

Strengths
Standard Library
Language

Weak Areas
Classes
Built-in Features

Ranking Information
----------------------------------------------------------------
.: 1 of 1 (99th Percentile)
.: 1 of 1 (99th Percentile)
.: 1 of 1 (99th Percentile)
Worldwide: 292 of 886 (67th Percentile)

First notice that my weakness as listed by BrainBench here are "classes and built-in features". Ironically after taking the test, i could identify these areas as well. Thiese are probably the same weaknesses that most seasoned programmers in other languages might encounter in the test.

This is because some of the ideas that we have learn from other languages, for eg, C++ naturally becomes the point of association and provides the learning platform when you learn other language. In my case, i skip most of the specific features of Python Classes since i believe i have a strong grasp of the C++ classes and thus didnt bother to pay much attention to it. The built-in features of Python are often viewed and subjected to my understanding of other languages and again which i ignored because i have never used them in practical programming.

For example the usage of : yield and function generators in python, now that is something you are not constantly exposed to in C++ and thus i have the tendency to ignore and skip over such topics, however in the test, these weaknesses are revealed.

This kind of remind me of Forbidden Kingdom, yes the girl is chun (Liu Yi Fei), but i am referring to the drunken master asking the disciple's cup to be emptied before he could teach him. There are some element of truths to this.

After coding in C# for the last few weeks, i found myself doing this in my C++ code :-

std::list listStrs = new std::list()

which is of course an error, since objects in C++ does not need to be "new" unless its a reference or pointer, but in both python and C#, you cannot instantiate any class member like a built in type.

Overall, i would say the BrainBench certification provides a good assessment since i agreed with the results and cheats are avoided due to supervision.

Malaysia Downloads

2008-06-12T17:44:00.001-07:00

Sometimes as programmers we need to download a large project from the internet for R&D or use. Often some of these are commercial frameworks or Open source projects.

Today i bought the Wing-ide 3.1.1 professional version and found the download speed from Malaysia is just 2.5kb and it will take 2 hours to download a 17 mb file.

So the trick is to use an alternate proxy site that has much faster connection to this download and at the same time a fast connection with Malaysia .

I tried : www.proxysite.org and it was fantastic. The download bumped from 2.5kb to 99kb/s.

The whole download took just 2 minutes. Now that is what i call productive bandwidth ;-)

RHB Internet Banking Programmers

2008-05-14T04:53:00.001-07:00

There are sentiments that local programmers sucks. Sometimes its unavoidable that such
biased sentiments are expressed towards local talents and here is one such example.

I found out lately that i could not login to http://www.rhbbank.com.my/ to do my online banking, that means i have to queue up to pay for utilities and astro..(hehe)

The reason i could not login is diplayed on the page :

The following error has occurred.Result Code: 1
Result Message: *ERR-onlynum

This message implies 2 things :-

i. the programmers working on this project has implemented a check routine that
detects if your password is all "numbers" and to discourage you to continue using such "unsafe" practices it blatantly displays this message.

They even display on the login page that you should login to change ur password to alphanumeric for security reasons....but what they forgot was....

ii. the programmers working on this project was dumb enought to FORGET to test this system on users that have such passwords and the system did not allow the user to even login to change the password. So if you cannot LOGIN to the system, how in the world do they expect you to change ur password? This kind of logic i thought was something i witness during the first 2 years of my junior programming projects , but to see it happening on a PUBLIC website for a BANK is just unbelievable.

After a number of calls to the customer service, they said they need a WEEK to solve this problem on their IT dept. One wonders if a 5 minutes solution requires ONE week to solve and the very fact that they did not test the system for such cheap bugs is amazing.

If i was the Project director, i would have fired the team responsible and hired a whole bunch of new programmers , the chances of a team working on a Bank website to have such error is like hitting a lottery.

Linux is still dumb....inherited Shell genes

2008-04-15T21:44:00.000-07:00

If you are working on both windows and linux at the same time, one of the first annoying thing that you will discover is that linux's shell (bash) is very unforgiving when it comes to "\r" or
carriage return or ctrl-ascii-13 ...whatever you call it.

When you use an ide to write some scripts and when you copied it to Linux, you get a really dumb message : "bad intepreter: No such file or directory" . It is really amazing at the time and age when software are supposed to be smarter and better, we have linux still stuck with dumb shells like bash who dont understand nor have to common sense to ignore "\r" as part of the new line. Although on windows if you open via notepad any text files written in unix, you will be similarly annoyed due to the missing \r, but i have yet to find a Windows program that is unable to process a file that is missing \r or created by unix text editors. Thus i can say Linux is still pretty much dumb due to the fact that it inherited some cow legacy while Windows have smartly sided with the users.

Meanwhile, you have to live with doing commands like this :

mv original.file backup.file; tr -d '\r' <> original.file

whenever you move some scripsts over from Windows.

IDE rules the day

2008-04-08T21:28:00.000-07:00

I tried 2 Python IDE, Komodo 4.3 and Wing IDE 3.0.
Komodo has support for other languages like php and ruby while Wing IDE is on Python itself.

I must say Wing IDE's "intellisense" works much better than Komodo and the latter had a problem running on Vista with UAC on (even with admin rights) while Wing running under Admin was good.

However there are some things that both IDE could not address in Python coding that makes
the standard lower compared with VS 2005 IDE (we wont even talk on VS 2008).

When you key in python class for eg that inherits from another class, none of the ide manage to display the base class members methods when you are coding the new class. Maybe this is the problem with Python language itself make it much harder for the IDE to present the valid context for showing lists of members methods.

For eg if you have a base class "Dog" and it has 2 methods :

class Dog () :
def DigGround (self, where, depth) :
pass

def Jump (self, objecttojump, height) :
pass

class Basset (Dog) :
(expect no help here if you plan to override DigGround, you will need to look at the api reference as none of the IDEs could show you)

I know it a minor thing but if you are using Twisted for example, there are many instance where you would need to refer to the documentation for the apis if you plan overriding or calling some of these attributes which in the process is really frustrating. Unless you memorized stuffs like "one of the method you can override is def connectionMade(self): or def dataReceived (self, data)" it would be hard pressed to ask the IDE to show you these methods implemented at the those libraries you are using. (Twisted is becoming more of an essential library then option)

Updated : Incorrect, the Wing IDE source browser could show the base class methods even though the autocomplete does not provide much help. Definitely a good enuf reference to program productively in Python . Only on Wing IDE PRO. If you buy Wing IDE personal, too bad life goes on..

Ultimately the IDE makes a big difference in your productivity. Python is a very productive code however without an equivalent productive IDE, Windows Programmers will find VS offering an irreplaceable IDE to work with.

However, having said that, if you are into PHP and ruby as well, then there probably is no better ide than Komodo for "multi-open-source" language editor (apart from Eclipse which you need to download the plugins ). As for Wing IDE, it is still one of the best Python editor around, so if you are into serious Python (which i do hope to see more Windows commercial project on this), then its an excellent choice.

C++ or Python

2008-03-30T04:08:00.000-07:00

After playing with python and implemented it in a commercial product lately, i am quite happy with familiarity i have with the language now.

So the question is, will Python replace C++ in my new work? Unfortunately, i must say as long as i am in Windows, C++ is still one of the best language to use for shipping solid, tight and resource friendly code. I can't hire a team of C++ as easily as i can hire and train programmers in python and C#, the latter being a preference in Windows also.

So whats all these fuss of Python rocks etc?
Python does rocks, however one has to taken into consideration at the base level its still pretty much a bytecode interpreted language. C++ however gives you total power and its limited only by programmer's skill and knowledge.

In today's programming, one needs to be practical. If i need to hire good C++ programmers, it would take much effort and its nearly impossible to keep them. If their skills are amaturish we end up with a ton of bugs which needs time to debug and fix and like habits , some programmers takes a long time to fix their coding practice. This is where languages like C# and Python comes in, it makes forming a team fast, it lowers the penalty and possibility of bugs in programming (Python even enforces tabs and code readability to some extent).

Although there are success stories in Pyton, eg the bittorrrent, one must not forget the current popular bittorrent client is utorrrent, which i believe given the size, is probably written in C++. So how does a C++ product replace a Python product given that the Python product was already in the market lead?

I leave that question to you.

Cheap hacks

2008-03-19T00:23:00.000-07:00

Yup, i dont like cheap hacks. I dont like the idea of parroting what the internet community proclaimed hackers has done and presenting it as though as one discover it.

Various security seminars held here in Malaysia consists of more than 90% security exploits that are nothing more that copy-cat / parrotting.

Most real exploits are discovered by real programmers, not hackers. Programmers who understand how the system works and why it would fail resulting in an exploit discovery, sometimes its discovered accidently.

The way some folks use the work "hack" you probably think these folks are hackers, in actual truth however, these folks are just wanna-be. Ask them to show you one original exploit that they discover themselves and you get a blank stare.

Do i hack? No, maybe just some games vs the computer which today are no longer needed due to built in cheat codes. Hacking should be reserved for breaking into security systems.

Just the other day i saw on CNN how some chinese teens contacted CNN and promised to reveal hacking information . While i pity the reporter since he has not much idea whats going on and seeing a "ping x.x.x.x" running in a cmd console seems to impart the "hacking" feel, i doubt those teens have anything solid. Later when CNN interview a guy from Verisign defense , he gently says something along the line of "i dont think those guys are in the echelon to do what they claimed... ". In reality many don't, many just want to project the cool impression that they hack, but they are just running some tools that they cut + paste from somewhere.

On that note :
http://www.rtm.net.my/streaming/login.php

Type ' OR ' , as the name and password, including the ' , and congratulations you just hacked RTM streaming. "Oh wow! new site hack, lets spread the word!!" <- exactly the kind of stuffs cheap trills are made of. SQL injection has been around for ages, no big deal, it will take less than 2 minutes to fix the site up. But these are the kind of demos you get by self proclaimed Malaysian hackers.

Python Take III

2008-03-13T11:34:00.000-07:00

These are my first 3 book on Python. Having read them from cover to cover.
The philosophy of Python makes all the difference here. It summarizes most of the tenets of programming as far as coding is concerned that i believe and practiced although mostly on other languages that i worked on...

If you start the Python intepreter and type "import this" you will get the following beautiful summary of what programming should be :-

The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Python Take II

2008-02-25T19:26:00.000-08:00

Ok.

Here is the verdict. Python simply rocks.
I am going to move one of my project from C/C++ to full Python.

There are a few things that makes Python a real magnet to programmers like myself :-

i - enforcement of readability and tabbings
ii - unit testing is naturally the way of python, each module is developed and tested easily
iii - rapid development for "engine" type of development is fast
iv - superior basic types, like list, and dictionaries that are iterable
v - oreilly python books are all pink (hehhaha, no relevance to edison chen pink mac)
6 - calling C dlls is a walk in the park
7 - works on both windows and linux

I have 3 books in my collection now, Learning Python, Programming Python and Python Cookbook. The cookbook series is a must if you are moving to a new language and you are an experienced programmer. It beats scavaging the internet for junks codes written by amatures, cookbook codes on the other hand are of excellent quality.

Python From a Newbie Perspective

2008-02-09T08:37:00.000-08:00

The Python support for Win32 comes in the form of PyWin32.
However the documentation is very weak and lacks the standard that one normally feels when using products from "de" Giant software company. This is the difference between a product that is made by a company with $$$$ and one that is of voluntary work. No amount of voluntary work could compete with highly paid dedicated people that is assigned to a task.
(activepython on the other hand gives an overall much better starting point)

Although i have the utmost respect for the work done as an open source product and the maturity of python as a programming language, i do feel that such low quality documentations only deters new programmers from fully adopting Python, especially those that have tasted something better like Visual Studios.

Python has been around for nearly 17 years now, but C# has covered more mileage and devotees in a few years than Python.

Here are some sane arguments, as trivial as it is, i felt it is important and is a stumbling block to convert anyone who has been using MS products

Python documentation lacks examples
What better way for seasoned programmers to learn than to just take a peek at how a function
is used? The standard docs that comes with Python 2.5 lacks useful examples. You end up buying a book from Oreilly (Mark Lutz) or scavaging the internet. Alternatively fireup the interactive session and try.
The 2.5 documentation is however much better than the pythonwin documentation, the indexing itself proved to be different.
If you were to use the "index" on the chm file provided for pythonwin, be amazed that one keyword you will have say 10 listing and u have no idea which one is the full description.
Lets take a look at one simple example : DeviceIoControl (win32file, phyton Win)

win32file.DeviceIoControl

string = DeviceIoControl(hFile, dwIoControlCode , data , readSize , ol )
Call DeviceIoControl
Parameters
hFile : int
Handle to the file
dwIoControlCode : int
IOControl Code to use.
data : string
The data to write.
readSize : int
Size of the buffer to create for the read.
ol=None : PyOVERLAPPED
An overlapped structure

Any windows programmer knows that there is supposed to be 2 buffers in the original windows "DeviceIoControl" , one for read and the other for write. So even if PyWin decideds to make life easier by using some smart detection on the parameters and selecting to "return the read buffer" as a return value (string), you can see from the documentation above, no hints is given. One can only be left wondering what is "string" supposed to be, things like "hFile" :int is not a int handle but rather the PyHandle, again hardly mentioned. Examples for using these apis (not just this one) are as scarce as water in the desert.

* the activepython documentation was better since it merges all necessary documentation into one place, however pythonwin docs for win32 stuff still sucks.

* the string return value is the replacement for the read buffer. If the values were actually c-integer , well, you have to figure out how to convert the python string with raw data representing the c-integer into phyton int. Yeah, for seasoned pythoners working with w32, this might be easy, for new comers even with java, c++ or c# background, they would be more productive sticking with their proven tools.

Python has some unjustified oddities (noob bugs?)


x = 0

class Hello :
   def __init__ (self) :
   global x
   x = 100
   def __del__(self) :
   global x
   x = 0

def DoNothing() :
   xyz = 100

a = Hello()
b = DoNothing()
print x

What do you think the result would be? Run on a python and see for yourself, then comment out "DoNothing" and run it again .
Bug? By george, this is already Version 2.5 and if the garbage collector is still buggy with something as simple as this.....

*** The activeState Python however does NOT have this bug!!

Python PyWin32 GUI needs refinement
The autocomplete feature for example will come as a shock to many VS users, instead of the
natural convenience of pressing "enter" to select the autocomplete listing, you need to press "tab". The logic behind "enter" is simple, your right hand is already bz with pressing the "up" and "down" arrows and it won't be doing anything else so using that same hand to press the "enter" key which is just above those arrow keys is fast and easy. Pressing "Tab" on the other hand requires your left hand to move from its natural typing position while the "right" hand which is stuck doing nothing at the arrow keys . I don't think this could be modifed from the config.cfg file for the keyboard layouts.

Its hard to explain but that is what ergonomics is all about, refinement over routine habits that yields comfort and productivity as the goal.

Somehow VS just got it right, even Eclipse misses a mark. For eg, using the php plugin to Eclipse, when you enter [', as a corrective fix to ur expression, the ide will rudely add in ']', without any smart detection that you are typing between the expressions (eg correcting _GLOBAL$(hello'])

When you add up all the "annoyances" in phyton ide, its obvious they need to learn from MS.
But...But...it is an excellent ide for free.

Python needs a proper image
You will likely see a lot of python related stuff using "snake" icons and images as though that was the real meaning of the language. Unfortunately its actually named after "monty python" a comedy show. On the other hand you have O'reilly, the most respected programming book publisher using all kinds of images for its Python books, ironically "Learning Python" uses a "mouse" as its front cover and pink color as the scheme.

Ok, maybe Python theme is a humorous one, but many programmers would prefer if their career and livehood tools are serious cause not everyone gets employed by Google or use programming for academic purposes. (if u can find a company in Malaysia or Sg that will hire you for using lisp and fortran, pls post a comment here, i need to know why some languages i considered "offline" are still commercially viable)

So what are you trying to say ...

So, does this mean that python sucks and we should avoid it?

I suggest you take up this language when you are on some vacation and see for yourself.
The language survived for many years and it does have some really fun and powerful factors
such as list, tuples and dictionary built in types and i love the fact that the tabs/whitespaces are enforced into the language itself.
The interactive session also makes learning fun and unlike the VS that we all "had to love", the size and implementation dont experience the same "bloat" that you get from .Net.
I must say its a very unique programming language that kept me sleepless for few days playing with it, the sheer fun and joy of learning a new language.
I do recommend that you use ActivePython and not python.org (See the bug above, tested on windows vista, python 2.5)

Wifi Hell and Acer 4520 (open suse 10.3)

2008-02-05T22:31:00.000-08:00

Chinese New Year is just tomorrow and i was walking around tesco and i saw some really cheap notebooks, RM 2k for various brands. HPs. compaqs and Acers were the models displayed. The same offer you can find in Carrefour.

I notice it comes with "Linpus Linux", a fedora based linux that is shipped by a taiwan company and Acer is putting it in these low cost notebooks.

So i grabbed one since i am playing with phyton nowadays and i plan to have a dedicated Open Suse 10.3 instead of running it via my Vista vmware (ouch!). There is where the hell begins...

The Wifi wouldn't work and this is not some newbie problem. Although some manage to get it working perfectly on Mandriva and Ubuntu, it seems that the Wifi that comes with this ACER Aspire 4520 has a string of problem when it comes to getting it to work on 32 bit Open Suse.

The said wifi is Atheros Wifi.

The following are things that DON'T work :-

- using ndiswrapper and getting the Atheros wireless driver for XP
- acerpanam.com latest drivers and via ndiswrapper
- madwifi default packages and latest build (up to this blog entry, so feel free to try)
- any vista/xp driver attempting to be used via the ndiswrapper (ndisgtk + related stuff)

There was one Ubuntu forum that shows how this was done successfully on the Ubuntu, and somewhere else a guy in malaysia copied the instruction and posted it as his own...with no credits given. However, the exact steps doesnt work on Open Suse 10.3 even after i mapped the correct differences.

Finally 12 hours of non sleep attempt, here is your ANSWER :

1. Download the following files and transfer to ur linux via USB drive.

http://madwifi.org/attachment/ticket/1679/madwifi-ng-0933.ar2425.20071130.i386.patch?format=raw

http://snapshots.madwifi.org/madwifi-ng/madwifi-ng-r2756-20071018.tar.gz r2756-20071018.tar.gz

Read the How to Apply from here : http://madwifi.org/ticket/1679

2. Read the following and do :-
http://madwifi.org/wiki/UserDocs/FirstTimeHowTo

3. Run the "yast2 lan" or via ur yast2 (my computer, administrator setting) and set ur configuration from there, setting via the iwconfig etc won't work. After that use the icon on ur system tray and connect to ur SSID (router), re enter the WEP key if needed.

There. I just saved you 12 hours.

As for Nvidia drivers, just download http://www.nvidia.com/object/linux_display_ia32_169.09.html
and do the sh NVIDIA* instead of depending on the yast.

Kids Stuff - The ultimate in learning Programming

2008-01-14T22:45:00.000-08:00

When i was in form2, the teacher gave all of us an assignment. Write a report of a survey of any topic . My particular group consist of 3 guys and we decided to do a survey on "What kind of boys you like to be your boyfriend". Yeah, seems like we were taking advantage of the teachers assignment to satisfy our curiousity. ;-)

My school back then was St.John Institution, a school that was the envy of many boys school, not because St.John has the best atheletes back then (even though we claimed we were) but because we were surrounded by girls school.

Just across the road we have Convent Bukit Nanas (aka the "classy" girls), not far away we have St.Mary Girls' School (aka the "best" choice) and down the road we have Bukit Bintang Girl School (aka the "naughty" girls who didnt make it to the other 2 schools).

The survey was completed in 3 days since the questions was circulated via the school buses that conveniently provides transportation for most of these schools. I was given the solemn task of compiling and making the results into a report.
But instead making it a dull and serious report, i used a format back then that got people laughing. I cut various pictures from magazines and drew baloon dialogue boxes to present the points. Most thought i would be given a stern warning from the English teacher, but instead she praised me for creativity.

Fast forward to my current time now..Welcome to the new school where programmers needs to learn many stuffs with minimum time. The Dummies line of books has been very successful so much so that you can find all kinds of topics in it and why was it successful? Because the presentation was clear, and people are tired of reading dull scholarly words.

Just the other day i got myself a book on "Head Rush" by Oreilly on certain programming topics. After reading it for 2 days, i conclude that this kind of format is one of the best books available today for learning IT .

If you are curious about certain programming topics, like Ajax, C# or Javascript, before you jump all over the net gobbling up every garbage you find and end up being half-cooked, I suggest you take up "Head Rush" line of books for the topic you are interested in.

http://www.headfirstlabs.com/

Bad Security Programming

2007-12-15T03:26:00.000-08:00

I am often amazed on how much privacy and security violation that often times appear on
websites.

Here the latest one i notice just a few days back. It not an actual programming problem but more on common sense privacy violation.

Goto : www.pizzahut.com.my

Goto "Register", enter any phone number of a person that you know of, and chances
is the Pizza hut will show you the "address" where that person is staying or last used to order
pizza. Yes, this means you can get the address of anyone who happen to have registered with
PizzaHut EVEN if that person ordered via the PHONE last time.

What does this tells you?
Anyone can locate your house address and other information if you have ordered anything from Pizza Hut via phone or online.

Conclusion : Call up pizza hut to remove your information on their database, or if you havent registered or ordered via phone, DONT.

Alchepo Guide to Software Quality Assurance

2007-11-19T22:13:00.001-08:00

Face it. Most teams don't have time to do a proper quality check on their products, instead most depend on "trust". Its impossible to expect the Guru programmer to check everyone's source code to look for flaws and potential problem. Its equally insane to expect amature programmers to code like a pro equally.
So how does one implement quality inspection on the code? In many cases, its the end product that is installed and tested in a given time. Although end product testing is a must, i am talking about other stages where quality can be enforced and not just the end stage.

Here are some of my suggestion, mostly based on William Edwards Deming's key recommendations and paraphrased. Who is he? If you study on how Japan manufacturing and quality rose from the ashes, you will know who this person is. http://en.wikipedia.org/wiki/W._Edwards_Deming

One of the important rule you have to remember is :-

- Substitute personal supervision with.. Statistics.
This means you must ask your programmers for proof of quality instead of "have u tested it?" etc. For eg, ask him/her to run a set of tests and show u the result of those tests on his routine/function.
One alchepo way is to also record down the memory usage and handles of the program by simply running the task manager and recording the usage of that process before/during/after the tests are conducted. This simple practise ensures that no crazy leaks occur in the program which will result in those popular scenario where the program runs perfectly in day 1, but crashes in a week.
This should be conducted whenever they perform unit test on their new routine/program.

- Diverse data testing
One of the most evident proof of an amature programmer is when he/she says "it runs ok on my machine" where else it crashes just after deployment on the first client pc.
The product/routine must be tested in as diverse scenario as possible, you dont really need an infinite number of pc's to do this, just some range of diff setup. For eg, a new setup with no service pack or on Virtual pc, and some end users pc you have in your office with loads of rubbish loaded. Better still is the industry practise of providing a beta testing to selected clients or public.

This is by no means a complete or even decent article on software assurance, but if your team have no proper method anyway, these 2 steps alone will secure ur eternal gratitude to this blog . ;-)

Windows Mobile Dominance : Dopod / HTC = ?

2007-11-05T16:56:00.000-08:00

Around 2-3 years back, i got myself the new Xphone, powered by the SmartPhone OS which is basically Windows CE based. The idea that the phone ran on Windows was interesting so much so that the curiosity got the better of me and i parted with RM 1,800.

(photo from gsmarena)

Thats when the nightmare began, it seems like the phone was nothing more than a plastic shell that ran Windows with no regards to phone practicality. Although the contacts smart search (it searches both letter and numbers when you press the relevant keys) was a great design, the rest of the functions just falls out of place. The phone taught me how refined a normal Nokia was and how foolish was me to think that MS new venture (back then) could possibly be better.

One of the most irritating problem i recalled was the SMS ringtone. When you are using phones like nokia, alcatel, motorola and even Sharp, the sms ringtone are discreet or a soft vibrate. On the XPhone, it just "DING-DONG!" right into your ears not realizing that you are on the phone talking and your ears are like just milimeters from the speakers. It rang right out, without detecting the current state of the phone. That shows how "basic" the phone was. I did read a review on an Australia ZDnet before buying the phone, i guess the reviewer never really used the phone. The phone was sold under O2, but if you dont know by now, its all by HTC. Couple that with frequent hangs and reboots, the phone was sold off to a guy who wanted to use it as contact backup at a fraction of the price.

Time passed, so does my grudge against Xphone. This year 2007 during the CNY, i got myself the Dopod838 pro, surely the best of Window Mobile won't go wrong was my excuse this time.

(photo from dopodasia)

The camera was chun!, the video was perfect for recording impromptu stuffs. The Outlook synch worked perfectly well and the phone comes with a sliding keyboard, something that my fingers easily adapt to.

Then comes the problem :-

Its a 3G phone, when you are surfing on the phone, all incoming calls are dropped, you won't even hear a ring. You cant make any calls out either. Yes, one can argue that 3G uses the same bandwidth as the calls thus it was technically correct to make the phone work this way...GO TRY A NOKIA, it would smartly pause the 3G while you chat and resume when you hang up.

The sliding keyboard frequently hung, the screen was no receptive to anything you type on the keyboard, i ended up switching to the touch screen and pen.

You will need to turn "off" the screen by pressing the side buttons or the phone can be configured to "energy saving mode" when idle for minutes. But most likely you will just press the side button like all other Windows mobile devices (ver 5 and 6). Now this poses a problem, the device comes with so many other buttons which are accidentally triggered somehow even when the phone screen is already off. Thus you find a slew of applications running when you turn on the screen and it lags the device to a crawl. Solution: Assign all the buttons (except camera) to "none".

The front buttons have a paint that easily peels off, for a device that cost RM 3k (ouch!) this kind of quality is not expected. I wasn't expecting the VW golf 4 layer paint coat, but hey its just 3 months down the road and it peeled.

If you are new to PDA+phone and you like to quickly reply SMSs while you are stuck in a short traffic, you would want a phone with a real keypad.

I upgraded the OS to Windows Mobile 6 (official dopod838 pro patch) , yes the new menu's better but overall...the phone is still very much slacking in practicality as a phone. I find myself nagging the phone and getting impatient with its flaws and missing the good'ol Nokia 6630.

Well, thats me, on the corporate industry and IT world however ....

Windows Mobile is a very attractive choice.

MS Exchange comes with free push email (Direct Push). Blackberry, Visto, and Seven (Ericsson mobility) require a third party provider and expensive upfront payment. Then there is Funambol (http://www.funambol.com/) which claims to provide free push email Open Source. Funambol won't work in Malaysia, not on Maxis and other providers, it require an incoming connection directly from your Provider to your phone, which only means it can be implemented only by your provider. I did manage to make this work, by writing our own proxy clients (java for nokias, sonyerics and .net for WM) on the phone and modifying the server to provide straight push email. However, the market expectation of push email speed and funambol server polling design makes it non viable.

Vista beautiful support for Windows Mobile via the new Window Mobility Centre.

Thus i would say, as more people demand for push email becomes apparent, some would turn the blind eye to the Windows mobile flaws and embrace the Empire's offer. (imagine the evil star wars empire soundtrack playing....teee tee teee te te te te te te)

May the mails be with You....

Programming Job available

2007-11-01T21:43:00.001-07:00

There will be a booth in TARC this coming 14-15th Nov, 2007 held by my company.

The company wants to hire programmers, if you are a tarcian and attending the event, i gurantee u will not miss our booth (ha ha)...the posters that was done is out of this world.

"Code Wars" - Master Yoda poster

"Enter the Coders" - Bruce Lee poster

I hope the rightful parties wont be offended as the images are just used for this one occasion, for the same reason i wont be posting the images in its actual format.

Actually if you are looking for a programming job as your starting career or if you have 1-2 years experience, you just need to be able to write this program, and my company would love to hire you.

Program Test: (C# or C++)

1. Write a program that will write to a file, the following random objects

integers, alphanumberics, floats and seperated in a file. The file size must be around 20 MB.

Eg, 23213, 1.2312, helloworld, 88593, what is this, 88.22

2. Write a program that will read from this file and print the types of each token that was read from the file :

Eg, 23213 = integers

1.2313 = floats

helloworld = strings

Thats it, simple and straight forward. Just complete the 2 programs on your own.

Send the program project to the company at : onlythebest@internetnow.com.my complete with your resume .

Its all about Design

2007-10-29T20:24:00.000-07:00

What happens when two programmers of equivalent skills and same field exposure are hired to write their own version of the same sofware? E.g A Corporate Messenger or something as simple as a Leave application system.

The "Design" will be the distinctive line between the amature and the pros.

When i say design, i mean the way all the components and function calls interact. When i say design i mean how the UI will be presented to the users. When i say design, i mean how the product ends up in different modules/classes and processes?

Let me give you a real life example :

Write an application for XP that can retrieve MAC addresses given a list of IPS in the LAN.
A layer2 passthru packet driver is provided.

Programmer A# : Adds a function in the layer2 to provide ARP request. Adds a function in layer@ to process ARP reply. Adds a function in layer2 to receive the list of ips to test. Need to determine which Network interface to send the ARP request/reply. Send the information back to User mode .

Programmer B# : Adds a function in layer2 to receive the list of ips to test. Adds a function to match incoming ips and extract from packet header the mac. Send the information back to User mode.

While Programmer A# seems to have coded the more technical way, its also the more time consuming approach. Since most the the functions (arp handling) is also handled at the driver level, that means more vulnerable BSODs are bound to occur if any bugs were to surface.
That also means that you need to send the packets out in the first place and you will need
to determine which network interface is the one that talks to the LAN.

Programmer B# on the other hand, you might ask how will he ever get the MAC address since he did not do any ARP request/reply functions at the driver? He did that via the usermode....all he did was issue ping via the raw socket api, and the Windows XP naturally send the packet out via the correct network interface. So all he need to do at the layer2 driver, was to match the return ping replies and get the MAC address from there.

Why write your own ARP handling when Windows already paid $$$$ and hired XXXX programmers to provide the functionality? A choice of design.

Same product, 2 design.

Design is everything. I see "design" everywhere, from the way the texts are displayed when you boot up the system, Windows displaying its welcomed screen, the license agreement layout and presentation, how my power off button is used...everything.

Some design are just so dumb, it makes you wonder why the person/company that does it never saw the errands of it. There was a time back then when USB was just coming out of the oven and you see tons of manufacturers and ham-kei shops producing their desktop with USB support.

I notice a faulty design choice that many chose... Why in the world, was the USB port BEHIND?
Its was just a plain "everything else is at the back, so just put it there". While those com ports, printer ports and power connectors have always been conventionally at the back, the USB port which is a drastically different port was placed errornously with the rest at the back.
Of course, soon, some bright sparks soon realize the fallacy of the design and in just a matter of months, kazam! USB ports in front of all major Desktop brands.

The other day i was passing by USJ and i notice one big advertisement board near the Summit left turn : Giant 70,000 and more items. The problem is, people could not visualize how much is 70,000 and that figure doesnt impress anyone. The casual mind is just not able to associate how great or significant is 70,000 in a hypermarket item count. The mind could only remember 3 type of things, which if i were to do marketing, i would use these 3 factors to the fullest:

1. Humour / ridiculous factors
2. Great quantities (eg you see uncountable warriors in an epic movie)
3. Big sizes (imagine godzilla 10 times bigger coming to town)

These factors you can extract from books that teaches you how to improve your Memory.

People are bombarded with figures like 1,000,000++ in many marketing materials, what is 70,000 ? Thus putting that advertisement is a big waste of space, they might as well put there "GIANT" surrounded by small uncountable "giant" and it would strike a better impression.

Can i be a programmer for life?

2007-10-19T18:30:00.000-07:00

Quoted from jobstreet.com -

"The two greatest gurus of software development in Malaysia, Ralph Tee and Marcus Low will address your concerns about the career path of a programmer. Having charted many successes in their careers and earning the accolade of being the brains behind cutting-edge Malaysian technology which had penetrated the international market, Ralph and Marcus will cut to the bone, challenge your beliefs and present their outspoken views on the following:

1. Is there a future to being a software programmer? (As Ralph puts it, have you ever met a 50-years-old programmer? Is your programming career over after 35?)

2. What does it take to be the best code guru?

3. What should you do and not do when you are applying for a programming job? If you are a programmer worth your salt, if you are a software developer who sees heaven in lines of code, this is a forum you cannot afford to miss."

Years back i was invited by JobStreet to give a talk on career as a programmer. This is one subject that is closed to my heart. Having worked and interacted with all kinds of programmers in Malaysia, I have identified and listed out various "mental" stumbling blocks and attitude that prevents a person from being a successful programmer which leads to a frustrating career.

So is programming a job for you? Watch the Videos below and download my presentation.
You will need to watch the video to interpret the powerpoint, else its gibberish. The first half of the talk was conducted by Ralph Tee, a respected programming guru that was the author of Armour Anti-virus back in 1993. For those who didnt know, Armour was the best selling antivirus from 1993-1996 up to a point where in 1995 the US defense ministry uses 1.5 million license of it (in a diff brand name ^_^). Ralph wrote tons of other programs and recovery tools that he never made public but very interesting. He currently runs his own company doing virtual Telekom mapping projects, POS malaysia and other project based stuffs.

Someone in the seminar wrote me this "what qualifications you have that Jobstreet labelled you Malaysia Top gurus in programming" Frankly, there is no way u can get yourself such a title being conferred by any organization. Its the people in jobstreet and their personal evaluation ^_^. Personally i feel that anyone's programming skill and worth is only measurable when its weighted on a global scale. When the result of your product is matched againts the borderless world, the demand for your product or your skills is put on the real scale.

Amazingly, one of those mentality problems malaysian programmers have is to quick to compare with local talents when the real standard is out there. Its okay when someone says "This mr.foreigner is a great programmer, one of the best". The moment you hear "This malaysian programmer is one of the best", arrows and backstabbing comments come flying. What profit is there when 2 proton car engineer argued on who designed the better proton model in the line-up? Here comes VW...

I would recommend anyone downloading the powerpoint presentation to bear in mind the principles involved, but the material itself is 3 - 4 years back, and in IT that is one whole life cycle. Today i would recommend "C#" as part of programmers foundation to work on.

Video1 (113MB) (right click and save)
Video2 (13MB) (right click and save)
Presentation (ppt)

Windows Pro but Noob on Linux : Things you should know to move quickly in Linux

2007-10-18T19:15:00.000-07:00

Ur a hot shot programmer in Windows. Nothing is too hard for you in Windows.
Suddenly you received an email from your employers, "Our next project will run on a Linux Box".

Here are things that you should know that can save you time and get you moving quickly :-

1. If its going to be webbase thing, you can leverage your .net skills in Linux. You can even host ASP.net on Linux or write application in C# that will run on Linux. Check out this site for more info : http://www.mono-project.com/

Alternatively you can learn PHP, the syntax is similar to C/C++ so if you have experience in C#, this should be no problem. However you will soon notice that you need an IDE that recognizes PHP, i suggest you just use your Visual Studio .Net and get this php syntax highlighting FOC from this David Crumps : http://blog.cumps.be/visual-studio-2008-and-php-coloring/

Although there are commercial IDEs for PHP, but you can easily lookup the function reference
from the windows chm file you get from http://www.php.net/. In Visual Studio (2005 at least in my test), the Designer view is even better than some of these commercial PHP. This allows you to easily see the xhtml layout and code php at the same time.

If you going to use php, i suggest u start right on with smarty and php together. You can get smarty from http://www.smarty.net/ . Its a templating engine that will save u lots of time when you do php and its so much cleaner on your code/ui separation.

2. Learn about Daemons, Crontab and linux file permission. These info are easily available anywhere . For file permissions read this up : http://www.linuxcommand.org/lts0070.php
Linux processes and daemons (eq windows services) uses configuration files for their settings and configuration. These are the basics you should at least know and understand.

3. There are many linux distributions, use one and stick with that one. Depending on your linux project, if its a hardware, its easy, just choose any one of the latest Linux distribution and use that for all your deployment. Each distribution is trying to value add to the generic linux and thus its kinda problematic if you want to deploy to all versions.

Take one of the Anti-Virus my company uses for example, www.drweb.com . Its one of those few antiviruses that supports various linuxes, it has basically a diff scripts for installing to each distribution to protect its Samba folder (haha yeah, you're not really going to sell unless u are protecting the samba folders which many are used as file sharing with the the windows stations).

The popular choice would be Red Hat Enterprise Server, Fedora, Debian and CentOS. Now talking about CentOS, here is one event that i would labelled as the worst "User support" case ever, the kind of user that would be your technical team's nightmare : http://www.centos.org/modules/news/article.php?storyid=127

Linux vs Windows

2007-10-17T03:54:00.000-07:00

I have never been a fan to Linux vs Windows discussion. Yes, i have worked on Linux and Yes i know it has tons of network tools given to the admin that made win95/98 looks like a joke. But that was win95 back then.
The argument that linux is secure, stable due to the open source many "eyes" concept has never been accepted by me. Yeah, i do agree win3.1 and 95, 98 was a rough ride when it comes to security and even on 2k and xp, it was hackers haven.

But then consider this, if you load windows with just those essential drivers, modules like what came in linux, it would be as stable as it is. Heck, i have witness so many linux firewall hanging after adding this module and that module, its just no diff. I am talking about linux firewall that is retailing at RM 50k and above.

Stability is just using whatever that is standard, tested and proven by a party that practices safe programming, be it Windows, Linux, BEOS or MAC OS.

If u ask me, hehehe, i seriously think that most of the unstability in Windows is caused by security products itself! Such as antivirus and personal firewalls. These programs uses drivers (sys=nt,xp,2k, vxd=95/98) which if buggy can easily crash the system. That my fren, is no different than you adding some hooks into the kernel for linux and compile and use it when its buggy. It will crash.

In fact after giving myself a lot of time looking at those projects for linux, i can conclude only one thing.

Linux will still have a market on its own - that is in appliance.
Being free as in $$$$ is a reason so strong its absolutely ridiculous not to use it as an appliance OS. An open source kernel in which u can add.change anything you want, inspect anything, hook anything and the software components of Apache, php, java allows decent development to make the appliance works. These factors just makes Linux win easily as the choice OS for appliance.

Apart from that, design wise, there is nothing in apache, php that IIS and .net doesnt win.
Yes, PHP has tons of supporter, i use it too! However once u use both tools, you cant help but admire the design that the .Net has put in for web development and dynamic content. Basically its makes php looks really dated.

There is a reason why Windows was the best seller and popular choice OS for all time, for some, the reasons seems uncomprehensible, for others, its not a choice. For many programmers the choice is more emotional than logical, we all remember how MS crush netscape, how MS crush Borland C++, how MS this and that , deep down, nobody like to give in to a monopolizing Giant.

What if today MS suddenly declares that Vista will be sold at RM 50 and Windows
Server 2008 at RM 100? I bet the market share for MS will grow exponentially crazy.

Now...Linux is free isnt it?

Nothing beats Soft-Ice for live engineering

2007-10-16T21:30:00.000-07:00

Soft-ice, the mighty sword i have wielded for many generations of windows os and days of DOS.
How have thou fallen O my comrade, thy blade would no longer pierce the foe Vista. I mourn for thee Soft-ice, has thy maker abandon thee?

I love soft-ice. I have used it to reverse...hrummp... i mean analyse (i dont promote cheap hacking) many applications and malwares thru the generations. Its a loyal and trustworthy companion for any serious system programming. The fun i had in the Ultimas, Red-alert (super sonic tank), alone in the dark, , neverwinterII (using the cage as armour), ROM (256 rays!),diabloII (99% life leech...ouch), DungeonSiegeII (complete the mysterious quest, damage of 14124124) and those cant even recall endless list of games (did i mention no cheap hacks, i never mod life stats and str, mana etc...those can be done via trainers and even built in cmds, no fun if u cant die)

While we are at this, i got some confession to make, some naughty stuf i did in this IT world during my earlier reckless days...

- Ixxxxx college, around year 1993/94. My sincere apologies for crashing all the pcs for a duration of week. Shooting masses of msgs via the netware send api that refuses to go away when u press the ctrl-enter. Yeah, it was no virus...

- to all the cybercafe players in pandan indah, during 2003-2004, counter strike 1.5, 1.6. Miyu, Camper369 is actually a cheater = me. Yup, i wrote a client proxy via the blueprint from ogc, and i played in the cybercafe. Since the gun auto aims the "cookoo part" and hides all UI elements and triggers via the keypade keys, you guys never knew what hit u, even when you are standing behind me watching me play. That program is just...well... anti-anti-cheater and anti-ppl-stand-behind-to-see-if-i-cheated version. Thanks for all the fun and frags.

- sincere apologies to Tay cheng yik and his cousins and collegues. For i used the same program on the silversurf cybercafe when u organized the counter strike match . Yes, you were smart the cybercafe comes with the security of not allowing any downloads and processes....but i went there earlier at 6pm and well..."loosened" the security so my program would load. How? Well the security they granted to yahoo messenger was too high, so its just a matter of renaming and putting my program as yahoo messenger. Now u know, why i didnt agree to join WCG.

- QuakeII office match, wait a minute, i was a QuakeII master, what did u expect to read here, i never wrote any mods to help out in this one. Where did i put that rail gun...u just wait here.

Sob...Sob...

Soft-ice wont be available for vista. So what alternative you have. For one, u still have the windbg (yawn...), Ollydebug and Syster. I have tried the latter two and they do a good job in analysis but for live reverse....i mean terbalik engineering on live processes/drivers, nothing beats Soft-ice.

Soft-ice....(ok skip ahead, leave me mourning here for my loved ones..)

Microsoft Expression Blend (sept preview)

2007-10-15T22:51:00.000-07:00

What does an UI programmer, a System programmer and a Computer animator have in common? The UI complains "i am not a web designer, those cool flash sites you see, well they are more of a designer job". The System programmers proclaims "i only write engines, those UI coders are no fun anyway". The designers wish list "i wish these jokers understand i cant perform well unless i know the context of how this image will be used in their UI in detail".

Now everyone gets a chance to put themselves into everyones' else shoes.
In MEB, you can animate a control using a timeline....did i say animate...., yes
animate, animate, animate, and TIMELINE, what in the world is happening here? What does a programmer do with an animation timeline? and wait...layering on my toolbox controls? Layering as in Adobe photoshop kinda thing? Dreamweaver??

You bet...the MEB is one heck of an IDE providing all these functionality, wait maybe calling it an IDE is insuficient to provide the right impressin, a mega ide perhaps, a tool that integrates everything that powers an application. (isnt that what an normal ide does? heheh dont try to be smart mouth here...)

MEB is possibly one of the most radical IDE i have the chance of playing with. It infused great power (well not so great responsibility since generated app ran on .net 3.0) and promotes the WPF concept. I must say the first 30 minutes of playing with it caused my mind to ran back and forth thru time and space, to venture the possibility where no man has gone before.
Its one of the most exciting tool :-) for bored programmers.

Having said that, i was working on pHP and java and been using various IDE's (emacs, Nusphere, eclipse, netbeans) before i completed the linux box project. Well, if the linux community was happy with those IDEs they have (there are still some clueless ppl who champion the use of vi or notepad for programming but i am talking about professionals that maximizes a tool like an ide for dev in linux), the MEB would be a real eye opener.

Wait a minute, why are you talking about linux? You are talking about some Microsoft stuff here and that won run on linux, so it has nothing to do with the linux community.
Actually, i have this funny feeling that the borders are coming down fast, heard of mono? The .net being supported in linux also means hordes of windows coders will find their skill and knowledge workable on both worlds with minimum effort. One may argue the other way is true, for eg, java running on windows, well honestly, java apps running on windows not only looked weird but downright silly, they are just anything but cool. Remember there was once that Lotus Notes on the server ran on a command console box (dos like)? Yeah that feeling.

The developers have ensured that whatever u developed on MEB would ran on XP sp2 onward, now that isnt too bad of a criteria seeing that w2k is at least 7 years old now and most Xps could dl the sp2 anytime. U still need the 3.0 .net though.

And as expected, c# is the core language for using this tool. The MEB would launch your VS 2005 when you add the code behind the controls.

Here download it and watch some of the samples provided :
http://www.microsoft.com/expression/products/download.aspx?key=blend2preview

Winsock2 Replacement (ws2_32.dll)

2007-10-11T21:12:00.000-07:00

Ok one day i decided try something funny, basically i wrote a winsock2 dll (ws2_32.dll) that will intercept all the calls to the real ws2_32.dll. Sure there are plenty of winsock hooks around town, but a replacement dll? oh come on, it must be a big waste of time, but anyhow i did it.

The advantage of a replacement winsock is obvious for research reasons, for one, it only intercepts whichever application folder that it is running in, without interfering the rest of applications that uses winsock, thus you dont need to filter the logging of other apps.

How was it done?

First i wrote a pre-processor that will preprocess the winsock2 header files including the ws2spi.h from the sdk and churn out a cpp and a def file. These files are then used to create a project.

To use the DLL, just copy the ws2_32.dll to the directory where your application resides.
Eg, c:\temp
Then run your application from there, eg c:\temp\telnet.exe

Depending on what kind of logs you want to see, you can create the following at c:\ before
running the app.

c:\winsock2.dll.var (this must be created if u intend to start logging, then content of this file is not important, enter anything)
c:\winsock2.dll.log (create this file with any text as the first line, this file will contain logging from the replacement winsock)
c:\winsock2.dll.binary.var (create this file with any text as the first line, this file will inform the replacement winsock to also log binary data)

I have used this fun project to intercept and make modifications to some legacy apps running, for eg, some dump apps just insist of listening to port 8888 which some corporate IT admin refuse to forward that port, thus the replacement winsock replace the port to 80 (u get the idea).

You can also use it for eg, to learn about data and protocols, eg put in the firefox folder, or msn messenger...well i guess i leave the rest of the ideas to you.

(* one more thing though, last time i used it was ages ago, before vista came out, so dont ask me for vista version if it doesnt work)

Here, download the complete source code with built binaries of the dll. (no support for this of course, its freeware from me)

codemagnet.winsock2.zip

Have fun!

** Update : See remarks (from Martin) Modified version. (click on the winsock2_getaddrinfo.rar)

Apollo, Flash, Flex, (Ajax). Which way to go ?

2007-10-11T03:21:00.000-07:00

Well unless you are living in a tempurung, you probably agree that any cool development in the past few years are web based development. And you would also come to a conclusion that flash is the best dynamic content format that can be employed...well at least for the past few years.
(althought ajax does a decent job, heck who are we fooling, look at those flash sites, ajax btw is not product haha so its not a comparison but i am referring to the effects and results)

Anyway, many developers ask me this question as to whether they should venture full force into application development with flash. Instead of whamming a do-or-die opinion from me, read the following blog :

http://weblogs.asp.net/jezell/archive/2007/05/03/silverlight-vs-flash-the-developer-story.aspx

There....make your own decision.

Device Driver Woes

2007-10-11T02:43:00.000-07:00

If you are deploying an IM driver or some other device driver via the dpinst.exe, here are some problems and solution :

1. dpinst says updated, but nothing happens, the old drivers are still used.
Set the DriverVer to some bumped up values and date.

2. dpinst fails to update one of the driver.
Look at \windows\dpinst.log. The error codes are listed there, if the error is service name then you will need to remove that name from the registry under system/services. A reboot is required else you will get an error like 0x6xx.

3. try doing the dpinst /U xxxx.inf , on the previos version to force remove the old version
before you install the new one

4. Do a SnetCfg -l command on the service name (eg ms_passhtru) to replace the drivers.

Reading the entire section on *.INF format in MS doc is a necessity if you intend to do serious driver programming.

Moving to .NET : Multilanguage support

2007-10-10T21:04:00.000-07:00

Best MultiLanguage Example

The best example no doubt is the one provided by Microsoft :
General_LocalizedHelloWorld.exe

The rest as found in code project are some singalimgam saying and quoting each other and giving half-baked solution.

Multilanguage in .NET (C#)
You will meet a load of problems if you dont follow these steps.
Basically, u need to create Satellite (*.dlls) that contains the resource for each of the language. These dlls are stored in the local directory where the .EXE will be running.
Eg,
en-USAPopup.resources.dll
enAPpopup.resources.dll

The filenames are language-culture shortcut names to use. These are not just alias, it must be EXACT.
Then make sure your ASSEMBLY name and NAMESPACE is same. This is to avoid asking yourself funny questions later on like, what is the resource name to load. You can always rename the *.exe output yourself later on and it will work.

Your code will be put before the InitializeComponent:

Thread.CurrentThread.CurrentUICulture = Thread.CurrentThread.CurrentCulture = new System.Globalization.CultureInfo(“en-US”); // example, or “” for neutral
MyManager = new System.Resources.ResourceManager(“APpopup.Resources.MyResource”, System.Reflection.Assembly.GetExecutingAssembly());
string str = MyManager.GetString(“form2.renew”);
InitializeComponent();

On your C# projects you have to create 2 additional directories, 1 for resource (the resx files) and the other for satellites. You can do this using console but heck its a mess, btter use the IDE. You create this folder by :

1. Right click on your solution on the solution explorer, add new project, select VC++, empty project.Name your project “Resource” for example.
2. Edit the new project properties, set the type to “Utility”. This will get rid of the C++/Linker etc tabs which u wont be using.
3. Set the files need to clean up (eg if resources, *.resources)
4. Add the *.resx files here. You need the names to be like this. If your namespace, assembly is : APpopup. You might want to call it : APpopup.MyResource.resx for the main one. For english-USA (en-US), you call it : APpopup.MyResource.en-US.resx etc.
5. Under the custom build, command line : resgen.exe “$(InputPath)”

Under the output path : $(InputName).resources
Set the dependency of your main project to “Resource” (the above project) so that it will always build that one first before building ur main project.
Now do the same for Satellite folder (Step 1-3). Except you dont have any .resources files to add here. So you need to build the “Resource” project up there first. Now add all the “*.resources” to this project.

Now one more VERY important STEP.On the custom build, command line :
mkdir ../windowsapplication1/bin/$(OutDir)/en-US al.exe /culture:en-US /out:”../windowsapplication1/bin/$(OutDir)/en-US/APpopup.Resources.dll” /embed:”$(InputPath)”,”APpopup.Resources.$(InputFileName)” /template:”../windowsapplication1/obj/$(IntDir)/APpopup.exe” Output : ../windowsapplication1/bin/$(OutDir)/en-US/APpopup.Resources.dll

Create a “Resources” folder directly under your main project (not your solution which u already have created Resources Project). Under this folder put in the neutral resource, MyResource.resx or MyResource.resource. Without this you always get a "tut-tut-tut-censored (these are vulgar curses..LOL)" error on your application when your culture info is neutral : CultureInfo(”")

There, thats all you need to do.

Porting winpkfilter to Vista compilation

2007-10-10T19:54:00.000-07:00

Moving the old compilation of winpkfilter that was used for win2k,xp to the new Vista.
However note that from now on, its NDIS 5.1 and no longer 4.0 so it wont work on win2k.
I will be using the IM version and no longer the ndis hook.

1. Set the correct build environment
you willl need to install the winddk. Set a shortcut like this :-
C:\WINDOWS\system32\CMD.EXE /k c:\winddk\5483\bin\setenv.bat c:\winddk\5483 chk WXP

2. Correct the misc problems
- Apparently efilter.h and afilter.h is not used, thus create dummies for these.

- next the ndisrd.rc complains about afxres.h missing, duh... just copy the .h from /vc/atlmfc/include in ur 2005 Visual studio.

- copy the nmsym.exe, nmtrans.dll, msdia20.dll to your new vista pc folder, and make ur reference to these when generating nms for softice. (note to self : sice no longer works on vista move on, debug sice only on wxp machines)

- chkstk.obj is no longer used. So removed it from the sources file.

There, that should do it, type "build" and we're done here. The new driver will work on xp and vista as intermediate driver.