Tuesday, August 9, 2011

A dangerous Pitfall in using cPickle + import

 

Assuming you have a module : myfunc.py

from  defunc_single import *

And defunc_single have a class that is  :

class Hello :

       def __init__ (self) :

               self.name = “hello”

Now if you access this Hello via myfunc.py like this :

o = myfunc.Hello()

cPickle.dumps(o)


and write it using cPickle, the module defunc_single is also pickled into the file.



So that means along the way, if you felt that you wanted a new module : lovefunc_single.py  with a better Hello and you thought that you can just replace the import statement in myfunc.py to :



from lovefunc_single import *



The cPickle will fail  to load the existing data unless you stick to defunc_single.py

Friday, July 29, 2011

Zodb : What you should know.

 

A programmer recently was dismissed for failing to produce satisfactory results after working on a project for 4 months.

He told his employer that “zodb is just too hard”.

Now i have a similar project that i am working on and decided to use Zodb for my storage. As usual i started to google for some quick examples and documentation.

2 weeks into the project i realize that some of my data are not “persisting” as it should while it managed to fool the unit test . I wrote some small single modules to simulate the situation and realized that the same data are not persisting even though another variation of the same code does.

To cut the story short, if you are doing or intend to use Zodb as your storage, just follow the rules below, forget whatever “friendly” samples you have seen, they just don’t reflect the issues that can arise :

1. Any classes that you store in the Zodb, derive it from “Persistent”.

Eg :

from persistent import Persistent

class Whatever (Persistent) :

2. Any dictionary or List that you want to store in Zodb, don’t bet on your luck, use the following wrappers.

If you want to store dict, use PersistentMapping

If you want to store list, use PersistentList

from persistent.mapping import PersistentMapping

from persistent.list import PersistentList

root[secret_key] = PersistentMapping()

root[secret_key][“mykey”] = “yada”

3. The connection and root must be opened by the thread that wants to use it or only use 1 single thread

storage     = FileStorage(Path)

db          = DB(storage)       
            
connection  = db.open()
root        = connection.root()

while “storage” and “db” are shareable across threads, “connection” and “root” are not. Each new thread must do the “db.open()” and “connection.root()” call.

Alternatively you can assign one single thread to do all the above opening of handles and access the “root” and “connection” freely among multi-threads, but you must ensure that “single thread” you assigned will be the same one closing the database.

Tuesday, July 12, 2011

Irresponsible Sharing

Its pretty frustrating sometimes when you want some quick scripts to do your job and some of the given scripts are not only mediocre but non functioning.
Today i had the urge to quickly backup a directory i have to a remote ftp, so i figured that surely someone must have written a python script on this! After 30 minutes of searching and looking around, i found :
http://code.google.com/p/ftpsync2d/downloads/detail?name=ftpsync.py

But guess what, the code don't work on Windows. The bugs you get from it are very typical of newbie who have never coded in dual platform. For eg, it expect the path to be in "/" where else when it ran on windows it automatically converts to "\" and it bombed out on assertions.

Dumb? No. Just typical and if i knew i gonna spend 30 minutes to do this i might as well fire-up my FTP client and do it.

Gee guys/gals, if you want to post some codes and scripts on the Internet for the community, please at least raise its quality to an acceptable level. We have enough junk codes shared, we don't need any more of it, or the least, label the project that it only works on Linux/MAC/Windows.