I’m in a way of releasing few of my projects to bitbucket.
Starting now I’m releasing the Sqlalchemy model generator. A quick reminder, it’s a project that you
can generate models based on your database. So you don’t need to write models by yourself.
You can see the project here: http://bitbucket.org/marcinkuzminski/sqlalchemy_model_generator/
Any participants are welcome.

While I love many things about SQLAlchemy this was something I just missed sooooo much – defining model’s methods without having to pass session object.
While using Django ORM for such action it is extra easy as there is no session objects at all (well, there is connection object but it’s much simpler and less sophisticated).
Assume you have BlogEntry and BlogComment mappers and you want to define function at entry model which returns related comments – I know, it is as simple as it could be, but it’s only an example.

class BlogEntry(Base):
    __tablename__ = 'blog_entry'

    title = Column(String)
    content = Column(Text)
    created_at = Column(DateTime, default=datetime.datetime.now)

    def __repr__(self):
        return self.title

class BlogComment(Base):
    __tablename__ = 'blog_comment'

    content = Column(Text)
    created_at = Column(DateTime, default=datetime.datetime.now)
    entry_id = Column(Integer, ForeignKey('blog_entry.id'))
    entry = relation('BlogEntry',
        primaryjoin='BlogEntry.id==BlogComment.entry_id',
        backref='comment_set')

You now can of course get comments using comment_set attribute on blog entry. But sometimes we want start querying for other objects, for some reasons.
With Django ORM, no problem, just define method on the model and start querying – it just works. With SQLAlchemy on the other hand we need to create query
using Session object. But how does mapper use proper session if you call ‘comment_set’? Well, I’ve found the answer here: http://www.sqlalchemy.org/docs/05/reference/orm/sessions.html . There is ‘Session.object_session’ class method which returns session object binded to an model object.

So now we can simply define method for BlogComment:

    def get_related_comments(self):
        session = Session.object_session(self)
        comment_list = session.query(BlogComment)\
            .filter(BlogComment.entry_id==BlogEntry.id)\
            .all()
        return comment_list

Just remember that object which is passed to ‘Session.object_session’ method has to be binded with some session first.

Recently i had to create a lot of models using declarative mode with sqlalchemy 0.5.5 It was very time consuming especially for large databases. I started to think about an automated solution.

Sqlalchemy have this auto load feature which fetches info about tables by itself. I wanted to use this somehow to generate a string which can be easily  copy/paste to database model definition script. So i started a little project to write auto  model generator for sql alchemy based on auto load feature in sqlalchemy.

The script  is not perfect but it works and save a huge amount of time just codding the models. In fact is was able to setup a 186 table database model in 15minutes with this script.

Well the script helped me a lot and now on when i have to load some new existing database or changes i use it all the time.

The documentation is inside the script i tried to write it so anybody could use it out of the box.

I tested it on 4 schema 200+ tables database and i worked quite good. This script still have to be polished but current functionality is enough for most users.

My script is going to be also a part of django-alchemy project started by my friend Lukasz Balcerzak. You can visit the project site @ http://code.google.com/p/django-alchemy/

I used it with Pylons application and Django with modded sqlalchemy. but it can be basically used in any app which uses sqlalchemy. I did not test it with mysql though,but it should work, just specify mysql as dialect.

You can download this script from sqlalchemy model generator

One of my friends started an interesting project to integrate sqlalchemy into Django. Project name is django-alchemy.
You can visit it on Google code under http://code.google.com/p/django-alchemy/.

You may ask why to use sqlalchemy under Django if it already have an ORM ?
It’s very easy the django orm is not always the best choice. When using multiple schemas, or multiple database connection the Django ORM just does not provide such functionality.
SqlAlchemy have this built in so when you’re thinking of using multiple schemas or databases in your django project check out his project. Highly recommended

With almost 4 months late here it comes, Django 1.1. I’ve been waiting to see this at the final stage, as we use Django in our company, and I was sometimes forced to code down things that are already boundled now, like sql aggregation functionality for instance (well, in fact I just wrote a little django app using full force of SQLAlchemy instead of Django ORM). Additionally, some security patch was released yesterday, which is included with 1.1 too.

You may read more at release notes page . Stay tuned with Django and minor swing ;-)

Recently I’ve run into some optimizations problem which I once hoped I wouldn’t had to face. After some research on the subject I decided to split some process using threads. I hadn’t have much experience in that area except some classes at PJIIT (Polish-Japanese Institute of Information Technology) so start was painful… but not for long (as with everything in Python).

So I decided to share my thoughts with everyone who would ever try to use threads in Python programming.

Let’s imagine simple case: you want to check some host if it has opened any port from range 1-1000. We create simple script (use whatever host you like)

import socket
import time

def is_port_open(host, port):
    """
    Takes host param as string and port param as int.
    Returns true if port is open and false otherwise.
    """
    #print "[DEBUG] Checking host %s:%s" % (host, port)
    s = socket.socket()
    is_open = True
    try:
        s.connect( (host, port) )
    except socket.error:
        is_open = False
    s.close()
    return is_open

def main():
    start = time.time()

    host = 'some.host'
    ports = range(1,1000) 

    print "[INFO] Will now check host %s for ports between %s and %s" % \
        (host, ports[0], ports[-1]) 

    for port in ports:
        if is_port_open(host, port):
            print "[INFO] Port %s on host %s is open." % (port, host)

    print "Elapsed Time: %s" % (time.time() - start)

if __name__ == '__main__':
    main()

How long was it? Well, try this then:

import threading
import socket
import time
import Queue

THREAD_NUMBER = 20

def is_port_open(host, port):
    """
    Takes host param as string and port param as int.
    Returns true if port is open and false otherwise.
    """
    #print "[DEBUG] Checking host %s:%s" % (host, port)
    s = socket.socket()
    is_open = True
    try:
        s.connect( (host, port) )
    except socket.error:
        is_open = False
    s.close()
    return is_open

class PortChecker(threading.Thread):
    def __init__(self, queue):
        threading.Thread.__init__(self)
        self.queue = queue

    def run(self):
        while True:
            host, port = self.queue.get()
            if is_port_open(host, port):
                print "[INFO] Port %s on host %s is open." % (port, host)
            self.queue.task_done()

def main():
    start = time.time()

    host = 'some.host'
    ports = range(1,1000)
    queue = Queue.Queue()

    for i in xrange(THREAD_NUMBER):
        pc = PortChecker(queue)
        pc.setDaemon(True)
        pc.start()

    print "[INFO] Will now check host %s for ports between %s and %s" % \
        (host, ports[0], ports[-1]) 

    for port in ports:
        queue.put( (host, port) )

    queue.join()
    print "Elapsed Time: %s" % (time.time() - start)

if __name__ == '__main__':
    main()

Try with different port range/thread number. Ok, this is just an example. But it should give you an idea how to handle with some process that is done inside a loop. You have to define your “handler” (thread that will process on some object(s)), create empy queue and fill it with objects. Try to use only 1 thread (set THREAD_NUMBER to 1) and look at the time it took to finish whole process.

I know that programmer’s time is much more expensive than processor’s time and I try not to break this rule. But sometimes (i.e. when your script runs for over 12 hours) it is very nice if you can cut the time you can generate results. Just remember to catch exceptions inside threads – it is very important as if exception is risen by thread, it will just die without any notification.

Well, thats it. I will proceed tomorrow with some example with SQLAlchemy. Good night folks.