Release 1.5.1

• hg serve ( out of the box http server)
• it’s written in python
• has a very good eclipse plug-in (http://www.vectrace.com/mercurialeclipse/)

Cons: it’s slower than GIT ( but who cares :D )

So after installing mercurial and moving my svn repos with history with hgsvn ( which can also do push to svn )
I started to setup easy access to the repository using nginx. I’ll show you how to setup an SSL http based repository to do PUSH/PULL/CLONE what ever.

Firstly we have to make self signed ssl certificates.
I found on of the easiest tutorials available you can find it here: http://www.akadia.com/services/ssh_test_certificate.html

OK when we have the ssl certificates for our server. Put it somewhere on the server so nginx can access it.
We have to setup a new virtual host for nginx that will only do ssl connections
and have basic auth additionally.

Here’s the example config:

server {
    listen          443;
    server_name     hg.yourserver.com;
    ssl    on;
    ssl_certificate    /home/ssl_certs/hg_cert.crt;
    ssl_certificate_key     /home/ssl_certs/hg_cert.key;
    access_log      /var/log/nginx/hg.log;
    auth_basic      "mercurial server";
    auth_basic_user_file    /etc/nginx/.htpasswd;

    location / {
        proxy_pass      http://127.0.0.1:8001;
        #here's where the hg server runs
        include         /etc/nginx/proxy.conf;
    }
}

Few thing to notice.
.htpasswd file has to be in a format <username>:<cryptPassword> if you don’t have apache
installed you can use my password generator for generating crypt password. This username
and password will be used to do pull/push from console and eclipse.
Another important thing is that when you run hg serve you must specify the -a 127.0.0.1 option which is
for the address the mercurial server runs. Hg serve default is to start at all interfaces
so you have this port open outside and your ssl/passwd protection is for nothing…

I run my using hg serve –webdir-conf=/etc/hg/hgweb.conf -d -p 8001 -a 127.0.0.1

The hgweb.conf should be with

push_ssl = false
#since nginx is doing the SSL
allow_push = * #NGINX  is doing the auth
style = gitweb

O and one more thing remember that your repo should be accessible to hg serv.
I made a mistake and run hg serve as www-data and my repo was to my home user,
and i had internal server error when trying to do push to server.

So now you can have your repo via http with SSL and nginx authentication.

I’ve also put it on the PyPi (http://pypi.python.org/pypi/Djalog) so in order to install it you would execute:

easy_install Djalog

and you are ready. If you want to quickly check how it works just type:

import djalog
import logging

logging.debug("Log some debug information.")
logging.info("Log some information.")
logging.warn("Log some warning.")
logging.error("Log some error.")
logging.critical("Log some critical information.")

from djalog import configure
configure(LOG_USE_COLORS=True, LOG_SQL=True, LOG_LEVEL=5)

logging.debug("Log some debug information.")
logging.info("Log some information.")
logging.warn("Log some warning.")
logging.error("Log some error.")
logging.critical("Log some critical information.")

# Log sql query
logging.sql("SELECT * FROM users WHERE username = 'admin'")

This is beta but it should be changed to “final” in two weeks. Remember that you may update projects from CheeseShop using “-U” option:

easy_install -U Djalog

Enjoy!

class BlogEntry(Base):
    __tablename__ = 'blog_entry'

    title = Column(String)
    content = Column(Text)
    created_at = Column(DateTime, default=datetime.datetime.now)

    def __repr__(self):
        return self.title

class BlogComment(Base):
    __tablename__ = 'blog_comment'

    content = Column(Text)
    created_at = Column(DateTime, default=datetime.datetime.now)
    entry_id = Column(Integer, ForeignKey('blog_entry.id'))
    entry = relation('BlogEntry',
        primaryjoin='BlogEntry.id==BlogComment.entry_id',
        backref='comment_set')

You now can of course get comments using comment_set attribute on blog entry. But sometimes we want start querying for other objects, for some reasons.
With Django ORM, no problem, just define method on the model and start querying – it just works. With SQLAlchemy on the other hand we need to create query
using Session object. But how does mapper use proper session if you call ‘comment_set’? Well, I’ve found the answer here: http://www.sqlalchemy.org/docs/05/reference/orm/sessions.html . There is ‘Session.object_session’ class method which returns session object binded to an model object.

So now we can simply define method for BlogComment:

    def get_related_comments(self):
        session = Session.object_session(self)
        comment_list = session.query(BlogComment)\
            .filter(BlogComment.entry_id==BlogEntry.id)\
            .all()
        return comment_list

Just remember that object which is passed to ‘Session.object_session’ method has to be binded with some session first.

You may ask why to use sqlalchemy under Django if it already have an ORM ?
It’s very easy the django orm is not always the best choice. When using multiple schemas, or multiple database connection the Django ORM just does not provide such functionality.
SqlAlchemy have this built in so when you’re thinking of using multiple schemas or databases in your django project check out his project. Highly recommended

You may read more at release notes page . Stay tuned with Django and minor swing ;-)

Here are by my opinion three advantages over threading.

• Multiprocessing runs on processes not threads.
• Overcomes the GIL (global interpreter lock) that threading is using by using sub processes.
• Processes can be synchronized even remotely so we could write a concurrent calculations over the network

I made a simple example class that uses multiprocessing module to scan ports, just for testing i made the same thing that Lukasz made using threading. This example however is not more efficient than the one presented in http://www.python-blog.com/2009/07/01/python-threaded/ but when we could replace the function check_port with more CPU consuming function we could end up with performance grater by the number of cpus/cores we have. For example in one of my projects recently i made a calculations for popular gambling game in Poland, MultiMulti. I’d made up a calculations of most repeating number in last 50 games. To calculate combination of 9’s over 20’s 50 times with threading i’d got around 1200s with multiprocessing i was able to calculate it in around 700 s with my Core2Duo CPU. I wish i had core2 quad to check the performance :) So i’f you need to make some heavy calculations multiprocessing can give you that performance.

Here’s the code and you can download the port descriptions file port_list to match port with description.

from multiprocessing import Process, Queue, cpu_count, Lock
import socket, sys

class PortScanner(object):
    ''' multiprocessing port scanner with port description'''

    def __init__(self, host = '' , port_range = (1, 100), nr_processes = cpu_count(), port_list_file = ''):
        '''
        port_range=(start,stop) default 1,100
        nr_processes = int default cpu_count() '''
        q = Queue()
        l = Lock()
        port_list = []

        try:
            for i in open(port_list_file).readlines():
                port_list.append([x.strip() for x in i.split('\t')])

        except IOError:
            print 'no port list file specified'
            pass

        for _ in xrange(port_range[0], port_range[1]):
            q.put((host, _))

        #to stop all processes we have to put STOP to queue and break the loop for each process
        for _ in xrange(nr_processes):
            q.put('STOP')

        for _ in xrange(nr_processes):
            p = Process(target = self.check_port, args = (q, l, port_list))
            p.start()

    def check_port(self, q, l, port_list):
        ''' worker class invoked by process '''
        while True:
            queue_ret = q.get()

            if queue_ret == 'STOP':
                break

            s = socket.socket()

            try:

                s.connect((queue_ret))

                #lock for uncorrupted printing to console
                l.acquire()
                print "[INFO] %s on port %s is open" % (queue_ret)

                if len(port_list) > 1:
                    for i in port_list:
                        if int(i[0]) == int(queue_ret[1]):
                            for _ in i:sys.stdout.write(_ + " ")
                            print "\n\n"

                l.release()
            except socket.error:
                #print "[WARNING] %s on port %s is closed" % (queue_ret)
                pass
            s.close()

if __name__ == "__main__":

    PortScanner(host = 'example.com', port_range = (1, 60), nr_processes = 40, port_list_file = 'port_list.data')

Download the Python-Threads pdf. Whitch includes the basic, and some more advanced information and tutorials about the threads in python.

In the pdf you can useful find info about:

• thread and threading modules
• queue
• locks
• gil
• events
• debugging threads