Pylons based replacement for hgwebdir. Fully customizable,
with authentication, permissions. Based on vcs library.

• has it’s own middleware to handle mercurial protocol request each request can
  be logged and authenticated +threaded performance unlikely to hgweb. Middleware recognises a mercurial protocol
  and skips any functionality for pure web request
• mimics full functionality of hgwebdir including feeds,archives,diffs,raw-diffs,annotations,file history etc.
• mako templates let’s you customize look and feel of the application.
  Possibly visual merge with your company systems
• diffs annotations and source code all colored by pygments (see screenshots).
• admin interface for performing user/permission managements as well as repository
  managements. Out of beta version should have fully customizable permission system, with
  private repos, and repo permissions
• added cache with invalidation on push/repo managment for high performance and
  always upto date data.
• rss / atom feeds
• future support for git (thanks to vcs mercurial/git api should be 1:1
• based on pylons 1.0 / sqlalchemy 0.6 /sqlite(possibly mysql/postgres thanks to sqlalchemy models)

Below are the beta version screenshots. Now I’m not good at designing webapps, so excuse me if you feel the look&feel to be to simple or bad looking.
It’s all i could come up with. Most of my expiration was taken from hgwebdir/bitbucket and github. I tried to take all best parts from all of them and combine into this app.
It’s still in beta but i think it’s production ready in this stage I’m using it for few weeks, and had no troubles or crashes with it. The code can be grabbed at bitbucket repo http://bitbucket.org/marcinkuzminski/hg-app/

import multiprocessing

multiprocessing.cpu_count()
# this shows up 2 for my core2duo e9300

#so now if we iterate this like that

for i in xrange(multiprocessing.cpu_count()):
    print 'core',i+1

#this prints
#core 1
#core 2

Download the Python-Threads pdf. Whitch includes the basic, and some more advanced information and tutorials about the threads in python.

In the pdf you can useful find info about:

• thread and threading modules
• queue
• locks
• gil
• events
• debugging threads

So I decided to share my thoughts with everyone who would ever try to use threads in Python programming.

Let’s imagine simple case: you want to check some host if it has opened any port from range 1-1000. We create simple script (use whatever host you like)

import socket
import time

def is_port_open(host, port):
    """
    Takes host param as string and port param as int.
    Returns true if port is open and false otherwise.
    """
    #print "[DEBUG] Checking host %s:%s" % (host, port)
    s = socket.socket()
    is_open = True
    try:
        s.connect( (host, port) )
    except socket.error:
        is_open = False
    s.close()
    return is_open

def main():
    start = time.time()

    host = 'some.host'
    ports = range(1,1000) 

    print "[INFO] Will now check host %s for ports between %s and %s" % \
        (host, ports[0], ports[-1]) 

    for port in ports:
        if is_port_open(host, port):
            print "[INFO] Port %s on host %s is open." % (port, host)

    print "Elapsed Time: %s" % (time.time() - start)

if __name__ == '__main__':
    main()

How long was it? Well, try this then:

import threading
import socket
import time
import Queue

THREAD_NUMBER = 20

def is_port_open(host, port):
    """
    Takes host param as string and port param as int.
    Returns true if port is open and false otherwise.
    """
    #print "[DEBUG] Checking host %s:%s" % (host, port)
    s = socket.socket()
    is_open = True
    try:
        s.connect( (host, port) )
    except socket.error:
        is_open = False
    s.close()
    return is_open

class PortChecker(threading.Thread):
    def __init__(self, queue):
        threading.Thread.__init__(self)
        self.queue = queue

    def run(self):
        while True:
            host, port = self.queue.get()
            if is_port_open(host, port):
                print "[INFO] Port %s on host %s is open." % (port, host)
            self.queue.task_done()

def main():
    start = time.time()

    host = 'some.host'
    ports = range(1,1000)
    queue = Queue.Queue()

    for i in xrange(THREAD_NUMBER):
        pc = PortChecker(queue)
        pc.setDaemon(True)
        pc.start()

    print "[INFO] Will now check host %s for ports between %s and %s" % \
        (host, ports[0], ports[-1]) 

    for port in ports:
        queue.put( (host, port) )

    queue.join()
    print "Elapsed Time: %s" % (time.time() - start)

if __name__ == '__main__':
    main()

Try with different port range/thread number. Ok, this is just an example. But it should give you an idea how to handle with some process that is done inside a loop. You have to define your “handler” (thread that will process on some object(s)), create empy queue and fill it with objects. Try to use only 1 thread (set THREAD_NUMBER to 1) and look at the time it took to finish whole process.

I know that programmer’s time is much more expensive than processor’s time and I try not to break this rule. But sometimes (i.e. when your script runs for over 12 hours) it is very nice if you can cut the time you can generate results. Just remember to catch exceptions inside threads – it is very important as if exception is risen by thread, it will just die without any notification.

Well, thats it. I will proceed tomorrow with some example with SQLAlchemy. Good night folks.