Here are by my opinion three advantages over threading.

• Multiprocessing runs on processes not threads.
• Overcomes the GIL (global interpreter lock) that threading is using by using sub processes.
• Processes can be synchronized even remotely so we could write a concurrent calculations over the network

I made a simple example class that uses multiprocessing module to scan ports, just for testing i made the same thing that Lukasz made using threading. This example however is not more efficient than the one presented in http://www.python-blog.com/2009/07/01/python-threaded/ but when we could replace the function check_port with more CPU consuming function we could end up with performance grater by the number of cpus/cores we have. For example in one of my projects recently i made a calculations for popular gambling game in Poland, MultiMulti. I’d made up a calculations of most repeating number in last 50 games. To calculate combination of 9’s over 20’s 50 times with threading i’d got around 1200s with multiprocessing i was able to calculate it in around 700 s with my Core2Duo CPU. I wish i had core2 quad to check the performance :) So i’f you need to make some heavy calculations multiprocessing can give you that performance.

Here’s the code and you can download the port descriptions file port_list to match port with description.

from multiprocessing import Process, Queue, cpu_count, Lock
import socket, sys

class PortScanner(object):
    ''' multiprocessing port scanner with port description'''

    def __init__(self, host = '' , port_range = (1, 100), nr_processes = cpu_count(), port_list_file = ''):
        '''
        port_range=(start,stop) default 1,100
        nr_processes = int default cpu_count() '''
        q = Queue()
        l = Lock()
        port_list = []

        try:
            for i in open(port_list_file).readlines():
                port_list.append([x.strip() for x in i.split('\t')])

        except IOError:
            print 'no port list file specified'
            pass

        for _ in xrange(port_range[0], port_range[1]):
            q.put((host, _))

        #to stop all processes we have to put STOP to queue and break the loop for each process
        for _ in xrange(nr_processes):
            q.put('STOP')

        for _ in xrange(nr_processes):
            p = Process(target = self.check_port, args = (q, l, port_list))
            p.start()

    def check_port(self, q, l, port_list):
        ''' worker class invoked by process '''
        while True:
            queue_ret = q.get()

            if queue_ret == 'STOP':
                break

            s = socket.socket()

            try:

                s.connect((queue_ret))

                #lock for uncorrupted printing to console
                l.acquire()
                print "[INFO] %s on port %s is open" % (queue_ret)

                if len(port_list) > 1:
                    for i in port_list:
                        if int(i[0]) == int(queue_ret[1]):
                            for _ in i:sys.stdout.write(_ + " ")
                            print "\n\n"

                l.release()
            except socket.error:
                #print "[WARNING] %s on port %s is closed" % (queue_ret)
                pass
            s.close()

if __name__ == "__main__":

    PortScanner(host = 'example.com', port_range = (1, 60), nr_processes = 40, port_list_file = 'port_list.data')

Download the Python-Threads pdf. Whitch includes the basic, and some more advanced information and tutorials about the threads in python.

In the pdf you can useful find info about:

• thread and threading modules
• queue
• locks
• gil
• events
• debugging threads

• Package Explorer showing outline for python modules
• Mylyn integration (separate feature: org.python.pydev.mylyn.feature)
• Jython and Python support!
• Support for scripting Pydev with Jython
• Nice PYTHONPATH configuration!
• Project and module creation wizards
• Ctrl+Shift+O: Organizes imports or sorts selection alphabetically
• Ctrl+Shift+F: Autoformat your code (preferences can be set)
• Python 2.4, 2.5, 2.6, 3.0 syntax supported
• Pydev builders can be disabled
• Syntax highlighting
• Spell checking (depends on JDT for Eclipse 3.3)
• Hovering in errors shows descriptions
• Matching brackets highlighted.
• Parser errors marked in the task list
• Outline view with imports/functions/classes
• Tabs or spaces preference
• Smart indentation (indent and dedent)
• Navigation: keyboard shortcuts to previous or next function. Default: (Ctrl+Shift+Up and Ctrl+Shift+Down)
• Comment and uncomment commands (on the popup menu) and keybindings. Default: (Ctrl+3 and Ctrl+Shift+3)
• hyperlinks over functions//import statements
• Code folding
• Refactoring with bicycle repair man.
• Code Completion (Ctrl+Space)
• Templates Completion (Ctrl+Space Too)
• Go to definition with F3 (powered by bicycle repair man)
• Content Assistant (Ctrl+1)
• Code Coverage
• Pylint
• TODO tasks
• Background and current line color chooser.

I highly recommend using Pylint in pydev it really helps to keep the code good quality. Pylint analyzes your code and shows unnecessary variables and imports, provides helpers to clean up the code and many more.

To install pydev just add http://pydev.sourceforge.net/updates/ http://pydev.org/updates (see update below) link to eclipse by using software updates and addons option.
I’m using the nightly builds for eclipse (http://nightly.aptana.com/pydev/) because they always come with new features earlier than the main release but remember use it on your own responsibility. You can find how to setup pydev here. When your up and running with pydev download pylint from here don’t forget the necessary dependencies i.e: logilab-astng, logilab-common, links are on the pylint download site. The turn on pylint you have to add a path to lint.py file in pylint tab on pydev options in eclipse. After restart of eclipse you should see output from pylint on the eclipse console anytime when you save a python file.

Now enjoy the most powerful python IDE available :-)

UPDATE: since PyDev release 1.5 now the package have integrated a pyDev extensions… the new repository for eclipse containing the version 1.5 is :http://pydev.org/updates
I also added a post about the PyDev extensions see it here http://www.python-blog.com/2009/09/06/pydev-extension-now-open-source/