Archive for the ‘Python’ Category

Filed Under (Django, Python) by Łukasz Balcerzak on February-21-2010

We have no time for creating documentation for our project… wrong! Documentation and tests are crucial in all software projects. Sure, it is more important that our software would work fine but package of unit tests almost always save time (in future as less bugs are produced) and without documentation shouldn’t be treated as “user guide”.

It doesn’t matter if you are creating project for your company or you plan to release it as an open source – by providing documentation you make life easier for other contributors and end-users, of course. And if you know how to write restructuredText documents then you already own necessary skills to create documentation lighting fast!

Prepare!

First of all, you would need to have Distutils and Sphinx installed. You can install it using one of the Python package management tools, in example easy_install would do the job. I prefer to use pip:

pip install distutils
pip install sphinx

Simply use any project you’d like to create documentation for, I would use django-richtemplates. Create a new directory for your documentation and run sphinx-quickstart command:

hg clone https://bitbucket.org/lukaszb/django-richtemplates/ django-richtemplates-temp
cd django-richtemplates-temp
sphinx-quickstart

Just answer some questions – they are very straightforward. You should probably turn autodoc option on.
Sphinx would create some directories for internal usage and static content storage (_build, _static and _templates), configuration file (conf.py) and index file (index.html). Ok, enough of this crap, lets create our first documentation page!

First page

Create new file, lets call it installation.rst and write the document itself:

.. _installation:

============
Installation
============

Fake installation for `django-richtemplates`_ project.

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur.

.. note::
   Yeah, this would be a note.
   Very simple, indeed.

Some more crap
--------------

If you would like to highlight a code snippet, it is as easy as::

   from mycompany.auth import AuthenticationBackend

   class MyClass(object):
       """
       Docstring.
       """
       def __init__(self, *args, **kwargs):
           """
           Some *business* logic.
           """
           pass
       def authenticate(self, username, password):
           """
           External authentication process.
           """
           auth = AuthenticationBackend(username, password)
           return auth.is_authenticated()

Or choose language explicit:

.. code-block:: bash

   cd django-richtemplates
   sudo python setup.py install

Yeah, `restructuredText `_ is great!
And easy, too.

External-internal hyperlinks
----------------------------

You may also use ``sphinx`` features, like linking to other *documents*, like
this: go to :ref:`configuration`.

.. _django-richtemplates: http://bitbucket.org/lukaszb/django-richtemplates/

This is standard restructured text document. The only sphinx specific parts are first line (need to add relation with file itself – this is needed for table of contents generation) and reference to other file (which is done with ref:`configuration`). Yup, we haven’t created configuration.rst file, yet. So do this now, and paste following:

.. _configuration:

=========================
Configuring richtemplates
=========================

I hope you have already read chapter :ref:`installation`.

Lorem ipsum!
------------

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur.

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur.

Don’t forget about the main page!

I once hoped that the first page (namely index.rst) is generated automatically, yet, I was wrong. But in order
to supply needed data for sphinx generator you have to make very simple changes to the index.rst. Just add
:glob: option to the toctree directive. Finally, mark that all (*) restructuredText documents should
be used.

.. django-richtemplates documentation master file, created by
   sphinx-quickstart on Sat Feb 20 21:42:53 2010.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to django-richtemplates's documentation!
================================================

Contents:

.. toctree::
   :maxdepth: 2
   :glob:

   *

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

Now in mydocs directory run

make html

and open file located at mydocs/_build/html/index.html. You should see your documentation!

Final words

Ok, this was very simple and the effect is – for such a little work done – quite amazing if you’d ask me.
However, we still haven’t used autodoc Sphinx’s extension. Let’s do that now – create new file
api.rst and paste whats below:

.. _api:

===
API
===

You can find some basic API information here.

Forms API
---------

In this section we cover some basics about the `django-richtemplates`'s forms
API.

.. automodule:: richtemplates.forms
   :members:

.. _django-richtemplates: http://bitbucket.org/lukaszb/django-richtemplates/

Again, run make command and see your documentation. Ok, so where is your API, you ask? Well, if you look at the output
of make html command you’d see that there was a warning – richtemplates.forms couldn’t be found.
Which is completely correct, as Sphinx doesn’t try to guess where your modules are, you have to configure it by yourself.
Change sys.path by uncommenting existing line and change it to this:

sys.path.append(os.path.abspath('..'))

If you are not sure why why put .. there, it’s simple – Sphinx would go a one directory up (..) and try to find richtemplates there. Try to build your documentation now. Warning again? Yeah, Sphinx now complains about not
having DJANGO_SETTINGS_MODULE which is, again, totally correct. As importing richtemplates won’t
raise exception, but trying to get deeper into the module (richtemplates.forms) would be impossible without
being within Django context (same would happen if you try to run python interpreter and run from django import forms).
So add this line into your conf.py file:

os.environ['DJANGO_SETTINGS_MODULE'] = 'django.conf.global_settings'

More information about autodoc extension could be found here.

Hope this would help anyone trying to start with his/her own documentation :)
If you have any suggestions – let me know.

Take care and make documentation, not war.



Filed Under (Editors, Python) by Marcin Kuźmiński on February-10-2010

makers of a great ruby on rails IDE rubymine, and ever greater Java IDE intelliJ, released
another (probably) great idea product named pycham, a python IDE, written from scratch in Java.

Since IntelliJ is a really the best Java IDE out there, i presume their next product
dedicated to python will be worth switching.

It’s still in preview but as for now it provides:

  • Coding Assistance
  • Project Code Navigation
  • Google App Engine Support
  • Code Analysis
  • Python Refactoring
  • Web Development with Django (some really nice ones)
  • debugger
  • Integrated unit tests
  • Version control integration



I’d tested under linux, seems to be a littler slower than my eclipse home brew version, but I’d expect the final version faster. It has few nice features, easy configuration, code completion are doing fine.
You can grab a free 45 day copy from here.

I’ll for sure fallow this product development. Stay posted for more !



Filed Under (Python, SqlAlchemy) by Marcin Kuźmiński on February-9-2010

I’m in a way of releasing few of my projects to bitbucket.
Starting now I’m releasing the Sqlalchemy model generator. A quick reminder, it’s a project that you
can generate models based on your database. So you don’t need to write models by yourself.
You can see the project here: http://bitbucket.org/marcinkuzminski/sqlalchemy_model_generator/
Any participants are welcome.



Filed Under (Python) by Marcin Kuźmiński on February-8-2010

I’d created another simple application in pylons, final step was to translate it from English to Polish.

I used the babel extractor recommended by pylons and that went really smooth.

My application have a language selector available anytime from menu.
I saw that even this switch worked very well for mako templates and string inside controllers,
it did not work when displaying my custom messages that i over rid in validators
and also did not work in my own custom validators that inherit from FancyValidator.
So i started to investigate this.

The first problem of not translating custom error messages in the regular formencode validators was easily being fixed by using lazy_ugettex method.

def _(s):
    #fix for translation error messages
    return lazy_ugettext(s)

This method allowed me to use lazy_ugettext to translate.
So now when in the validators your translating a string using _(‘this string is for translate’) method it’s actually wrapper for lazy_ugettext function.
Lazy translation is when you translate a string when it is accessed, not when the _() or other functions are called.

The second problem of translating custom error messages in your own validator was a bigger problem

I was looking for a solution for some time, and found a nice trick. You have to use your own
state_obj with static _() function. As example below for custom validation function.

class ValidLoginNames(formencode.validators.FancyValidator):

    messages = {'invalid_name':_('you cannot use %(username)s as login')}

    def validate_python(self, value, state):
        banned_names = ['admin', 'administrator', 'root']
        if value in banned_names:
            raise formencode.Invalid(self.message('invalid_name', state = State_obj, username = value), value, state)

#this is needed to translate the messages using _()
class State_obj(object):
    _ = staticmethod(_)

Those two trick now let’s you translate custom validators without a problem.



Filed Under (Python) by Marcin Kuźmiński on February-7-2010

So again the blog was having some mail problems which I just fixed.
So anyone can register, post to the blog again.

Sorry for the problems.



Filed Under (Python) by Marcin Kuźmiński on February-7-2010

I was browsing a web for some videos about stackless python implementation.
And i found a Montreal Python blip tv channel. montreal python blip tv
They have few interesting podcast which i recommend everyone to watch. The two last ones,
are dedicated to Stackless python and C extensions. Those are montreal python #8 and #9.

Be sure to check them out.



Filed Under (Python) by Marcin Kuźmiński on January-2-2010

Reading a huge file (ie. 70gb) in python just using open() could get you in troubles :).
There is a clean and nice solution for reading a complex huge files or list.
Using the new with open() as statement and python generator function you could write an easy functions
which will read such files without taking whole computer resources. Each iteration on such a function will perform a read of
given size, and the with statement will make sure that file is closed when iteration is finished…

An example of such a function:

def read_large_file(filename, mode = 'rs', size = '1024'):
    '''
    A lazy generator functions that reads a file with a given chunk of data
    USAGE:
    for data_chunk in read_large_file('/tmp/huge.file','rb',10240):
        print data_chunk
    @param filename: a filename
    @param mode: read mode
    @param size: size to read at one iteration
    '''
    with open(filename, mode) as f:
        while 1:
            data = f.read(size)
            if not data:
                break
            yield data

Remember that with statement is supported out of the box from python 2.6 in python
2.5 you have to do

from __future__ import with_statement

before using it. If you have troubles using with statement i recommend reading
this link



Filed Under (Python) by Marcin Kuźmiński on January-1-2010

I always loved how the print_r() function worked in php. In python print does print everything but when large collections (ie huge dicts) are printed using print you could get lost in the output. I just discovered a little smarter print function called pprint from module pprint

from pprint import pprint

This is great for debugging something in collections. Pprint prints in a nicer human readable form types of data like : list,dict,tuples.

l = [x for x in range(100)]
l.insert(0, l[:])
pprint(l)
l = dict([(x, x) for x in range(100)])

pprint(l)
#this will print this list in more convinient way
{0: 0,
 1: 1,
 2: 2,
 3: 3,
 4: 4,
 5: 5,
 6: 6,
 7: 7,
 8: 8,
 9: 9,
 ...
 98: 98,
 99: 99}

There are two interesting arguments in pprint function the:
width
Attempted maximum number of columns in the output.
width=100 will wrap everything that outputs longer than 100 chars.
and
depth
The maximum depth to print out nested structures.
ie:

pprint.pprint(stuff, depth=3)
['aaaaaaaaaa',
 ('spam', ('eggs', (...))),
 ['aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'],
 ['cccccccccccccccccccc', 'dddddddddddddddddddd']]
#now as you can see the tuples at index 1 have a depth of more than 3 and #they are being reduced by the pprint. Imagine how this helps with debuggin a
#really complicated structures.


Filed Under (Editors, Python) by Marcin Kuźmiński on November-20-2009

Few days ago there was again a major update on pydev eclipse plugin.
The version 1.5.1. Comes with new refactoring engine PEPTIC, it’s way better than any one before.
Complete list below: visit http://pydev.org for download.

Release 1.5.1

  • Improvements in the AST rewriter
  • Improvements on the refactoring engine:
    • No longer using BRM
    • Merged with the latest PEPTIC
    • Inline local available
    • Extract method bug-fixes
    • Extract local on multi-line
    • Generating properties using coding style defined in preferences
    • Add after current method option added to extract method
    • A bunch of other corner-case situations were fixed
  • Bug-fixes:
    • Minor editor improvements
    • Adding default forced builtins on all platforms (e.g.: time, math, etc) which wouldn’t be on sys.builtin_module_names on some python installations
    • Adding ‘numpy’ and ‘Image’ to the forced builtins always
    • Ctrl+1: Generate docstring minor fixes
    • Ctrl+1: Assign to local now follows coding style preferences properly
    • Exponential with uppercase E working on code-formatting
    • When a set/get method is found in code-completion for a java class an NPE is no longer thrown
    • Backspace properly treated in block mode
    • Setting IRONPYTHONPATH when dealing with Iron Python (projects could not be referenced)
    • No longer giving spurious ’statement has no effect’ inside of lambda and decorators
    • Fixed new exec in python 3k
    • Fixed NPE when breakpoint is related to a resource in a removed project
    • Fixed import problem on regexp that could lead to a recursion.
    • No longer giving NPE when debugging with the register view open
    • List access be treated as __getitem__() in the list — patch from Tassilo Barth
    • Fix for invalid auto-self added when typing


Filed Under (Editors, Python) by Marcin Kuźmiński on November-15-2009

I installed new eclipse on ubuntu 9.10, and noticed that manu of FINISH,
NEXT buttons are not working correctly ( you can’t click them).

So it found out it’s eclipse bug, but fortunately there is a simple
solution to this problem.

Write a startup script:

#!/bin/sh
export GDK_NATIVE_WINDOWS=1
/path_to_eclipse/eclipse

Save it as eclipse-run.sh make it +x, and the buttons will work now if you
run your eclipse via the run script.