Setting up logging mechanism in Django

Coming from a Java world where projects and libraries without logging are unheard, it was a big surprise to see that all django apps that I have downloaded so far do not include logging, even though Python has a very good support for logging, it was inspired by the log4j. For a reason that I do not know they choose not to log, I suppose it is a kind of situation where everyone else knows but me!

So as a java developer used to work with a lot of logging I decided to put in place a logging mechanism for my django project and fortunately python has great support for that:
1. create a logging.conf file for the logging options
2. configuring the logging module
3. using the logger in my modules

So here is my simple set up:
1. create a logging.conf file for the logging options with the following content:

[loggers]
keys=root,core_view_logger

[handlers]
keys=consoleHandler,core_view_handler

[formatters]
keys=simpleFormatter

[logger_root]
level=NOTSET
handlers=consoleHandler

[logger_core_view_logger]
level=DEBUG
handlers=core_view_handler
qualname=core_view_logger
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[handler_core_view_handler]
class=handlers.RotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('logs/core_logger.log', 'a', 1000000, 4)

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

2. configuring the logging module in the settings.py

import logging
import logging.config

LOGGING_CONFIG = 'logging.conf' # logging configuration file
logging.config.fileConfig(LOGGING_CONFIG)

3.use the logger in your modules

import logging 

logger = logging.getLogger('core_view_logger');

def doSomething(request):
    logger.debug('This is a debug message')
    logger.info('This is an info message')
    logger.warning('This is a warning message')
    logger.error('This is an error message')
    logger.critical('Yes it is so simple :)')

That’s all folks!
But of course this is a simple set up and you can do more much more:

  • Logging to multiple destinations
  • Adding contextual information to the output
  • Send and receive logging information across network
  • Send logging information through email

Check the python official documentation for all the options http://docs.python.org/library/logging.html

Enjoy!

Advertisements

Organize django applications in packages and modules

When you create a django application you have one module to add your model classes the models.py and one module to add your views the views.py (also called actions in other web frameworks), and there are possibly other modules like forms.py and admin.py. This is perfectly fine when you have a small application with a couple of classes in your models.py and few functions  in your views.py. However, this files start to become unmanageable when adding more and more code. For example, in one of my projects the views.py is more than 600 lines and the models.py contains more than 20 classes. For that reason I decided to split the models.py and put each class in its won module, and also organize the views.py in more modules depending on related functionality.

To split the views.py in more modules it’s a no brain task, it is simply a python module and there is nothing that can prevent you from doing so. However, the models.py needs more attention and few more steps to accomplish the task since this module is used by the manage.py module.

Lets take an example and organize the code so it is more manageable.
Here’s how my application looks like at first, there is only one module for all model classes and one module for all views (actions):

This is how the models.py looks like
Note that the code example is from the book “Learning Website Development with Django”:

from django.db import models
from django.contrib.auth.models import User

class Link(models.Model):
    url = models.URLField(unique = True)

    def __str__(self):
        return self.url

class Bookmark(models.Model):
    title = models.CharField(max_length = 200)
    user = models.ForeignKey(User)
    link = models.ForeignKey(Link)

    def __str(self):
        return "%s, %s" % (self.user.username, self.link.url)

class Tag(models.Model):
    name = models.CharField(max_length = 64, unique = True)
    bookmarks = models.ManyToManyField(Bookmark)

    def __str__(self):
        return self.name

class SharedBookmark(models.Model):
    bookmark    = models.ForeignKey(Bookmark, unique = True)
    date        = models.DateTimeField(auto_now_add = True)
    votes       = models.IntegerField(default = 1)
    users_voted = models.ManyToManyField(User)

    def __str__(self):
        return '%s, %s' % self.bookmark, self.votes

This is only a small part of the views.py module, in the real example there are more than 200 lines of code:

from django.core.exceptions import ObjectDoesNotExist
from django.template import Context
from django.template import RequestContext
from django.template.loader import get_template
from django.http import HttpResponse, Http404, HttpResponseRedirect
from django.contrib.auth import logout
from django.contrib.auth.models import User
from django.contrib.auth.decorators import login_required
from django.shortcuts import render_to_response
from django.shortcuts import get_object_or_404
from bookmarks.forms import *
from bookmarks.models import * 

def main_page(request):
    shared_bookmarks = SharedBookmark.objects.order_by('-date')[:10]
    variables = RequestContext(request,
                {'shared_bookmarks': shared_bookmarks})
    return render_to_response('main_page.html', variables)

@login_required()
def user_page(request, username):
    user = get_object_or_404(User, username=username)
    bookmarks = user.bookmark_set.order_by('-id')
    variables = RequestContext(request,
                {'bookmarks': bookmarks,
                 'username': username,
                 'show_tags': True,
                 'show_edit': username == request.user.username, })
    return render_to_response('user_page.html', variables)

def logout_page(request):
    logout(request)
    return HttpResponseRedirect('/')

def register_page(request):
    form = RegistrationForm(request.GET)
    if request.method == 'POST':
        form = RegistrationForm(request.POST)
    if form.is_valid():
        user = User.objects.create_user(username=form.cleaned_data['username'],
                                        password=form.cleaned_data['password1'],
                                        email=form.cleaned_data['email'])
        return HttpResponseRedirect('/register/success/')
    else:
        form = RegistrationForm()
        variables = RequestContext(request, {'form': form})
    return render_to_response('registration/register.html', variables)

So I will take a more Java approach and create two packages one for the model and one for the views:

As I said, there is no problem whatsoever to organize the views.py into more modules, the authentication.py module is an example of the separated functionality:

from django.template import RequestContext
from django.template.loader import get_template
from django.http import HttpResponse, Http404, HttpResponseRedirect
from django.contrib.auth import logout
from django.contrib.auth.models import User
from django.contrib.auth.decorators import login_required
from django.shortcuts import render_to_response
from django.shortcuts import get_object_or_404

@login_required()
def user_page(request, username):
    user = get_object_or_404(User, username=username)
    bookmarks = user.bookmark_set.order_by('-id')
    variables = RequestContext(request,
                {'bookmarks': bookmarks,
                 'username': username,
                 'show_tags': True,
                 'show_edit': username == request.user.username,})
    return render_to_response('user_page.html', variables)

def logout_page(request):
    logout(request)
    return HttpResponseRedirect('/')

Separating model classes and placing them into their won module follows, for brevity only the Bookmark class will be shown but all the other classes are separated in similar way. The difference now is that we have added a class inside Bookmark class called Meta with the field called app_label where we define the name of the application, if you do not add this field then the separation will not work, there is a workaround which requires to modify the the manage.py. Note that the Meta class can be used for a lot more things, like the ordering of objects for example. Here is the bookmark module:

from django.db import models
from django.contrib.auth.models import User

from media.bookmarks import *

class Bookmark(models.Model):
    title = models.CharField(max_length = 200)
    user = models.ForeignKey(User)
    link = models.ForeignKey(Link)

    def __str(self):
        return "%s, %s" % (self.user.username, self.link.url)

    class Meta:
        app_label = 'bookmarks'

Another thing that should be done is to add the following import lines in the __init__.py, this way the manage.py can find your classes and you can import your model classes as you did previously:

from bookmarks import Bookmark
from link import Link
from sharedbookmark import SharedBookmark
from tag import Tag

The same for the views, I use the * here since I want to import every function from the modules but you can define the function names there:

from authentication import *
from registration import *
from search import *

Of course the admin.py and the forms.py modules can be organized in the same way.

The concept of django applications is to be small and focused on certain and related functionality so that they can be plugged in different projects without tweaking and maybe that is the reason that the code is organized in a simple modules instead of packages.  But I think this way the application looks cleaner and more manageable even for the small ones, or maybe I have been exposed to Java for a long time and it looks more like a Java application…

Enjoy!

django books

Django is a popular python web framework. It is relatively easy and you can learn it from the tutorials that are provided by the django team. Although the tutorials are very good, for a more professional and in depth learning one should consider to buy a book or two. In the market there are excellent books for django.

Apress has already three titles and two of them are preparing their second edition:

The Definitive Guide to Django: Web Development Done Right

The first part of the book introduces Django fundamentals like installation and configuration. You’ll learn about creating the components that power a Django–driven web site. The second part delves into the more sophisticated features of Django, like outputting non–HTML content (such as RSS feeds and PDFs), plus caching and user management. The third part serves as a detailed reference to Django’s many configuration options and commands. The book even includes seven appendixes for looking up configurations options and commands. Anyone that does serious development with django must have this book. In June 2009 the second edition is coming out.

Practical Django Projects

Build a django content management system, blog, and social networking site with James Bennett as he introduces the popular Django framework.
You’ll work through the development of each project, implementing and running the applications while learning new features along the way. In June 2009 the second edition is coming out.

Pro Django

Pro Django more or less picks up where The Definitive Guide to Django left off and examines in greater detail the unusual and complex problems that Python web application developers can face and how to solve them. This book is for companies looking for a framework capable of supporting enterprise needs, as well as advanced Python or web developers looking to solve unusual, complex problems.

Packt Publishing has also three titles:

Learning Website Development with Django

This is an excellent book to begin djano, it also contain a chapter on how to integrate django with jQuery. This book will show you how to assemble Django’s features and leverage its power to design, develop, and deploy a fully-featured website. It will walk you through the creation of an example web application, with lots of code. Each chapter will add new features to the site, and show what parts of Django to work on to build these features.

Django 1.0 Template Development

This book will help you to master the Django template system. Built-in template tags and filters are explained with examples and usage notes, as well as information on building custom tags and filters to extend the system for your needs. You will learn to use inheritance to create modular templates that are easy to maintain. You will learn how to serve multiple templates from within the same Django project, with an example of how to serve different templates for a mobile version of your site without having to change any code in your views. Pagination, internationalization, caching, and customization of the automatic admin application are also covered.

Django 1.0 Website Development
This book will show you how to assemble Django’s features and take advantage of its power to design, develop, and deploy a fully-featured web site. It will walk you through the creation of an example web application, with lots of code examples. Specially revised for version 1.0 of Django, the book starts by introducing the main design concepts in Django. Next, it leads you through the process of installing Django on your system. After that, you will start right away on building your social bookmarking application using Django. Various Django 1.0 components and sub-frameworks will be explained during this process, and you will learn about them by example.

Addison-Wesley Professional:

Python Web Development with Django
This complete guide starts by introducing Python, Django, and Web development concepts, then dives into the Django framework, providing a deep understanding of its major components (models, views, templates), and how they come together to form complete Web applications. After a discussion of four independent working Django applications, coverage turns to advanced topics, such as caching, extending the template system, syndication, admin customization, and testing. Valuable reference appendices cover using the command-line, installing and configuring Django, development tools, exploring existing Django applications, the Google App Engine, and how to get more involved with the Django community.

Sams Publishing:
Sams Teach Yourself Django in 24 Hours
Designed for experienced website developers who have at least some familiarity with the Python programming language, this book uses a straightforward, step-by-step approach. Each lesson builds on the previous ones, enabling you to learn the essentials of implementing the Django framework on a website from the ground up.

django template tag for active CSS class

Navigation bar is an important part of the web application since it is the map that guides the users in different parts of the application. For better user experience navigation bars should indication which option is currently selected by the user. A common way to indicate which option is selected is by simply changing the CSS style of the selected option.

Vincent Foley has a very good article in his blog of how to implement a navigation bar in django. I include the latest paragraph of his post here, so if you are not familiar with url patterns and django template tags please read the full article of Vincent:

## tags.py
@register.simple_tag
def active(request, pattern):
    import re
    if re.search(pattern, request.path):
        return 'active'
    return ''

<!-- navigation -->

{% load tags %}
<div id="navigation">
    <a class="{% active request "^/$" %}" href="/">Home</a>
    <a class="{% active request "^/services/" %}" href="/services/">Services</a>
    <a class="{% active request "^/contact/" %}" href="/contact/">Contact</a>
</div>

This is a pretty good implementation but from my point of view it has one problem. It binds the url patterns with the template page. If you need to change the url pattern you must not forget to change the template also. Furthermore, what happens if you ship the application and the client decides to change the urls of the application.

Fortunately django has two features that allows to push this implementation one step further.
The first feature is that in django we can name the the url patterns, so the urls.py will look something like this:

## urls.py
urlpatterns += patterns('',
(r'/$', view_home_method, 'home_url_name'),
(r'/services/$', view_services_method, 'services_url_name'),
(r'/contact/$', view_contact_method, 'contact_url_name'),
)

The second feature is that we can create variables in the template page, the following code creates three variables (home, services and contact) these variables holds the url patterns for the respective names:

{% url home_url_name as home %}
{% url services_url_name as services %}
{% url contact_url_name as contact %}

Combining these two features of django we can modify the code of Vincent as follows:

{% load tags %}

{% url home_url_name as home %}
{% url services_url_name as services %}
{% url contact_url_name as contact %}

<div id="navigation">
    <a class="{% active request home %}" href="home">Home</a>
    <a class="{% active request services %}" href="services">Services</a>
    <a class="{% active request contact %}" href="contact">Contact</a>
</div>

As you can see we have totally decoupled the url patters from the template page. I know we have coupled it with the url names but there are no reasons whatsoever to change the url names since they are not visible anywhere, from now on we can change the url patterns as we wish.
Also note that the href tag has changed to the new variable that we created so it points to the correct url.

Important: There is not need to change the template tag.

Enjoy.

javafx on google app engine

Two weeks ago I started to work with python and the django framework in order to deploy a web application in google app engine. (I am really amazed of how fast you can build web applications with python & django – but this will be another post :)

Since JavaFX is my favorite RIA technology I was wondering if we can deploy javafx applications on google app engine. So I started a new project and after 20 minutes it was ready. What surprised me the most is the fact that it is really easy to deploy javafx applications into google app engine. Keep in mind, however, that this is a simple example and I don’t  access the database or other services that are offered by google app engine (maybe I will try that next year ;) .

What you really need to do is to pack your javafx application as an applet and place it under a folder in the google app engine application, declare that folder as a static resource in the app.yaml configuration file and finally use python to write it to the response.

Lets go through it step by step:

1. Create your javafx application and pack it as applet. For this example I will use the SpringAnimation of Josh Marinacci which you can find it here http://javafx.com/samples/SpringAnimation/index.html

2. Create a google app engine application. Instructions on how to create an app engine application can be found here
I have ported the google app engine application into eclipse and it looks something like this:

The most important files are the app.yaml where we need to declare that the javafx folder will serve static files and the mainapp.py python file which will serve our application.

Lets see the app.yaml file

So in the app.yaml configuration file we tell app engine that the application name is javafxtest, we declare the static folder under the handlers and we configure all the urls to be forwarded to the python file mainapp.py

Lets have a look at the python file

The content of this file is straightforward, we just write html code to the response. Note how we refer to the jar file and the folder that servers the static files. In the same folder with the jar file there is also the SpringAnimation_browser.jnlp file, we have to edit this file and change the codebase to the appropriate one (for me it was codebase=”http://localhost:9999/javafx/“) to test the application locally before uploading it, when you are about to upload it, change the codebase to the url of the app engine (for me it is codebase=”http://javafxtest.appspot.com/javafx&#8221;)

3. Upload the application in the app engine. The command to upload the application is python appcfg.py update <PROJECT_PATH>\JavaFXonGAE\src enter the email address and the password. You need to create an account if you do not have one.
You can find more info on how to deploy app engine applications here

You can access the application at http://javafxtest.appspot.com/

Resources:
Google app engine: http://code.google.com/appengine/docs/gettingstarted/
JavaFX: http://javafx.com/samples/SpringAnimation/index.html