Lance McNearney
lance@mcnearney.net

I've been experimenting with different caching methods ever since reading this post on the performance differences between no caching, built-in Django caching, memcached, static files, and Varnish. Varnish and static files were the fastest by a large margin.

Since I'd like to avoid adding another proxy layer to my setup (already using both Nginx and Apache), I looked into the caching capabilities built-in to Nginx. It seems that originally if you wanted to use caching with Nginx you used ncache. This is no longer the case since that project has been built into the Nginx core.

To add support for purging a cached item, you need to install the ngx_cache_purge module. Unfortunately, installing an Nginx module isn't as easy as installing an Apache module. It requires a re-compile and when you're used to just installing your software through apt-get, that can be a little intimidating. I also didn't want to lose the special setup that the package installer provided including using the /sites-available/ and /sites-enabled/ folders, the init script, etc.

Here's a quick guide on re-compiling while still remaining compatible with the default package:

Install Nginx through aptitude

This is really easy (and you probably already did this):

apt-get install nginx

Install compile tools

A few of the compile libraries needed by Nginx aren't installed by default. To install them, use the following command:

aptitude -y install build-essential libc6 libpcre3 libpcre3-dev libpcrecpp0 libssl0.9.8 libssl-dev zlib1g zlib1g-dev lsb-base

Download and extract the source

Download and extract the source of both the newest stable Nginx and the cache_purge module:

cd /usr/src/
wget http://www.nginx.org/download/nginx-0.7.65.tar.gz
wget http://labs.frickle.com/files/ngx_cache_purge-1.0.tar.gz
tar -xvf nginx-0.7.65.tar.gz
tar -xvf ngx_cache_purge-1.0.tar.gz
cd nginx-0.7.65/

Configure compile options

You can view the packaged Nginx's compile options by using:

nginx -V

For my server, I kept generally the same options but added the cache_purge module to the end using --add-module:

./configure --sbin-path=/usr/sbin --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/body --http-proxy-temp-path=/var/lib/nginx/proxy --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --with-debug --with-http_stub_status_module --with-http_flv_module --with-http_ssl_module --with-http_dav_module --with-ipv6 --add-module=/usr/src/ngx_cache_purge-1.0

This command will write out a summary of the configured options.

Compile Nginx

Compiling Nginx can be done with this command:

make && make install

A lot of text will scroll by during the compilation. In my case, I hadn't stopped my Nginx service before compiling and an error occurred when the install process attempted to copy the new Nginx executable over the current one. To fix this, I had to manually stop, copy and restart Nginx:

/etc/init.d/nginx stop
cp objs/nginx /usr/sbin/nginx
/etc/init.d/nginx start

If everything worked, you can view the current version and compile information of the running Nginx by using:

nginx -V

And the final result should be:

nginx version: nginx/0.7.65
built by gcc 4.4.1 (Ubuntu 4.4.1-4ubuntu8)
TLS SNI support enabled
configure arguments: --sbin-path=/usr/sbin --conf-path=/etc/nginx/nginx.conf 
--error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid 
--lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log 
--http-client-body-temp-path=/var/lib/nginx/body 
--http-proxy-temp-path=/var/lib/nginx/proxy 
--http-fastcgi-temp-path=/var/lib/nginx/fastcgi --with-debug 
--with-http_stub_status_module --with-http_flv_module 
--with-http_ssl_module --with-http_dav_module --with-ipv6 
--add-module=/usr/src/ngx_cache_purge-1.0

This post is a continuation of the first post on creating a gravatar template tag.

My test script was fairly straight forward to get working as a template tag. The only gotcha was not realizing that any values passed into the template tag are simple strings and that it was my job to resolve any variables being passed in. Here's an example:

Template:

{% gravatar comment.user_email 64 as gravatar %}

Tag code:

class GravatarNode(Node):   
    def __init__(self, email, size, varname):
        self.email = email

    def render(self, context):
        return self.email

@register.tag
def gravatar(parser, token):
    bits = token.contents.split()
    return GravatarNode(bits[1], bits[2], bits[4])

This template tag will render:

comment.user_email

Since I was hashing this value and passing it to the gravatar service it took me a little while to figure out what was happening.

To get it to resolve the actual value of the comment.user_email variable, you have to wrap the string variable in a template.Variable() and then call the resolve(context) method of the variable inside the tag's render() method:

class GravatarNode(Node):   
    def __init__(self, email, size, varname):
        self.email = template.Variable(email)

    def render(self, context):
        return self.email.resolve(context)

@register.tag
def gravatar(parser, token):
    bits = token.contents.split()
    return GravatarNode(bits[1], bits[2], bits[4])

Now it will return the comment user's e-mail address.

Here's the completed code for the template tag. You can see how it's used in a template based on the main tag's comments:

from django import template
from django.template import Library, Node, TemplateSyntaxError
import urllib, hashlib, httplib

register = Library()

GRAVATAR_DOMAIN = 'gravatar.com'
GRAVATAR_PATH = '/avatar/'

class GravatarNode(Node):   
    def __init__(self, email, size, varname):
        self.email = template.Variable(email)
        self.size = int(size)
        self.varname = varname

    def render(self, context):
        hash = hashlib.md5(self.email.resolve(context)).hexdigest()
        context[self.varname] = get_gravatar(hash, self.size)
        return ''

def is_default(hash):
    '''
    Returns whether the specified gravatar is returning the default icon.
    '''
    try:
        query = urllib.urlencode({
            'gravatar_id': hash,
            's': 1, # Smallest size available
            'default': '/' # Causes a re-direct when the gravatar is missing
        })
        full_path = '%s?%s' % (GRAVATAR_PATH, query)

        # Create connection and test for 302 redirect
        conn = httplib.HTTPConnection(GRAVATAR_DOMAIN)
        conn.request('HEAD', full_path)
        response = conn.getresponse()

        return response.status == 302
    except:
        return True

def get_gravatar(hash, size):
    '''
    Returns an object with the gravatar information.
    '''
    query = urllib.urlencode({
        'gravatar_id': hash,
        's': size
    })

    return {
        'src': 'http://%s%s?%s' % (GRAVATAR_DOMAIN, GRAVATAR_PATH, query),
        'width': size,
        'height': size,
        'is_default': is_default(hash)
    }

@register.tag
def gravatar(parser, token):
    '''
    This tag is used for rendering a gravatar icon.

    You can also check that the gravatar exists by checking the is_default property.

    Usage::
        {% gravatar email_address size as variable_name %}

    Example::
        {% gravatar user.email 64 as icon %}
        {% if not icon.is_default %}
            <img src="{{ icon.src }}" width="{{ icon.width }}" 
            height="{{ icon.height}}" alt=""/>
        {% endif %}
    '''
    bits = token.contents.split()
    if len(bits) != 5:
        raise TemplateSyntaxError, 'gravatar takes exactly four arguments'
    if bits[3] != 'as':
        raise TemplateSyntaxError, 'third argument must be "as"'
    return GravatarNode(bits[1], bits[2], bits[4])

You can download full template tag here. Since I'm still fairly new to Django and Python I would appreciate any feedback anyone has to offer. Please add a comment or contact me directly. Thanks and enjoy!

I spent the last couple of days implementing Django's comment framework for my blog. Since making a comment requires the user's e-mail address, I wanted to display the user's gravatar icon next to their name (but only if they have a gravatar setup).

How Gravatar Works

Gravatar is a free service that lets a user associate a graphic with their e-mail address. The only thing required for sites to implement the service is to create a md5 hash of the user's e-mail address. You then make a request to retrieve that user's icon:

http://gravatar.com/avatar/?gravatar_id=1440826ae9908cf7c415dbfd3a66b240&s=64

The above request returns my gravatar icon at 64x64 pixels (the little graphic to the left).

Existing Django/Python Implementations

Searching for existing implementations of Gravatar template tags turned up the following two results:

• Django-gravatar
• Implementing Gravatar in Django

The first project is aimed at extending Django's User model. It also renders the gravatar as an img tag without providing any context variables for me to integrate into my own template HTML.

The blog post by Drew Engelson was a much better implementation. The only piece missing was that I wanted to check to see if a gravatar exists for an e-mail address instead of just providing a default image.

Since neither one of these solutions supported checking if a gravatar exists, I set out to create my own template tag.

Does a Gravatar Exist?

The gravatar service will return its own "default" image for users who don't have a gravatar setup. You can also pass a "default" parameter to the service with your request and it will issue a 302 re-direct to that address instead of returning the service's default image.

My goal was to issue a simple HEAD request to the service with a default parameter and if it issued a 302 re-direct response I knew the gravatar didn't exist for the specified e-mail address. Unfortunately, the urllib library in Python doesn't support HEAD requests and it also automatically follows re-directs to the new address (and doesn't tell you that it followed the re-direct). Because of this, I needed to use the lower level httplib library.

Here's a simple test script that implements the functionality I wanted to achieve:

#!/usr/bin/env python
import sys
import urllib, hashlib, httplib

GRAVATAR_DOMAIN = 'gravatar.com'
GRAVATAR_PATH = '/avatar/'

def main():
    email = sys.argv[1]
    hash = hashlib.md5(email).hexdigest()
    print email + ' = ' + hash

    query = urllib.urlencode({
        'gravatar_id': hash,
        's': 1, # Smallest size available
        'default': '/' # Causes a re-direct when the gravatar is missing
    })
    full_path = '%s?%s' % (GRAVATAR_PATH, query)
    print full_path

    # Create connection and test for 302 redirect
    conn = httplib.HTTPConnection(GRAVATAR_DOMAIN)
    conn.request('HEAD', full_path)
    response = conn.getresponse()
    print response.status

    if response.status == 302:
        print 'No gravatar :('
    else:
        print 'Gravatar found :)'

if __name__ == '__main__':
    main()

You can download the full gravatar test script if you're interested.

With my e-mail address passed to the script it found a valid gravatar:

$ gravatar.py lance@mcnearney.net
lance@mcnearney.net = 1440826ae9908cf7c415dbfd3a66b240
/avatar/?default=%2F&s=1&gravatar_id=1440826ae9908cf7c415dbfd3a66b240
200
Gravatar found :)

And testing with a bogus e-mail address returned the following:

$ gravatar.py bogus@example.com
bogus@example.com = 42d33ed1a79b32726b43ade0460c7ff8
/avatar/?default=%2F&s=1&gravatar_id=42d33ed1a79b32726b43ade0460c7ff8
302
No gravatar :(

Now I just needed to turn this example code into a Django template tag...

Read part two of this post for the template tag implementation.

I've gotten used to using markdown because of Stack Overflow. One of the coolest features is being able to have your code automatically highlighted. I discovered this is really easy to implement in Django:

Download Libraries

Install the following two libraries:

• Markdown
• Pygments

I did it the easy way by using easy_install:

easy_install pygments
easy_install markdown

Update Models

You need to import the markdown library in your models.py. In my case, I added an additional TextField (body_html) to my Post model to store the rendered HTML. This field will get updated every time the Post is saved. The codehilite extension for the markdown library adds the pygments code highlighting support:

from markdown import markdown

class Post(models.Model):
    title = models.CharField(max_length=128)
    slug = models.SlugField(unique_for_date='publish')
    body = models.TextField()
    body_html = models.TextField(editable=False, blank=True, null=True)

    def save(self):
        self.body_html = markdown(self.body, ['codehilite'])
        super(Post, self).save()

CSS Styling

Take a look at the pygments demo to find a color scheme that you find appealing. I picked the "tango" theme.

The next step is to generate a stylesheet file for your theme:

• Manually generate the file using the "pygmentize" command-line options
• Be lazy (like me) and just download them all from pygments-css

Since I downloaded the files, I had to do a search/replace for ".highlight" and replace it with ".codehilite" since that is the class that wraps the <pre> code when code blocks are generated.

I then included the stylesheet in my base template for my site:

<link type="text/css" rel="stylesheet" href="{{ MEDIA_URL }}css/pygments/tango.css"/>

I also added some additional styling to the make the code blocks look better in my blog posts (this is SASS code):

.codehilite
    pre
        +round_less
        padding: .5em
        margin: 1em 0
        background-color: #fafafa
        border: solid 1px #ddd
        width: auto
        overflow: auto
        font-family: Consolas, Monaco, Courier New

All Done

It was really that simple. I think I spent more time tweaking the css styling than actually implementing the code.