Using Pylons with Durus

Author: Mike Orr
Date: 2006-12-20

Abstract

This article shows how to modify Pylons' QuickWiki tutorial to use the Durus object database instead of SQLAlchemy. It's based on QuickWiki 0.1.2 and Durus 3.6.

Contents

Background

Every Durus database has a root object which emulates a Python dict. Modifying this dict automatically pickles the changes to the database file. Reading the dict automatically loads the data from the file as needed. You can store any combination of scalar values, lists, dicts, class instances, dicts of lists of dicts -- anything pickleable -- in the root object. You can use "Persistent" objects to fine-tune how much gets loaded and saved at a time: each Persistent object creates a separate pickle.

The simplest equivalent of a SQL database table is a Python dict, Durus PersistentDict, or Durus btree. The latter two emulate a dict but have different persistance rules which we won't get into here. The value represents a database record; the key represents its primary key. Normally the value would be an instance of a class, but in QuickWiki the record would have only one string field, so we represent it as a string directly.

Durus is transactional so you must call a connection.commit() method after making changes to save them permanently and make them visible to other users.

A real Durus application would also pack the database occasionally to purge data that is no longer used.

Instructions

First, install the QuickWiki tutorial as described in Pylons Execution Analysis. We'll modify the application in place because we'd have to change package names in many places if we made a separate copy.

Install Durus by running "easy_install durus".

Modify the QuickWiki modules to match the code in the "QuickWiki Code Changes" section.

In a separate shell window, start the database server and leave it running:

$ durus -s --file=qwiki.durus --host=localhost --port=5001

In your quick-wiki.ini file, comment out the "sqlalchemy.dburi" line and add the following line:

durus.main = client://localhost:5001

Run "paster setup-app quick-wiki.ini".

Test the database with Durus's command-line utility:

$ durus -c --host=localhost --port=5001
Durus qwiki.durus
    connection -> the Connection
    root       -> the root instance
>>> root["pages"].items()
[('FrontPage', 'Welcome to the QuickWiki front page.')]
>>>

Press ctrl-d to end the test.

Run "paster serve quick-wiki.ini". The application should behave identically to the SQLAlchemy version.

Press ctrl-c in both the paster serve window and the database server window to quit them.

QuickWiki Code Changes

$LIB/pylons_durus.py

Download pylons_durus.py to your $LIB directory. Documentation is in the module docstring.

$LIB/QuickWiki*.egg/quickwiki/websetup.py

import os
from durus.persistent_dict import PersistentDict
from pylons_durus import get_connection
from quickwiki.models import *
from paste.deploy import appconfig

def setup_config(command, filename, section, vars):
    app_conf = appconfig("config:" + filename)
    try:
        url = app_conf["durus.main"]
    except KeyError:
        raise KeyError("no 'durus.main' entry in config file")
    conn = get_connection(url)
    root = conn.get_root()

    print "Creating tables"
    root.clear()
    root["pages"] = PersistentDict()

    print "Adding front page data"
    pages = root["pages"]
    title = 'FrontPage'
    content = 'Welcome to the QuickWiki front page.'
    pages[title] = content

    print "Committing changes"
    conn.commit()

    print "Successfully setup"

$LIB/QuickWiki*.egg/models/__init__.py

import re
import quickwiki.lib.helpers as h
wikiwords = re.compile(r"\b([A-Z]\w+[A-Z]+\w+)")
from docutils.core import publish_parts

def get_wiki_content(content):
    content = publish_parts(content, writer_name="html")["html_body"]
    titles = wikiwords.findall(content)
    for title in titles:
        content = content.replace(title, h.link_to(title,
                h.url_for(controller='page', action='index',
                    title=title)))
    return content

Commentary

get_wiki_content is a function now because there's no Page class to attach it to. Our equivalent of Page is a string value in the "pages" dict, so we just pass it directly. As a side effect, this module does not depend on Durus at all. A more complex application would define classes here (Persistent or not) which would be stored in the database.

$LIB/QuickWiki*.egg/quickwiki/lib/base.py

from pylons import Response, c, g, cache, request, session
from pylons.controllers import WSGIController
from pylons.decorators import jsonify, rest, validate
from pylons.templating import render, render_response
from pylons.helpers import abort, redirect_to, etag_cache
from pylons_durus import get_connection
from pylons.util import _
import quickwiki.models as model
import quickwiki.lib.helpers as h

class BaseController(WSGIController):
    def __call__(self, environ, start_response):
        # Insert any code to be run per request here. The Routes match
        # is under environ['pylons.routes_dict'] should you want to check
        # the action or route vars here
        return WSGIController.__call__(self, environ, start_response)

class DBController(BaseController):
    def __call__(self, environ, start_response):
        cfg = environ["paste.config"]["app_conf"]
        url = cfg["durus.main"]   # Raises KeyError.
        self.db_connection = get_connection(url)
        self.db_connection.abort()  # Roll back any uncommitted transaction.
        return BaseController.__call__(self, environ, start_response)

Commentary

We add a DBController base class for our controllers that need database access.

$LIB/QuickWiki*.egg/quickwiki/controllers/page.py

from quickwiki.lib.base import *
from quickwiki.models import *

class PageController(DBController):
    def index(self, title):
        root = self.db_connection.get_root()
        content = root["pages"].get(title)
        if content:
            c.content = get_wiki_content(content)
            return render_response('page')
        elif model.wikiwords.match(title):
            return render_response('new_page')
        abort(404)

    def edit(self, title):
        root = self.db_connection.get_root()
        content = root["pages"].get(title)
        if content:
            c.content = content
        return render_response('edit')

    def save(self, title):
        root = self.db_connection.get_root()
        root["pages"][title] = request.params["content"]
        self.db_connection.commit()
        c.title = title
        c.content = get_wiki_content(root["pages"][title])
        c.message = 'Successfully saved'
        return render_response('page')

    def list(self):
        root = self.db_connection.get_root()
        c.titles = self._get_titles(root)
        return render_response('titles')

    def delete(self):
        root = self.db_connection.get_root()
        title = request.params['id'][5:]
        try:
            del root["pages"][title]
        except KeyError:
            pass
        else:
            self.db_connection.commit()
        c.titles = self._get_titles(root)
        return render_response('list', fragment=True)

    def _get_titles(self, root):
        titles = root["pages"].keys()
        titles.sort()
        return titles

Commentary

The controller is adjusted to use self.db_connection, and to access the title and content in the Durus manner.