Smorgasbord

A Universal Templating Front-End for Python

Author: Mike Orr <sluggoster@gmail.com>
License:MIT -- http://www.opensource.org/osi3.0/licenses/mit-license.php
Copyright: 2007-06-03

Status

Initial test for Mako/Cheetah/Genshi/Kid/string.Template passes. No template options supported yet.

TODO:

Introduction

Smorgasbord is a common front end to Mako, Genshi, Cheetah, Kid, and other template engines. It allows Python applications and frameworks to access any of these engines, switch between them easily, and use multiple engines in the same application. Smorgasbord is inspired by Buffet but corrects design deficiencies and scalability problems in Buffet 1.0, and has better documentation. All internal variables are named Buffet to allow this project to merge into Buffet if the Buffet team permits. The API and plugins are _not_ compatible with Buffet 1.0. Basic usage:

# Assume a Mako template /myapp/templates/hello.txt contains
# "Hello, ${what}!"  Output will be "Hello, Mako world!"
from smorgasbord import Buffet
buffet = Buffet()
buffet.init_engine("mako", dirs=["/myapp/templates"], naming="uri")
info = {"what": "Mako world"}
print buffet.render(None, "/hello.txt", info)

At startup, Smorgasbord scans all installed Python packages looking for template plugins. Plugins are small classes that translate between the Smorgasbord API and an engine's native API. The plugin's entry point (a short string) becomes the engine's name as seen by the user. Note that the same underlying engine may have multiple entry points for different modes; e.g., "genshi" and "genshi-text". Smorgasbord includes plugins for several popular engines. Other engines may include a plugin in their distribution, or the plugin may be in a separate Python package.

Smorgasbord ships with plugins for the following template engines:

Engine name Underlying template engine
mako Mako
genshi Genshi
genshi-text Genshi
cheetah Cheetah
kid Kid
myghty Myghty
breve Breve
string string.Template

string.Template is part of the Python standard library so it's always available. To use the other engines, you must install the appropriate Python package (e.g., "easy_install Mako").

Usage in applications and frameworks

See the above example for basic usage.

Constructor:  Buffet()   -- no arguments

.init_engine(engine_name, **options)
    Initialize a plugin for rendering.
    'engine_name' is the plugin's entry point (a short string).
    '**options' are engine-specific template options.
    Exceptions:
        TemplateEngineMissingError
        NotImplementedError (for options not recognized by the plugin)

.render(engine_name, template, info, **options)
    Invoke the plugin to render the template and return the result.
    You must call .init_engine before calling this!
    'engine_name' is the plugin's entry point, or None to use the default
    engine.
    'template' is a string identifying the template, normally its
    filesystem path or URI.
    '**options' are engine-specific rendering options.
    The return value is normally string or unicode, though the engine
    may return whatever it wants.
    Exceptions:
        TemplateEngineNotInitializedError
        NoDefaultTemplateEngineError
        TypeError (for options not recognized by the plugin)
        OSError (if the template file does not exist)

.set_default_engine(engine_name)
    Make this engine the default engine.  You must call .init_engine
    before calling this.
    'engine_name' is the plugin's entry point.
    Exceptions:
        TemplateEngineNotInitializedError

Buffet objects have the following instance attributes:

.available
Dict of engine name : plugin class. These are all the plugins Smorgasbord knows about. You can modify this attribute if you have a plugin without an entry point, or to use a different version than the one Python discovered.
.engines`
Dict of engine name : plugin instance. These are all the plugins that are ready for rendering (i.e., buffet.init_engine has been called).
.default_engine
Plugin instance.

There is also a class attribute .entry_point_group specifying which group to look for plugins under. Currently the value is "python.smorgasbord.plugins".

Pylons and TurboGears

I'm working on a Pylons patch for Smorgasbord. When it's completed you'll put this at the top of your applications middleware.py, above the config.init_app(...) line:

from smorgasbord.patches import make_pylons_use_smorgasbord

Eventually the code may be merged into Pylons itself. We're looking for a TurboGears programmer to write a similar patch for TurboGears.

BuffetCached

The BuffetCached subclass saves compiled template in a memory cache for quick reuse. The cache is a simple dict that never expires items. If you have so many large templates you're worried about running out of memory, subclass it and and implement a different cache algorithm. Some engines (Kid) can't produce a thread-safe template object, so their templates won't be cached.

The cache disappears when the application process ends. If you want a persistent cache, you can try pickling the .cache attribute before exiting and unpickle it when the application starts again. (Still need to research which engines produce pickleable templates. This may become a requirement for caching.)

Options

Plugins accept both template options and rendering options. Template options are keyword arguments to .init_engine and modify the engine's general behavior (e.g., character set). Rendering options are keyword arguments to .render and modify the specific output (e.g., HTML or text format). All plugins shipped with Smorgasbord have a common set of standard options which are translated to the appropriate engine-specific options. In addition, some plugins have engine-specific options. We generally try to make all features of the underlying engine accessible if possible.

Note

Of the options below, only 'dirs', 'naming', and 'exts' are currently implemented. The others are potential options extrapolated from Buffet 1.0 and the engines' method signatures, but cannot currently be specified.

Standard template options

Template options:

  • dirs: a list of directories to search for templates in.
  • naming: This defines how template files are looked up.
    • "path": Identifiers are in native pathname syntax ("foo/bar.html"). If the path is absolute, use it as is. Otherwise look in every directory on the dirs path until a template file is found.
    • "uri" (default): Identifiers are in URI syntax ("/foo/bar.html"). Drop the intial slash if specified, run the remainder through os.path.normpath, then proceed as with "path" naming.
    • "dotted": Identifiers are in Python module syntax ("foo.bar") but map to an engine-specific filename ("foo/bar.kid"). This mode is not implemented yet, and may never be. It's intended to provide TurboGears behavior for Genshi, Kid, and Cheetah, if that is considered necessary.
    • "module": Identifiers are in dotted syntax and represent a Python module that should be imported, relative to the 'dirs' path. This mode is not implemented yet, and may never be. This mode is valid only with engines that compile templates into importable Python modules; i.e., Kid and Cheetah. If the 'precompiled' option is passed and is true, only Python modules will be considered. Otherwise "dotted" mode is also tried, and if both a Python module and a source template are found, the one with the latest modification date takes precedence.
    • "literal": The identifier is the template itself.
  • exts: a list of filename extensions that will be appended to the template filename in turn if the indicated template does not exist.
  • module: an optional package prefix added to the template identifier if 'naming' is "dotted"' or "module".
  • output_encoding: the encoding of the rendered result; e.g., "utf-8". Pass "unicode" to get Unicode output if the engine supports it. Unsure whether to call this option 'encoding'.

Standard rendering options

  • fragment: default False. If true, render the specified template without any site template that would normally be rendered around it. Kid and Genshi handle this natively, and perhaps Myghty also. Cheetah and Mako mimic this behavior by calling a template method called "fragment"; it's up to the template to define this method.

Mako

Template options:

  • ?? (Not using TemplateLookup so are there any?)
Rendering options:
  • fragment: default False. If true, call a '<%def name="render()">' method instead of the normal rendering. The method must be defined in the template. This mimics Kid/Genshi's 'fragment' option, which renders the specified template alone without any site template around it.

Myghty

Template options:

  • A few dozen scattered throughout the Pylons source (pylons/config.py, pylons/templating.py, and myapp/environment.py).

Rendering options:

  • fragment?

Cheetah

Rendering options:

  • fragment: default False. If true, call a #def render" method instead of the normal rendering. The method must be defined in the template. This mimics Kid/Genshi's 'fragment' option, which renders the specified template alone without any site template around it.

Kid

Template options:

  • encoding: default "utf-8".
  • sitetemplate: automatically apply a site template around every template?
  • assume_encoding: default "utf-8". Default encoding for source template files?
  • i18n.run_template_filter: something to do with multilingual templates?

Rendering options:

  • Needs more research. Probably 'format'.
  • fragment: if true, render the template without any site template that would normally be rendered around it.

Genshi

Both "genshi" and "genshi-text" recognize these template options:

  • default_encoding: default "utf-8".
  • auto_reload: default True. Maybe, if the functionality is in the underlying engine rather than in the plugin.
  • search_path: colon-separated list of directories to search for templates in. If empty use sys.path.
  • max_cache_size: default 25.

"genshi" additionally recognizes these template options:

  • default_doctype: any of "html", "html-strict", "html-transitional", "xhtml", "xhtml-strict", "xhtml-transitional".
  • default_format: any of "html", "xhtml", "xml", "text".

Rendering options for "genshi" and "genshi-text":

  • Needs more research.
  • fragment: if true, render the template without any site template would normally be rendered around it.

Test suite

Smorgasbord comes with a Nose test suite. To run the tests you'll have to install Nose, Unipath, and all the template engines used in the tests:

$ easy_install 'Nose>0.9.3'
$ easy_install Unipath
$ easy_install Mako
$ easy_install Cheetah
$ easy_install Genshi
$ easy_install Kid
$ cd PARENT_OF_SMORGASBORD_PACKAGE
$ nosetests smorgasbord/tests/test.py

Nose often has difficult finding tests; the last two lines are what work for me. If it reports "Ran 0 tests", it didn't find the Smorgasbord tests. In that case you may have to read Nose's help or ask a Nose expert for advice.

The plugin interface

Engine plugins have the following methods:

def __init__(self, **options):
    """
    '**options' are plugin-specific template options.
    Raise TypeError if you don't recognize an option, unless you're sure
    you can ignore it.
    """
    self.options = options

def load(self, template):
    """
    Return a compiled object that can be passed to render in multiple
    threads simultaneously, and ideally pickled too.  If the engine can't
    produce this kind of object, return None.
    'template' is a template identifier (path, URI, dotted module name,
    database key, etc).
    """
    return None

def render(self, template, info, generic_options=None, **render_options):
    """
    Return the rendered version of the template.
    'template' is the return value of .load, or the original template
    identifier of ``.load`` returned None.
    'info' is a dict of data values to be applied to the template (aka
    "placeholder values" or "context values").
    '**options' are plugin-specific rendering options.
    Raise TypeError if you don't recognize an option, unless you're sure
    you can ignore it.
    """
    raise NotImplementedError()

A sample base class is in smorgasbord/plugins/__init__.py. Use it if it's convenient; ignore it otherwise.

Convenience functions

The smorgasbord.utils module includes several functions useful in applications.

.find_plugins(entry_point_group)

Called by the Buffet constructor to initialize the .available attribute. This function scans any entry point group and returns a dict of the entry points found, in the form {entry point : implementing class}.

.get_options(prefix, options_dict)

Extract options from a general configuration dict such as Pylons app_conf. This looks for keys starting with 'prefix' plus a period, and returns a new dict containing those keys without the prefix. This function is implemented but not yet tested. Example:

app_conf = {"kid.encoding": "utf-8", "foo": "bar"}
kid_options = buffet.get_options_for_engine("kid", app_conf)
# 'kid_options' is {"encoding": "utf-8"}
buffet.init_engine("kid", **kid_options)

configure_template_engines(buffet, options_dict, fallback=None, framework_options=None)

This is all you need to do to initialize templates in a Pylons application or any application that uses a general configuration dict. This function is not implemented yet. Arguments:

'buffet': ``Buffet`` instance.
'options_dict': Configuration options to parse.
'fallback': Name of a template engine to initialize if no engine is
    specified in the config file.
'framework_options': a dict of additional generic options to pass to
    all engines.  These are to enforce the framework's common look &
    feel, and do not originate in the config file.  Options found in
    'options_dict' override these.

The function looks for an 'options_dict' key "smorgasbord.engines", and parses the value into a comma-separated list of engine names. All other keys with a "smorgasbord." prefix turn into generic options for all engines. All keys beginning with a prefix mentioned in the engines list are turned into engine-specific template options. The engines are then initialized, and the first engine listed becomes the default engine.

If "smorgasbord.engines" is not present or the value is empty, 'fallback' engine is initialized instead and becomes the default engine. If 'fallback' is None too, the function does nothing. This will cause an can lead to an exception later if no engine is initialized and buffet.render is called. To guard against this, after calling this function check to see if buffet.default_engine is None, and abort the application if it is.

find_file(search_path, filename, is_uri)

Implements the naming="uri" and naming="path" template options. 'search_path' is a list of directories; if empty the current directory is used. 'filename' is the file we're looking for, which may have a directory prefix. If 'is_uri' is true, strip any initial slash from 'filename' and run it through os.path.normpath before searching for it. The return value is the path of the found file, or None if the file was not found.

find_module(search_path, identifier, ext="", in_module=None)

Implements the naming="dotted" template option. This function is not implemented yet, and may never be. Both 'identifier' and 'in_module' should be in Python dotted "package.module" format. Create a filesystem path by prefixing 'identifier' with 'in_module', turning the dots into path separators, and adding extension 'ext'. Call 'find_file()' on the result. Or should this function actually import the Python module? There seem to be different answers for Cheetah, Kid, and Genshi, especially if the 'precompiled' option is considered, in which case we'd have to import the .py template module ourselves because the engine won't.

Design decisions

Smorgasbord does not directly handle templates calling other templates (include, inheritance, site template). In Mako's case, the 'dirs' option is translated directly to Mako's "directories" argument, and this is used for <%include> (or it will be when the option is implemented). In Cheetah's case, #include and #extends do not have anything akin to 'root_paths', and #extends uses dotted notation and imports a .py template module directly. There is nothing Buffet can do about this.

Both Cheetah and Kid have inconsistencies with URI notation, because their internal include mechanisms use dotted notation and import an actual Python module (though Kid can compile it from a .kid file). Using dotted notation for template lookup and putting valid __init__.py files in the template directories clears up these inconsistencies, but introduces another inconsistency for Kid and Genshi. Kid and Genshi in Buffet 1 use dotted notation and require __init__.py files, but they're really doing a path lookup for ".kid" or ".html" files, which are not legal Python module names, so why the __init__.py files? Oh, they're needed for include and inheritance. Right.