Monday, October 17, 2005

Emergency Axiom 0.1

An object-relational (sort of) database, implemented in Python, backed by SQLite, and informed by the development of Atop, twisted.world and others: Axiom is still young (hardly three months have gone by since its conception), but it already supports a significant portion of the features present in the databases which shaped its development — and has for longer than not.

Axiom's goal is to provide an efficient, non-SQL interface to an existing relational database. Unlike other libraries in this area, Axiom discourages inefficient operations (such as loading a single object at a time - possibly building it up from multiple queries against different tables): instead it encourages a style where many objects are loaded with a small, often fixed, number of queries (sometimes the number being fixed at one). This lets the underlying relational database do what it's good at: yank data into memory out of underlying storage, perhaps based on some limiting criteria, or sorted in a particular way. From the Python side, you get genuine Python objects with Python-native attributes: Axiom is object-based, not table-, column-, or row-based. Objects stored in an Axiom database can refer to any other object stored in the same database (regardless of type - just like normal Python references), and attributes can be instances of any type Axiom natively supports (currently booleans, integers, byte strings, character strings, or timestamps). Additionally, support for new attribute types can be added by defining new attribute classes. Objects stored in an Axiom database provide a schema definition (as part of their class definition) which allows in-memory constraints to be enforced, just as they will be in the database, as well as providing necessary information for the upgrade mechanism (ie, you can change your schema without invalidating an existing database).

Of course, Axiom also integrates with common Twisted practices:

  • an Axiom store is also a Twisted IServiceCollection provider. A Store can be started and stopped, each of which propagates the appropriate event down to any in-database objects which care to receive them. This allows (if you will excuse the extremely hypothetical example) a web service to be implemented as an Axiom-stored object: when the object receives startup notification, it can bind port 80 and start handling requests; when it receives shutdown notification, it can close the port and delay actual shutdown until it has given each outstanding request a fair chance to be completed. By letting services exist as database objects, any database manipulation tools (such as applications or user- or administrative-configuration interfaces can persistently alter application configuration - goodbye, configuration files).

  • Axiom comes with an implementation of Twisted's IRealm interface. This means an Axiom store can be given to a portal to allow authentication and authorization against it, using the standard Twisted Cred APIs. Any Twisted protocol implementation which supports Twisted Cred (so, SMTP, POP3, IMAP4, SIP, HTTP, IRC and PB to name a few) will work with an Axiom Store backend.

  • Axiom makes heavy use of plugins in several ways. Features for an Axiom database can be provided and configured via a particular interface: IAxiomaticCommand. Once a plugin is installed and its Python modules are importable, it will be available to the database administrator (be that a sysadmin on the command line or an application admin via a web interface). Objects in the database can also act as powerups for other objects in the database. Powerups provide additional functionality in a modular and pluggable way, and can be queried for in a simple and efficient manner.

  • Axiom includes a wall-clock scheduler as well, made available to application-level code as a powerup on an Axiom Store. By loading up this powerup and calling its methods (such as the surprisingly-named schedule), application-level code can arrange to have its own code invoked at a particular point in the future, be it seconds, hours, or months away.

These features make it very straightforward to use Axiom in a Twisted application.

Currently, Divmod has several applications in development which use Axiom, and several more in the pipeline. Though Axiom is far from complete, even in its current form it has demonstrated its utility for a variety of applications, both client- and server-side.

It's difficult to include a demonstration of Axiom's abilities here: only the most trivial of database applications can reasonably fit into a blog post, yet such examples are rarely very interesting or informative. Instead, here's a brief transcript of an interaction (in which I create myself a new user account and disable an existing account) with a database being used by a running server, using the command line tool included in Axiom, axiomatic:

$ axiomatic --dbdir cc.axiom/ userbase list | grep exarkun                                 
exarkun@divmod.com
$ axiomatic --dbdir cc.axiom/ userbase create exarkun divmod.org
Enter new AXIOM password:
Repeat to verify:
$ axiomatic --dbdir cc.axiom/ userbase list | grep exarkun
exarkun@divmod.com
exarkun@divmod.org
$ axiomatic --dbdir cc.axiom/ userbase --help
Usage: axiomatic [options] userbase [options]
Options:
--version
--help Display this help and exit.
Commands:
install Install UserBase on an Axiom database
create Create a new user
disable Disable an existing user
list List users in an Axiom database

$ axiomatic --dbdir cc.axiom/ userbase disable exarkun divmod.com
$ axiomatic --dbdir cc.axiom/ userbase list
exarkun@divmod.com [DISABLED]
exarkun@divmod.org
$
userbase is a plugin included with Axiom for user management; the axiomatic script pulls it in and makes it available to me on the command line, making the above possible.

Download Axiom 0.1, visit the project page on divmod.org, or check out the Axiom wiki page.

3 comments:

  1. Can it be used outside Twisted? For example, as a CGI app?

    ReplyDelete
  2. Yes. But you don't want to do that, do you? :)

    ReplyDelete
  3. Errrr. There is an important word missing at the end of that sentence, I guess. "manager". Your understand is correct - Axiom expects to completely control the database, but it interfaces to an existing database manager, rather than trying to do everything itself.

    Data migration? Between different versions of Axiom? Between servers running the same Axiom on different hosts? Replication of live servers?

    Some stuff in this area is definitely planned. The general rule of thumb we try to follow is that if a task can be automated, it should be automated. Preferably by Axiom. :)

    ReplyDelete