Friday, December 29, 2006

November / December reading list




I've done some things different with this list. First, I've ordered them in terms of relative objective quality from best to worst. Second, of those which, were I to be struck in the head and lose the last two months of my memories (but only after having written a note to myself suggesting reading material), I would read again, I have linked via my Amazon Affiliate account (most likely a waste of time - but it is an experiment!) to a web page where each may be purchased.




Oh and here is one more which I could not, in good conscience, include at any position in the above list, even though the ordering is mostly a joke:



Relativity: The Special and the General Theory. Albert Einstein.

Sunday, December 17, 2006

A man walks into a bar. What'll it be? asks th...

A man walks into a bar.

What'll it be? asks the bartender.

Replies the man, Tell me, where could I find a non-alcoholic drink which will get me just as inebriated as that fine scotch just behind you?

Ahh, I don't know, but perhaps you'd like some of that scotch?

No, I don't drink alcohol. Goodbye.

If something about this conversation strikes you as odd, then you are dissimilar to a good number of programmers in at least one way.

Friday, December 8, 2006

An Hour of My Life Wasted

Python's warnings module stashes internal state tracking information in a dictionary which it jams into the global scope of one of the functions on the stack at the time the warning is emitted.

Brilliant idea, guys.

Sunday, November 19, 2006

Imaginary (An architectural overview)

View from 20 klicks




Imaginary is going to be a simulation engine. Simulation engine is used here to refer to any software library which is capable of modelling (not necessarily in a manner which mirrors reality) interactions between a number of systems (not necessarily systems taken from reality). Some examples of such systems are physical interaction between different objects, chemical change in a material due to the introduction or removal of energy, and alterations in the propagation of electromagnetic radiation by intermediate transmission substrates.




Behaviors implemented by a single system must be specified completely and in detail in order to achieve somewhat predictable outputs for a given set of inputs. Likewise, interaction between different systems needs to be specified somehow. Explicitly defining the interactions of each system with every other system requires N! such definitions, where N represents the total number of systems. As N increases, N! increases prohibitively more quickly, preventing a large number of systems from being supported.
An important aspect of a successful implementation of a simulation engine is avoidance of this combinatorial explosion.




Currently, it would be most accurate to call Imaginary part of a research project for exploring possible ways to implement a simulation engine. It is the 5th or 6th (or perhaps 7th or 8th, depending on how you count) generation implementation by the team of developers involved in the project.




What it can do now is support multiuser interaction in a single persistent world, accept and parse a limited number of commands for interacting with the world, and transmit a limited amount of information to each user when an observable part of the world changes somehow.



Containing Framework




In its current incarnation, Imaginary is a Mantissa offering. Mantissa offerings are beyond the scope of this post, but you can read a bit about them on the Divmod Wiki or in previous blog entries of mine. If you are interested in Imaginary, then you can probably skip that for now. The salient points are simply that:



  • Imaginary ignores signup because it's taken care of.


  • Imaginary uses Axiom for persistence because that's what Mantissa encourages.


  • Imaginary uses Twisted for networking.




Taking Action




User interaction is close to the primary purpose of Imaginary. The results of the simulation need to be exposed, and changes to the state of the simulation world must be enactable by users. The current user input system for Imaginary is a simple text based interface exposed over SSH. Text input is parsed into higher level structures known as actions. Actions define their own parsing rules and are associated with code for inspecting or manipulating the game world.




Examples of actions are opening or closing a container, picking up or dropping an object, speaking, moving around, or examining some aspect of the world. When an action has completed, observers are notified using the event system. Sometimes, the implementation of an action is responsible for emitting these events. Other times, the game system which the action has interacted with will emit them. However, actions are not always successful. An object may be too heavy to be taken or a container may already already be in the state into which opening or closing it would have put it. These are some of the simpler cases which Imaginary can handle: the implementation of the action handles exceptions from the game system it is manipulating and emits the appropriate event (for example, instead of broadcasting that Alice has lifted a boulder above her shoulders, it would broadcast that Alice struggles and strains but is unable to budge the boulder).




Certain actions can result in more complex interactions. For example, movement between two locations involves two events being broadcast: one to the observers at the location being vacated, one to the observers at the location being occupied. To deal failures with this kind of action, other projects have often adopted the look before you leap approach: the action implementation can perform checks to determine if the actions is going to fail; if it is, it can take care to broadcast only a failure event to the location the user attempted to leave (not broadcasting any event to the destination location, since no observable change has occurred there). The drawback to this solution is that it requires the movement action to be completely aware of every interaction which might prevent the movement from succeeding.




Instead, Imaginary's action system uses a transactional event broadcasting mechanism. When the movement action begins, the departure event is broadcast at the location being vacated. Next, the action invokes the actual code in the movement system (or containment system). If the movement is not actually possible, this will raise an exception, preventing further action code from running and causing the entire action to be aborted. Events broadcast as part of an aborted action are not sent to observers. If the containment system allows the movement, the action continues to execute, resulting in an arrival event being broadcast to the destination. Once the action has finished running without aborting, all of the events it emitted are delivered to observers.



The Observable World




I've mentioned events above quite a bit. Events are the basic unit of information which moves around an Imaginary world. When someone walks into a room, if you can see it happen, it is because you received an event that described the change in the state of the world. If you can't see it happen, the event has failed to reach you for some reason. If you're in a dark room, for example, events which convey visual information may be unavailable to you.




Blindfolds, colored glass boxes, parabolic dishes and other world features which modify the propagation of an event are implemented by giving objects an opportunity to wrap a proxy around an event as it propagates. When an event is finally observed, it may have several layers of proxying wrapped around it which change what the observer actually perceives. A red glass box may make red items inside it invisible and make green items appear black. The red and green balls rolling around inside the box are unaware of the visible light red/yellow/blue color system, though. This leaves the implementation of colored balls simple and unfettered by all of the complications which may arise between the balls themselves and other actors in the world which observe or manipulate them.



Graph Traversal




Many of the concepts discussed above rely on what should probably be considered the core of Imaginary: a set of APIs which allow the object graph representing the simulation world to be queried and traversed in a structured way. All physical objects in the simulation world are represented by an instance of a class named Thing. Thing has a couple important methods: links and findProviders.




links defines the relationships a Thing has with other Things in the world. Everything is linked to itself and its location. Boxes, for example, are also linked to their contents. A room with a door is linked to the room into which the door leads. The implementation of links is extensible using a feature of Axiom called powerups. This is reminiscent of the Componentized idea from previous iterations of the research project.




findProviders implements a breadth-first, depth- and interface-limited search of a Thing's links. It also applies proxies as it traverses the object graph. This method supports both the action system and the event system in a major way. The input parser uses it to resolve string names into concrete object references. The event broadcast system uses it to find observers who should receive events.



Get Involved






  • You can read more in the Reality archives: http://twistedmatrix.com/pipermail/reality/


  • You can check out the code: svn co svn+ssh://divmod.org/svn/Divmod/trunk Divmod


  • You can join us on IRC in #imagination on irc.freenode.net


Thursday, November 16, 2006

Tuesday, November 7, 2006

Reading List for September/October

The Space Opera Renaissance. Edited by David G. Hartwell and Kathryn Cramer.

This is a collection of shorts. The stated goal of the work is to offer a definition of the term space opera by way of example.

The opening volley consists of these four stories:

The Star Stealers. Edmond Hamilton.
The Prince of Space. Jack Williamson.
Enchantress of Venus. Leigh Brackett.
The Swordsmen of Varnis. Clive Jackson.

I suppose because I am just a little kid, the first two of these nearly turned me off the entire book. They are absolutely typical of the worst stereotypes conjured to my mind by the label of space opera. The names are strange. The plots are contrived while being uninteresting recasts of stories from other genres of fiction. The characters are flat. To be fair, these are stories from 1929 and 1931 respectively. They are pioneering works in the genre of science fiction overall. Still, literary precedents and contextual considerations aren't enough to save them for me. Fortunately I pressed on. Brackett's story, written in 1949, is much more enjoyable. It has a historic feel similar to that of the previous two, and the setting is completely implausible as a science fiction setting, but the writing is well done and the story is actually interesting. It feels like a space adventure. The Swordsmen of Varnis is short and silly, more an extended joke than an example of space opera, or even science fiction. I'm not sure why it was included. However, at a page and a half it's not much of a disruption.

The next three stories are taken from the 60s:

The Game of Rat and Dragon. Cordwainer Smith.
Empire Star. Samuel R. Delany.
Zirn Left Unguarded, the Jenghik Palace in Flames, Jon Westerley Dead. Robert Sheckley.

Just moving forward a decade or two gives a striking change in the level of refinement of the science involved as compared to the first four. The Game of Rat and Dragon has quite an interesting premise and carries it through in an enjoyable style, but doesn't have much of a story. On the other end of the spectrum, the much longer Empire Star involves a very involved sequence of events and follows three characters over a much longer span of time. The universe Delany reveals is subtle and not fully explained, which is certainly part of what grabbed my interest as I read it. Shekley's story is as mystifying as the title might suggest. Perhaps he was making references which simply went over my head, but I basically found the story, if it can be called that, incomprehensible.

The 70s and 80s are carried by the following:

Temptation. David Brin.
Ranks of Bronze. David Drake.
Weatherman. Lois McMaster Bujold.
A Gift from the Culture. Iain M. Banks.

Temptation, Weatherman, and A Gift from the Culture each share their setting with at least one full length novel by the same author. Many people, I'm sure, are familiar with the Uplift and Culture series. I haven't read either of them myself but I suspect I will in the near future. The characters in Temptation were not particularly appealing (they weren't even humanoid, which always makes it tough to decide if their motivations are nonsensical or just incomprehensible - not fun either way, really), but the setting seems interesting. In stark contrast, Weatherman's character development, though brief, was very good, and made more interesting by the fact that the protagonist, Miles Vorkosigan is not a particularly likable fellow. I've heard a lot about Banks' Culture universe, so I would have gotten around to his novels eventually. The Culture doesn't appear much in this short, at least not directly, so I'm not sure how it compares to the rest of the works in this universe. The story was enjoyable though, and well written. Ranks of Bronze is a stand-alone, and has the distinction of being the only science fiction I can think of which includes a legion of Roman centurions (Hmm, Dr Who must have visited the Roman empire at some point, but I can't recall a specific episode).

The 90s is much more of a mixed bag:

Orphans of the Helix. Dan Simmons.
The Well Wishers. Colin Greenland.
Escape Route. Peter F. Hamilton.
Ms. Midshipwoman Harrington. David Weber.
Aurora in Four Voices. Catherine Asaro.
Ring rats. R. Garcia y Robertson.
The Death of Captain Future. Allen Steele.

If you like Hyperion, I guess you'll like Simmons' short story set in the same universe, post-Aeneanism. I mostly found Hyperion to be frustrating, confusing, and often pointless (though perhaps not entirely without redeeming virtues). Orphans of the Helix is okay, but also strikes me as similarly pointless. The Well Wishers also doesn't do much for me. The setting is strange but very uninteresting (which is somewhat intentional, I think, but that doesn't make me enjoy it any more) and the characters fairly flat. Little action takes place during the story as well, which one ultimately learns is a mystery (just in time to learn the resolution). Escape Route is somewhat hard scifi, reminiscent of some of Niven's work. Unfortunately, some of the prose is as obtuse as Niven's. About half the characters are also truly stupid, to a degree which stretches credibility (hmm, another similarity to Niven). It is enjoyable in the way that many hard scifi stories are enjoyable: some cool gadgets makes up for most or all of the other shortcomings. The Death of Capture Future might be more interesting to someone who has read anything about Captain Future before. Since I haven't, it didn't interest me too much. I related to roughly none of the characters, particularly the protagonist, who catches some unlucky breaks and deals with this by being a completely unlikable jackass.

Weber's story reminds me very strongly of Walter Jon Williams' Dread Empire's Fall. They are both serious space opera territory: big fleets zooming through space, classic naval military organizations, battles fought over vast distances between major stellar empires. This is the other kind of story which space opera often summons to my mind.

Ring Rats also has some of this feel to it. The story and setting are totally different from those of Ms. Midshipwoman Harrington, but it has the same feel of big events happening around individual people. In space.

I'm going to skip the last 15 stories since this post is pretty long already. There is some good stuff towards the end, though. If you haven't guessed, the collection is ordered chronologically, so the later stuff is from the late 90s up to the present. One of the stories features a monk, an anthropomorphic (literally) ferret, and the psychological equivalent of a VLA telescope. Another follows most of the life of a cowardly Kzin (set in known space, not by Niven) and his rise to power. Another about the love affair between an adolescent girl and the fledgling AI she (first unintentionally, then intentionally) helps raise to the level of sentience and self-awareness.

There's definitely some good stuff in here. I heartily recommend it to any scifi fan. If you think you may be a space opera junkie, then this is a must read.

Saturday, September 23, 2006

xserve g5

As Glyph announced on the Twisted mailing list, Apple has donated a machine to be used as a buildslave for Twisted's automated testing system. As he suggested would come to pass, here are some pictures of it.

DSCN0199
DSCN0201
DSCN0205
DSCN0202
DSCN0206

Friday, September 8, 2006

Twisted has 3000+ unit tests. That is not the mo...

Twisted has 3000+ unit tests. That is not the most tests of any project I know, but dang it is a bunch.

Wednesday, September 6, 2006

Reading List for August




  • Catcher in the Rye. J. D. Salinger.



  • Engine City. Ken MacLeod.



  • Jonathan Strange And Mr Norrell. Susanna Clarke.


Thursday, August 24, 2006

Web Development

Web-Development

There is a long story that goes with this, involving libc6-dbg, xlibs-dbg, valgrind, gdb, venkman, tightvncserver, firefox crashing, and xorg crashing. Maybe I'll write it down sometime (but probably not, living it was bad enough).

Sunday, August 13, 2006

Minuteman

Thirteen miles up the minuteman trail yesterday. Beautiful day, lots of people out. The trail was a little crowded, but not quite inconveniently so. About twenty seven miles total, as near as I can figure. I should probably get an odometer. I actually thought to bring my camera for once, but I didn't check the batteries first. I only got two pictures out of it before it gave up. There are a couple giant fields of heather and cat-tails, these are both of the one a couple miles past Lexington. (Click to get larger versions)

DSCN0171
DSCN0170

Thursday, August 3, 2006

Reading List for July

This is the easiest kind of blog post, and apparently the only one I have energy for:

Watchmen. Alan Moore, Dave Gibbons.
the curious incident of the dog in the night-time. Mark Haddon.
Lunar Park. Bret Easton Ellis.
Building Harlequin's Moon. Larry Niven, Brenda Cooper.
American Gods. Neil Gaiman.
Anthem. Ayn Rand.
Manta's Gift. Timothy Zahn.
Animal Farm. George Orwell. (Which apparently I left on a plane.)

Tuesday, June 20, 2006

January through June reading list

The Atrocity Archives. Charles Stross.
The Star Fraction. Ken MacLeod.
How to Write Science Fiction and Fantasy. Orson Scott Card.
The Sky Road. Ken MacLeod.
Learning the World. Ken MacLeod.
Wild Seed. Octavia Butler.
The Swords of Corum. Michael Moorcock.
Elric of Melniboné. Michael Moorcock.
We the Living. Ayn Rand.

Friday, May 26, 2006

A Note for Python Users

If you use Twisted (or any other network software which uses non-blocking sockets) and you handle large numbers of concurrent connections, you probably want to avoid Python 2.4.3:


exarkun@kunai:~$ python
fPython 2.4.3 (#2, Apr 27 2006, 14:43:58)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> files = []
>>> for i in range(1024):
... files.append(file('/dev/null'))
...
>>> import socket
>>> s = socket.socket()
>>> s.connect(('www.google.com', 80))
>>> s.send('GET / HTTP/1.1\r\n')
Traceback (most recent call last):
File "", line 1, in ?
socket.error: unable to select on socket
>>>


Hopefully this will be fixed in Python 2.4.4 and Python 2.5.0. Monitor the progress of this bug on the sourceforge bug tracker.

Monday, May 22, 2006

Limiting Parallelism

Concurrency can be a great way to speed things up, but what happens when you have too much concurrency? Overloading a system or a network can be detrimental to performance. Often there is a peak in performance at a particular level of concurrency. Executing a particular number of tasks in parallel will be easier than ever with Twisted 2.5 and Python 2.5:


from twisted.internet import defer, task

def parallel(iterable, count, callable, *args, **named):
coop = task.Cooperator()
work = (callable(elem, *args, **named) for elem in iterable)
return defer.DeferredList([coop.coiterate(work) for i in xrange(count)])


Here's an example of using this to save the contents of a bunch of URLs which are listed one per line in a text file, downloading at most fifty at a time:


from twisted.python import log
from twisted.internet import reactor
from twisted.web import client

def download((url, fileName)):
return client.downloadPage(url, file(fileName, 'wb'))

urls = [(url, str(n)) for (n, url) in enumerate(file('urls.txt'))]
finished = parallel(urls, 50, download)
finished.addErrback(log.err)
finished.addCallback(lambda ign: reactor.stop())
reactor.run()


[Edit: The original generator expression in this post was of the form ((yield foo()) for x in y). The yield here is completely superfluous, of course, so I have removed it.]



[Edit: The original post talked about Twisted 2.4 and Python 2.5. It has since turned out that Python 2.5 is too disimilar to Python 2.4 for Twisted 2.4 to run on it. Twisted 2.5 is required to use Python 2.5.]

Monday, May 15, 2006

Saturday, April 29, 2006

Right now, today, here's what you can do with I...


Right now, today, here's what you can do with Imaginary:



  • Create and launch a new Imaginary server using Divmod's axiomatic command line tool:


    axiomatic -d imaginary.axiom offering install imaginary
    axiomatic -d imaginary.axiom start




  • Connect to that server using a telnet or ssh client



  • Create a new character in the fully persistent world or log in to an existing character.



  • Create new objects or locations (actually the number of objects is currently fairly limited: you can create a quiche, a vending machine, a quarter, a shirt, a pair of pants, and a sword).



  • Walk around, look at things, talk to people, attack people, pick things up, put things down, wear clothing, take clothing off, browse the help files, write custom descriptions for things, and a handful of other trivial activities.




Transactions



Persistence in Imaginary is handled by Axiom, an object database with excellent support for transactions. All world manipulation is protected with transactions, preventing the world from entering an inconsistent state, even if a programmer error causes an exception partway through a complex mutation.



Powerups



Every game object (for example, a chair, a player, or a sentient energy cloud) is represented as a Thing. Thing defines basic behaviors such as location and some aspects of appearance. For any additional behaviors, a powerup is used. Powerups can be added to or removed from a Thing at any time, and combined in novel or unexpected ways, creating game objects which have behavior not explicitly accounted for by the author of the powerups involved.



Concepts



Whenever it is necessary to convey information, whether to a player or an NPC AI, a Concept is used. A concept abstractly represents some information (for example, the fact that a wild boar just charged into the clearing where you are standing) and is responsible for determining the most suitable representation of this information when expressing it to a Thing. A player connected to the system via telnet will have concepts expressed to him as a text string, while a player with a rich graphical client might see a 3d model of a boar run into view.



Events



Of course, since Concepts are mainly used to convey information about change in the world, they are closely related to events. The event system keeps track of one or more concepts as a transaction executes and the world is modified. If the transaction completes successfully, those concepts are broadcast to any observers in range. If the transaction fails, the failure is turned into a concept and that is broadcast instead. This prevents players from seeing the consequences of events which are ultimately reverted.



Actions



Where do Events come from? Actions, obviously! In Imaginary, an Action is a representation of a high-level change to the world. Most of the things in the list at the top of this post are examples of Actions. Each Action is defined by a class with several methods, but one really important one: do. When someone or something instigates an Action (for example, a player types "go west"), the appropriate Action class is instantiated and its do method invoked in a transaction. This results in some events, which in turn rely on Concepts, which are sent out to Things which may have Powerups which react to them.



So that, in a nutshell (a large nutshell, perhaps a coconut) is Imaginary as it stands right now. If this was interesting to you, consider subscribing to our mailing list

Friday, April 7, 2006

Divmod Nevow 0.8.0 Released

Hey folks, we've got a new Nevow release for you. Tons of great changes have gone in since 0.7.0. Athena in particular has gotten a lot of attention and here are some of the new things you'll find in 0.8.0:



  • Nevow Interactive Tests - a test runner which lets you exercise code in a browser and collect results in an structured manner.


  • JavaScript source lines in tracebacks logged on the server.


  • A JavaScript implementation of DeferredList and gatherResults from Twisted.


  • Direct use of Zope Interface (meaning everything will be faster from avoiding the compatibility layer provided by Twisted).


  • Transparent, structured argument passing for Widget.__init__.




There are plenty of other minor bug fixes and feature enhancements, too. Check out the ChangeLog for all the details. What are you waiting for?

Thursday, March 9, 2006

Reading List


Over about the last year, I've read:




  1. Collapse: How Societies Choose To Fail Or Succeed. Jared Diamond.



  2. Twisty Little Passages: An Approach to Interactive Fiction. Nick Montfort.



  3. The Legacy Of Heorot. Larry Niven, Jerry Pournelle, Steven Barnes.



  4. Choke. Chuck Palahniuk.



  5. The Extravagant Universe. Robert P. Kirshner.



  6. Knife Of Dreams. Robert Jordan.



  7. Dread Empire's Fall: Conventions of War. Walter Jon Williams.



  8. Cosmonaut Keep. Ken Macleod.



  9. The Island of the Day Before. Umberto Eco.



  10. Paths To Otherwhere. James P. Hogan.



  11. Moonseed. Stephen Baxter.



  12. The Zap Gun. Phillip K. Dick.



  13. Germinal. Émile Zola.







These are pretty much all worth a read, although if you read just 12 of them, I'd give The Island of the Day Before a miss. Germinal is probably the best of the bunch, but Collapse is the most educational, Knife of Dreams the most indulgent, and Conventions of War the most fun (but it's the last in a trilogy).




By way of meaningless comparison, about 46KLOC that I've checked into source control over the same period of time has survived to this evening.

Wednesday, March 8, 2006

Axiom Concurrency


Over the last several days, Axiom's concurrency story has come together quite a bit. This is something Divmod has put off, fairly confident that it would be easy once the time came. And hey, we were actually right (that's always fun). Quotient's fulltext indexer is now implemented as a second process which is automatically brought up and down with the main server and which is handed work to do through the Axiom database. This is done in a generic fashion with Axiom framework code doing most of the heavy lifting (like repeatedly adding one to an integer and comparing two numbers to see which is greater), so the actual application-level code in Quotient ends up doing something like this (actually, exactly this) in order to take advantage of the multi-process aspect of the system:


def installOn(self, other):
super(SyncIndexer, self).installOn(other)
self.store.findUnique(mail.MessageSource).addReliableListener(self, iaxiom.REMOTE)

def indexMessage(self, message):
# Do the actual full-text indexing of `message'
processItem = indexMessage




The code barely even has to be aware of the fact that anything is going on in a separate process: it simply declares its interest in MessageSource and the fact that it has no objection to a potentially esoteric execution context. Any application can take advantage of this feature just as easily to split computationally expensive or high-volume disk-bound tasks into a separate operating system process, where they will not interefere with code that is latency-sensitive (like a network server) and where they can take advantage of additional CPU or disks, providing for greater parallelization of the task.




Of course, like any user of the reliable listener system, be they local or remote, the indexer:



  • will be run as quickly as the system can manage, taking into account the interactive requirements being placed on the server as well as the other reliable listeners which have been created and are also vying for runtime;


  • can be throttled back if another task takes precedence;


  • can be monitored and reported on for user feedback or administrator introspection;

  • can be prioritized relative to other kinds of listeners;


  • can be reset entirely and allowed to re-process all old messages (useful when, for example, the fulltext index becomes corrupt or is replaced by a superior system);


  • or just messages which caused an error and due to code changes may now be processable without one.



All this is due to the fact that the fulltext indexer is not directly responsible for finding messages to index, but has merely subscribed to a source of messages, and it is up to that subscription to decide what and when messages are given to it to be indexed.

Monday, March 6, 2006

JavaScript source lines in Python tracebacks


Added support to Athena today for generating tracebacks like this one:


2006/03/06 19:44 EST [HTTPChannel,250,127.0.0.1] Traceback (most recent call last):
File "", line 0, in

--- ---
File "", line 0, in

File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/runtime.js", line 39, in ()
d.callback(result);
File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/divmod.js", line 136, in ([object Object])
return methodFunction.apply(this, args);
File "", line 0, in apply([object Object],[object Array])

File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/defer.js", line 137, in callback([object Object],[object Object])
self._startRunCallbacks(result);
File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/divmod.js", line 136, in ([object Object])
return methodFunction.apply(this, args);
File "", line 0, in apply([object Object],[object Array])

File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/defer.js", line 134, in _startRunCallbacks([object Object],[object Object])
self._runCallbacks();
File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/divmod.js", line 136, in ()
return methodFunction.apply(this, args);
File "", line 0, in apply([object Object],[object Array])

File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/defer.js", line 108, in _runCallbacks([object Object])
self._result = callback.apply(null, args);
File "", line 0, in apply(null,[object Array])

File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/athena.js", line 259, in ([object Object])
action.apply(null, actionArgs);
File "", line 0, in apply(null,[object Array])

File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/athena.js", line 191, in ("Nevow.Athena.callByAthenaID","s2c241",[object Array])
result = funcObj.apply(null, funcArgs);
File "", line 0, in apply(null,[object Array])

File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/athena.js", line 609, in (14,"setStatus",[object Array])
var widget = Nevow.Athena.Widget.fromAthenaID(athenaID);
File "/home/exarkun/Projects/Divmod/trunk/Nevow/nevow/athena.js", line 601, in (14)
throw new Error(nodes.length + " nodes with athena id " + widgetId);
File "", line 0, in Error("0 nodes with athena id 14")

nevow.athena.JSException: Error: 0 nodes with athena id 14




Still a bit unsightly: function arguments probably shouldn't show up, anonymous functions should be reported as "" instead of "", every other frame should be omitted (or Athena shouldn't insert those in the first place, ah but it really needs to). It'd also be nice to have the first stanza filled out (the bit before --- ---), as it is with real Python exceptions.




Anyway, the coolest part is how emacs can parse these and automatically open the appropriate JavaScript file for me when an unhandled error occurs in the browser during debugging.

Sunday, March 5, 2006

Saturday, March 4, 2006

cProfile and kcachegrind

Armin Rigo's cProfile is pretty cool, and will be in Python 2.5. KCacheGrind is basically the most awesome profile analysis tool I have ever seen. But how can you use cProfile with KCacheGrind? Itamar and I took a patch against lsprof (cProfile's immediate ancestor) from David Allouche's blog, changed it to be a stand-alone module and updated it to work with cProfile. It's pretty awesome. If you are profiling Python you almost certainly cannot live without it.

Friday, February 17, 2006

Athena Tracebacks


A branch by idnar for Athena landed a few days ago. I just noticed some output on a testing server which shows it off pretty nicely:


2006/02/17 20:07 EST [HTTPChannel,13,66.92.68.248] Traceback (most recent call last):
File "", line 0, in

--- <exception caught here> ---
File "", line 0, in

File "https://<hostname>/private/jsmodule/Divmod.Runtime", line 39, in ()

File "https://<hostname>/private/jsmodule/Divmod", line 135, in ([object Object])

File "", line 0, in apply([object Object],[object Array])

File "https://<hostname>/private/jsmodule/Divmod.Defer", line 137, in callback([object Object],[object Object])

File "https://<hostname>/private/jsmodule/Divmod", line 135, in ([object Object])

File "", line 0, in apply([object Object],[object Array])

File "https://<hostname>/private/jsmodule/Divmod.Defer", line 134, in _startRunCallbacks([object Object],[object Object])

File "https://<hostname>/private/jsmodule/Divmod", line 135, in ()

File "", line 0, in apply([object Object],[object Array])

File "https://<hostname>/private/jsmodule/Divmod.Defer", line 108, in _runCallbacks([object Object])

File "", line 0, in apply(null,[object Array])

File "https://<hostname>/private/jsmodule/Nevow.Athena", line 259, in ([object Object])

File "", line 0, in apply(null,[object Array])

File "https://<hostname>/private/jsmodule/Nevow.Athena", line 191, in ("Nevow.Athena.callByAthenaID","s2c0",[object Array])

File "", line 0, in apply(null,[object Array])

File "https://<hostname>/private/jsmodule/Nevow.Athena", line 608, in (16,"setStatus",[object Array])

File "https://<hostname>/private/jsmodule/Nevow.Athena", line 600, in (16)

File "", line 0, in Error("0 nodes with athena id 16")

nevow.athena.JSException: Error: 0 nodes with athena id 16

Next step, source lines....

Tuesday, February 7, 2006

Athena Update


I wrote the beginning of a tutorial for Athena last weekend. I started on a similar piece for writing tests for Athena apps, but haven't gotten anywhere interesting yet. This latter will cover nit in some detail when it is finished, which might be this weekend (but don't hold your breath).




Also on the Athena front, I added a minor feature yesterday, client-side notifyOnDisconnect, the purpose of which you can likely guess. The first thing Glyph and I did with it was add a default handler for that event which makes it extremely obvious to the user that the connection has gone away.




The logging/debug console is also a bit nicer now that it doesn't spew 100 copies of every message. ;) And if you haven't taken a look at the new admin REPL in Mantissa SVN, you probably should. This will likely work its way backwards into Nevow as a generic terminal-esque or text-editing widget of some sort.

Friday, January 27, 2006

Knife Of Dreams (Take Two)


Okay, I made a snarky post a couple weeks ago about Knife of Dreams. I finished it last night. It was awesome. Seriously an excellent fantasy novel. Some of Jordan's previous works may have been mind numbingly dull, but not this one. I couldn't have asked for anything more (except maybe another thousand pages, oh man how am I going to wait for the next one?). Hard to post anything else, because I'm not one for spoilers, and pretty much everything I have to say at the moment takes the form of "Oh my God, spoiler spoiler huge spoiler!"

Friday, January 20, 2006

Year In Review - Twisted Bugs

Change summaries on a semi-random (but mostly weekly) basis:

Bugs opened: 5 Bugs closed: 2 Total open bugs: 339 (+3)
Bugs opened: 12 Bugs closed: 17 Total open bugs: 334 (-5)
Bugs opened: 9 Bugs closed: 17 Total open bugs: 326 (-8)
Bugs opened: 17 Bugs closed: 10 Total open bugs: 333 (+7)
Bugs opened: 13 Bugs closed: 13 Total open bugs: 333 (+0)
Bugs opened: 17 Bugs closed: 18 Total open bugs: 332 (-1)
Bugs opened: 7 Bugs closed: 2 Total open bugs: 337 (+5)
Bugs opened: 29 Bugs closed: 20 Total open bugs: 346 (+9)
Bugs opened: 11 Bugs closed: 7 Total open bugs: 350 (+4)
Bugs opened: 16 Bugs closed: 2 Total open bugs: 364 (+14)
Bugs opened: 20 Bugs closed: 15 Total open bugs: 369 (+5)
Bugs opened: 20 Bugs closed: 12 Total open bugs: 377 (+8)
Bugs opened: 14 Bugs closed: 3 Total open bugs: 388 (+11)
Bugs opened: 9 Bugs closed: 1 Total open bugs: 396 (+8)
Bugs opened: 5 Bugs closed: 5 Total open bugs: 396 (+0)
Bugs opened: 8 Bugs closed: 6 Total open bugs: 398 (+2)
Bugs opened: 3 Bugs closed: 0 Total open bugs: 401 (+3)
Bugs opened: 15 Bugs closed: 4 Total open bugs: 412 (+11)
Bugs opened: 7 Bugs closed: 2 Total open bugs: 417 (+5)
Bugs opened: 29 Bugs closed: 28 Total open bugs: 418 (+1)
Bugs opened: 6 Bugs closed: 1 Total open bugs: 423 (+5)
Bugs opened: 9 Bugs closed: 9 Total open bugs: 423 (+0)
Bugs opened: 33 Bugs closed: 7 Total open bugs: 449 (+26)
Bugs opened: 16 Bugs closed: 8 Total open bugs: 457 (+8)
Bugs opened: 26 Bugs closed: 12 Total open bugs: 471 (+14)
Bugs opened: 27 Bugs closed: 23 Total open bugs: 475 (+4)
Bugs opened: 6 Bugs closed: 3 Total open bugs: 478 (+3)
Bugs opened: 7 Bugs closed: 3 Total open bugs: 482 (+4)
Bugs opened: 21 Bugs closed: 17 Total open bugs: 486 (+4)
Bugs opened: 15 Bugs closed: 21 Total open bugs: 480 (-6)
Bugs opened: 14 Bugs closed: 11 Total open bugs: 483 (+3)
Bugs opened: 4 Bugs closed: 6 Total open bugs: 481 (-2)
Bugs opened: 6 Bugs closed: 3 Total open bugs: 484 (+3)
Bugs opened: 11 Bugs closed: 2 Total open bugs: 493 (+9)
Bugs opened: 9 Bugs closed: 2 Total open bugs: 500 (+7)
Bugs opened: 10 Bugs closed: 9 Total open bugs: 501 (+1)
Bugs opened: 10 Bugs closed: 5 Total open bugs: 506 (+5)
Bugs opened: 20 Bugs closed: 8 Total open bugs: 518 (+12)
Bugs opened: 13 Bugs closed: 7 Total open bugs: 524 (+6)
Bugs opened: 12 Bugs closed: 3 Total open bugs: 533 (+9)

Congratulations to everyone involved for stemming the tide - 347 bugs resolved last year! Maybe we can break even this year ;)

Tuesday, January 10, 2006

Some days, it's all worth it


Hi Jean-Paul,

Thank you for the advice, I appreciate it, my university training was
next to useless so I learn how to write good programs by the advice of
helpful people in online communities. I'm really thankfull when people
take the time to tell me when I'm doing something wrong and show me a
better way.

And I was still curious, so thank you for explaining how to distinguish
iterables from non-iterables.

Have a great day!

-Sandra

Knife Of Dreams


I started Knife of Dreams, book eleven in the Wheel of Time, a couple nights ago. I've just gotten through the hundred page prologue, but at the rate things are going, this is going to be one of the best Robert Jordan books yet. Spoilers in the next paragraph.




On page 50, Javindhra smooths her skirt (it's deep red). On page 95, Egwene is allowed to wipe her chin with a handkerchief by Silviana (the Mistress of Novices in the Tower).

Wednesday, January 4, 2006

Handy Bookmarklet


I cannot present this material in the manner which I desire. LiveJournal scrubs links, removing important structure. So, instead of one instruction, here are eight:





  1. If you are a Firefox user, right-click on the area below the location bar but above the row of tab labels.


  2. Select "New Bookmark...".


  3. Name the bookmark "Look It Up".


  4. Set the location of the bookmark to be
    javascript:var s = String(window.getSelection()); if (s.length) { window.open('http://www.dict.org/bin/Dict?Form=Dict2&Database=*&Query=' + encodeURI(s)); } else { alert('Select a word to look up.'); } void(0);



  5. Visit some web page which contains a word the definition of which you would like to know.


  6. Select the word in the document (for example, double click on it).


  7. Click the "Look It Up" bookmarklet you just added.


  8. Savor the novelty (it wears off fast, so savor it likewise).





Maybe this is convenient for something. However, it does demonstrate document range selections, which have some other interesting uses. For example, I would like someone to write a Mantissa application which tracks quotes this way. People should be able to vote for quotes they find on the web. The quote can easily be selected and submitted to the server for tracking. The server can keep running tallies for various time periods and report them. It can also do things like indicate the first reporter of a quote, long-persisting quotes, etc. Communities like comp.lang.python should find this useful, as people are always responding to posts with messages that consist of nothing more than "+1 QOTW".



This would be an extremely trivial Mantissa application. Who would like to write it?



Props to radix for reminding me that dict.org is way better than dictionary.com.