Friday, December 21, 2007

Filesystem structure of a Python project

Do:
  • name the directory something related to your project. For example, if your project is named "Twisted", name the top-level directory for its source files Twisted. When you do releases, you should include a version number suffix: Twisted-2.5.
  • create a directory Twisted/bin and put your executables there, if you have any. Don't give them a .py extension, even if they are Python source files. Don't put any code in them except an import of and call to a main function defined somewhere else in your projects. (Slight wrinkle: since on Windows, the interpreter is selected by the file extension, your Windows users actually do want the .py extension. So, when you package for Windows, you may want to add it. Unfortunately there's no easy distutils trick that I know of to automate this process. Considering that on POSIX the .py extension is a only a wart, whereas on Windows the lack is an actual bug, if your userbase includes Windows users, you may want to opt to just have the .py extension everywhere.)
  • If your project is expressable as a single Python source file, then put it into the directory and name it something related to your project. For example, Twisted/twisted.py. If you need multiple source files, create a package instead (Twisted/twisted/, with an empty Twisted/twisted/__init__.py) and place your source files in it. For example, Twisted/twisted/internet.py.
  • put your unit tests in a sub-package of your package (note - this means that the single Python source file option above was a trick - you always need at least one other file for your unit tests). For example, Twisted/twisted/test/. Of course, make it a package with Twisted/twisted/test/__init__.py. Place tests in files like Twisted/twisted/test/test_internet.py.
  • add Twisted/README and Twisted/setup.py to explain and install your software, respectively, if you're feeling nice.
Don't:
  • put your source in a directory called src or lib. This makes it hard to run without installing.
  • put your tests outside of your Python package. This makes it hard to run the tests against an installed version.
  • create a package that only has a __init__.py and then put all your code into __init__.py. Just make a module instead of a package, it's simpler.
  • try to come up with magical hacks to make Python able to import your module or package without having the user add the directory containing it to their import path (either via PYTHONPATH or some other mechanism). You will not correctly handle all cases and users will get angry at you when your software doesn't work in their environment.

Monday, December 3, 2007

Incompatabilities between classic and new-style Python classes

radix asked me if I had a blog post about why changing a class from classic to new-style is a bad idea. After I told him that I didn't he insisted that I should write one. Since doing so will take less work than finding something else to distract his attention, here it is:






  • attribute lookup


    Attributes on instances of classic classes override attributes of the same name on their class. For example:


    >>> class SimpleDescriptor(object):
    ... def __get__(self, instance, type):
    ... print 'getting'
    ... return instance.__dict__['simple']
    ... def __set__(self, instance, value):
    ... print 'setting'
    ... instance.__dict__['simple'] = value
    ...
    >>> class classic:
    ... simple = SimpleDescriptor()
    ...
    >>> x = classic()
    >>> x.simple = 10
    >>> x.simple
    10
    >>> class newstyle(object):
    ... simple = SimpleDescriptor()
    ...
    >>> x = newstyle()
    >>> x.simple
    getting
    Traceback (most recent call last):
    File "", line 1, in ?
    File "", line 4, in __get__
    KeyError: 'simple'
    >>> x.simple = 10
    setting
    >>> x.simple
    getting
    10

    As you can see, the descriptor on the new-style class can both handle the setattr and continue to operate after shadowing itself with an instance variable.




  • Special methods


    A particularly interesting consequence of the previous point is that the behavior of special methods changes in some cases:


    >>> class x:
    ... def __init__(self):
    ... self.__eq__ = lambda other: True
    ...
    >>> x() == 10
    True
    >>> class x(object):
    ... def __init__(self):
    ... self.__eq__ = lambda other: True
    ...
    >>> x() == 10
    False
    >>>




  • MRO


    Rules for determining the method resolution forbid new-style classes in places where classic classes are acceptable. Consider:


    >>> class x: pass
    ...
    >>> class y(object, x): pass
    ...
    >>> class x(object): pass
    ...
    >>> class y(object, x): pass
    ...
    Traceback (most recent call last):
    File "", line 1, in ?
    TypeError: Error when calling the metaclass bases
    Cannot create a consistent method resolution
    order (MRO) for bases object, x
    >>>







Hopefully that's enough to satisfy radix. Know of other incompatibilities? Please comment.

Tuesday, October 2, 2007

Hair today...

Some pictures of Quechee Gorge up. Including a 360° panorama I assembled. Which took forever with stupid gimp. And it didn't come out very well anyway. Stupid parallax, I'll get you next time.

Off to San Antonio tomorrow, back next week. If there's anything worth taking pictures of (other than my brother) maybe those'll show up in a week or so.

Sunday, September 30, 2007

September Reading List

By way of The Book of Jhereg:

  • Jhereg Steven Brust.

  • Yendi Steven Brust.

  • Teckla Steven Brust.



Also some Milton, for class: On the Morning of Christ's Nativity, At a Solemn Music, L'Allegro, Il Penseroso, Lycidas, On the New Forcers of Conscience, To Mr. Cyriack Skinner Upon His Blindness, and sonnets 11, 12, 15, 16, 19.

Tuesday, September 11, 2007

chevre, pine nut, raisin, onion, garlic, bacon, d...

chevre, pine nut, raisin, onion, garlic, bacon, duck pizza

Saturday, September 1, 2007

August Reading List

Not much going on in August:


  • Player Piano. Kurt Vonnegut, Jr..

  • The Execution Channel. Ken MacLeod.

  • Rainbows End. Vernor Vinge.

Tuesday, August 21, 2007

Twisted Trial performance improvements

Trial, Twisted's xUnit(-esque) testing package (library, discovery tool, runner, etc), has long inherited its overall testing flow from the Python standard library unittest module. It's quite simple, actually: iterate over a sequence of test cases and invoke each one with the result object. Lately, this has actually led to some noticeable problems with its performance. Creating a test case instance for each test method isn't extremely unreasonable, but in the process of running them all, more and more objects are created and the process balloons to an unreasonable size. The Twisted test suite typically uses about 800MB of RAM by the time the last test method is run.



So, in order to be able to run the Twisted tests on machines without 800MB of free memory, we changed our TestSuite so that it drops each test after it runs it. The suite now takes about one quarter as much memory to run. As an unexpected bonus, it also runs almost 50% faster (66% for trial --force-gc which calls gc.collect in between each test method). I can only explain the speedup as time saved inside the garbage collector itself due there being far fewer objects to examine (this is not a completely satisfying explanation, but I cannot think of a better one).



If you're using trial to run your test suite, you may notice reduced memory requirements and reduced overall runtime, too. :)

Friday, August 17, 2007

Typespeed v0.4.4 Best score was: Rank: ...


Typespeed v0.4.4

Best score was:

Rank: Computer
Score: 1187
Total CPS: 8.492
Correct CPS: 8.352
Typo ratio: 1.6%
Typorank: Secretary


exarkun@charm:~$

Thursday, August 16, 2007

Child process improvements in Twisted

Thomas Hervé merged his process/pty-process branch into Twisted trunk today. He's been working on integrating the two separate implementations of POSIX child process creation in Twisted, one for child processes connected to a PTY and one for child processes connected to a set of pipes. A long time ago, someone copied an entire class full of code and started twiddling it to support PTYs. This led to a lot of duplication, obviously, and some bugs where the two classes diverged.



With this re-unification, some behavior which was inconsistent between the two variations has been removed, some bugs which were present in one or the other of the two classes have been fixed, and a lot of duplicate code has been completely eliminated. Best of all, there is now some actual unit test coverage for this code (it was originally developed long before Twisted's current high testing requirements) and a test double which will make it easier to write more and better unit tests for this code in the future.



One other piece of fall out is that on Windows, if application code tries to signal an already-exited process, an exception will be raised indicating that it is not possible to do so. This brings Windows process support closer still to POSIX process support.

Thursday, August 9, 2007

It's Official

http://www.guardian.co.uk/environment/2007/aug/08/endangeredspecies.conservation

Unfortunately, this is hardly surprising. It's been pretty clear for at least a decade that this would be the outcome. Maybe the saddest part is that it shouldn't have been inevitable.

Saturday, August 4, 2007

Side-by-side Twisted plugins installation issue fixed


Since its introduction a little over two years ago, the "new" Twisted plugin system has had a minor bug which manifest whenever two or more separate copies of a plugin packaged were available (ie, in sys.path). Normally, the first of these should obscure the second. However, the plugin system would allow both to be discovered. This could lead to confusing results in some cases.




This bug is now fixed in trunk@HEAD: only the first plugin package on sys.path will be considered for plugins. Plugin directories (not packages) will still be considered.

Thursday, August 2, 2007

The Internet Is Filled With Wonderful Things

You were probably just wondering about these effects, anyway. Of course, I don't see the text online anywhere, so you'll have to keep wondering.

Wednesday, August 1, 2007

Twisted system event triggers fix

One of the older, darker corners of Twisted, system event triggers (part of the core reactor interface) have long had slightly poor handling of some corner cases. Fortunately most use of these triggers is via the application service system which avoids all of these cases, but there is another startup notification API through which one might have encountered some weird behavior. As of now, a bug which could cause a trigger to not be executed has been fixed. Code which triggered this behavior was probably buggy itself, since it was asking the reactor to do something impossible; it will receive a warning from now until we turn it into a real exception, a couple releases from now.

Monday, July 30, 2007

random update plus reading list for june/july

I was on vacation last week. I took some pictures. Go look at them.



Also, not related, I beat Guitar Hero II on hard tonight.



Books:





  • Name of the Rose. Umberto Eco.


  • Glasshouse. Charles Stross.


  • Children of the Lens. Edward E. Smith, Ph.D.


  • Linguistics and Politics. Noam Chomsky & Mitsou Ronat.


  • The New Space Opera. Edited by Gardner Dozois & Jonathan Strahan. Includes stories by Gwyneth Jones, Ian McDonald, Robert Reed, Paul J. McAuley, Greg Egan, Kage Baker, Peter F. Hamilton, Ken Macleod, Tony Daniel, James Patrick Kelly, Alastair Reynolds, Mary Rosenblum, Stephen Baxter, Robert Silverberg, Gregory Benford, Walter Jon Williams, Nancy Kress, and Dan Simmons (curse you, Dan Simmons).




I also started to read Man-Kzin Wars III which I bought accidentally (in various sense of the word), but realized halfway through the first story that I'd read it already.

Sunday, July 29, 2007

setUpClass and tearDownClass are probably no...

setUpClass
and
tearDownClass
are
probably
not
a
good
idea
and
might
be
deprecated
soon.

Of course, if they are, there will be a suitable period of support for them before they are removed, as dictated by Twisted's strict backwards compatibility policy.

Friday, July 13, 2007

How to override comparison operators in Python

Python, like many languages, allows the behavior of operators to be
customized using a scheme based on the types of objects they are applied to.
The precise rules and intricacies of this customization are fairly involved,
though, and most people are unaware of their full scope. While it is sometimes
valuable to be able to control the behavior of an operator to the full extent
supported by Python, quite often the complexity which this results in spills
over into simpler applications. This is visible as a general tendency on the
part of Python programmers to implement customizations which are correct for
the narrow case which they have in mind at the moment, but are incorrect when
considered in a broader context. Since many parts of the runtime and standard
library rely on the behavior of these operators, this is a somewhat more
egregious than the case of a similar offense made in an application-specific
method, where the author can simply claim that behavior beyond what was
intended is unsupported and behaves in an undefined manner.



So, with my long-winded introduction out of the way, here are the basic
rules for the customization of ==, !=, <, >, <=, and >=:





  • For all six of the above operators, if __cmp__ is defined on
    the left-hand argument, it is called with the right-hand argument. A result
    of -1 indicates the LHS is less than the RHS. A result of 0 indicates they
    are equal. A result of 1 indicates the LHS is greater than the RHS.


  • For ==, if __eq__ is defined on the left-hand argument, it
    is called with the right hand argument. A result of True indicates the
    objects are equal. A result of False indicates they are not equal. A result
    of NotImplemented indicates that the left-hand argument doesn't
    know how to test for equality with the given right-hand argument.
    __eq__ is not used for !=.


  • For !=, the special method __ne__ is used. The rules for
    its behavior are similar to those of __eq__, with the obvious
    adjustments.


  • For <, __lt__ is used. For >, __gt__.
    For <= and >=, __le__ and __ge__
    respectively.




So how should these be applied? This is best explained with an example.
While __cmp__ is often useful, I am going to ignore it for the
rest of this post, since it is easier to get right, particularly once
NotImplemented (which I will talk about) is understood.




class A(object):
def __init__(self, foo):
self.foo = foo
def __eq__(self, other):
if isinstance(other, A):
return self.foo == other.foo
return NotImplemented
def __ne__(self, other):
result = self.__eq__(other)
if result is NotImplemented:
return result
return not result


That's it (because I'm not going to define the other four methods to make
<, >, <=, and >= work. They follow basically the same rules as
__eq__ and __ne__, though). Pretty straightforward,
but there are some points which are not always obvious:





  • __eq__ does an isinstance test on its argument. This lets
    it know if it is dealing with another object which is like itself. In the
    case of this example, I have implemented A to only know how to compare itself
    with other instances of A. If it is called with something which is not an A,
    it returns NotImplemented. I'll explain what the consequences
    of this are below.


  • __ne__ is also implemented, but only in terms of
    __eq__. If you implement __eq__ but not
    __ne__, then == and != will behave somewhat strangely, since the
    default implementation of __ne__ is based on identity, not the
    negation of equality. Quite often a class with only __eq__ will
    appear to work properly with !=, but it fails for various corner-cases (for
    example, an object which does not compare equal to itself, such as NaN).




The major remaining point is NotImplemented: what is that
thing? NotImplemented signals to the runtime that it should ask
someone else to satisfy the operation. In the expression a == b,
if a.__eq__(b) returns NotImplemented, then Python
tries b.__eq__(a). If b knows enough to return True or False,
then the expression can succeed. If it doesn't, then the runtime will fall
back to the built-in behavior (which is based on identity for == and !=).



Here's another class which customizes equality:




class B(object):
def __init__(self, bar):
self.bar = bar
def __eq__(self, other):
if isinstance(other, B):
return self.bar == other.bar
elif isinstance(other, A):
return self.bar + 3 == other.foo
else:
return NotImplemented
def __ne__(self, other):
result = self.__eq__(other)
if result is NotImplemented:
return result
return not result


Here we have a class which can compare instances of itself to both instances
itself and to instances of A. Now, what would happen if we weren't careful
about returning NotImplemented at the right times?



One way it might go is...




>>> class A(object):
... def __init__(self, foo):
... self.foo = foo
... def __eq__(self, other):
... return self.foo == other.foo
...
>>> class B(object):
... def __init__(self, bar):
... self.bar = bar
...
>>> A(5) == B(6)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 5, in __eq__
AttributeError: 'B' object has no attribute 'foo'
>>>


Another way it could go is...




>>> class A(object):
... def __init__(self, foo):
... self.foo = foo
... def __eq__(self, other):
... if isinstance(other, A):
... return self.foo == other.foo
...
>>> class B(object):
... def __init__(self, bar):
... self.bar = bar
... def __eq__(self, other):
... if isinstance(other, A):
... return self.bar + 3 == other.foo
... else:
... return self.bar == other.bar
...
>>> print A(3) == B(0)
None
>>> print B(0) == A(3)
True
>>>


That one's particularly nasty. ;) But here's what we get with correct
NotImplemented use:




>>> class A(object):
... def __init__(self, foo):
... self.foo = foo
... def __eq__(self, other):
... if isinstance(other, A):
... return self.foo == other.foo
... return NotImplemented
...
>>> class B(object):
... def __init__(self, bar):
... self.bar = bar
... def __eq__(self, other):
... if isinstance(other, A):
... return self.bar + 3 == other.foo
... elif isinstance(other, B):
... return self.bar == other.bar
... else:
... return NotImplemented
...
>>> print A(3) == B(0)
True
>>> print B(0) == A(3)
True
>>>


Ahh, excellent. NotImplemented has uses for other operators in
Python as well. For example, if the + override, __add__, returns
it, then __radd__ is tried on the right-hand argument. These can
be useful as well, though equality and inequality are by far more common use
cases.



If you follow these examples, then in the general case you'll find yourself
with more consistently behaving objects. You may even want to implement a
mixin which provides the __ne__ implementation (and one of
__lt__ or __gt__), since it gets pretty boring typing
that out after a few times. ;)



Of course, there are plenty of special cases where it makes sense to deviate
from this pattern. However, they are special. For most objects, this
is the behavior you want.



You can read about all the gory details of Python's operator overloading
system on the Python website:
http://docs.python.org/ref/specialnames.html

Sunday, July 1, 2007

Charles River Bikeway

yesterday's bike ride with itamar

Minuteman

today's bike ride

and google maps is neat, but the ui is terrible and it constantly throws away data :(

Wednesday, June 27, 2007

Longhammer IPA and orange juice popsicle

Effective weapons in the local fight against global warming (it better not be 37°C again tomorrow though).

Friday, June 8, 2007

Warnings unit test helper

Trial's TestCase offers a new helper method in Twisted trunk@HEAD, assertWarns. This makes it trivial to test that a particular function call emits a warning which satisfies some condition. For example, this is useful for verifying that an API which is supposed to be deprecated is actually deprecated:


from twisted.trial.unittest import TestCase

def someFunction(input):
"""
Map inputs to outputs.
"""


class FunctionTests(TestCase):
"""
Tests for L{someFunction}.
"""

def test_deprecation(self):
"""
Calling L{someFunction} should emit a L{DeprecationWarning}.
"""
self.assertWarns(
DeprecationWarning,
("someFunction is deprecated, use anotherFunction instead",),
__file__,
someFunction, 3)


Before adding the warning, this fails:


$ trial assertwarns.py
Running 1 tests.
assertwarns
FunctionTests
test_deprecation ... [FAIL]

===============================================================================
[FAIL]: assertwarns.FunctionTests.test_deprecation

Traceback (most recent call last):
File "/home/exarkun/assertwarns.py", line 22, in test_deprecation
someFunction, 3)
File "/home/exarkun/Projects/Twisted/trunk/twisted/trial/unittest.py", line 346, in failUnlessWarns
self.assertEqual(len(warningsShown), 1, pformat(warningsShown))
twisted.trial.unittest.FailTest: []
-------------------------------------------------------------------------------
Ran 1 tests in 0.009s

FAILED (failures=1)


After changing the definition of someFunction to this:


import warnings

def someFunction(input):
"""
Map inputs to outputs.
"""
warnings.warn(
"someFunction is deprecated, use anotherFunction instead",
category=DeprecationWarning)


the tests pass:


$ trial assertwarns.py
Running 1 tests.
assertwarns
FunctionTests
test_deprecation ... [OK]

-------------------------------------------------------------------------------
Ran 1 tests in 0.003s

PASSED (successes=1)

Friday, June 1, 2007

May Reading List

Measuring the World. Daniel Kehlmann. Translated by Carol Brown Janeway.
The Terror. Dan Simmons.

After Hyperion and the sequels I wasn't very interested in reading any more Simmons; reading a couple of his short stories didn't do a lot to change this (rather, it solidified the opinion). It's hardly a spoiler to share the fact that the Shrike plays a major role in The Terror but it otherwise bears little resemblance to Hyperion et al. radix is harassing me to go to dinner now or I'd say more.

Tuesday, May 1, 2007

March - April reading

Second Stage Lensmen. E. E. "Doc" Smith.
Necromancer. Gordon R. Dickson.
Tactics of Mistake. Gordon R. Dickson.
Dorsai!. Gordon R. Dickson.
The Rediscovery of Man. Cordwainer Smith.
The Jennifer Morgue. Charles Stross.
babel-17. Samuel R. Delany.
The Epic of Gilgamesh.
Lolita. Vladamir Nabokov.

Monday, April 30, 2007

Macintosh why do I loathe thee

Okay, I suppose Apple deserves credit for making their filesystem UTF-8. They even deserve credit for storing strings in normalized form. Why do they store NFD though? And how do I make this play nicely with any Linux system? Or am I supposed to want NFD on Linux too? Why wouldn't I want NFC? And how do I tell CIFS not to screw everything up when it runs into this?

Saturday, April 21, 2007

SAX excitement

Consider the unsurprising:


$ python
Python 2.4.3 (#2, Oct 6 2006, 07:52:30)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.sax
>>> xml.sax.parse('pypi.rss', xml.sax.ContentHandler())
>>>


Now predict the behavior of the preceeding code snippet after this shell interaction:


# ifdown eth1


If you guessed this:


$ python
Python 2.4.3 (#2, Oct 6 2006, 07:52:30)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.sax
>>> xml.sax.parse('pypi.rss', xml.sax.ContentHandler())
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/__init__.py", line 31, in parse
parser.parse(filename_or_stream)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 109, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, in parse
self.feed(buffer)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 216, in feed
self._parser.Parse(data, isFinal)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 395, in external_entity_ref
self._source.getSystemId() or
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/saxutils.py", line 523, in prepare_input_source
f = urllib2.urlopen(source.getSystemId())
File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen
return _opener.open(url, data)
File "/usr/lib/python2.4/urllib2.py", line 358, in open
response = self._open(req, data)
File "/usr/lib/python2.4/urllib2.py", line 376, in _open
'_open', req)
File "/usr/lib/python2.4/urllib2.py", line 337, in _call_chain
result = func(*args)
File "/usr/lib/python2.4/urllib2.py", line 1021, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.4/urllib2.py", line 996, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error (-2, 'Name or service not known')>


Give yourself a pat on the back.

Saturday, March 24, 2007

<u>Proper Care and Feeding of Your Neural Network</u>

I am seeking this instructional booklet. If anyone has any information on where it might be obtained, please let me know.

Setting "process name" with twistd

Little known twistd feature:


exarkun@boson:~$ cat foo.tac
from twisted.application.service import Application, IProcess

application = Application("Demonstrate twistd process name support")
IProcess(application).processName = "name-demo"
exarkun@boson:~$ twistd -y foo.tac
exarkun@boson:~$ pidof name-demo
11097
exarkun@boson:~$

Monday, March 12, 2007

Fallen

I played my first MMO tonight.

I am Kuran Kal the Minmatar. Say hello if you see me.

Teas

Peach Rooibos is terrible. Blech!@ Though given that all other rooibos are also awful, I shouldn't be surprised.

Lapsang is good. So is jasmine. Even white is pretty nice. I should stick to those. No more rooibos ever.

Adventures in routing


exarkun@charm:~$ ping 10.0.2.200
PING 10.0.2.200 (10.0.2.200) 56(84) bytes of data.
From 10.0.2.129: icmp_seq=1270 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129: icmp_seq=1271 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129: icmp_seq=1272 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129: icmp_seq=1273 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129: icmp_seq=1274 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129: icmp_seq=1275 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129 icmp_seq=1273 Destination Host Unreachable
From 10.0.2.129 icmp_seq=1274 Destination Host Unreachable
From 10.0.2.129 icmp_seq=1275 Destination Host Unreachable
From 10.0.2.129: icmp_seq=1276 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129: icmp_seq=1277 Redirect Host(New nexthop: 10.0.2.200)
From 10.0.2.129 icmp_seq=1276 Destination Host Unreachable
From 10.0.2.129 icmp_seq=1277 Destination Host Unreachable
From 10.0.2.1 icmp_seq=1278 Destination Host Unreachable
From 10.0.2.1 icmp_seq=1279 Destination Host Unreachable
From 10.0.2.1 icmp_seq=1280 Destination Host Unreachable

--- 10.0.2.200 ping statistics ---
1282 packets transmitted, 0 received, +8 errors, 100% packet loss, time 1281599ms, pipe 5
exarkun@charm:~$

Thursday, March 1, 2007

January/February Reading List

The Terminal Experiments. Robert J. Sawyer.
Darwin's Radio. Greg Bear.
Men Without Women. Ernest Hemingway.
The Dreamthief's Daughter. Michael Moorcock.
Go Tell It on the Mountain. James Baldwin.

Monday, February 26, 2007

JavaScript unit tests


Athena has had Nit for some time now. Nit comes in handy when code which interacts in complex ways with the runtime needs to be tested (that is, when you really want Firefox running your code, to make sure your code works with Firefox, because Firefox does something ridiculously insane that you need to account for). This isn't really the common case for unit testing, though. In the common case, you want nice, cleanly factored units, each with a well defined test suite with nice, simple tests which don't require a web browser to run.




A little while ago jml threw together an xUnit implementation for JavaScript to include in Nevow with Athena. Here's a sample of the result:


// Copyright (c) 2006 Divmod.
// See LICENSE for details.

/**
* Tests for L{Mantissa.ScrollTable.PlaceholderModel}
*/

// import Divmod.UnitTest
// import Mantissa.ScrollTable

Mantissa.Test.TestPlaceholder.PlaceholderTests = Divmod.UnitTest.TestCase.subclass(
'Mantissa.Test.TestPlaceholder.PlaceholderTests');
Mantissa.Test.TestPlaceholder.PlaceholderTests.methods(
/**
* Set up a placeholder model with an initial placeholder which extends as far
* as C{totalRows}.
*
* @type totalRows: integer
* @rtype: L{Mantissa.ScrollTable.PlaceholderModel}
*/
function setUp(self) {
var totalRows = 5;
self.model = Mantissa.ScrollTable.PlaceholderModel();
self.model.registerInitialPlaceholder(totalRows, null);
},


/**
* Set up a placeholder model with as many placeholders are there are entries
* in C{ranges}, each start and stop indices equal to the first and second
* entries in the corresponding pair.
*
* @param ranges: pairs of [start index, stop index]
* @type ranges: sorted C{Array} of C{Array}
*
* @rtype: L{Mantissa.ScrollTable.PlaceholderModel}
*/
function createPlaceholderLayout(self, ranges) {
var model = Mantissa.ScrollTable.PlaceholderModel();
for (var i = 0; i < ranges.length; i++) {
model._placeholderRanges.push(
model.createPlaceholder(ranges[i][0], ranges[i][1], null));
}
return model;
},

/**
* Test L{Mantissa.ScrollTable.PlaceholderModel.createPlaceholder}
*/
function test_createPlaceholder(self) {
var p = self.model.createPlaceholder(0, 1, null);

self.assertIdentical(p.start, 0, "expected a start of 0");
self.assertIdentical(p.stop, 1, "expected a stop of 1");
self.assertIdentical(p.node, null, "expected a null node");

p = self.model.createPlaceholder(5, 11, 6);

self.assertIdentical(p.start, 5, "expected a start of 5");
self.assertIdentical(p.stop, 11, "expected a stop of 11");
self.assertIdentical(p.node, 6, "expected a node of 6");
},

// ...
);


These can be run with trial, although the integration still has a couple obvious defects:


exarkun@boson:~$ trial xmantissa.test.test_javascript
Running 1 tests.
subunit
RemotedTestCase
test_createPlaceholder ... [OK]
test_dividePlaceholder ... [OK]
test_findFirstPlaceholderIndexAfterRowIndexMultiplePlaceholders ... [OK]
test_findFirstPlaceholderIndexAfterRowIndexMultiplePlaceholders2 ... [OK]
test_findFirstPlaceholderIndexAfterRowIndexMultiplePlaceholders3 ... [OK]
test_findFirstPlaceholderIndexAfterRowIndexMultiplePlaceholders4 ... [OK]
test_findFirstPlaceholderIndexAfterRowIndexMultiplePlaceholders5 ... [OK]
test_findFirstPlaceholderIndexAfterRowIndexOnePlaceholderNeg ... [OK]
test_findFirstPlaceholderIndexAfterRowIndexOnePlaceholderNeg2 ... [OK]
test_findPlaceholderIndexForRowIndexMultiplePlaceholders ... [OK]
test_findPlaceholderIndexForRowIndexMultiplePlaceholdersNeg ... [OK]
test_findPlaceholderIndexForRowIndexOnePlaceholder ... [OK]
test_registerInitialPlaceholder ... [OK]
test_removedRow ... [OK]
test_removedRowNeg ... [OK]
test_replacePlaceholder ... [OK]

-------------------------------------------------------------------------------
Ran 16 tests in 2.435s

PASSED (successes=16)
exarkun@boson:~$



This kind of testing means faster development cycles, better code factoring, and a more reliable final product.

Friday, January 12, 2007

Python thread debugging

Python's Misc/gdbinit provides a helper to print the Python call stack. Because of some assumptions it makes, it won't work for any thread except the main thread. This patch fixes that:


Index: gdbinit
===================================================================
--- gdbinit (revision 53403)
+++ gdbinit (working copy)
@@ -119,7 +119,7 @@

# print the entire Python call stack
define pystack
- while $pc < Py_Main || $pc > Py_GetArgcArgv
+ while ($pc < Py_Main || $pc > Py_GetArgcArgv) && ($pc < t_bootstrap || $pc > thread_PyThread_start_new_thread)
if $pc > PyEval_EvalFrame && $pc < PyEval_EvalCodeEx
pyframe
end


A similar change is required for pystackv.