May - December Reading List

• The History of the Peloponnesian War. Thucydides. (Books 2 - 8)
• Root Cellaring: Natural Cold Storage of Fruits & Vegetables. Mike and Nancy Bubel.
• The Fatal Shore: The Epic of Australia's Founding. Robert Hughes.
• Special Topics in Calamity Physics. Marisha Pessl.
• The Dirty Life: A Memoir of Farming, Food, and Love. Kristin Kimball.
• The Worst Hard Time: The Untold Story of Those Who Survived the Great American Dust Bowl. Timothy Egan.
• The Children of the Sky (Zones of Thought). Vernor Vinge.
• The Gathering Storm (Wheel of Time). Robert Jordan and Brandon Sanderson.
• Oresteia: Agamemnon, The Libation Bearers, and The Eumenides. Eschylus.
• Towers of Midnight (The Wheel of Time). Robert Jordan and Brandon Sanderson.
• The Clouds. Aristophanes.
• Saturn's Children. Charles Stross.
• The Fuller Memorandum (A Laundry Files Novel). Charles Stross.
• Scratch Monkey. Charles Stross.

Don't Use Buildbot EC2 Features

I just noticed that Buildbot spun up one EC2 instance 31 days ago and another one 14 days ago and left them both running.

Garlic Followup

Previously I shared some pictures of a garden bed Jericho and I have been working on. Over the long weekend we had some more time to spend on it. While we were away, we got some help preparing the rest of the garden, which would have otherwise taken weeks or months to do by hand:

You can see the oats we put in at the beginning of September in the garlic ged on the left side there. They didn't grow as much as I had hoped:

Perhaps due to some nutrient or mineral deficiency. To rectify that (and based on a soil test), we spread a number of amendments, starting with greensand to provide potassium:

We also spread rock phosphate (for phosphorus) and pelletized lime for calcium and to adjust the pH to be less acidic. And, importantly, compost - about 1 cubic yard over the entire bed (with which task my dad helped us out):

As you can see, we just left the oats in place. They are not cold hardy and will die soon enough without any help. With the bed thusly prepped, we began breaking up our "seed" garlic:

Garlic is most often grown by sowing cloves in the autumn for harvest the following summer. The winter encourages the clove to split and grow into a new bulb. We planted four varieties of garlic, but mostly Inchelium, a softneck variety.

These seed bulbs each had around a dozen cloves in them.

We planted the largest undamaged cloves. We also planted three varieties of hardneck garlic. Compared to the inchelium, these all look pretty similar to each other. Here's some Siberian Red:

The hardneck varieties have bulbs with fewer, larger cloves. After we broke up the cloves, we planted them! While one of us dropped cloves in pre-marked locations, the other followed behind and planted them.

The cloves are planted right-side-up about one inch deep. Finally we mulched them with straw to even out temperature variations and retain moisture.

Now the garlic sits tight until next year.

Garden Bed Prep

Over the long weekend, Jericho and I made a garden bed. We picked a plot a few minutes walk from the new orchard and started by mowing a 100' x 4' area.

That's Lucy, my mom's lab, down near the end. Next we cut sod with shovels.

I dug too, but Jericho doesn't take as many pictures as I do.

After that, we flipped sod. First one row of it:

And then the next row:

After all the sod was out, we dug a little more.

Then we put the sod at the bottom of the hole, upside-down, where it will hopefully die and contribute organic material to the soil.

And then we shoveled that dirt off the tarp, back into the hole on top of the sod, and raked it flat.

And again.

Until all the dirt was back in the bed.

This will be a garlic bed. We'll plant the garlic in October. Until then, we put in a quick cover crop of oats.

Then I rubbed some dirt on my shirt to make it look like I helped too.

A few hours after we finished seeding, a nice thunderstorm rolled in and watered everything for us.

If all goes well, in about a month we'll have some nice young oats to mow down before planting garlic in the bed.

Releasing Python Software is Tedious

I released pyOpenSSL 0.13 a few days ago. Apart from making sure it actually worked on various platforms, updating the version number, regenerating the documentation, and sending out the release announcement, I also had to upload release files to the Python Package Index.

Uploading release files to PyPI is the part of the release process I hate the most. pyOpenSSL 0.13 had 15 files to upload to PyPI. There is no usable automated interface for uploading files to PyPI. Before I can even begin to upload them, I have to download them from the build farm where they're generated. Then, uploading just one file to PyPI requires at least 8 mouse clicks. The clicks vary depending on which file is being uploaded. I have to select a file, select its type, specify which Python version it's for, and then submit a form.

15 files, 8 clicks per file. Well, you do the math. It's not a pleasant experience. PyPI would be a much better resource if it didn't force me to specify a ton of redundant, mostly useless information every time I do a release. It would be a much better resource if it had a programmatic interface so I don't have to spend 20 minutes clicking buttons in web browser. It would be a much better resource if it didn't try to hard to discourage me from releasing software.

Buildin' a fence

Earlier this year Jericho, myself, and a bunch of friends and family planted a little orchard at the farm in Maine. This week I built a fence around it.

Happily mowing was mostly taken care of before I arrived:




So it was easy to work around the trees and along the eventual fence-line.

I started at the corners of the area and drove 8 foot steel t-posts:



Using a post driver:



This was actually a lot easier than I expected, perhaps because of the heavy rains on the preceding evenings. I strung 12 gauge monofilament wire between these to define a straight line:



I used the line to place intermediate t-posts:



After 15 t-posts were up and connected with monofilament, I hung deer netting from it:



This part seemed harder than driving the t-posts, maybe because I was already somewhat tired from that earlier activity, and maybe because this activity mostly involved holding my arms above my head for long periods of time doing more close-detail work.

I started by zip-tying the net to the monofilament to temporarily fight sag between posts:



Then I used inline tensioners to take up the slack:



The bottom of the net is held down with sod staples:




The netting quickly vanishes as you move away from it, so I also flagged each stretch:



Some loose ends still need to be wrapped up. I found some ferrules at the hardware store to secure the monofilament, but I haven't tried crimping them yet. The zip-ties need to be replaced with a more robust solution. And the deer net needs to be secured to each t-post to keep it from billowing so much.

Redhumped Caterpillar

At my mom's for the week (at least), house-sitting and putting up fence. I had some warning about these:



However seeing the almost stripped tree they were feeding on in person was still a shock:



I had tentatively identified them as codling moth larvae, which didn't quite make sense since these typically grow and feed inside fruit, not on leaves. My revised identification is redhumped caterpillar. I just sent an email to Maine Cooperative Extension to see what they think.

The bright side is that only one tree appears affected, and there seems to be a fair chance that it will recover.

Twisted Conch in 60 Seconds: Protocols

Welcome once more to Twisted Conch in 60 Seconds, the tutorial series about writing SSH applications with Twisted.

Over the past several articles, I've introduced the APIs for letting clients establish a new logical connection to your SSH server, generating output on those connections, accepting input on those connections, and detecting the end of those connections. Taken together, these four activities map almost exactly onto the standard Twisted protocol abstraction (represented and documented by the IProtocol interface). In this article, I'll show you how to use any IProtocol implementation from Twisted to interact with an SSH channel.

The previous example implemented an echo-ish type of server by customizing the session. I'll duplicate that functionality here, but in an IProtocol implementation:

class EchoProtocol(Protocol):
    def connectionMade(self):
        self.transport.write("Echo protocol connected\r\n")

    def dataReceived(self, bytes):
        self.transport.write("echo: " + repr(bytes) + "\r\n")

    def connectionLost(self, reason):
        print 'Connection lost', reason

Each of the three events a protocol might receive is handled here - connection made and data received are handled by writing something to the connection, and connection lost is handled by writing something to stdout (writing to the connection is no longer an option after it has been lost, of course). This is a simple, fairly typical Twisted-based protocol implementation; you'll find protocols like this all over the place.
The major part of the requirements for using this is having a transport to which to connect it. The SSHSession class you're now familiar with can serve in just that role. Only a little code to put it together with a protocol is required. Again I'll create a "session" channel which accepts but ignores pty requests:

class SimpleSession(SSHSession):
    name = 'session'

    def request_pty_req(self, data):
        return True

And again I'll override request_shell so that the protocol is connected to the channel/transport as soon as the client requests a shell.

    def request_shell(self, data):
        protocol = EchoProtocol()

All I've done so far here is make an instance of the protocol defined above. Next I'll create the transport object and hook it up to this protocol. This part uses a couple helpers from twisted.conch.ssh.session, SSHSessionProcessProtocol and wrapProtocol:

        transport = SSHSessionProcessProtocol(self)
        protocol.makeConnection(transport)
        transport.makeConnection(wrapProtocol(protocol))
        self.client = transport
        return True

Each step here is necessary, but the specifics aren't very interesting or important. Suffice it to say it hooks objects up as necessary so that bytes can be passed from the SSHSession to the protocol and vice versa.

The rest of the code for this version is the same as it has been in the previous versions (except for new imports). Here's the full code listing:

from twisted.internet.protocol import Protocol
from twisted.cred.portal import Portal
from twisted.cred.checkers import FilePasswordDB
from twisted.conch.ssh.factory import SSHFactory
from twisted.internet import reactor
from twisted.conch.ssh.keys import Key
from twisted.conch.interfaces import IConchUser
from twisted.conch.avatar import ConchUser
from twisted.conch.ssh.session import (
    SSHSession, SSHSessionProcessProtocol, wrapProtocol)

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

class EchoProtocol(Protocol):
    def connectionMade(self):
        self.transport.write("Echo protocol connected\r\n")

    def dataReceived(self, bytes):
        self.transport.write("echo: " + repr(bytes) + "\r\n")

    def connectionLost(self, reason):
        print 'Connection lost', reason

def nothing():
    pass

class SimpleSession(SSHSession):
    name = 'session'

    def request_pty_req(self, data):
        return True

    def request_shell(self, data):
        protocol = EchoProtocol()
        transport = SSHSessionProcessProtocol(self)
        protocol.makeConnection(transport)
        transport.makeConnection(wrapProtocol(protocol))
        self.client = transport
        return True

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        user = ConchUser()
        user.channelLookup['session'] = SimpleSession
        return IConchUser, user, nothing

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(SimpleRealm())
factory.portal.registerChecker(FilePasswordDB("ssh-passwords"))

reactor.listenTCP(2022, factory)
reactor.run()

Connect to this server with an ssh client and type away, you should see something like this:

exarkun@boson:~$ ssh -p 2022 localhost
exarkun@localhost's password:
Echo protocol connected
echo: 'h'
echo: 'e'
echo: 'l'
echo: 'l'
echo: 'o'
echo: ','
echo: ' '
echo: 'w'
echo: 'o'
echo: 'r'
echo: 'l'
echo: 'd'
echo: '.'

Tune in next time for a surprise topic!

Twisted Conch in 60 Seconds: Detecting EOF on input

Greetings patient readers.  Welcome once again to Twisted Conch in 60 Seconds, a series of articles in which I attempt to convey the simplicity of writing SSH applications using Twisted.  For those of you who have been reading since the beginning, thank you for bearing with me through the unfortunate hiatus.

When last we parted, I had just explained how to accept input on a custom server channel. In this edition, I'll explain the various callbacks you can expect when your channel is not going to receive any further data.

Recall the channel class which echoes all of its input back to the client from the previous example:

class SimpleSession(SSHChannel):
    name = 'session'

    def request_pty_req(self, data):
        return True

    def request_shell(self, data):
        return True

    def dataReceived(self, bytes):
        self.write("echo: " + repr(bytes) + "\r\n")

When the client disconnects, the SimpleSession instance will get garbage collected (as long as we have no extra code of our own keeping a reference to it, thus keeping it alive). You can also receive a callback when this disconnection happens, though, before the instance is discarded. Actually, you can receive one or more of three different callbacks:

    def eofReceived(self):
        self.write("eof received\r\n")

This eofReceived callback happens when the client sends an End-Of-File notification for this channel. You might see this if the client is invoked like this:

$ ssh username@host < input-file

After the contents of input-file have been sent, the client will send the EOF notification and wait for server output. However, if the client is invoked in this more common manner:

$ ssh username@host

then it's more likely that it will exit via the magic ~. sequence, or by being killed (explicitly or by a SIGHUP when the user logs out). In this case, there is no EOF notification. Instead, the SSH connection is just closed. This results in the channel being closed as well, which can be handled like this:

    def closed(self):
        print "Channel closed"

This notification may also happen after an EOF notification, so be prepared to handle them both. Finally, it's possible for the client to request that just one particular channel be closed. This can be handled like this:

    def closeReceived(self):
        print "Close received"

However, this condition is difficult to trigger with the standard ssh command line client (in fact, I don't know how to trigger it). A custom SSH client might want to do this, though - for example if it creates and destroys many different channels in the lifetime of a single SSH connection.

As a rule of thumb, closed is probably the most commonly useful of these three callbacks. You can use it to reliably clean up resources when a channel is no longer in use and be confident that it is going to be called.

Here's a complete server using the echo channel definition from the previous post, plus the three new callbacks introduced in this post:

 

from twisted.cred.portal import Portal
from twisted.cred.checkers import FilePasswordDB
from twisted.conch.ssh.factory import SSHFactory
from twisted.internet import reactor
from twisted.conch.ssh.keys import Key
from twisted.conch.interfaces import IConchUser
from twisted.conch.avatar import ConchUser
from twisted.conch.ssh.channel import SSHChannel

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

def nothing():
    pass

class SimpleSession(SSHChannel):
    name = 'session'

    def request_pty_req(self, data):
        return True

    def request_shell(self, data):
        return True

    def dataReceived(self, bytes):
        self.write("echo: " + repr(bytes) + "\r\n")

    def eofReceived(self):
        print 'connection lost'

    def closed(self):
        print 'closed'

    def closeReceived(self):
        print 'closeReceived'

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        user = ConchUser()
        user.channelLookup['session'] = SimpleSession
        return IConchUser, user, nothing

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(SimpleRealm())
factory.portal.registerChecker(FilePasswordDB("ssh-passwords"))

reactor.listenTCP(2022, factory)
reactor.run()

By this point, I've explained enough to make a complete (if simple) SSH server application - authentication, output, pty requests, input, and disconnection handling. In the next entry, I'll explain how you can hook a normal Protocol instance up to a channel, useful for abstracting your application logic somewhat away from the SSH details and (therefore) for re-using existing Twisted-based protocol implementations with SSH as their transport.

The "example.com" of OIDs

I happened to find myself wanting to write a unit test for some ASN.1 related code today. Someone else had already written the success case for this code, in fact, and I only needed to write a unit test for the failure mode. This involved receiving some data including an unrecognized OID. I though this might be a case where a lookup function would return NULL instead of a filled out structure. The code in question would then continue on to dereference the NULL and things would go downhill from there.

So of course I wanted my test case to throw an OID at the implementation that I could be sure it would never recognize. If you're familiar with DNS, you may know that there are names like "example.com" which everyone has agreed won't really mean anything (in fact, "example.com" does mean something, and a better comparison is with the less widely used "invalid" TLD - eg "example.invalid") - I wanted the OID equivalent of this.

First I refreshed my acquaintance with the OID registry of which I was already aware. Later I came across the somewhat more cleverly named site, http://www.oid-info.com/. Then I poked around for a while trying to find an existing allocation which fit the bill.

I discovered that the LDAPv3 people seem to think that 1.1 (scroll down a bit) can be used for this purpose. This despite the fact that 1.1 appears to have been allocated to the ISO registration authority, but maybe this use has now been discarded. LDAPv3's appropriation of this OID seemed a misuse to me, so I kept looking.

I found a publication the contents of which apparently would tell me how I could get a new arc (as these things are called) beneath {iso(1) identified-organization(3)}. I didn't feel like shelling out CHF 50,00 for the privilege of learning that, though.

Next I noticed another registration authority at {itu-t(0) identified-organization(4)}. But this all seemed like the wrong path to me. I just want a guaranteed unknown OID to use, I don't want to register an organization with an international standards body.

Then I started looking beneath {joint-iso-itu-t(2)}. Perhaps by their powers combined, ISO and ITU-T had imagined this eventuality and allocated something. Lo! Immediately beheld I {joint-iso-itu-t(2) example(999)}. Not quite invalid but at this point in my searches, close enough (I didn't initially notice that this arc is not in fact currently allocated, merely proposed; after I noticed that it appears it will be allocated later this year I still thought it was good enough).

Only, no. What's the ASN.1 DER encoding of {joint-iso-itu-t(2) example(999)}? There doesn't appear to be one. It appears to be invalid to have 999 as the second component of the arc. As the third component? No problem. As any component after that? Sure thing. But... not as the second (not as the first either; and the exact rules governing the maximum values of the first two components are complicated beyond my understand, so I'm not going to try to explain).

So there's an example OID (pending actual allocation at two separate meetings of two separated international standards organizations). Just not one that can actually be encoded. So I can't actually use it for my test.

Ah well, screw all this crap. 4.5.6.7 seems like a good test value to me.

Addendum: I couldn't leave the loose thread of 2.999 being unencodable alone. So I tracked down the ASN.1 DER encoding rule for this. It's actually the same as the ASN.1 BER rule, which is: "The first octet has value 40 * value1 + value2. (This is unambiguous, since value1 is limited to values 0, 1, and 2; value2 is limited to the range 0 to 39 when value1 is 0 or 1; and, according to X.208, n is always at least 2.)". So there you go. ASN.1 DER really cannot encode 2.999. Perhaps someone will point this out at one of the two upcoming standards bodies meetings.

Twisted Conch in 60 Seconds will return soon

Hello all,

I expect some of you have been eagerly awaiting the next installment of Twisted Conch in 60 seconds. My goal was to produce one of these a week, but clearly I've missed that the last few weeks. The series will return, but it may be another week or two (optimistically) before the next post. Sorry about the delay!

Planting trees, day 6

Wait, for real, six days of this? I wonder how I survived. Today was a day to wrap up loose ends, but there were a lot of them. First we measured and recorded the circumference of all the trees either just above their graft union or at ground level (I measured some heights too before being reminded of what comes next). Then we pruned. We pruned a lot. It was a very uncomfortable experience. We cut back every apple and pear tree to 36" and no branches. We cut back every cherry and plum tree to 28" and no branches. It's a little stick orchard now. If we did it all right, then maybe we'll see some leaves next year. Next we grafted ten scions brought by Jericho's parents from their home orchard. We grafted on to Malus Antonovka (aka antonovka rootstock). This is the same rootstock all of the other apple trees came grafted on to. It's a standard rootstock, which means it wants to grow a 30' tree. In this zone, hopefully it will come out more like 15' - we'll see! We planted these little guys down by the garden with their roots wrapped in t-shirts. In a couple years they should be big enough to join the rest of the apple trees. After that, we whitewashed all the other fruit trees. If this lasts through winter, it will help prevent sun scald. Until then, it will make apple borer damage more visible which should help head off the worst of it. After that, we spread the last of the ramial chipped wood. The smeared out pile from yesterday actually came to almost a cubic yard (probably about 22 cubic feet), which is about what we needed to mulch all of the trees that got landscape fabric instead of gravel. This was an unexpected win; the pile looked much too small to cover everything. Now we won't need to buy woodchips and spread them in June.

And... that's it! All told, now growing outside are four hardy kiwis, two walnuts, two pears, three cherries, four crab apples, seven plums, and 26 apple trees (plus the ~forest that was here before - but I won't count that).

Planting trees, day 5

Today was definitely the day. The weather was great: partly sunny, nice breeze, temperature just right. We had plenty of great food for breakfast. Then, everyone was eager to start early, so we had more than half the holes dug by the time we thought we would be getting started. Four more helpers showed up the morning, and Jericho's parents returned the shredder for us. Then Jericho and I did a demonstration planting. And then everyone was off to plant! It all went very fast. Each tree got some lime, rock phosphate, Azomite, Pro-Gro, compost, and ramial chipped wood (except towards the end we ran out of rock phosphate so a few trees missed out on that). Some got a mycorrhiza inoculant. some got gravel as mulch, some got landscape fabric. After we planted 20 apple trees, 7 plums, 3 cherries, 2 pears, and 2 walnuts, we paused for lunch. Then we dug four post holes (which unfortunately struck... bedrock? 20" down) for an arbor and planted the 4 hardy kiwi next to the posts. The turnout was great (14 diggers +/- 2 small children) so everything went amazingly quickly. The only tasks left for tomorrow are pruning and grafting.

Planting trees, day 4

Almost! Today we rented a bushhog and a chipper/shredder. We cleared (almost) all the brush where the trees will go and we shredded what we cut earlier in the week for woodchips. The shredder was a slight disappointment; it couldn't really keep up most of the time, it went through gas like mad, and there was a mixup about its oil levels that left it out of commission for an hour or two. The bushhog was great, though. We also did a little final cleaning at the cabin. We bought all of the lime the hardware store had (it was only 200 pounds). We also wrote up instruction cards for helpers tomorrow. Jericho's parents arrived with the trees and a lot of food. The trees are taller than I expected. We planted one to run through the instructions (a honey crab). Five other planting helpers showed up after dinner and we got them situated. Planting TOMORROW!

Planting trees, day 3

Today was a very non-agricultural day. It was supposed to thunderstorm, but it did not (it never thunderstorms anymore, it thunderstormed much more when I was a child). In any case, we cleaned out the cabin (it needed it, a lot) so people can stay there while they help plant. We also cleaned the outhouse and cut a few saplings that had grown up in front of it. And we bought a lot of food to fuel the planting, too. One day until planting!

Planting trees, day 2

Successful second day preparing for planting. Cut some wood for chipping in the morning. Took a break from that to measure out and mark tree locations before lunch. Then back to cutting wood for the afternoon. We probably cut enough wood to make ramial chipped wood for most of the trees. On Friday we'll have a chainsaw to cut wood to chip for mulch. Two days to go until planting!

Planting trees, day 1

Day one complete! Jericho and I made it to Maine, borrowed the truck, picked up 14 cubic feet of gravel (~1500 lbs) from Agway. We also reshuffled plans for the rest of the week to better fit the weather forecast (rainy every day, thunderstorms on Thursday). Should be an adventure. Three days to go until planting.

August - April Reading List

Been a while since I let you all know what I've been reading:

• Containment. Christian Cantrell.
• Human Legacy Project. Christian Cantrell.
• The Long Emergency. James Howard Kunstler.
• Sethra Lavode. Steven Brust.
• Star Soldier. Vaughn Heppner.
• BIO-WEAPON. Vaughn Heppner.
• Dzur. Steven Brust.
• Fragile and Distant Suns: A Poul Anderson Collection. Poul Anderson.
• Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software. Scott Rosenberg.
• Spell Games. T. A. Pratt.
• Blood Engines. T. A. Pratt.
• Poison Sleep. T. A. Pratt.
• Dead Reign. T. A. Pratt.
• Bone Shop. T. A. Pratt.
• Makers. Cory Doctorow.
• Microcosm: E. coli and the New Science of Life. Carl Zimmer.
• The Theory of Almost Everything. Robert Oerter.
• Broken Mirrors. T. A. Pratt.
• The Warrior's Apprentice. Lois McMaster Bujold
• The Mountains of Mourning. Lois McMaster Bujold
• The history of Herodotus.  Herodotus. (except books 2 and 3)
• Surely You're Joking, Mister Feynman!  Richard P. Feynman.
• The Name of the Wind. Patrick Rothfuss.
• The Wise Man's Fear. Patrick Rothfuss.
• Reluctant Swordsman. Dave Duncan.
• The Golden Age of Science Fiction. Various
• History of the Peloponnesian War. Thucydides. (Book 1)

 

pyOpenSSL on PyPy

You may know that I'm the maintainer of pyOpenSSL, a Python extension library which wraps some OpenSSL APIs. pyOpenSSL was started in mid 2001, around the time of Python 2.1, by AB Strakt (now Open End). Shortly afterwards Twisted picked it up as a dependency for its SSL features (the standard library SSL support was unsuitable for non-blocking use). When Twisted bumped into some of pyOpenSSL's limitations and no one else was around to address them, I decided to take responsibility for the project.

Fast forward almost a decade. pyOpenSSL now runs on Python 2.4 through Python 3.2. And soon I hope it will run on PyPy, too.

This post is about some of the things I learned while getting pyOpenSSL to work on PyPy.  All of this work is made possible, of course, by the "cpyext" module of PyPy which implements CPython's C extension API for PyPy. 

PyModule_AddObject steals references

When an extension module wants to expose a name, the most common way it does this is by using PyModule_AddObject. This adds a new attribute to the module, given a char* name and a PyObject* object. CPython uses reference counting, so the PyObject* has a counter on it recording how many different pieces of code still want it to remain alive. When populating a new module, the PyObject* you have generally have a reference count of 1. PyModule_AddObject steals this reference: it doesn't increase the reference count, it just assumes that the caller is giving up its interest in the object remaining alive; now it is the module to which the PyObject* was added which has that interest. So the reference count is still 1.

pyOpenSSL exposes a few names which are just aliases for other names (for example, X509Name and X509NameType refer to the same object). It does this by calling PyModule_AddObject twice with the same PyObject* but different char* names. Considering what wrote above about reference counts above, you might guess that this needs extra work to get the reference counting to work correctly. Otherwise the second PyModule_AddObject would steal a reference from the first PyModule_AddObject, since that's what stole it from the module initialization code. This wouldn't work very well, since there really are two references to the PyObject* now, not just one.

Though, it turns out that on CPython, it doesn't really matter. The reference count for one of these types, say X509Name again, ends up at around 20 by the time everything is initialized. Being off by one doesn't make a difference, because most of those 20 references last for the entire process lifetime. The value never gets close to 0, so the missing reference is never noticed. However, on PyPy, it turns out the missing reference does matter. I won't try to explain how PyPy manages to support CPython's C extension API, nor how it manages to make a reference counting system play together with a fully garbage collected runtime (ie, PyPy doesn't normally do reference counting). Suffice it to say that on PyPy, sometimes the reference count does get close to 0, and at those times, being off by 1 can be important - because it might mean that the reference count is exactly 0 when it was supposed to be 1. When that happens, PyPy frees the object, but other code continues to use it, and after that the behavior you get is difficult to predict due to memory corruption - but it's certainly not correct.

The fix for this one is simple - add a Py_INCREF before PyModule_AddObject. This was by far the most pervasive bug in pyOpenSSL which needed to be fixed, since it was repeated for each aliased type pyOpenSSL exposed. I added 28 Py_INCREF calls in total to address these.

PyPy doesn't support tp_setattr (yet?)

The type I mentioned above, X509Name, customizes attribute access. It needs to delegate to OpenSSL to determine if an attribute is valid or not, and if so what its current value is. It does this by implementing two C functions, one for the tp_setattr slot and one for the tp_getattro slot. No, that o isn't a typo. The CPython C extension API provides two different ways to customized attribute access. Using one way, tp_setattr and tp_getattr, CPython hands the extension function the name of the attribute as a char*. Using the other way, tp_setattro or tp_getattro, a PyObject* is passed in, instead of a char*.

So far, PyPy only implements tp_setattro and tp_getattro, not tp_setattr and tp_getattr. It would have been nice to implement this missing feature for PyPy, but instead I switched pyOpenSSL over to the already supported mechanism. This was a very simple change, since most of the lookup code is the same either way, there's just a little extra code at the beginning of the function to convert from PyObject* to char*.

I also learned about a quirk of the tp_setattro API while doing this. I expected setattr(name, u"name", "value") to pass in a PyUnicodeObject*. However, CPython actually encodes unicode to ascii and passes in a PyStringObject* instead.

X509Name.__setattr__ was missing some cleanup code for the AttributeError case

While making the switch to tp_setattro, I noticed a bug in pyOpenSSL where it failed to flush the OpenSSL error queue properly, causing a spurious OpenSSL.SSL.Error to be raised whenever an attempt was made to set an invalid attribute on an X509Name instance. This was easy to fix by adding a call to the function which flushes the error queue.

PyPy doesn't yet support all of the PyArg_ParseTuple format specifiers

Finally, I had to work PyPy itself a little bit to implement the s* and t# format specifiers for PyArg_ParseTuple. PyArg_ParseTuple is how C extension functions unpack the arguments passed to them. A call to this function looks something like PyArg_ParseTuple(args, "s*|i:send", &pbuf, &flags). The string specifies how many and what type of arguments are expected, and the values are unpacked from args into the rest of the arguments passed in. PyPy did not yet support a couple argument types which pyOpenSSL relies on, so I added this support. This code is still in a branch of PyPy, but I hope it will be merged into the default branch soon.

Remaining work

There is one thing left to do before pyOpenSSL will be 100% supported on PyPy. Though I said I implemented s* for PyArg_ParseTuple, I actually only implemented part of it. My code will handle the case where a str is passed in, but not the case where a memoryview is supplied instead. Handling memoryview involves a bit more work and a bit more understanding than I currently have of how PyPy's CPython bridge works. Fortunately there are many useful things that can be done with pyOpenSSL on PyPy even without this feature (when was the last time you constructed a memoryview? :), so I'm still very happy with where things currently stand.

The code

As of this posting, the PyPy code needed to make this work is in the pyarg-parsebuffer-new branch and the pyOpenSSL code is in the run-on-pypy branch. I'll be psyched if the PyPy branch can be merged in time for PyPy 1.5 so that the next pyOpenSSL release can work with the next PyPy release - we'll see!

Twisted Conch in 60 Seconds: Accepting Input

Welcome back to Twisted Conch in 60 Seconds, the documentation series about writing SSH servers (and eventually, clients) with Twisted. In earlier entries, I've covered some of the basics of accepting client connections and generating output. In this edition, I'll cover accepting input from the client.

Recall that in the previous two example programs, a SSHChannel subclass was responsible for sending some output to the client connection. The same object is going to have input from the client delivered to it. Some of you may not even be surprised to learn that the way this is done is that the channel has its dataReceived method called with a string:

class SimpleSession(SSHChannel):
    def dataReceived(self, bytes):
        self.write("echo: " + repr(bytes) + "\r\n")

The single argument to dataReceived, bytes, is a str containing the bytes sent from the client. This simple implementation of dataReceived escapes the received data with repr so it's easy to see what bytes were actually received and then sends them back with a little formatting. As you might expect, dataReceived is being passed bytes from a reliable, ordered, stream-oriented connection. That is, it's a lot like TCP. This means you need to be careful about message boundaries, possibly buffering up several calls with of data before handling it. Unlike TCP, of course, these bytes were sent encrypted over the network. This is an SSH tutorial, after all!

Aside from this method, it's still necessary to acknowledge the PTY request the client will send:

    def request_pty_req(self, data):
        return True

But since this example doesn't make use of the terminal name, size, or mode information all the method needs to do is return True to indicate that the request was successful. Similarly, the shell request must be allowed:

    def request_shell(self, data):
        return True

Again, nothing going on here except a positive acknowledgement of the request so the client will be happy and move on. That's all of the code that's changed since the last example. The full code listing looks like this:

from twisted.cred.portal import Portal
from twisted.cred.checkers import FilePasswordDB
from twisted.conch.ssh.factory import SSHFactory
from twisted.internet import reactor
from twisted.conch.ssh.keys import Key
from twisted.conch.interfaces import IConchUser
from twisted.conch.avatar import ConchUser
from twisted.conch.ssh.channel import SSHChannel

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

def nothing():
    pass

class SimpleSession(SSHChannel):
    name = 'session'

    def request_pty_req(self, data):
        return True

    def request_shell(self, data):
        return True

    def dataReceived(self, bytes):
        self.write("echo: " + repr(bytes) + "\r\n")

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        user = ConchUser()
        user.channelLookup['session'] = SimpleSession
        return IConchUser, user, nothing

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(SimpleRealm())
factory.portal.registerChecker(FilePasswordDB("ssh-passwords"))

reactor.listenTCP(2022, factory)
reactor.run()

Et voilà, a custom SSH server which accepts input and generates output. Next time, the exciting topic of detecting EOF on that input stream...

Twisted Conch in 60 Seconds: PTY Requests

Greetings and welcome once again to Twisted Conch in 60 Seconds, a series introducing SSH development with Twisted. If you're just joining me, you probably want to go back to the beginning before reading this article.

The previous example constructed a server which could present some data to clients which managed to authenticate successfully. However, it also produced an error message which I didn't explain:

PTY allocation request failed on channel 0

This is an error message the client software displays (so you may see something different depending on which SSH client you use - I am using OpenSSH 5.1). What the client is trying to tell us is that it asked the server to allocate a PTY - a pseudo-terminal - and the server would not do it. The client doesn't think this is a fatal error though, so it continues on to request a shell, which it gets.

It turns out that PTY requests are another very common channel request (that is, a request that is scoped to a particular channel, as the shell request is). Almost any time you expect clients to show up at your server and request a shell, it is somewhat likely that they'll request a PTY as well, so you probably want to handle this case.

Recall how the shell request was handled in the previous example:

class SimpleSession(SSHChannel):
    name = 'session'

    def request_shell(self, data):
        self.write("This session is very simple.  Goodbye!\r\n")
        self.loseConnection()
        return True

As a PTY request is just another type of channel request, I can extend this class with a new method, request_pty_req, in order to handle this additional request type.

    def request_pty_req(self, data):

I didn't talk about the data parameter last time because the shell request makes no use of it. However, a PTY request does use it. The client packs some information about its capabilities into a particular string format and that's what this method will receive as an argument. Conch provides a method for parsing this string into a more manageable form:

        self.terminalName, self.windowSize, modes = parseRequest_pty_req(data)

terminalName is a string giving the name of the terminal. This is typically the value of the TERM environment variable on the client - which might not be what you expect, and is often simply set to "xterm".

More useful than the terminal name is the windowSize tuple. This tuple has four elements. The first two give the number of rows and columns available on the client. The third and fourth give the terminal width and height in pixels. For many clients, the latter isn't available or useful so these values are set to 0.

The last value, modes, is rather complicated. You can read about it in RFC 4254, section 8. Or you can take my word for it for now that you probably don't care about it, at least for now.

Notice that I saved some of that data. I'm going to use it to make request_shell a little more interesting. Before moving on to that, there's one last thing to do in request_pty_req though. I need to indicate that the PTY request was successful:

        return True

Now, here's a new version of request_shell which uses those two new attributes:

    def request_shell(self, data):
        self.write(
            "Your terminal name is %r.  "
            "Your terminal is %d columns wide and %d rows tall." % (
                self.terminalName, self.windowSize[0], self.windowSize[1]))
        self.loseConnection()
        return True

Now when clients connect to the server, they will be able to see what terminal they're running and how big it is. Here's the full code listing for this version of the example, including a new import for parseRequest_pty_req:

from twisted.cred.portal import Portal
from twisted.cred.checkers import FilePasswordDB
from twisted.conch.ssh.factory import SSHFactory
from twisted.internet import reactor
from twisted.conch.ssh.keys import Key
from twisted.conch.interfaces import IConchUser
from twisted.conch.avatar import ConchUser
from twisted.conch.ssh.channel import SSHChannel
from twisted.conch.ssh.session import parseRequest_pty_req

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

def nothing():
    pass

class SimpleSession(SSHChannel):
    name = 'session'

    def request_pty_req(self, data):
        self.terminalName, self.windowSize, modes = parseRequest_pty_req(data)
        return True

    def request_shell(self, data):
        self.write(
            "Your terminal name is %r.\r\n"
            "Your terminal is %d columns wide and %d rows tall.\r\n" % (
                self.terminalName, self.windowSize[1], self.windowSize[0]))
        self.loseConnection()
        return True

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        user = ConchUser()
        user.channelLookup['session'] = SimpleSession
        return IConchUser, user, nothing

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(SimpleRealm())
factory.portal.registerChecker(FilePasswordDB("ssh-passwords"))

reactor.listenTCP(2022, factory)
reactor.run()

Compared to the last example, only the methods of SimpleSession have changed. If you run this server, connect to it, and authenticate then you'll be rewarded with a little information about your terminal. Or, if your SSH client does not request a PTY for some reason (for example, if you pass -T to the OpenSSH client) then you'll see an error when the shell request fails, since terminalName and windowSize won't have been set.

At this point, I have explained enough of Conch that you could almost write a very simple application with it. For example, you could add an SSH server to your Twisted Web server which sends request logs to the client. This would just involve calling the write repeatedly, once for each request which was handled. However, for many applications, you may actually want to accept input from the client. Stick around for the next article in which I'll cover precisely that.

Twisted Conch in 60 Seconds: A Trivial Channel

Welcome to the first nearly-useful edition of Twisted Conch in 60 Seconds. In previous posts, I demonstrated how to create an SSH server no one could log in to, and then how to create an SSH server with password authentication but no content beyond that. Today I'll show you how to present some data to clients which connect and successfully authenticate.

Recall that in the last example, I added SimpleRealm to the SSH server. The SimpleRealm was responsible for creating a user (or avatar) to handle authenticated connections. Since Conch provides a very simple user class, ConchUser, the realm was quite simple:

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        return IConchUser, ConchUser(), nothing

Almost any behavior you might want to implement in a custom SSH server is going to go into the user object. For example, after an SSH client authenticates, in almost all cases the first thing it does is open a channel. SSH can multiplex many logical connections over a single TCP connection. Channels are the basis of this multiplexing. When an SSH client opens a channel, Conch turns this into a method call onto the user object, lookupChannel. Channels come in different types, but ConchUser doesn't define behavior for any of them by default. This explains the output of the previous example:

channel 0: open failed: unknown channel type: unknown channel

The channel I'm going to define in this example will be of the session type. This is the kind of channel almost all SSH clients request after authentication succeeds. Channels should typically subclass SSHChannel:

from twisted.conch.ssh.channel import SSHChannel

class SimpleSession(SSHChannel):

Channels also need to specify their type or name:

    name = 'session'

Aside from just passing uninterpreted bytes back and forth, as is done when you use SSH to get a remote shell, channels also support structured requests. SSHChannel lets you handle these requests by defining request_-prefixed methods. When a foo request is received, request_foo is called to handle it. One common request, and the only one I'm going to handle in this example, is a request for a shell. However, to keep things simple, I won't implement it to set up a real shell:

    def request_shell(self, data):
        self.write("This session is very simple.  Goodbye!\r\n")
        self.loseConnection()
        return True

Since the most common thing to use the command line ssh tool for is to get a remote shell, the client issues a shell request by default. After this, ssh is just passing bytes back and forth. It's up to the user and the shell process (such as /bin/bash) to make up and interpret those bytes.

However, this shell request handler does generate some bytes. It sends them to the client using the write method. Then it closes the channel (not the entire SSH connection) with the loseConnection method. Finally, it returns True to indicate that the request was successful.

The only thing left to do is make ConchUser capable of using SimpleSession to satisfy requests to open channels of type session. The channelLookup attribute of ConchUser makes it easy to do this. It's a dictionary the keys of which are channel types and the values of which are SSHChannel subclasses. This means SimpleRealm can be tweaked slightly to produce users which support session channels:

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        user = ConchUser()
        user.channelLookup['session'] = SimpleSession
        return IConchUser, user, nothing

As before, we have a requestAvatar which returns a three-tuple of interface, avatar, and logout callable. However, this version of the realm also adds an item to the user's channelLookup dictionary before returning it. These users can therefore open new channels of type session.

Here's a complete code listing for this version of the example. Only SimpleSession is new, and only SimpleRealm.requestAvatar has changed.

from twisted.cred.portal import Portal
from twisted.cred.checkers import FilePasswordDB
from twisted.conch.ssh.factory import SSHFactory
from twisted.internet import reactor
from twisted.conch.ssh.keys import Key
from twisted.conch.interfaces import IConchUser
from twisted.conch.avatar import ConchUser
from twisted.conch.ssh.channel import SSHChannel

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

def nothing():
    pass

class SimpleSession(SSHChannel):
    name = 'session'

    def request_shell(self, data):
        self.write("This session is very simple.  Goodbye!\r\n")
        self.loseConnection()
        return True

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        user = ConchUser()
        user.channelLookup['session'] = SimpleSession
        return IConchUser, user, nothing

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(SimpleRealm())

factory.portal.registerChecker(FilePasswordDB("ssh-passwords"))

reactor.listenTCP(2022, factory)
reactor.run()

Run this server and connect to it (ssh -p 2022 localhost) and you should see output like this:

PTY allocation request failed on channel 0
This session is very simple.  Goodbye!
Connection to localhost closed.

Next time I'll talk about what that first line of output means and ways to avoid it.

Twisted Conch in 60 Seconds: Password Authentication

Welcome back to Twisted Conch in 60 Seconds, a series about writing custom SSH software with Twisted.  Last time, I showed you how to set up a basic SSH server which could accept connections and then reject all authentication attempts.  This time we'll extend that server to support password authentication.
Let's consider the factory created in the first example:

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(None)

The object in this snippet responsible for authenticating the client is Portal(None). However, this portal is missing two things it really needs in order to be useful. First, it's missing a username/password checker. Twisted includes one that's easy to use and reads credentials out of a file, so we'll start with that one:

from twisted.cred.checkers import FilePasswordDB

factory.portal.registerChecker(FilePasswordDB("ssh-passwords"))

FilePasswordDB will now try to read information from ssh-passwords in order to authenticate SSH client connection attempts. The file should be populated with lines like this:

alice:goodpassword
bob:badpassword
jpcalderone:supersecretpassword

The second thing the Portal needs is a Realm. After the FilePasswordDB says a user supplied the correct username/password combination, Conch needs an object that will represent the user who just authenticated. This object will be used to determine what actions the SSH user is allowed to take, and what consequences they will have. A realm only needs to implement one method:

from twisted.conch.avatar import ConchUser

def nothing():
    pass

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        return IConchUser, ConchUser(), nothing

After authentication succeeds, requestAvatar is called. The avatarId parameter tells the realm the name of the user who just authenticated successfully. The mind isn't used by Conch. The interfaces indicate what kind of user is being requested; in this case, it will include twisted.conch.interfaces.IConchUser (and we just assume that it does for now). The method must return a three-tuple. The first element is the kind of user the realm decided to give back (this must be one of the requested interfaces - again, we're assuming IConchUser for now). This just lets the calling code know what kind of user it ended up with. The second element is the user object itself. Conch conveniently provides us with a basic user class that implements almost no behavior, so it's suitable to be used directly in this simple example. The final element of the tuple is a logout callable. This will be invoked for us when the user logs out. This example has no custom logout logic, so we return a no-op function.
Portal construction will now look like this:

factory.portal = Portal(SimpleRealm())

These are all the pieces necessary to do username/password authentication of SSH users. Here's the full code listing for this version:

from twisted.cred.portal import Portal
from twisted.cred.checkers import FilePasswordDB
from twisted.internet import reactor
from twisted.conch.ssh.factory import SSHFactory
from twisted.conch.ssh.keys import Key
from twisted.conch.interfaces import IConchUser
from twisted.conch.avatar import ConchUser

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

def nothing():
    pass

class SimpleRealm(object):
    def requestAvatar(self, avatarId, mind, *interfaces):
        return IConchUser, ConchUser(), nothing

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(SimpleRealm())

factory.portal.registerChecker(FilePasswordDB("ssh-passwords"))

reactor.listenTCP(2022, factory)
reactor.run()

If you run this server and connect to it with one of the credentials in the password file you provide, then you should receive an extremely gratifying result along the lines of:

channel 0: open failed: unknown channel type: unknown channel

Sorry! Tune in next time to learn what a channel is and how to define one.

Twisted Conch in 60 Seconds: Introduction to an SSH server

Greetings once more, readers.  A year ago I introduced you to the simplicity of web programming with Twisted.  Since then, I've been eager to try to duplicate the success of the series for another topic.  Welcome to Twisted Conch in 60 Seconds.

Twisted Conch implements the SSH protocol for both clients and servers.  It also implements client and server applications.  It is also a library for writing custom SSH client and server software.  This series will focus on using Twisted Conch as an SSH library, and the first articles will cover writing custom SSH servers, with clients covered later.

In this first installment, the goal is to set up an SSH server which will accept SSH client connections but
which cannot actually authenticate any users (so no one will be able to log in).

Before getting into any code, the first thing we need for an SSH server is a server key.  This can be generated using ssh-keygen from OpenSSH or with Conch's own key generation tool, ckeygen.  Since ample examples of ssh-keygen can be found elsewhere, here's an example of generating a key with ckeygen:

exarkun@boson:/tmp$ ckeygen -t rsa -f id_rsa
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in id_rsa
Your public key has been saved in id_rsa.pub
The key fingerprint is:
d5:1d:ba:47:a4:75:1a:fe:d4:ba:46:2e:44:67:0d:8b
exarkun@boson:/tmp$ ls -l id_rsa id_rsa.pub
-rw------- 1 1000 1000 886 2011-03-23 22:59 id_rsa
-rwxr-xr-x 1 1000 1000 226 2011-03-23 22:59 id_rsa.pub
exarkun@boson:/tmp$ 

Here's where the actual Conch code begins.  First, some imports:

from twisted.conch.ssh.factory import SSHFactory

SSHFactory is an IProtocolFactory which is used to create protocol instances to handle new connections that arrive at a listening port.  SSHFactory in particular creates protocol instances that can carry on the server side of an SSH connection.  We need an instance to hook up to a listening port:

factory = SSHFactory()

Nothing very exciting going on there.  SSHFactory doesn't take any initializer arguments.  It does have some useful attributes you can set, though.  Two important attributes are publicKeys and privateKeys.  These define what key the server uses to identify itself to clients.  A server can have multiple keys, so these attributes are bound to dictionaries.  In this example we'll just give the server one key though.  For this, we need to have a couple Key objects:

from twisted.conch.ssh.keys import Key

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

The Key.fromString method can parse several formats, including the standard format keys are kept in by OpenSSH (which is also the format ckeygen emits).

Now we can set up the factory to know about these keys:

factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}

Clients connecting to this server will get to see this public key and then use it to challenge the server to prove it also has the private key by signing some random data correctly.

One more attribute needs to be set on the factory.  privateKeys and publicKeys let clients identify the server.  Now the server needs some way to identify clients.  This is done using a Portal from twisted.cred.  However, I said this server wouldn't be able to authenticate users, so this example will only set a placeholder value here which will fail all authentication attempts:

from twisted.cred.portal import Portal
factory.portal = Portal(None)

Later we'll revisit this attribute so that users can actually log in to the server.

The example is almost done now.  The only thing left to do is listen on a port with this factory and run the reactor.  For this we'll need to import the reactor.  We have several options for how to listen on a port, but for this example I'll use the classic reactor.listenTCP:

from twisted.internet import reactor
reactor.listenTCP(2022, factory)

This listens on TCP port 2022.  Whenever a connection arrives, factory will be used to create a new protocol instance to handle it.  This factory is going to create SSHServerTransport instances, but don't worry about that for now.

Since the reactor is the mainloop that drives any Twisted-based application, nothing actually happens until we run it:

reactor.run()

With that, you now have an SSH server.  It's functional enough to prove its identity to clients that connect to it and even accept authentication attempts from them, but it will always reject their attempts.  So it's not the most useful SSH server (but add a little logging and maybe you have a simple SSH honey pot!), but if you tune in next time, I'll demonstrate how it can be extended to support some more useful authentication options.  Here's a full code listing for this example:

from twisted.cred.portal import Portal
from twisted.conch.ssh.factory import SSHFactory
from twisted.internet import reactor
from twisted.conch.ssh.keys import Key

with open('id_rsa') as privateBlobFile:
    privateBlob = privateBlobFile.read()
    privateKey = Key.fromString(data=privateBlob)

with open('id_rsa.pub') as publicBlobFile:
    publicBlob = publicBlobFile.read()
    publicKey = Key.fromString(data=publicBlob)

factory = SSHFactory()
factory.privateKeys = {'ssh-rsa': privateKey}
factory.publicKeys = {'ssh-rsa': publicKey}
factory.portal = Portal(None)

reactor.listenTCP(2022, factory)
reactor.run()

Question About Hardware

What does it mean if, when nothing but main and EPS power and a CPU are connected to my Asus P5K Premium motherboard (as well as when all my usual peripherals are connected), it resets itself apparently at random, typically between one to ten seconds after powering on?  Or what can I do to find out what it means?