Friday, May 26, 2006

A Note for Python Users

If you use Twisted (or any other network software which uses non-blocking sockets) and you handle large numbers of concurrent connections, you probably want to avoid Python 2.4.3:


exarkun@kunai:~$ python
fPython 2.4.3 (#2, Apr 27 2006, 14:43:58)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> files = []
>>> for i in range(1024):
... files.append(file('/dev/null'))
...
>>> import socket
>>> s = socket.socket()
>>> s.connect(('www.google.com', 80))
>>> s.send('GET / HTTP/1.1\r\n')
Traceback (most recent call last):
File "", line 1, in ?
socket.error: unable to select on socket
>>>


Hopefully this will be fixed in Python 2.4.4 and Python 2.5.0. Monitor the progress of this bug on the sourceforge bug tracker.

4 comments:

  1. Do you use ulimit to increase max of open fd?

    ReplyDelete
  2. I've been struggling with the same problem. Apparently this LJ page has higher google search rank than the actual bug report!

    Very frustrating bug, and twisted handles it really badly: it just closes the connection after the first send() or recv() with a generic "the connection closed" error.

    ReplyDelete
  3. The link in the post doesn't work. This one does: http://bugs.python.org/issue1494314

    ReplyDelete