Matts Sanctuary

After seeing the failed Google Summer of Code project of integrating journaling support for BSD's FFS, I decided that I'd quite like to work on that project. So, time and university work allowing, I'm going to try and make a start on the code soon. Stay tuned for updates.

p.s I also received my SoC t-shirt and certificate yesterday, they're both pretty cool.

Seeing as this year's SoC is over I've been spending some time working on a project for Netbsd. As some of you may know, Elad Efrat has been working on a new authorization model for the NetBSD kernel, kauth(9). It's based on the kauth framework from MacOSX and it's my job to help Ober to produce a regression test framework to ensure that the transition from a traditional security model (refered to in literature as "bsd44") has no adverse side-effects (such as allowing a non-superuser to reboot the system) because the kauth framework hasn't implemented something properly. I won't discuss how the framework works here, refer to the kauth(9) man page for more details and the kernel authorization technical note from apple here.

The real purpose of this blog entry is to stress the need for regression tests in every project :P The regression framework and tests are written in Python as I found it the quickest language to write designs up in. I wrote the framework in two days and began implementing the tests and running them with a kernel that had Elad's patches compiled in. Within a few runs we'd found bugs in Elad's implementation and in fact, in the NetBSD date(1) command. This bug in the date(1) had been around for years and no one had ever reported it before. These tests have been invaluable and it still fails to amaze me how important good regression tests are to the success of a project, especially when you're implementing new policies or changing existing implementations of interfaces. The tests are also adaptable for different models.

Everyone write unit tests ;-)

Well that's it now. Summer of Code is over for another year. I've had a great time working on it and managed to get some nice stuff into pdb/pydb. Whilst I didn't complete all of the goals I'd set myself there's definately been some progress. One new thing that pydb has is gdb-like signal handling.

* gdb-like signal handling

Also yet to be merged but working is the ability to debug a process from another process and the ability to debug a process remotely. These will be neat features when they go in.

The thing that I didn't complete though, that the ability to debug threaded applications. This proved to be much more involved than either I or my mentor had first thought. However, we're still working on this and hope to get it into pydb soon.

If anyone's thinking about taking part next year, I whole-heartedly recommend it! Having a mentor to guide you is great, and you get to take advantage of that fountain of experience. A big thanks to my mentor, Rocky!

Matt

I spent most of today debugging a deadlock case in the Python Standard Library. I'm not sure if anyone's ever tried, but if you write a tracing function and pass it to threading.settrace, and within this function call 'threading.currentThread()', when the trace functions traces the thread code through threading.Thread.__delete, if will try to call threading.currentThread() *AFTER* _active[_get_ident()] has been deleted (threading.currentThread() just returns _active[_get_ident()]). Now the Python developers thought of the possibilty of asking for the currentThread() when it had been removed from _active, so they decided to create a _DummyThread object. Only this object tries to acquire a lock held in the threading.Thread.__delete method. *BOOM* deadlock.

Anywho, I'm tired, it's nearly 2am...

Been working on allowing debugging by a debugger running in a separate process for most of this week. I pushed this forward in the list of things to do (in front of thread debugging) as I'd spent quite a long time working on allowing thread debugging and was getting a little frustrated. Ok,
so, 'debugging an already running process?', I hear you say. Yes! I'll give a quick overview, if you're interested, check out the documentation in the sandbox/pdb/Doc directory.

So, you've got your simple chat application, that allows clients to connect to a server and messages are broadcast to each of the clients. But you're a little unsure if the code you've written is totally stable, you'd better be able to debug it. But you realise you want to debug the program from a different terminal, but still on the same machine (think, logging in to a machine at work from home). Add this little bit of code to enable debugging from another process

import mpdb ; mpdb.process_debugging()

What this does is sets up a signal handler and when it catches a signal a pdbserver is started. You can specify the pdbserver parameters by passing them to process_debugging. The default
connection parameters are a FIFO connection with a filename composed of the temp directory on your system (as returned by tempfile.gettempdir()), the PID of the process and the word 'mpdb'. For, for example, running this on my NetBSD machine I get,

>>> import mpdb ; mpdb.process_debugging()
'mconnection.MConnectionServerFIFO /tmp/25813mpdb'

But say, you wanted to start a pdbserver using TCP on localhost:8000 (you can only use localhost as a hostname for a TCP connection with process_debugging),

>>> mpdb.process_debugging(protocol='tcp', addr=':8000')
'tcp :8000'

Now a debugger can start a debugging session by issuing the 'attach' command, which takes a PID as an argument and sends (by default) a SIGUSR1 signal to that PID. This gets handled by the handler in mpdb and starts a pdbserver. The debugger then connects. *BAM* have a debugging-good-time. By the way, use the 'detach' command to detach from the debugger, as it allows the program being debugged to continue execution. Typing 'quit' quits the program :P

Tim posted a message to python-dev today asking if he could get some changes into the threading code of Python, namely, exposing a mechanism for mapping thread names to thread frame objects in a dict. This is the original conversation back in March 2005, http://mail.python.org/pipermail/python-dev/2005-March/051856.html.

Since the Python trunk is in "feature-freeze" and Tim's patches can't be incorporated, it looks as though he's made his own branch for these features, tim-current_frames. The actual code is going to be used to get the functionality of a module called 'threadframe' into the Python core. This could really be useful to my SoC project for using these features to help debug threaded applications :-) Who knows, if I can think of some useful things to add, maybe Tim will accept them?

I'll keep y'all posted.

Well yesterday one of my lecturers at university emailed me to tell me that he's changed my degree programme to computer science from computing, which was at my request. This is a good thing for me a) because computer science has much more interesting classes than computing b) I would have wasted option-points choosing comp sci classes and would still have had to do the boring computing classes! So, all in all, I'm a happy man..