Matts Sanctuary

For the past few days now I've been hiding myself away from the world. As I said last post, I've chosen to work on a compressed cache for NetBSD's virtual memory system. Seeing as there's a linux implementation I decided to look through the code and see how it was interacting with the Linux memory manager (mm). So for most of this past week I've been sifting through the source code and the patches provided at http://linuxcompresed.sourceforge.net.

Let me break down the way that the cache is going to work (and currently does work in the linux patches):

Usually, when memory in a computer system becomes scarce, the mm starts swapping pages to disk. Now, over the past few years the difference in speed between the CPU accessing data from main memory and accessing data from the hard disk has widened. Thus, swapping pages to disk is a very 'slow' process. This is where the compressed cache comes in.

When the system starts up, a portion of memory is put aside for the compressed cache. Ideally, this cache has an initial size and should be able to grow or shrink depending on how much it is being used (adaptability policy), because naturally, if there are no compressed pages in the compressed cache it's a complete waste of RAM space and should be 'shrunk'.

Ok, so we've got some space put aside for our compresssed cache.. and the system is chugging along nicely using up more and more free pages but not really freeing any. This can be caused by having lots of large processes in memory at once time, each requesting more and more pages. Naturally, there are only enough a finite amout of pages free. Suddenly the pagedaemon (the process that handles paging in NetBSD) can't keep up with the demand any more, it can't free pages fast enough and other processes are demanding more and more free pages all the time. Something must be done...

So the pagedaemon comes in and starts swapping pages out to 'backing store'. Now, this is a very slow process so what happens is, instead of writing these pages straight to disk, they're compressed and placed in the compressed cache. The page that contains the data can now be freed and used to meet the free page demand.

When the compressed cache becomes full the compressed pages are decompressed and finally written to swap. This has the advantage that if the free page demand is satisfied, and a process pagefaults on a page that we have stored in the compressed cache, we've just saved ourself an expensive write to disk..

I _think_ that makes sense. Please feel free to leave any comments if you didn't understand or (and this is totally probable) that I got something wrong.

Until next time...

Woo hoo! I got gxemul working with a NetBSD/cats image last night. Going to use this emulator to work on NetBSD's vm manager in an embedded architecture environment. It was actually pretty simple to set up as the documentation leading from the NetBSD software emulators page gave a great walk-through. Also, I've compiled the latest NetBSD -current source tree (after fixing a problem with libc) and now have a -current kernel running on an emulated cats CPU.

And just in case anyone was wondering, no, I didn't build the source tree on the cats CPU, I built it on my AMD Athlon 64-bit 3000+ machine ;]

Also, the reason I'm working on this embedded system is that for this year's Google Summer Of Code I'm wanting to work on a compressed cache subsystem for NetBSD's virtual memory layer. This idea is along the lines of http://linuxcompressed.sourceforge.net

So fingers crossed that I get accepted! Even if I don't, I'm sure I'll have some information to post here soon.

As the title no doubt suggests.. this post is a games related one. I've been playing Elder Scrolls IV - Oblivion practically non-stop since it's been released! It's an excellent game, there's just so much to do, so many little quests that seem to fill your time! Not only that, but the 'main quest' is enough for any person. Having lost one of the items that is a key part of the main quest, I'v recently restarted with a new character.

Not usually being one to play games over and over again, the fact that I even thought of starting this game with a new character and redoing all those quests that I've spent around 40 hours so far completing, is obviously a sure sign of a classic.

I also got my hands on a copy of Final Fantasy VII, (having never played any of the Final Fantasy series) I'm hoping how this one turns out. A friend of mine told me that this one in particular was the best one of the series, so we shall certainly see.

I'm currently reading The Hitchiker's guide To The Galaxy, it's a funny book, although I'm slightly annoyed at how watching the movie _before_ reading the book has seemed to lessen the enjoyment that I might have actually gotten out of it had I read it first. Oh well..

Until next time.

NADD may sound like a severe disease, but its not all bad, it means I have embraced the multimedia age. After reading this post I've realised that I exhibit similar symptoms. Do you have more than 10 windows open on your desktop right now? Might want to check it out and see if you too, are affliced with NADD.

Well yesterday's Python bug day was fun.. apparently more people turned up then previous years which is cool! I had quite a good time, spent about 10 hours on it in total. Will definately go to the rest.