RamDisk Backed By Real Disk?

Ramback looks like an interesting project.

Essentially, it's a device-mapper layer that sits in between a backing store (i.e. regular block device) and a virtual DM device.

In most cases, it acts as a virtual RAM disk of the same size as the device.  Writes automagically happen to the RAM disk and it flushing stuff out to disk as massive (albeit unsafe) linear writes.  The idea is that your system is embedded and battery backed up.  When line power goes, you go into an emergency mode where everything gets flushed to disk.

I've been scheming about something like:

AoE <--> DRBD <--> RamBack <--> AoE SAN (staggered across physical shelves)

This would be cool to have ultra-fast but still reliable storage for things like MySQL transaction logs, BDB transaction logs, JFS/Ext3/ReiserFS external journals, and the like.  Rather than being backed up by batteries (to provide RamBack with assurance of runtime), DRBD would provide the redundancy against single failure (although separate power feeds would be nice).

When it's SMP-ready I might take a stab at shoving it in production.

UPDATE:  I meant DRBD, not GNBD.  Also, I recently discovered Write-Intent Bitmaps for MD devices, which also would be great for this kind of backing store.

Documentation and Testing

Big changes to the Agent Architecture as of late.  In particular the security infrastructure is getting fleshed out a bit.  It doesn't feel like an afterthought, which is nice.

Unfortunately I'm not getting as much code done as I'd like.  I've spent a lot of time revising the documentation and working out examples.  Writing examples is slow going, as there's a lot to document.  However, I'm committed to it, since the examples are going to become my unit tests.  It's all in the wiki.  I can't seem to access the wiki without logging in, so I probably need to ask somebody when we are opening that part up.

Speaking of unit tests, with a messaging protocol it's absolutely amazing how much infrastructure has to be in place to test some of this stuff.  Some of the original design was really poor for testing against.  Getting a good test environment still eludes me a bit, although I'm getting there.

Erlang packaging is still a little fuzzy for me.  Ezra sent me a link to this exciting Erlang P2P project called Monsoon that's on GitHub.  I've actually learned a bit about packaging an Erlang App from them, so that makes me feel good at least.

Well, back to the grindstone.

Entrepot, Docs, Testing, and Collaboration!

So I'm currently doing some work on Entrepôt.  I won't go too into detail, but it's part of our management infrastructure.  It's been my focus for a while.

I took a break tonight to move some of the design docs over to the new wiki.  I'm unsure when it will be available to the public, but I hope it's soon.  I'll be sure to mention it here.

Before I tear back into things too hard, I think I'm going to work on the protocol examples a little more.  If you're familiar with the project, you'll know that's just extensions to XMPP.  At any rate, that shouldn't take too long, but I expect it will be useful for anyone who wants to collaborate.  It'll also be helpful for me, since I really need some more tests once I get a good framework together for testing our XMPP infrastructure.

I'm meeting with some people later this week that may like to help, and that's always nice.

Body Shop Open For Business

Well, my old blog is currently under wraps, and I wanted somewhere to document the work we're doing, so I opened this sucker up.  In a perfect world, I may even update it regularly!  We'll just have to see.

We've been doing some exciting design work.  I must admit I was kind of shocked when I found out how much of it we were going to open source.  I mean, every high-minded code-jockey thinks "I'm going to get THIS employer to open source my code." but invariably never gets anywhere.

These guys are different.  Of course, it helps that I helped to found it with Tom and Ezra, but I'm continually surprised how committed that our founders, employees, and even our investors are to opening this stuff up.  Talk about a group of people that really "get it".

That's enough of my sentimental ramblings.  Back into the hole for me...