In the Folks project, we've managed pretty good buildability as we've been developing the project. So far, that's mostly been through us checking buildability of patches during reviews and fixing any build breaks immediately when we hit them on our own machines. As I said in my FOSDEM talk, I don't think this "spot checking" is sufficient for the kind of reliability GNOME projects really need given our reliance upon jhbuild (which, by its design, does not cope well with unbuildable master branches).
A couple weeks ago, Martin Pitt announced the jhbuild continuous build server he has been working on. It's a great setup and clearly details which modules are failing to build and why. It also keeps useful metrics like "time of last successful build", "time of last failed build", and runs and reports on the results of "make check".
This is a great safety net for GNOME as a whole, since it continuously checks our entire stack in a common environment. Once maintainers get reliably notified about any breaks, I think we'll see broad benefits. When our code reliably builds and passes its unit tests, it's a lot more attractive to outside (and inside!) developers.
With, thankfully little work, Folks is now reliably building and passing its unit tests on this server, and I intend to keep it that way :)
I'd like to thank Martin for setting this all up and him and Jean-Baptiste Lallement for helping debug some LXC-related issues which were causing some of our tests to fail on this server (but not for us maintainers, making it hard for us to help).
In the process of fixing these issues, the build server proved its usefulness for our project already because it caught a build break I didn't hit on my own system (because I had an older version of EDS build). Normally, this could have taken days or longer for me to notice because I tend not to re-build the stack below Folks very often. If I'm working on a large feature, I want to get it all working before I potentially break my build environment with unstable changes in our dependencies. A continuous integration system should also make frequently updating dependencies less of a risk (which is just more good news for everyone).
Folks building and passing tests consistently
February 24th, 2013