Machine slavery - compat-drivers build box
I've written before on how the compat-drivers project got started, designed to help automatically backport the Linux kernel, and our ideas to help optimize backporting collateral evolutions, and of our move of releases onto kernel.org. I now want to thank the Linux Foundation, HP and SUSE for providing to the project with a build box to help us do our build tests prior to making releases. After a few years of making releases I determined it didn't make sense to make any releases to the community that were not at least build tested over a slew of supported kernels. As it is right now we support building compat-drivers down to the 2.6.24 kernel, Linus is now on v3.9 so that means we have 31 kernels to test against. Our builds tests used to take approximately 120 minutes, thanks to the server our builds were reduced to 23 minutes. The above picture is of htop running first on my laptop running ckmake, then on an 8-core build box I built at home to help with the project, and later the monster 32-core build box supplied by HP for the project. This thing has 32 cores and 236 GiB of RAM.
Thanks HP!
mcgrof@drvbp1 ~ $ df -h /pub/mem/*
Filesystem Size Used Avail Use% Mounted on
tmpfs 20G 5.4G 15G 27% /pub/mem/hauke
tmpfs 20G 6.3G 14G 32% /pub/mem/mcgrof
tmpfs 20G 5.4G 15G 27% /pub/mem/ozancag
Given that we have so much RAM the build system allows each of our developers to run their entire work space in RAM: linux-next, linux-stable, compat, compat-drivers, ccache. This means we skip all the IO on a slow disk. To take advantage of the number of cores available and use as much RAM as possible ckmake, our cross kernel compilation utility, is now multithreaded and runs a build for each kernel on its own thread. It was rewritten to Python and uses ncurses to allow us to get an update of each compile as it completes.
You may have heared kernel.org has a shiny new page now, its using Pelican for its new face lift. compat-drivers is no different, but Pelican is far too complex for what we need, so I wrote a small Python script called rel-html that parses naked project pages and generates an HTML5 release page for you based on simple configuration file. As with other new HTML5 pages the rel-html HTML5 code is based on the HTML5 initializr boilerplate project. Right now we have the release page on a temporary page but with time hope to coordinate a release page somewhere to be hosted on kernel.org that aligns itself with the current Pelican usage. The rel-html code also currently supports others projects, including the Linux kernel releases. What this has made obvious is how many projects lack digitally signing releases in a secure manner. Sample release pages for the different rel-html supported projects below. Whether or not each project embraces the tool, its up to them. Oh yeah, and rel-html is licensed under the AGPL, patches welcomed ;)
Comments