Monday, August 25, 2014

Hacking on systemd with OpenSUSE

I had recently had no other option but to hack on systemd :*( and found there wasn't any documentation on how to do this on OpenSUSE. Replacing your /sbin/init isn't as simple as it used to be back in the day, eventually I figured things out with a few hiccups but apart from the actual ability to hack and install systemd I also picked up a bit of good best practices you can use to help while testing, and dealt with installing kdbus as I was tired of seeing those pesky warnings from systemd without it. My first assumption that things would just work if I installed over my base install proved incorrect, so avoid that ;), I'll cover doing this with containers. While I don't yet have access to edit the freedesktop wiki I figured I'd document my steps here and later move that documentation once and if granted access.

First you need the equivalent of a deboostrap a la OpenSUSE. Since OpenSUSE is now a rolling distribution this documentation will focus on using those repositories. Since OpenSUSE embraces btrfs fully and it has copy-on-write bells and whistles to help you save space this small little guide will also provide instructions on using the btrfs snapshot capability of btrfs to help you use a base OpenSUSE install for further "branch" type of hacking. This will let your copies of the original install share the same base size / blocks on the hard drive and only make changes once you've modified the system. If you don't want to use the btrfs snapshot feature just ignore the btrfs commands and create a directory instead of the creation commands. This should let you hack without using up gobs of space. This should be considered a small supplement on hacking and testing systemd in a virtualized environment. As of 2014-08-05 the instructions here will create a small container for you that will take about 333 MiB of space.

First get your repos set up with the latest rolling distribution repo, if using btrfs might as well use the btrfs snapshot feature:

$ sudo btrfs sub create /opt/opensuse/
# If you don't want to use the snapshot just create the directory
$ sudo mkdir -p /opt/opensuse

This will let you install package binaries with zypper install

$ sudo zypper --root /opt/opensuse/ ar repo-oss

Quite a bit of packages require /dev/zero to be available. 

$ sudo mkdir /opt/opensuse/dev/
$ sudo mknod /opt/opensuse/dev/zero c 1 5
$ sudo chmod 666 /opt/opensuse/dev/zero

Then install a minimal set for hacking:

$ sudo zypper --root /opt/opensuse/ install rpm zypper wget vim sudo

Now get qemu-kvm and load then the kvm module.

$ sudo zypper install qemu-kvm
$ sudo modprobe kvm-intel

 Next you should launch systemd-nspawn (the systemd chroot equivalent) and change your root password before booting into it, and enable root login from the console.

$ sudo systemd-nspawn -D /opt/opensuse
Timezone America/New_York does not exist in container, not updating container timezone.
Directory: /root
Tue Aug 5 17:39:47 UTC 2014
opensuse:~ # passwd
New password:
Retype new password:
passwd: password updated successfully

By default OpenSUSE won't let you log in to the console as root, to enable that do:

opensuse:~ # echo console >> /etc/securetty

opensuse:~ # sed -i 's/session\s*required\s*    required' /etc/pam.d/login

To make it easier to hack on it'd be ideal to also just enable root access without a password, the involves making some PAM changes, and disabling the password for root, this still doesn't work for me so this is incomplete for now, ignore the next steps for now, I leave them here if anyone wants to continue to chug on that route and figure out the other steps.

opensuse:~ # sed -i 's/root:.*:\([0-9]*\)::::::/root::\1::::::/' /etc/shadow

Now you should be able to just boot into it using a container, shut down the container you were just in:

opensuse:~ # systemctl halt

Now give your new container a fresh spin with -b

$ sudo systemd-nspawn -bD /opt/opensuse 3
The -b will tell systemd to run init on the container and the number 3 tells systemd to launch the various services required for the A target is a way to group up required services. You should be able to log in using root.

Eventually you want to list and manage any deployed container, this includes killing them. For that you can use machinectl within your own system, not within the container.

$ machinectl
MACHINE                          CONTAINER SERVICE        
opensuse                         container nspawn         

1 machines listed.

To kill the one you just started for example:

$ sudo machinectl terminate opensuse
$ machinectl
MACHINE                          CONTAINER SERVICE        

0 machines listed.

To start hacking create new snapshot based on the original. This will let us easily create new OpenSUSE containers to hack on. Kill the base container first with machinectl before doing this though.

$ sudo btrfs sub snap /opt/opensuse /opt/opensuse-hack1

And then go at it on /opt/opensuse-hack1 to hack on your stuff. You can now follow the instructions on the freedesktop wiki on hacking on systemd on virtualized environment but it doesn't tell you to uninstall the distribution's version of systemd -- this is recommended, at least I ran into issues without doing this. To do that just remove the files the rpm installs. You can do this several ways:

From within your system, targeting the new container path:

$ rpm -ql --root /opt/opensuse-hack1/ systemd | sed -e 's|\(.*\)|/opt/opensuse-hack1\1|' | xargs rm -f
Something a bit more safe if you don't trust the above:

$ CONT="/opt/opensuse-hack1/"
$ for i in $(rpm -ql --root $CONT systemd); do if [[ -f $CONT/$i ]]; then sudo rm -f $CONT/$i ; fi ; done

And finally another simpler / secure way to do this from within the container, your container will just become useless after this though so you'll have to kill it from your system with machinectl after this.

linux:~ # rpm -ql systemd | xargs rm -f

All you need now is to compile systemd from sources from your system locally and then use DESTDIR=/opt/opensuse-hack1/, but be very sure to also the --with-rootprefix= option as by default systemd will leave it blank.

$ ./
$ ./configure CFLAGS='-g -O0 -ftrapv' --enable-compat-libs --enable-kdbus --sysconfdir=/etc --localstatedir=/var --libdir=/usr/lib64 --enable-gtk-doc --with-rootprefix=/usr/ --with-rootlibdir=/lib64  
$ sudo DESTDIR=/opt/opensuse-hack1/ make install

As of 2014-08-05 systemd from source by default will want the shiny new kdbus. Go read up on the lwn kdbus article, then since kdbus is not yet in the kernel you'll want to compile a fresh vanilla kernel (I don't provide instructions here obviously), install that and later compile and install the kdbus as a module form the external repo:

git clone
cd kdbus
# Use a known compilable version at least if you're on v3.16.0-rc7
git reset --hard 1f63f96686f9398eedde86b4e08581d14c6e403a
sudo make install

Finally you can now give your container a spin.

$ sudo systemd-nspawn -bD /opt/opensuse-hack1

To be sure you are getting a new systemd you can test the version systemd --version from the container.