Friday, February 10, 2012

Using iperf with Linux containers

At times you may want to hack on a driver and test it on the same machine without having to require a switch in the middle, or even another box. In my case I wanted to test a new Atheros Ethernet driver we're working on, alx, with a 1 GB/s link through my free PCI-E slot and hook it up to my with an RJ-45 to my laptop's own internal 1 GB Intel Ethernet card, that uses the e1000e driver.

If you try assign IP addresses to these guys, and use 'iperf -s' on one and 'iperf -c' on another you'll end up transmitting using the loopback won't actually get metrics you want on the device.

To work around this you may be inclined at first to work with chroot, but that won't give you the separation you want with hardware. There are some ARP tricks you can work on to accomplish this, and another alternative is to use some -E iperf flag which is not merged. I also tried iperf -B but that didn't work either. In the end I decided to give Linux containers a shot and it worked nicely, albeit, with a bit of work too. Below is a few thing you can use to follow to get your setup with it.

First, go and get yourself a debootstrap setup for Linux containers and use a simple twist to that guide by using the lxc "phys" type for the network device as illustrated below. After this just cp -a the debootstrap'd thingy to another dir to get yourself a second box. You may be able to share the same files for a lxc run, but I haven't tested this. Then create two lxc.conf files, one for each of your containers which will use their own dedicated network device. I use /etc/lxc/sid-01.conf and /etc/lxc/sid-02.conf, below is my /etc/lxc/sid-01.conf:
lxc.tty = 6
lxc.pts = 1024
lxc.rootfs = /var/lib/lxc/sid-01
lxc.cgroup.devices.deny = a
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rwm
# network = phys = up = eth1 = = eth1 = 00:FF:12:34:56:78
For your sid-02.conf just change the to eth4 or whatever, and also change the IP address and hw address slightly. That's all. After this running 'iperf -s' on one box and 'iperf -c' on the other should produce the results you expect.

From I am mcgrof's smirking revenge

All the fun things you can get done now with just one box :)
Post a Comment