NOTE: This doesn’t work with the latest versions of LXCFS and cgmanager! Please see Alpine Linux and Systemd Containers (Round 2) for the new process.
I once got a computer from my uncle that I wanted to use as a virtualization server. However, I found out the processor in the box did not support virtualization, so I settled on containers. I started out using OpenVZ for containers,. This required a custom, and much older kernel. However, I learned about LXC, which was in the mainline kernel and up and coming at the time. I decided to give it a shot.
For the distro, I settled on Alpine Linux. It’s small, lightweight, and doesn’t use systemd (I’m not fond of systemd, for example, see my srvc project). With a little work, I set the system up with unprivileged containers, and things were working nicely. I mostly ran non-systemd based distos to make servers for testing different services.
However, I ran into an issue. I wanted to test my srvc project on a container, and attempted to bring up a systemd-based container. The container refused to work properly, and with a little research, I found that to run systemd unprivileged containers, you have to jump through more hoops (Which makes another example of why I don’t like systemd). Other distros had methods already in place to run systemd-based containers using things like lxcfs, but nobody had done it for Alpine Linux.
After a lot of work and frustrating debugging, I finally got a process down that allowed systemd-based containers in Alpine Linux.
1. Dependencies
First you’ll need to install some dependencies:
apk add alpine-sdk automake m4 autoconf libtool fuse fuse-dev linux-vanilla-dev linux-headers libnih-dev linux-pam-dev
2. Install LXCFS
Lxcfs has to be installed from source, since it is currently not in the Alpine repos. Download the zip from Github (https://github.com/lxc/lxcfs), unzip it, and cd to the unzipped directory. Then run the following:
./bootstrap.sh
./configure --prefix=/usr
make
make install
3. Install CGManager
Cgmanager also must be installed from source, as it also is currently not in the Alpine repos. Download the zip from GitHub (https://github.com/lxc/cgmanager), unzip and cd to the unzipped directory. Then run the following:
./bootstrap.sh
./configure --prefix=/ --with-init-script=openrc
make
make install
Then chmod the init script:
chmod +x /etc/init.d/cgmanager
The init script for cgmanager is bugged, and needs you’ll need to change the script to execute /sbin/cgmanager instead of /usr/sbin/cgmanager. Example:
#!/sbin/openrc-run
# Copyright 1999-2015 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Id$
description=.Control Group manager daemon.
pidfile=./run/cgmanager.pid.
command=./sbin/cgmanager.
command.args=.-m name=systemd.
command.background=.yes.
retry=.TERM/45.
depend() {
before cgproxy
}
Then start the service. This creates the necessary namespaces
service cgmanager start
Set it to start on boot.
rc-update add cgmanager
4. Load Kernel Modules
Load some kernel modules:
modprobe fuse
modprobe autofs4
Add them to the /etc/modules to have them load on boot.
5. Start LXCFS
Run lxcfs. Don’t start this until after cgmanager is started. First create the necessary directory for lxcfs to mount on.
mkdir -p /usr/var/lib/lxcfs
To run lxcfs, there are two options.
Manual Option (Good for testing)
You can run this in the command line with the following command (WARNING: ctrl-c won’t stop this process, you’ll have to use the kill command).
lxcfs -s -f -o allow.other /usr/var/lib/lxcfs
If you choose to use this method, you’ll have to do some stuff manually. You’ll need to change the permissions of certain directories. LXC.ROOT is the root user for the unprivileged containers, usually 100000:100000.
chown -R LXC.ROOT:LXC.ROOT /run/lxcfs
chmod -R 700 /run/lxcfs/controllers
Also, once lxcfs is shut down, you’ll have to unmount a few things as well.
umount /run/lxcfs/controllers/.
umount /run/lxcfs/controllers
umount /usr/var/lib/lxcfs
Init Script
I created an Openrc init script to make things easier to manage. It seems to work well enough, but it hasn’t been rigiously tested. It’s available here: https://gist.github.com/bocajspear1/cf36b33a3eb88b6b21589eba7fcc9a13
6. Patch lxc-ls Command
Due to our extra mounts woth lxcfs, lxc-ls breaks when listing active domains and requires a patch. Change the following line in the lxc-ls script:
mountpoint=$(awk -v subsysregex=.(^|,)$subsystems(,|..$). ..
'$3 == .cgroup. && $4 ~ subsysregex {print $2}' /proc/self/mounts)
To
mountpoint=$(awk -v subsysregex=.(^|,)$subsystems(,|..$). ..
'$3 == .cgroup. && $4 ~ subsysregex {print $2}' /proc/self/mounts | grep 'cgroup')
All you’re doing is adding . | grep ‘cgroup’ . to the end of the subcommand.
7. Finishing and Testing
Once this is complete, systemd-based containers should work. I’m not sure of the security of this setup, but it should be fine if you’re the only person using the system.
Troubleshooting
If you’re having issues starting lxcfs, check the mount command to see if things like /run/lxcfs/controllers/ are mounted and unmount them using the following commands:
umount /run/lxcfs/controllers/.
umount /run/lxcfs/controllers
umount /usr/var/lib/lxcfs