(Updated March 2016)
Containers fall into two broad types: - containers that contain one process - containers that contain multiple processes on a minimal OS install
Advantages of containers (to various degrees, depending on container type): - enhanced security - isolation (library versions, etc.) - ease of packaging and deployment
A lot of container systems seem to store their images in /var/lib/(docker|lxc|machines|whatever). Why /var/lib? No idea.
chroot is the simplest form of isolation. chroot temporarily sets the root directory (“/”) as another directory, which keeps processes from moving up into the real system root. This provides some security, and is useful for testing.
chroot is a “full system” type of container.
# apt-get install binutils debootstrap # mkdir -p /var/containers/mychroot # debootstrap --arch=amd64 jessie /var/containers/mychroot http://httpredir.debian.org/debian # chroot /var/chroot/mychroot
Configure the chroot. Of course, systemd makes this more complicated. Create a service file /etc/systemd/system/mychroot.service, like:
[Unit] Description=A chroot()ed Service [Service] RootDirectory=/var/containers/mychroot ExecStartPre=/usr/local/bin/setup-foo-chroot.sh ExecStart=/usr/bin/food RootDirectoryStartOnly=yes
The ExecStartPre script can be used to do things like mount /proc and /sys inside the chroot, if needed. As alternatives to RootDirectory, note ReadOnlyDirecctories and InaccessibleDirectories.
On systemd boxes, a chroot alternative is systemd-nspawn:
# apt-get install binutils debootstrap systemd-container # mkdir -p /var/containers/mynspawn # debootstrap --arch=amd64 jessie /var/containers/mynspawn http://httpredir.debian.org/debian # systemd-nspawn -D /var/containers/mynspawn
This dumps us into a shell prompt in the container. systemd-nspawn automatically mounts a number of filesystems read-only (e.g. /sys, /proc/sys). systemd-nspawn provides somewhat more isolation than pure chroot.
(On the back-end, systemd-nspawn uses kernel LXC.)
systemd-nspawn can be either a single process or full system container.
-b flag to
init to “boot” the container, instead of just running a shell.
machinectl to start,stop, and connect to systemd-nspawn containers.
Docker is a container system, often used for packaging and distributing software, particularly microservice components.
Docker is a single process container.
Note that docker containers are stateless, so persistent data must be handled outside the container (mounted filesystem, database, etc.).
(It’s possible to have multiple processes or to attach persistent storage to a Docker container, but that’s fighting against the tide.)
Docker can use a number of back ends (libcontainer, libvirt, LXC, systemd-nspawn, etc.).
Docker has a Docker Server with containers as clients. Optionally, a third component, the Docker Registry, stores Docker images and metadata.
# apt-get install docker.io # docker version
A note about how file systems normally work in Docker: a normal Docker container has a read-only file system based on the Docker image from which it was created. Above this, the container has a read-write layer that stores differences from the read-only original image layer. However, when the container is destroyed, the changes in the read-write layer are discarded; future spin-ups of the Docker image start fresh with the original read-only layer. Docker calls this the Union file system.
Docker has two ways to implement persistent storage (although the default assumption is still that containers will be non-persistent):
docker rm -v mycontainer). Find dangling volumes with
docker volume ls -f dangling=true; remove them with
docker volume rm myvolume.
NOTE: newer versions of Docker (1.9+) have
For most use cases, this supersedes data-only containers.
LXC is the native linux kernel container. LXC is a full system type of container.
# apt-get install lxc # lxc-checkconfig # lxc-create -n mycontainer -t debian # lxc-ls -f # lxc-info mycontainer # lxc-attach -n mycontainer
lxc comes with a number of templates for new containers. On Debian, these are found in /usr/share/lxc/templates/.
lxc containers can have various types of backing stores (btrfs, zfs, lvm, etc.); by default, they’re created on the file system under /var/lib/lxc/.
Unlike Docker, which is focused on containing a single process without persistent storage, LXC contains a more traditional fully system, like FreeBSD jails.