Linux Recovery

(February 2017)

Minimally, linux only needs three things to boot:

The initramfs is the “initial RAM filesystem”. It’s a cpio (“copy in and out”) archive of a self-contained root filesystem.

Normally, linux maintains an in-memory cache of pages of data read from the storage device; these pages can be discarded or written back to disk if linux needs to free RAM. With a tmpfs, linux effectively mounts this in-memory cache as a filesystem. The rootfs is a special instance of tempfs populated by the initramfs contents.

On boot, Linux copies the contents of the intitramfs cpio archive into the built-in rootfs. Linux looks for a file called init in the rootfs, and executes it as PID 1. Init must complete certain tasks (e.g. leading crypto fs drivers) before linux over-mounts rootfs with the real/final root filesystem.

The initramfs must contain all device drivers and tools to mount the final root file system.

         loads      kernel       initramfs     kernel

boot –> kernel –> extracts –> fills –> runs –> init loads drivers, loader image initramfs rootfs “init” over-mounts real root /

Can’t boot. What now?

Get a bootable image on a USB stick, like the recovery/install image for the distribution. The Debian installer has a “recovery” mode selectable from the boot menu; this is the same as typing rescue at the boot: prompt. Boot from recovery media. Mount the machine’s real root filesystem somewhere (e.g. /mnt). Use chroot to swap out the recovery rootfs with the machine’s real root filesystem. Make repairs.

In more detail:

If using the Debian recovery mode, the menu should present a list of partitions from which to choose the root.

If not, find the real root filesystem. The /sbin/ of our recovery rootfs should contain fdisk. fdisk -l shows the partitions.

# cd /
# mount -t ext4 /dev/sda2 /mnt
# mount -t proc proc /mnt/proc
# mount -t sysfs sys /mnt/sys
# mount -o bind /dev /mnt/dev

If /boot is on a different partition from /, and we need to fix it (e.g., working with GRUB, performing a kernel upgrade, etc.), mount that partition too:

# mount -t ext4 /dev/sda1 /mnt/boot

Do the same for other filesystems (/var, /usr) on separate partitions to which we’ll need access.


# chroot /mnt /bin/bash