https://www.freebsd.org/doc/handbook/jails.html https://wiki.freebsd.org/Jails
Also see jail(8), which may be better at this point (2015) than the handbook.
Jails improve the chroot concept by limiting not only what part of the filesystem a process can access, but also virtualizing a set of users and networking.
They work well as light-weight virtualization/containers.
Elements of a jail:
If we have only a single public IP, we can NAT to private IP’s for the jails.
Add to /etc/rc.conf (modified as appropriate for our particular jails and NIC):
gateway_enable="YES"
firewall_enable="YES"
firewall_script="/etc/ipfw.rules"
firewall_nat_enable="YES"
firewall_nat_interface="vtnet0"
cloned_interfaces="lo1"
ifconfig_lo1="inet 172.16.0.1/24"
Add nat rules to /etc/ipfw.rules (or wherever we keep our ipfw rules), with outbound rules that should include our jails using ‘skipto’ rather than ‘pass’:
add="ipfw -q add"
ipfw -q -f flush
pubnic="vtnet0"
pubip="10.0.1.55"
ipfw -q nat 1 config if $pubnic reset
$add 10 pass all from any to any via lo0
$add 25 pass all from any to any via lo1
$add 50 deny ip from any to any not antispoof in
$add 100 nat 1 ip from any to any via $pubnic in
$add 110 check-state
$add 1000 skipto 10000 udp from any to any 53 out via $pubnic keep-state
$add 1010 skipto 10000 tcp from any to any 53 out via $pubnic keep-state
$add 1100 skipto 10000 tcp from any to any 80,443 out via $pubnic keep-state
$add 1300 skipto 10000 udp from any to any 123 out via $pubnic keep-state
$add 2000 pass tcp from any to any established
$add 5200 pass tcp from any to any 80 keep-state
$add 5300 pass tcp from any to any 443 keep-state
$add 6100 pass tcp from any to $pubip 22 in keep-state
$add 6200 pass tcp from me to any 25 out keep-state
$add 6300 pass icmp from any to any via $pubnic out keep-state
$add 9999 deny all from any to any
$add 10000 nat 1 ip from any to any via $pubnic out
$add 10001 pass ip from any to any
In /etc/sysctl.conf:
net.inet.ip.fw.one_pass=0
iocage(8) https://github.com/iocage/iocage https://dan.langille.org/2015/03/07/getting-started-with-iocage-for-jails-on-freebsd/
iocage is newer than ezjail. Like ezjail, iocage simplified management of jails. iocage requires zfs.
[EDIT: ezjail can also add IP addresses ad hoc.] Unlike with ezjail, iocage handles setting up its own ip aliases, so don’t add ifconfig_lo1_alias lines to /etc/rc.conf.
# pkg install iocage
# iocage activate MYZFSPOOLNAME
# iocage fetch
# iocage create -c tag=myjail ip4_address="lo1|172.16.0.2/24"
# iocage list
If we need to change anything before spinning up the jail, we can chroot into it:
# iocage chroot myjail
Start the jail:
# iocage start myjail
Access the console of a running jail:
# iocage console myjail
Set the jail to start at boot:
# iocage set boot=on myjail
Delete a jail:
# iocage destroy myjail
See ezjail(7).
The ezjail utility simplifies setup and administration of jails significantly.
# pkg install ezjail
Add to /etc/rc.conf:
ezjail_enable="YES"
Create a jail base and template with:
# sudo ezjail-admin install
By default, it fetches files by ftp. As an alternative to ftp, install from the disc1.iso:
# wget http://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.2/FreeBSD-10.2-RELEASE-amd64-disc1.iso
# mdconfig -a -t vnode -f FreeBSD-10.2-RELEASE-amd64-disc1.iso
# mount -t cd9660 /dev/md0 /mnt
# ezjail-admin install -h file://mnt/usr/freebsd-dist
# umount /mnt
# mdconfig -d -u 0
Unless $ezjail_jaildir specifies otherwise, jails will reside under /usr/jails, so keep in mind the disk space of /usr.
Create a jail:
# ezjail-admin create test1 'lo1|172.16.0.2'
Start the jail:
# ezjail-admin start test1
Connect to the jail’s console:
# exjail-admin console test1
Create /etc/resolv.conf in the jail to get name resolution.
Remember to change the jail’s root password!
# ezjail-admin console dnsjail
# passwd
Changing local password for root
New Password:
Retype New Password:
And set the jail’s timezone with tzsetup.
Create a second jail: test2 with IP 172.16.0.3, etc.
Update the jail base to the latest patched version of the host:
# ezjail-admin update -u
If we have a single public IP and want to host multiple jailed, nat’d web apps without resorting to alternate port numbers, we can use HAProxy to handle the requests.
Public IP
|
+-------+--------- FreeBSD Host -------------------+
| | |
| NAT----------+---------+------HAProxy |
| | | |
| | | |
| +-----Jail1--+--+ +--+--Jail2-----+ |
| | | | | |
| | nginx | | nginx | |
| | | | | |
| +---------------+ +---------------+ |
| |
+--------------------------------------------------+
Install nginx in the jails:
# pkg install nginx
Edit/create the jails’ /etc/rc.conf:
nginx_enable="YES"
Edit the jails’ /usr/local/etc/nginx/nginx.conf:
user www;
worker_processes 1; # Set to number of cpu's.
Start the jails’ nginx:
# service nginx start
Now, on the host, we install haproxy:
# pkg install haproxy
Edit the host’s /etc/rc.conf:
haproxy_enable="YES"
Note the existence of /usr/local/share/doc/haproxy/.
HAProxy config in /usr/local/etc/haproxy.conf:
global
daemon
maxconn 1024
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend http
bind 10.0.1.55:80
default_backend test1
acl test1 hdr(host) -i test1.example.com
acl test2 hdr(host) -i test2.example.com
use_backend test1 if test1
use_backend test2 if test2
backend test1
server test1 172.16.0.2:80
backend test2
server test2 172.16.0.3:80
And on our host:
# service haproxy start
For testing purposes, we’ll add this to our workstation’s /etc/hosts (where 10.0.1.55 is the public IP of our FreeBSD box):
10.0.1.55 test1.example.com
10.0.1.55 test2.example.com
Edit /usr/local/www/nginx/index.html in each of the jails, so we can tell that different pages are actually getting served, and point our browser at http://test1.example.com and http://test2.example.com.
In some cases, unprivileged users can escape from a jail by cooperating with a user outside the jail. Mitigate this by denying access to the jail root for unprivileged users on the host system (and privileged users in the jail should not have accounts on the host system).