FreeBSD Jails ============= https://www.freebsd.org/doc/handbook/jails.html https://wiki.freebsd.org/Jails Also see jail(8), which may be better at this point (2015) than the handbook. Jails improve the chroot concept by limiting not only what part of the filesystem a process can access, but also virtualizing a set of users and networking. They work well as light-weight virtualization/containers. Elements of a jail: * a directory tree (like chroot) * a host name for the jail * an IP address for the jail (often an alias for a real interface) * an executable to run inside the jail (its path relative to the jail root) ## IPFW and NAT for Jails ### If we have only a single public IP, we can NAT to private IP's for the jails. Add to /etc/rc.conf (modified as appropriate for our particular jails and NIC): gateway_enable="YES" firewall_enable="YES" firewall_script="/etc/ipfw.rules" firewall_nat_enable="YES" firewall_nat_interface="vtnet0" cloned_interfaces="lo1" ifconfig_lo1="inet 172.16.0.1/24" Add nat rules to /etc/ipfw.rules (or wherever we keep our ipfw rules), with outbound rules that should include our jails using 'skipto' rather than 'pass': add="ipfw -q add" ipfw -q -f flush pubnic="vtnet0" pubip="10.0.1.55" ipfw -q nat 1 config if $pubnic reset $add 10 pass all from any to any via lo0 $add 25 pass all from any to any via lo1 $add 50 deny ip from any to any not antispoof in $add 100 nat 1 ip from any to any via $pubnic in $add 110 check-state $add 1000 skipto 10000 udp from any to any 53 out via $pubnic keep-state $add 1010 skipto 10000 tcp from any to any 53 out via $pubnic keep-state $add 1100 skipto 10000 tcp from any to any 80,443 out via $pubnic keep-state $add 1300 skipto 10000 udp from any to any 123 out via $pubnic keep-state $add 2000 pass tcp from any to any established $add 5200 pass tcp from any to any 80 keep-state $add 5300 pass tcp from any to any 443 keep-state $add 6100 pass tcp from any to $pubip 22 in keep-state $add 6200 pass tcp from me to any 25 out keep-state $add 6300 pass icmp from any to any via $pubnic out keep-state $add 9999 deny all from any to any $add 10000 nat 1 ip from any to any via $pubnic out $add 10001 pass ip from any to any In /etc/sysctl.conf: net.inet.ip.fw.one_pass=0 ## iocage ## iocage(8) https://github.com/iocage/iocage https://dan.langille.org/2015/03/07/getting-started-with-iocage-for-jails-on-freebsd/ iocage is newer than ezjail. Like ezjail, iocage simplified management of jails. iocage requires zfs. [EDIT: ezjail can also add IP addresses ad hoc.] Unlike with ezjail, iocage handles setting up its own ip aliases, so _don't_ add `ifconfig_lo1_alias` lines to /etc/rc.conf. # pkg install iocage # iocage activate MYZFSPOOLNAME # iocage fetch # iocage create -c tag=myjail ip4_address="lo1|172.16.0.2/24" # iocage list If we need to change anything before spinning up the jail, we can chroot into it: # iocage chroot myjail Start the jail: # iocage start myjail Access the console of a running jail: # iocage console myjail Set the jail to start at boot: # iocage set boot=on myjail Delete a jail: # iocage destroy myjail ## ezjail ## See ezjail(7). The ezjail utility simplifies setup and administration of jails significantly. # pkg install ezjail Add to /etc/rc.conf: ezjail_enable="YES" Create a jail base and template with: # sudo ezjail-admin install By default, it fetches files by ftp. As an alternative to ftp, install from the disc1.iso: # wget http://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.2/FreeBSD-10.2-RELEASE-amd64-disc1.iso # mdconfig -a -t vnode -f FreeBSD-10.2-RELEASE-amd64-disc1.iso # mount -t cd9660 /dev/md0 /mnt # ezjail-admin install -h file://mnt/usr/freebsd-dist # umount /mnt # mdconfig -d -u 0 Unless $ezjail_jaildir specifies otherwise, jails will reside under /usr/jails, so keep in mind the disk space of /usr. Create a jail: # ezjail-admin create test1 'lo1|172.16.0.2' Start the jail: # ezjail-admin start test1 Connect to the jail's console: # exjail-admin console test1 Create /etc/resolv.conf in the jail to get name resolution. Remember to change the jail's root password! # ezjail-admin console dnsjail # passwd Changing local password for root New Password: Retype New Password: And set the jail's timezone with `tzsetup`. Create a second jail: test2 with IP 172.16.0.3, etc. Update the jail base to the latest patched version of the host: # ezjail-admin update -u ## HAProxy to Two Jailed nginxes ## If we have a single public IP and want to host multiple jailed, nat'd web apps without resorting to alternate port numbers, we can use HAProxy to handle the requests. Public IP | +-------+--------- FreeBSD Host -------------------+ | | | | NAT----------+---------+------HAProxy | | | | | | | | | | +-----Jail1--+--+ +--+--Jail2-----+ | | | | | | | | | nginx | | nginx | | | | | | | | | +---------------+ +---------------+ | | | +--------------------------------------------------+ Install nginx in the jails: # pkg install nginx Edit/create the jails' /etc/rc.conf: nginx_enable="YES" Edit the jails' /usr/local/etc/nginx/nginx.conf: user www; worker_processes 1; # Set to number of cpu's. Start the jails' nginx: # service nginx start Now, on the host, we install haproxy: # pkg install haproxy Edit the host's /etc/rc.conf: haproxy_enable="YES" Note the existence of /usr/local/share/doc/haproxy/. HAProxy config in /usr/local/etc/haproxy.conf: global daemon maxconn 1024 defaults mode http timeout connect 5000ms timeout client 50000ms timeout server 50000ms frontend http bind 10.0.1.55:80 default_backend test1 acl test1 hdr(host) -i test1.example.com acl test2 hdr(host) -i test2.example.com use_backend test1 if test1 use_backend test2 if test2 backend test1 server test1 172.16.0.2:80 backend test2 server test2 172.16.0.3:80 And on our host: # service haproxy start For testing purposes, we'll add this to our workstation's /etc/hosts (where 10.0.1.55 is the public IP of our FreeBSD box): 10.0.1.55 test1.example.com 10.0.1.55 test2.example.com Edit /usr/local/www/nginx/index.html in each of the jails, so we can tell that different pages are actually getting served, and point our browser at http://test1.example.com and http://test2.example.com. ## Further Security ## In some cases, unprivileged users can escape from a jail by cooperating with a user outside the jail. Mitigate this by denying access to the jail root for unprivileged users on the host system (and privileged users in the jail should not have accounts on the host system).