DigitalOcean FreeBSD Runbook

Note: to access the DigitalOcean web console, open outbound tcp port 5000.

Create a droplet through the DO web tool.

Add our ssh keys.

Log in via ssh, using the user name ‘freebsd’:

% ssh -l freebsd nnn.nnn.nnn.nnn

Add a new user (and make them part of the wheel group):

% sudo adduser

(sudo is install by default, and wheel are sudoers.)

Add ssh keys to new user account:

% sudo mkdir /home/myuser/.ssh
% sudo cp ~/.ssh/authorized_keys /home/myuser/.ssh/
% sudo chown myuser:myuser /home/myuser/.ssh
% sudo chown myuser:myuser /home/myuser/.ssh/authorized_keys

Test to sure we can ssh in and sudo with this account.

Generate ssh keys for this account:

% ssh-keygen -b 4096

Add a minimal .tcshrc for the new account:

set promptchars="%#"
set prompt = "--- %m %h %c %# "
setenv EDITOR vim
setenv VISUAL vim
setenv PAGER less
set autolist
alias l 'ls -FG'
alias la 'ls -FGa'
alias ll 'ls -FGlha'
alias h 'history 60 | sort -k2 | uniq -f2 | sort -bn'
setenv LSCOLORS "gxfxcxdxbxxggdabagacad"
bindkey "^W" backward-delete-word

Install updates:

% sudo freebsd-update fetch
% sudo freebsd-update install
% sudo pkg update
% sudo pkg upgrade

Install out some packages:

% sudo pkg install vim-lite git-lite zsh sshguard-ipfw curl tmux alpine python ezjail haproxy wget

Edit /etc/rc.conf:


Edit /etc/sysctl.conf:


Edit /etc/ipfw.rules:

ipfw -q -f flush
add="ipfw -q add"
ipfw -q nat 1 config if $oif reset unreg_only same_ports
$add 10 allow all from any to any via lo0
$add 20 allow all from any to any via $jails
$add 100 deny all from any to
$add 110 deny ip from to any
$add 120 deny all from any to ::1
$add 130 deny all from ::1 to any
$add 500 deny all from any to any not antispoof in
$add 510 deny all from any to via $oif
$add 520 deny all from any to via $oif
$add 530 deny all from any to via $oif
$add 540 deny all from any to via $oif
$add 550 deny all from any to via $oif
$add 560 deny all from any to via $oif
$add 570 deny all from any to via $oif
# Work main office connection:
$add 990 allow tcp from NNN.N.NNN.NNN to me ssh,https setup keep-state
# sshguard (dyamic rules are added to table 22):
$add 1000 deny ip from table\(22\) to any
$add 2000 allow tcp from any to me ssh setup keep-state
$add 2100 allow tcp from any to me http,https setup keep-state
$add 5000 nat 1 all from any to any in recv $oif
$add 5001 check-state
$add 6000 skipto $natout tcp from any to any $tcpout out xmit $oif keep-state
$add 6100 skipto $natout udp from any to any $udpout out xmit $oif keep-state
$add 8999 deny log all from any to any
$add 9000 nat 1 all from any to any out xmit $oif
$add 9001 allow all from any to any

Set the timezone:

% sudo tzsetup

Select “No” for “Is this machine’s CMOS clock set to UTC?”.

Disable “freebsd” user account:

% sudo pw lock freebsd

Edit /etc/ssh/sshd_config:

PermitRootLogin no

Edit /etc/aliases to send system mail to an account we’ll read:

root: myuser

…and then rebuild the aliases:

% sudo newaliases


Set up jails.

Create the ezjail base:

# wget
# mdconfig -a -t vnode -f FreeBSD-10.2-RELEASE-amd64-disc1.iso
# mount -t cd9660 /dev/md0 /mnt
# ezjail-admin install -h file://mnt/usr/freebsd-dist
# umount /mnt
# mdconfig -d -u 0

Update base and port for jail base:

# ezjail-admin update -u
# ezjail-admin update -P

Create a first jail for MySQL/MariaDB:

# ezjail-admin create 'lo1|'
# ezjail-admin start
# ezjail-admin console

(It’s important that each jail has a FQDN, or weird stuff will go wrong, like local mail delivery.)

Basic jail config:

root@mysql:~ # echo 'nameserver' > /etc/resolv.conf
root@mysql:~ # echo 'nameserver' >> /etc/resolv.conf
root@mysql:~ # tzsetup

And set the root password for the jail (passwd).

# ezjail-admin console mysql
root@mysql:~ # pkg install mariadb100-server mariadb100-client
root@mysql:~ # echo 'mysql_enable="YES"' >> /etc/rc.conf
root@mysql:~ # service mysql-server start

Set up a jail for our default website:

# ezjail-admin create 'lo1|'
# ezjail-admin start

Set the nameservers, timezone, and root password.

root@www:~ # pkg install php56 php56-extensions php56-mysqli php56-mbstring php56-mcrypt nginx vim-lite
root@www:~ # echo 'nginx_enable="YES"' >> /etc/rc.conf
root@www:~ # echo 'php_fpm_enable="YES"' >> /etc/rc.conf

In /usr/local/etc/php-fpm.conf:

pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 2


root@www:~ # service php-fpm start
root@www:~ # service nginx start

Create SSL certs on host (outside jails)

# openssl genrsa -rand -genkey -out /etc/ssl/cert.key 2048
# mkdir /etc/ssl/haproxy
# openssl req -new -x509 -days 2190 -key /etc/ssl/cert.key -out /etc/ssl/ -sha256
# openssl req -new -x509 -days 2190 -key /etc/ssl/cert.key -out /etc/ssl/ -sha256
# cat /etc/ssl/cert.key > /etc/ssl/haproxy/
# cat /etc/ssl/cert.key > /etc/ssl/haproxy/

(We must supply the correct FQDN for cert generation when asked by openssl.)

Edit /usr/local/etc/nginx/nginx.conf:

-        location / {
-            root   /usr/local/www/nginx;
-            index  index.html index.htm;
-        }
+       root   /usr/local/www/;
+       index  index.html index.htm index.php;
+       autoindex on;
+       location ~ \.php$ {
+           include        fastcgi_params;
+           fastcgi_pass;
+           fastcgi_index  index.php;
+           fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
+       }

And actually do mkdir /usr/local/www/

Further nginx considerations:

- No Apache-like .htaccess
- For basic auth, see:
- The order of nginx.conf entries matters. E.g., it stops processing at the *first* matching location with a matching regex.
- We forbid access to sensitive files in /usr/local/etc/nginx/nginx.conf:
- We do only http (80) with nginx. haproxy handles https (443).

    location = /config.php {
        deny all;

Do the same thing to set up the second site:

Outside the jail, edit /usr/local/etc/haproxy.conf:

    maxconn 1024

    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend http
    bind ssl crt /etc/ssl/haproxy/
    acl hdr(host) -i
    acl hdr(host) -i
    use_backend if
    use_backend if



Create databases and users.

# ezjail-admin console mysql
root@mysql:~ # mysql -uroot
MariaDB [(none)]> CREATE DATABASE blog;
MariaDB [(none)]> CREATE DATABASE wp;
MariaDB [(none)]> CREATE DATABASE ttrss;
MariaDB [(none)]> CREATE USER 'blog'@'localhost' IDENTIFIED BY '****************';
MariaDB [(none)]> GRANT SELECT, INSERT, UPDATE, DELETE ON blog.* TO 'blog'@'';
MariaDB [(none)]> CREATE USER 'wp'@'' IDENTIFIED BY '********************';
MariaDB [(none)]> GRANT ALL ON wp.* TO 'wp'@'';
MariaDB [(none)]> CREATE USER 'ttrss'@'' IDENTIFIED BY '********************';
MariaDB [(none)]> GRANT ALL ON ttrss.* TO 'ttrss'@'';

Restore databases:

root@mysql:~ # mysql -u root -p blog < blog-db.sql
root@mysql:~ # mysql -u root -p wp < devilghost-wp-db.sql
root@mysql:~ # mysql -u root -p ttrss < devilghost-ttrss-db.sql

Set MariaDB root password:

MariaDB [(none)]> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('***********************');

Create the .htpasswd file:

--- 174 ~ # echo -n 'paulgorman:' >> /usr/local/etc/nginx/.htpasswd
--- 177 ~ # openssl passwd -apr1 >> /usr/local/etc/nginx/.htpasswd
--- 16 ~ # echo -n 'paulgorman:' >> /usr/local/etc/nginx/.htpasswd
--- 17 ~ # openssl passwd -apr1 >> /usr/local/etc/nginx/.htpasswd

Set up rewrite rule for by adding a location to /usr/local/etc/nginx/nginx.conf:

location ~* /blog/(new|save|all|delete) {
    auth_basic           "Login";
    auth_basic_user_file /usr/local/etc/nginx/.htpasswd;
    rewrite ^/blog/((\w|\/)+)$ /blog/index.php?url=$1;
location ~ /blog {
    rewrite ^/blog/((\w|\/)+)$ /blog/index.php?url=$1;

If we want to add a cron job for a system user in a jail, had root edit the user’s crontabl:

$ crontab -u www -e

Set up backups.

Create /root/bin/, and make it executable:

DAY=$(date +'%d')
tar -czf /home/paulgorman/backups/backup-inky-${DAY}.tgz \
/root/bin \
/etc/rc.conf \
/etc/ipfw.rules \
/etc/sysctl.conf \
/etc/ssl \
/etc/ssh \
/etc/aliases \
/var/cron \
/home/paulgorman/.ssh \
/home/paulgorman/repo \
/usr/local/etc/ \
/usr/jails/*/etc/rc.conf \
/usr/jails/*/root/bin \
/usr/jails/*/usr/local/etc/ \
/usr/jails/*/var/cron \
/usr/jails/*/usr/local/www/ \
chmod 640 /root/backup-inky-${DAY}.tgz

Create a root cron job:

 1  3  2,16  *  *    /root/bin/

Create the target directory:

# sudo mkdir /home/paulgorman/backups
# sudo chmod 740 /home/paulgorman/backups

Set up a cron job on another machine to fetch the backup files.

In the mysql jail, create /root/bin/sqlbackup.sql, and make it executable:

mysqldump -u root --password='***********************' --all-databases > /root/inky-alldatabases.sql
chmod 640 /root/inky-alldatabases.sql

In the mysql jail, create a root cron job:

1  2  2,16  *  *    /root/bin/

.tcshrc for jails

set promptchars=“%#” set prompt = “— %M %h %c %# ” setenv EDITOR vim setenv VISUAL vim setenv PAGER less set autolist alias l ‘ls -FG’ alias la ‘ls -FGa’ alias ll ‘ls -FGlha’ alias h ‘history 60 | sort -k2 | uniq -f2 | sort -bn’ setenv LSCOLORS “gxfxcxdxbxxggdabagacad” bindkey “^W” backward-delete-word

.vimrc for jails

set nocompatible set encoding=utf-8 set viminfo=‘500,f1,<200,:200,@200,/200,s20,h set t_Co=256 syntax off set ignorecase set smartcase set incsearch set showmatch set wildmenu set autoindent set nosmartindent set tabstop=4 set shiftwidth=4 set expandtab set backspace=indent,eol,start set smarttab set ruler set showmode set nu set rnu let mapleader=" " " Default is \ nmap h :set hlsearch! nmap n :set nu! rnu! nmap o ok nmap O Oj imap ii


Add to /etc/rc.conf:


Stop sendmail, and install opensmtpd:

# service sendmail stop
# killall sendmail
# pkg install opensmtpd

Edit /etc/mail/mailer.conf:

sendmail        /usr/local/sbin/smtpctl
send-mail       /usr/local/sbin/smtpctl
mailq           /usr/local/sbin/smtpctl
makemap         /usr/local/libexec/opensmtpd/makemap
newaliases      /usr/local/libexec/opensmtpd/makemap

Edit /usr/local/etc/mail/smtpd.conf:

# See smtpd.conf(5) for more information.
# To accept external mail, replace with: listen on all
listen on localhost
# If you edit the file, you have to run "smtpctl update table aliases"
table aliases file:/etc/mail/aliases
# Uncomment the following to accept external mail for domain ""
#accept from any for domain "" alias <aliases> deliver to mbox
accept for local alias <aliases> deliver to mbox
accept for any relay

Start opensmtpd:

# service smtpd start