OpenBSD Virtual Server on Vultr Runbook
=======================================

I started these notes in May 2016.
Last revised January 2018.

This runbook describes a relatively simple OpenBSD web server set up as a small vultr.com instance.
We use Let's Encrypt for SSL certificates, and Nginx to do SSL termination and as a reverse proxy.


UPDATES
------------------------------------------------------------------------

OpenBSD 6.1 introduced `syspatch`, and official binary patch system for core.
This eliminates the need for the third-party `openup` utility.

	$ doas syspatch

To update non-core binary packages:

	$ doas pkg_add -Uu

- http://man.openbsd.org/syspatch
- http://man.openbsd.org/pkg_add


Initial VM Setup
------------------------------------------------------------------------

Create a Vultr account, and fund it.

Add an ISO to Vultr, such as:
http://openbsd.cs.toronto.edu/pub/OpenBSD/5.9/i386/install59.iso

Install through the Vultr noVNC web console.
We need to connect to a high port for noVNC to work.

Configure the new VM's network via DHCP.

Use a custom disklabel partition scheme, since disk space is relatively tight on our VPS:

    /        1G 4.2BSD
    swap     1G swap
    /usr     4G 4.2BSD
    /home    2G 4.2BSD
    /var     7G (the remainder) 4.2BSD

Since we don't want to install X stuff, do `-x*` during set selection.

Add a user during install.
The install should make that first user part of the `wheel` group.

After install, remove the ISO in the Vultr control panel.
Servers -> Instance -> Server Information -> Settings -> Custom ISO -> Remove ISO

Restart.


Post-install
------------------------------------------------------------------------

Read `afterboot(8)`.

Set up `doas` (the OpenBSD `sudo` replacment) by editing `/etc/doas.conf`:

    permit nopass keepenv { PKG_PATH ENV PS1 SSH_AUTH_SOCK } :wheel

(Omit the `nopass` if we want it to prompt for passwords.)

Add our client's ssh keys to `~/.ssh/authorized_keys`.

Generate an ssh key for our user:

    $ ssh-keygen -b 4096

Disable password-based authentication by editing /etc/ssh/sshd_config:

    PasswordAuthentication no

Reload the sshd config:

    $ doas /etc/rc.d/sshd reload

Write a /etc/pf.conf:

    # $OpenBSD: pf.conf,v 1.54 2014/08/23 05:49:42 deraadt Exp $
    # See pf.conf(5) and /etc/examples/pf.conf
    ext_if = "vio0"
    tcp_pass_in = "{ 80 443 22 }"
    tcp_pass_out = "{ 80 443 22 53 123 43 67 68 }"
    udp_pass_out = "{ 53 123 67 68 }"
    table <friendly_ip_addrs> { nnn.0.nnn.153, nnn.148.nnn.214, nn.49.nn.170 }
    set skip on lo0
    match in all scrub (no-df random-id max-mss 1440)
    block in all
    pass out all
    pass in proto tcp to port $tcp_pass_in
    pass in on $ext_if from <friendly_ip_addrs>

(Outbound should probably be more restrictive!)

Check the syntax and reload pf:

    $ doas pfctl -nf /etc/pf.conf
    $ doas pfctl -f /etc/pf.conf

Install the ports collection:

    $ cd /tmp
    $ ftp http://ftp.openbsd.org/pub/OpenBSD/$(uname -r)/ports.tar.gz
    $ ftp http://ftp.openbsd.org/pub/OpenBSD/$(uname -r)/SHA256.sig
    $ signify -Cp /etc/signify/openbsd-$(uname -r | cut -c 1,3)-base.pub -x SHA256.sig ports.tar.gz
    $ cd /usr
    $ doas tar xzf /tmp/ports.tar.gz

(The ports tree should use less than 400 MB at install.)

Edit ~/.profile:

    PATH=$HOME/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games:.
    export PATH HOME TERM
    export PKG_PATH=http://openbsd.cs.toronto.edu/pub/OpenBSD/$(uname -r)/packages/$(machine -a)/
    export HISTSIZE=1000
    export HISTFILE=$HOME/.history
    export PS1='--- \! --- \h \w \$ '
    export PAGER='less -gJ'
    export LC_CTYPE=en_US.UTF-8
    export LSCOLORS=gxfxcxdxbxxggdabagacad
    alias l='ls -F'
    alias ll='ls -Fl'
    alias la='ls -Fa'
    alias h='history -60 | sort -k2 | uniq -f2 | sort -bn'

After we set $PKG_PATH in our ~/.profile:

    $ cd /usr/ports
    $ make search key='^vim.*no_x11'
    $ doas pkg_add vim-7.4.900-no_x11
    $ doas pkg_add git pftop wget

(Note that package installation notes/warning/instructions are saved in /usr/local/share/doc/pkg-readmes/.)

Create ~/.tmux.conf:

    set-option -g default-terminal "screen-256color"
    set-option -g default-shell /bin/ksh
    set -g status-bg "colour143"
    set -g status-fg "black"
    set -g prefix C-a
    unbind C-b
    bind-key C-a send-prefix
    set -g history-limit 10000
    set -g base-index 1
    set-window-option -g mode-keys vi
    bind-key -t vi-copy 'v' begin-selection
    bind-key -t vi-copy 'y' copy-selection
    unbind-key j
    bind-key j select-pane -D
    unbind-key k
    bind-key k select-pane -U
    unbind-key h
    bind-key h select-pane -L
    unbind-key l
    bind-key l select-pane -R
    bind-key -n M-F1 select-window -t :1
    bind-key -n M-F2 select-window -t :2
    bind-key -n M-F3 select-window -t :3
    bind-key -n M-F4 select-window -t :4
    bind-key -n M-F5 select-window -t :5
    bind-key -n M-F6 select-window -t :6
    bind-key -n M-F7 select-window -t :7
    bind-key -n M-F8 select-window -t :8
    bind-key -n M-F9 select-window -t :9

Set up mail alias to our user account. Edit /etc/mail/aliases:

    root: paulgorman

and run:

    $ doas newaliases


Web Server Setup
------------------------------------------------------------------------

[These notes were revised in January 2018 when I started using Nginx as a reverse proxy for OpenBSD's httpd.]

Set up Nginx to do SSL termination and reverse proxying:

	$ doas pkg_add nginx

	$ doas vim /etc/nginx/nginx.conf
	worker_processes  3;

	worker_rlimit_nofile 1024;
	events {
		worker_connections  800;
	}

	http {
		include       mime.types;
		default_type  application/octet-stream;
		charset       utf-8;
		index         index.html index.htm;

		keepalive_timeout  65;

		server_tokens off;

		server {
			listen       80;
			listen       [::]:80;

			location ^~ /.well-known/acme-challenge/ {
				proxy_pass   http://127.0.0.1$request_uri;
			}

			location / {
				return       301 https://$host$request_uri;
			}
		}

		server {
			listen       443;
			listen       [::]:443;

			ssl                  on;
			ssl_certificate      /etc/ssl/letsencrypt/fullchain.pem;
			ssl_certificate_key  /etc/ssl/letsencrypt/private/privkey.pem;
			ssl_session_timeout  5m;
			ssl_session_cache    shared:SSL:1m;
			ssl_ciphers  HIGH:!aNULL:!MD5:!RC4;
			ssl_prefer_server_ciphers   on;

			location / {
				proxy_set_header Host $http_host;
				proxy_pass   http://127.0.0.1/;
			}
		}
	}

	$ doas rcctl enable nginx
	$ doas rcctl start nginx

Set up OpenBSD's httpd (see `httpd.conf(5)` and `/etc/examples/httpd.conf`):

	$ doas vim /etc/httpd.conf
	#### Macros ####
	ext_addr="127.0.0.1"
	#### Global Options ###
	prefork 3
	types {
		include "/usr/share/misc/mime.types"
	}
	#### Servers ####
	server "example.com" {
		root "/example.com"
		listen on $ext_addr port 80
		directory {
			auto index,
			index "index.html"
		}
		location "/" {
			block return 302 "/blog/"
		}
		location "/index.html" {
			block return 302 "/blog/"
		}
		location "*.php" {
			fastcgi socket "/run/php-fpm.sock"
		}
		location "/ttrss/*" {
			authenticate with "/htpasswd"
			directory index "index.php"
		}
		location "/.well-known/acme-challenge/*" {
			root {
				"/acme"
				strip 2
			}
		}
	}
	server "example.org" {
		root "/example.org"
		listen on $ext_addr port 80
		directory {
			auto index,
			index "index.html"
		}
		location "/" {
			block return 302 "/technical/blog/"
		}
		location "/index.html" {
			block return 302 "/technical/blog/"
		}
		location "*.php" {
			fastcgi socket "/run/php-fpm.sock"
		}
		location "/blog/*" {
			directory index "index.php"
		}
		location "/.well-known/acme-challenge/*" {
			root {
				"/acme"
				strip 2
			}
		}
	}
	server "www.example.com" {
		listen on $ext_addr port 80
		block return 301 "http://example.com$REQUEST_URI"
	}
	server "www.example.org" {
		listen on $ext_addr port 80
		block return 301 "http://example.org$REQUEST_URI"
	}

	$ doas rcctl enable httpd
	$ doas rcctl start httpd


### PHP Setup ###

It seems that php-fpm was merged into the php packages in OpenBSD 5.9.
Don't choose the "-ap2" php packages; they're for Apache.

    $ cd /usr/ports
    $ make search key='^php' | grep '^Port' | sort
    $ doas pkg_add php-5.6.18 mariadb-server mariadb-client php-mysql-5.6.18 php-mcrypt-5.6.18 php-curl-5.6.18
    $ doas rcctl enable php56_fpm

Enable MySQL and mcrypt and curl support in PHP:

    $ doas ln -s /etc/php-5.6.sample/mysql.ini /etc/php-5.6/mysql.ini
    $ doas ln -s /etc/php-5.6.sample/mcrypt.ini /etc/php-5.6/mcrypt.ini
    $ doas ln -s /etc/php-5.6.sample/curl.ini /etc/php-5.6/curl.ini

Add to `/etc/my.cnf`:

	expire_logs_days = 3
	max_binlog_size = 25600000

Start php and restart httpd:

    $ doas rcctl start php56_fpm
    $ doas rcctl restart httpd


### Let's Encrypt SSL ###

[As of OpenBSD 6.0 (late 2016) the below stuff, especially Let's Encrypt, is changing. Until I can revise these notes, see https://paulgorman.org/diary/20161013.html.]

Install Let's Encrypt:

    $ doas pkg_add letsencrypt
    $ doas letsencrypt certonly --webroot \
    -w /var/www/example.com/ -d example.com -d www.example.com \
    -w /var/www/example.org/ -d example.org -d www.example.org -d test.example.org

The certs are saved under `/etc/letsencrypt/live/example.com/`.


### Tiny Tiny RSS ###

Generate a test htpasswd file. The file must be readable by the webserver user.

    $ doas htpasswd /var/www/htpasswd myuser
    $ doas chown root:www /var/www/htpasswd
    $ doas chmod 640 /var/www/htpasswd

And:

    $ doas /usr/local/bin/mysql_install_db
    $ doas rcctl enable mysqld
    $ doas rcctl start mysqld
    $ doas /usr/local/bin/mysql_secure_installation


Install tt-rss.

    $ mysql -uroot -p
    MariaDB [(none)]> CREATE DATABASE ttrss;
    MariaDB [(none)]> CREATE USER 'ttrss'@'127.0.0.1' IDENTIFIED BY '*******************';
    MariaDB [(none)]> GRANT ALL ON ttrss.* TO 'ttrss'@'127.0.0.1';
    MariaDB [(none)]> FLUSH PRIVILEGES;
    $ mysql -uroot -p ttrss < ttrss.sql
    $ tar -zxf ttrss-html.tgz
    $ doas mv ttrss /var/www/rss.devilghost.com
    $ doas chown -R root:daemon /var/www/rss.devilghost.com
    $ doas htpasswd /var/www/htpasswd myuser
    $ doas chown root:daemon /var/www/htpasswd
    $ doas chmod g+r /var/www/htpasswd
    $ doas mkdir /var/www/letsencrypt-rss.devilghost.com

(Use "127.0.0.1" rather than "localhost". 127.0.0.1 uses TCP sockets; localhost uses unix sockets. Unix sockets would be more efficient, but needs a lot more configuration because httpd is chroot'd. We'd have to tell MySQL to write its socket file to /var/www/run/mysql/mysql.sock, and tell PHP to look there.)

Set up a cron job for the www user to update the tt-rss feeds:

    $ doas crontab -u www -e

    */89  *   *    *    *   cd /var/www/devilghost.com/ttrss && /usr/local/bin/php-5.6 /var/www/devilghost.com/ttrss/update.php --feeds --qui
et >/dev/null 2>&1

(Sigh. This will break when the php version changes, but I don't see an obvious way I can avoid it.)

For convenience:

    $ doas chown -R paulgorman:www /var/www/paulgorman.org/
    $ doas chown -R paulgorman:www /var/www/devilghost.com/


Backups
------------------------------------------------------------------------

Set up backups. Create ~/bin/backup.sh:

    #!/bin/sh
    BOX=$(hostname -s)
    DAY=$(date +'%d')
    /usr/local/bin/mysqldump -u root --password='***********************' --all-databases > /var/backups/${BOX}-alldatabases.sql
    chmod 640 /var/backups/${BOX}-alldatabases.sql
    tar -czf /var/backups/backup-${BOX}-${DAY}.tgz \
    /etc \
    /var/cron \
    /var/www/paulgorman.org \
    /var/www/devilghost.com \
    /var/www/cgi-bin \
    /var/www/htpasswd \
    /var/backups/${BOX}-alldatabases.sql \
    /home/paulgorman/.ssh \
    /var/repo \
    /home/paulgorman/bin
    chmod 640 /var/backups/backup-${BOX}-${DAY}.tgz

Create root's cron jobs:

    1  3  2,16  *  *    /home/paulgorman/bin/backup.sh

One the remote machine, we create a cron job to pull the backup:

    1 0 3,17 * * scp clyde.devilghost.com:/var/backups/backup-clyde\*tgz /data/share/backups/


Miscellaneous
------------------------------------------------------------------------

Create home for git repositories:

    $ doas mkdir /var/repo
    $ doas chown paulgorman:paulgorman /var/repo/
    $ ln -s /var/repo /home/paulgorman/repo


General OpenBSD Stuff
------------------------------------------------------------------------

Use rcctl(8) to enable and disable services. `rcctl get foo` will show what rc knows about service foo.

Get info on current pf state:

    $ doas pfctl -sr
    $ doas pfctl -ss
    $ doas pfctl -sa

Finding packages:

    $ cd /usr/ports && make search key='foo' | grep 'Port:' && cd -
    $ pkg_info -Q foo

(The first searches all package field for the key; the second only searches package names.)