paulgorman.org/technical

OpenBSD as an IPsec VPN Server

UPDATE Some of the below is slightly wrong or outdated. See my IPsec notes instead.

(2015)

These notes describe setting up an OpenBSD IPsec VPN server as a Linux KVM guest. The OpenBSD box will be purely an IPsec/VPN gateway. The firewall of our LAN forwards traffic from the remote peers to our internal OpenBSD box, which acts as the VPN end point.

             Remote Routers/VPN Peers
     +--------+    +--------+    +--------+
     | Peer 1 |    | Peer 2 |    | Peer 3 |    (any IPsec peers, Cisco routers, etc.)
     +--------+    +--------+    +--------+
          |             |            |
          |             |            |
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        (       Public Internet        )
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                        |
                        |
                +----------------+
                |    Firewall    |    (doing port forwarding and NAT, etc.)
                |    10.0.0.1    |
                +----------------+
                        |
     ~ Port forward traffic from VPN Peers ~
                        |
      +-----------------------------------+
      |    OpenBSD VPN Endpoint/Server    |    (a virtual machine)
      |             10.0.0.50             |
      +-----------------------------------+
                        |
                   ~~~~~~~~~~~
                  (    LAN    )
                   ~~~~~~~~~~~

IPsec Overview/Background

IPsec includes a pair of protocols: ESP (encapsulating security payload) and AH (authentication header). ESP provides encryption, authentication, and integrity (or optionally it can provide only encryption while relying on AH for authentication and integrity). AH provides only authentication and integrity (and can optionally work with ESP for encryption).

IPsec can operate in either Transport or Tunnel mode.

Transport mode is generally used for end-to-end (i.e. client-server or client-client) communication. In Transport mode, AH provides authentication and integrity guarantees for packets, but not confidentiality (it doesn’t encrypt IP headers or data payloads) — the traffic can be seen by third parties but not modified. In Transport mode, ESP provides confidentiality by encrypting the data payload of packets; it can configured to also provide authentication and integrity itself, or it can rely upon AH for authentication and integrity. Transport mode (with AH or ESP) does not encrypt the IP headers at all.

Tunnel mode is generally used for network-to-network or host-to-network communication (e.g. VPN’s between routers). Tunnel mode encrypts both the IP headers and payload. Tunnel mode can use ESP alone to provide encryption, authentication, and integrity; or ESP with AH to provide authentication and integrity.

We are concerned in these notes with Tunnel mode using ESP.

An SA (security association) describes a simplex connection. A pair of SA’s are needed to connect two boxes (one for A to B, and a second for B to A). An SA is comprised of: an SPI (security parameter index), a destination address, and a protocol (ESP or AH). SA’s are established by ISAKMP (internet security association and key management protocol, which is the messaging format for IKE).

IPsec keeps an SAD (security association database) and an SPD (security policy database). The SPD holds all the IPsec rules†; and reviews every packet (including non-IPsec traffic). The SAD stores all active SA’s.

During traffic processing, each packet is evaluated according to the rules in the SPD. If a rule identifies a packet for IPsec processing, the SAD is asked for a matching SA. If SAD returns a matching SA, the packet undergoes IPsec processing. If no matching SA is found in the SAD, the packet is dropped but IKE (internet key exchange) is kicked off to create the SA. (IKE is the protocol for key exchange and negotiation of SA’s during Phase 1 and Phase 2.)

† OpenBSD calls the policies in the SPD “flows”. Here we can see the flows/SPD entries and the SAD entries:

bsd #  ipsecctl -s all 
FLOWS:
flow esp in from 10.0.46.0/24 to 10.0.0.0/24 peer 20n.nn.219.190 srcid 10.0.0.50/32 dstid 20n.nn.219.190/32 type use
flow esp out from 10.0.0.0/24 to 10.0.46.0/24 peer 20n.nn.219.190 srcid 10.0.0.50/32 dstid 20n.nn.219.190/32 type require
flow esp in from 10.0.10.0/24 to 10.0.0.0/24 peer 20n.nn.217.230 srcid 10.0.0.50/32 dstid 20n.nn.217.230/32 type use
flow esp out from 10.0.0.0/24 to 10.0.10.0/24 peer 20n.nn.217.230 srcid 10.0.0.50/32 dstid 20n.nn.217.230/32 type require
flow esp in from 10.0.1.0/24 to 10.0.0.0/24 peer 20n.nn.204.134 srcid 10.0.0.50/32 dstid 20n.nn.204.134/32 type use
flow esp out from 10.0.0.0/24 to 10.0.1.0/24 peer 20n.nn.204.134 srcid 10.0.0.50/32 dstid 20n.nn.204.134/32 type require

SAD:
esp tunnel from 20n.nn.217.230 to 10.0.0.50 spi 0x338d7414 auth hmac-sha1 enc 3des-cbc
esp tunnel from 10.0.0.50 to 20n.nn.217.230 spi 0x58db0d95 auth hmac-sha1 enc 3des-cbc
esp tunnel from 20n.nn.204.134 to 10.0.0.50 spi 0x5dfa6164 auth hmac-sha1 enc des-cbc
esp tunnel from 20n.nn.219.190 to 10.0.0.50 spi 0x7247a33a auth hmac-sha1 enc aes
esp tunnel from 10.0.0.50 to 20n.nn.204.134 spi 0x96c0978c auth hmac-sha1 enc des-cbc
esp tunnel from 10.0.0.50 to 20n.nn.219.190 spi 0xff779a61 auth hmac-sha1 enc aes

Life of an IPsec Tunnel

  1. “Interesting” traffic kicks off an IPsec session. Traffic is considered interesting when it matches rules (pf rules, IOS ACL’s, whatever) on the IPsec peers.
  2. In IKE phase 1, the peers authenticate each other, and set up a secure channel for phase 2. Phase 1 can occur in one of two modes:
    1. Main mode has three exchanges: negotiate which algorithms/hashes and lifetimes to use, generate seeds and nonces for later secret key creation (this is where the Diffie-Hellman groups like “group 1” or “group 5” come in), and verify the other side’s identity based on IP address. These three exchanges serve to establish a protected channel for subsequent ISAKMP exchanges between the peers.
    2. Aggressive mode jams all that into a single exchange. It’s slightly faster than main mode, but slightly less secure, because eavesdroppers could sniff some information, like the unencrypted ID payload that contains the peer’s IP address.
  3. In IKE phase 2, each peer negotiates details of the pair of SA’s, and adds the SA’s to their SAD’s.
  4. Communication happens between peers. Data is exchanged. Packets are encrypted and decrypted as specified by the stored SA’s.
  5. The IPsec session terminates. This happens by deletion of the SA’s or session timeout. Note that it is possible for for new SA’s to be generated in advance of an anticipated session timeout, so that flow continues uninterrupted.

Installing the Virtual Machine

Download cd56.iso from a nearby OpenBSD mirror. See also Getting the OpenBSD distribution.

Read the “Requirements & Getting Started” and “Network Setup” sections from my Linux Virtualization (KVM) notes.

Create a logical volume to hold the virtual machine:

linux #  vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "vg0" using metadata type lvm2
linux #  lvcreate --size 6G --name vpn-server vg0
  Logical volume "vpn-server" created

Now, start the installer:

linux #  virt-install --connect qemu:///system --name=openbsd-vpn-server --ram=2048 --vcpus=1,maxvcpus=2 \
	--cdrom=/home/me/Downloads/openbsd-5.6.iso --disk path=/dev/vg0/vpn-server --network bridge=br0,mac=RANDOM --cpu=host

The OpenBSD installer is straightforward. The only mildly tricky part is disk partitioning. For our purposes, we’ll make one large root partition.

Once we complete the basic installation, complete general post-install setup and install any additional packages preferred, such as:

bsd # export PKG_PATH=ftp://openbsd.mirrors.pair.com/5.6/packages/`machine -a`/
bsd # pkg_add vim-7.4.135p2-no_x11-perl-python-ruby
bsd # pkg_add pftop
bsd # pkg_add git
bsd # pkg_add colorls

(tmux is already part of the base install.)

or adding a ~/.kshrc:

export PS1='--- \# --- \h \w \$ '
export PAGER='less -gJ'
export LC_CTYPE=en_US.UTF-8
alias l='colorls -G'
alias ll='colorls -Gl'
alias la='colorls -Ga'
alias h='history -60 | sort -k2 | uniq -f2 | sort -bn'

(You may need a line like export ENV=$HOME/.kshrc in your ~/.profile.)

OpenBSD has extremely helpful man pages. The /etc/examples directory has useful config examples.

OpenBSD release, stable, and current

OpenBSD make a Release every six months, usually in May and November. Small fixes and urgent security updates are made to Stable. Have a look at the Release Errata (i.e. the changes between Release and Stable), and see the instructions for following Stable. The bleeding edge nightly developer builds are Current.

Do an initial pull of the tree:

#  cd /usr 
#  cvs -qd anoncvs@anoncvs.ca.openbsd.org:/cvs get -rOPENBSD_5_6 -P src

Whenever you want to update the tree with the latest changes thereafter:

# cd /usr/src
# cvs -d anoncvs@anoncvs1.usa.openbsd.org:/cvs up -rOPENBSD_5_6 -Pd

The upgrade/update involves two steps: upgrading the kernel, and upgrading userland. You may need to reboot between upgrading the kernel and upgrading userland. Note that GENERIC is the single processor kernel; I believe GENERIC.MP is the multi-processor one.

# cd /usr/src/sys/arch/amd64/conf/
# /usr/sbin/config GENERIC
# cd /usr/src/sys/arch/amd64/compile/GENERIC
# make clean && make
# cd /usr/src/sys/arch/amd64/compile/GENERIC
# make install
# reboot

After reboot, rebuild the binaries:

# rm -rf /usr/obj/*
# cd /usr/src
# make obj
# cd /usr/src/etc && env DESTDIR=/ make distrib-dirs
# cd /usr/src
# make build

Preparing the pf packet filter for IPsec

Add to /etc/pf.conf:

bsd # Don't filter IPsec tunnel traffic (at least until we know VPNs work!) 
set skip on enc0

bsd # Customize this to match your network card:
ext_if = "re0"

bsd # The "any" below assumes we're behind a firewall in a trusted network. This
bsd # would be better as an explicit # list of peer IP's.
ipsec_peers = "any"

pass in on $ext_if proto udp from $ipsec_peers to ($ext_if) port { 500 4500 }
pass out on $ext_if proto udp from $ext_if to $ipsec_peers port { 500 4500 }

pass in on $ext_if proto esp from $ipsec_peers to ($ext_if)
pass out on $ext_if proto esp from $ext_if to $ipsec_peers

Test the config file with: pfctl -nf /etc/pf.conf. If it doesn’t return any warnings, load the new config with pfctl -f /etc/pf.conf.

Show the current pf rules: pfctl -sr.

Show the current pf states: pfctl -ss.

Show the pf stats and counters: pfctl -si.

Show a ton of crap: pfctl -sa.

For testing purposes, pf can be disabled with pfctl -d and enabled with pfctl -e.

Perimeter Firewall Settings

You likely also need to forward some ports on your firewall from the external interface to our new VPN server on the inside. Forward port 500 UDP traffic (ISAKMP), port 4500 UDP traffic (NAT-T), and ESP traffic (IP protocol 50).

You may also need to add or modify routes, so the machines on your network know that the OpenBSD server is the gateway for the remote VPN sites.

Configuring IPsec

Stock OpenBSD ships with all the necessary bits to support IPsec VPN’s. Here are the components of interest:

(If you are interested in IPsec gateway failover, have a look at the manual pages for sasyncd and sasyncd.conf.)

Edit /etc/sysctl to make sure the options get turned on at boot (though the first two should be activated by default):

net.inet.esp.enable=1     # Enable the ESP IPsec protocol
net.inet.ah.enable=1      # Enable the AH IPsec protocol
net.inet.ip.forwarding=1  # Enable IP forwarding for the host. Set it to '2' to forward only IPsec traffic

sysctl displays and sets kernel options. For example, `sysctl net.inet.ip.forwarding will show you the current state of these switches. Change it on the fly like sysctl net.inet.ip.forwarding=0. Such changes will not survive a reboot unless they’re added to /etc/sysctl.

See man enc. The enc interface is a virtual interface for IPsec traffic. It allows packet filtering using pf; prior to encapsulation and after decapsulation, packets can be monitored with tcpdump.

bsd # ifconfig enc0 up
bsd # echo "up" & /etc/hostname.enc0

We’ll be using a pre-shared key for authentication to keep this simple (though this isn’t the best option for production). Edit /etc/ipsec.conf.

ike esp from 10.0.0.0/24 to 10.0.10.0/24 \
    main auth hmac-sha1 enc 3des group modp1024 lifetime 86400 \
    quick auth hmac-sha1 enc 3des group modp1024 lifetime 3600 \
    peer 192.148.217.230 \
    psk 1234AAA-SeCrEt-kEY-ABB1234 \
    tag my_site_name

Make sure to chmod 0600 /etc/ipsec.conf, or ipsecctl will bitch about it.

Also, the phase 1 and phase 2 lifetimes must match the other end of the tunnel, or you could run into problems with dropped/hung VPN’s.

Have ipsecctl tell isakmpd to establish the VPN:

$  ipsecctl -f /etc/ipsec.conf

Errors will be logged to /var/log/daemon.

Add to /etc/rc.conf.local, so this isn’t lost after a reboot:

isakmpd_flags="-K"    # Avoid keynote(4) policy checking
ipsec=YES             # Load ipsec.conf(5) rules

Monitoring and Status

tail /var/log/daemon
ipsecctl -s all
ipsecctl -vs all
pfctl -s info
pfctl -vgs rules
tcpdump -netttr /var/log/pflog

Routes

VPN routes should be created automatically. To see such routes:

$  netstat -rnf encap

Managing VPN’s

Take down all the tunnels with ipsecctl -F (i.e. Flush). This shuts down the flows. Rules are dumped from the SPD, and SA’s are flushed from the SAD.

Start all the tunnels with ipsecctl -f /etc/ipsec.conf.

If you want to manage individual tunnels, it’s important to break them out during setup from ipsec.conf into their own config files (and then include each of those config files in /etc/ipsec.conf like include "/etc/IPsec/foo.conf"). Remember to chmod 600 foo.conf!

To start one tunnel: ipsecctl -f /etc/ipsec/foo.conf. This assumes the tunnel is cleanly down (no flows, no SA’s).

Assuming you have each tunnel broken out into its own config file as mentioned above, kill the tunnel with:

$  ipsecctl -df /etc/ipsec/foo.conf

If that doesn’t work, however, the tunnel might have hung during phase 1 or phase 2, which makes killing it more difficult. In that case, you have to kill it old-style, by sending signals to isakmpd. See this and man isakmpd.

Moving to the Virtual Machine to Production

If we’ve been building this virtual machine on a test host, and now want to move it to a production host, first shutdown our OpenBSD guest, and then:

linux #  dd if=/dev/vg0/vpn-server of=/tmp/vpnserver.img bs=16M

We can then copy the .img file to our production host, and dd the .img onto a new logical volume. The /etc/libvirt/qemu/machinename.xml may need some changes, particularly the CPU section and the storage location. virt-xml-validate machinename.xml tests the validity of the XML file.

”” linux $ virt-xml-validate /tmp/openbsd-vpn-server.xml /tmp/openbsd-vpn-server.xml validates linux # cp /tmp/openbsd-vpn-server.xml /etc/libvirt/qemu/ linux # virsh define /etc/libvirt/qemu/openbsd-vpn-server.xml Domain openbsd-vpn-server defined from openbsd-vpn-server.xml

12:50 kosmokrator ~ $ sudo virsh list –all

Id Name State

1 terminal running 2 openbsd-vpn-server shut off

linux # virsh start openbsd-vpn-server “`

Of course, it’s also possible (and possibly easier) to use virsh migrate.

Cisco

If your other end is a Cisco router, these might be useful:

show crypto engine connections active

References

OpenBSD man pages tend to be helpful. See the man pages for ipsec, ipsecctl, ipsec.conf, and isakmpd.