paulgorman.org/technical

OpenBSD Mail Smarthost

(April 2018)

Configure OpenBSD as virtual (KVM/libvirt) mail smarthost. Screen for mail spam, viruses, and other badness, then pass the mail to our internal mail server.

Requirements:

Initial VM Install

Grab the OpenBSD ISO (https://www.openbsd.org/ftp.html).

$  touch create-openbsd-smarthost-vm.sh
$  chmod u+x create-openbsd-smarthost-vm.sh
$  cat create-openbsd-smarthost-vm.sh
#!/bin/sh
set -euf
vm=openbsd-6.3-smarthost
qemu-img create -f qcow2 -o preallocation=metadata ~/libvirt/"$vm".qcow2 20G
virt-install \
        --name="$vm" \
        --ram=2048 \
        --vcpus=1 \
        --network bridge=br0,model=e1000 \
        --os-variant openbsd6.2 \
        --cdrom ~/libvirt/openbsd-install63.iso \
        --disk ~/libvirt/"$vm".qcow2

During the brief install process:

Initial Post-Install Setup

If we added a non-root user during install, add them to the wheel group:

#  usermod -G wheel paul

Add an additional user if desired:

#  useradd -mG wheel scott

Create /etc/doas.conf:

permit nopass keepenv :wheel

Create /etc/installurl:

https://ftp.openbsd.org/pub/OpenBSD

Run updates:

#  syspatch
#  pkg_add -Uu
#  pkg_info -Q vim
#  pkg_add vim-8.0.1589-no_x11 bash coreutils amavisd-new amavisd-new-utils p5-Mail-SpamAssassin clamav unrar lz4 tnef

Create /etc/pf.conf (which we’ll add to in the “spamd” section below):

set block-policy drop
set skip on { lo }
block drop in quick from urpf-failed to any
block drop log all
pass in inet proto tcp from any to any port { 22 25 443 587 }
pass in inet proto icmp from 10.0.0.0/24 to self
pass out on egress proto { tcp udp icmp icmp6 } all modulate state

Test the PF rules, then reload:

#  pfctl -vnf /etc/pf.conf
#  pfctl -F all -f /etc/pf.conf

Install any SSH keys, if desired.

In /etc/ssh/sshd_config disable root logins:

PermitRootLogin no

Generate an SSH key for root:

#  ssh-keygen -b 4096

Create /etc/daily.local. See daily(8).

VERBOSESTATUS=0

What services do we need to configure?

$  rcctl ls all | grep -i -e spam -e clam -e amavis -e smtp
amavis_mc
amavisd
clamav_milter
clamd
freshclam
smtpd
spamassassin
spamd
spamd_black
spamlogd

AD/LDAP Integration

If we wanted to integrate with Active Directory, or a similar LDAP service, we could using the openldap-client package. After configuring LDAP, do something like this in smtpd.conf:

table vusers ldap:/etc/mail/ldap.conf
table vdomains ldap:/etc/mail/ldap.conf

However, since we’re using the smarthost as a chokepoint that does not forward mail for just any AD user, we forgo LDAP here.


Spamd

Read the spamd(8) man page.

spamd has three important intervals set with the -G flag, as decribed in the man page spamd(8).

The default values are “25:4:864”.

When a foreign mail server connects for the first time, spamd saves a tupple of foreign server IP address, sender email address, and recipient address. Then, it closes the connection with a temporary error. The “25” value means that spamd ignores further connection attempts from that host for twenty-five minutes. Spamd calls this the passtime.

After twenty-five minutes elapses, spamd will watch for connections matching the recorded tuple for four hours from the original attempt (the second argument to -G, the “4”). If the foreign host tries to resend with the same tuple, between twenty-five minutes and four hours following the original attempt, spamd adds the foreign server IP address to its whitelist. Spamd calls this the greyexp.

A host remains in spamd’s whitelist for 864 hours (i.e., thirty-six days). Spamd calls this the whiteexp.

To change the default values with which spamd starts, add arguments to /etc/rc.conf.local, like:

spamd_flags="-v -4 -G 25:4:864 -h mail.example.com -n \"Sendmail 8.14.4/8.14.4\" -C /etc/ssl/example.com.crt -K /etc/ssl/private/example.com.key"

Edit /etc/pf.conf to add spamd-specific rules:

table <spamd-white> persist
table <nospamd> {10.0.0.11}
set block-policy drop
set skip on { lo }
block drop in quick from urpf-failed to any
block drop log all
pass in on egress inet proto tcp from any to any port smtp divert-to 127.0.0.1 port spamd
pass in on egress proto tcp from <nospamd> to any port smtp
pass in log on egress proto tcp from <spamd-white> to any port smtp
pass in inet proto tcp from any to any port { 22 443 587 }
pass in inet proto icmp from 10.0.0.0/24 to self
pass out on egress proto { tcp udp icmp icmp6 } all modulate state
pass out log on egress proto tcp to any port smtp

Edit /etc/mail/spamd.conf:

all:\
    :nixspam:whitelist:blacklist:whitelist:

nixspam:\
    :black:\
    :msg="Your address %A is in the nixspam list\n\
    See http://www.heise.de/ix/nixspam/dnsbl_en/ for details":\
    :method=http:\
    :file=www.openbsd.org/spamd/nixspam.gz

whitelist:\
    :white:\
    :method=file:\
    :file=/etc/mail/nospamd:

blacklist:\
    :black:\
    :msg="Your address %A is blacklisted by this mail server":\
    :method=file:\
    :file=/etc/mail/blacklist:

(Indent with spaces not tabs.)

Create stub files for the white and black lists:

#  touch /etc/mail/nospamd
#  touch /etc/mail/blacklist

Add to /etc/syslog.conf:

!spamd
*.*                      /var/log/maillog

Turn on spamd:

#  rcctl enable spamd

Edit root’s crontab to enable updates of the spamd lists:

0       *       *       *       *       sleep $((RANDOM \% 2048)) && /usr/libexec/spamd-setup

The daemon spamlogd (see spamlogd(8)) watches the PF log. It’s responsible for adding IP’s to the spamd whitelist in /var/db/spamd. The spamlogd daemon only notices connections logged by PF, so make sure to tag the appropriate rules in /etc/pf.conf with log! Beyond that, spamlogd requires no configuration.

OpenSMTPD

Edit /etc/mail/aliases:

root: alert@example.com

And run:

# newaliases

Since we want to HELO to other mail servers with a name other than our true hostname, create /etc/mail/mailname:

mail.example.com

Since we prefer to work with uncompressed mail logs, and keep logs for longer, edit /etc/newsyslog.conf:

- /var/log/maillog                           640  7     *    24    Z
+ /var/log/maillog                           640  31    *    24

Edit /etc/mail/smtpd.conf:

table aliases file:/etc/mail/aliases
listen on all
accept from any for domain "example.com" relay

(Note that we add to the smtpd.conf as these notes progress.)

If necessary (without a proper internal MX record, for example) we can specify the relay:

accept from any for domain "example.com" relay via "postoffice.example.com"

Restart OpenSMTPD:

# rcctl restart smtpd

At this point, we’re relaying mail.

Now, set up secure mail submission on port 587. Since we’re only doing this for a handful of machine/admin accounts, we authenticate against a simple file Use the smtpctl encrypt command to encrypt passwords for the file.

Create the file /etc/mail/passwd-smtps:

mailuser $2b$09$M1451uuhfn6ZD0Gafd3NPOWioR3gdEodzXn/ZI/Y4sSdjbkm74DNi

And:

#  chmod 0660 /etc/mail/passwd-smtps

Edit /etc/mail/smtpd.conf:

pki example certificate "/etc/ssl/example.com.crt"
pki example key "/etc/ssl/private/example.com.key"
table aliases file:/etc/mail/aliases
table passwd-smtps file:/etc/mail/passwd-smtps
listen on all
listen on all port 587 tls-require pki example auth <passwd-smtps>
accept from any for domain "example.com" relay

Amavis, ClamAV, and SpamAssassin

Why Amavis? Amavis provides a unified interface between the MTA, which speaks SMTP, and various filter programs, many of which do not speak SMTP.

The package installer created an _vscan user and group.

Edit /etc/amavisd.conf. Verify:

$daemon_user  = '_vscan';
$daemon_group = '_vscan';
$inet_socket_port = 10024;

And set:

$mydomain = 'example.com'
$log_level = 1;
$log_recip_templ = $log_verbose_templ;
$sa_tag_level_deflt  = -999.0
$sa_tag2_level_deflt = 3.0;
...
@score_sender_maps = ({
	...
	read_hash("/etc/mail/amavis-senders"),
	...
)};
...
@av_scanners = (
 ['ClamAV-clamd',
   \&ask_daemon, ["CONTSCAN {}\n", "/tmp/clamd.sock"],
   qr/\bOK$/m, qr/\bFOUND$/m,
   qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],
...

Do:

#  touch /etc/mail/amavis-senders

The (many) other options can be left alone or tweaked as desired.

$  cat /etc/amavisd.conf | sed '/^[[:space:]]*#/d' | sed '/^$/d' | wc -l
     315

ClamAV

We have several new config files:

Edit /etc/freshclam.conf.

$  cat /etc/freshclam.conf | sed '/^#/d' | sed '/^$/d'
DatabaseMirror db.us.clamav.net
DatabaseMirror database.clamav.net

Set up a root cron job to update the malware definitions:

20  */4 *   *   *   /usr/local/bin/freshclam >/dev/null 2>&1

Edit /etc/clamd.conf.

$  cat /etc/clamd.conf | sed '/^#/d' | sed '/^$/d'
LocalSocket /tmp/clamd.sock
LocalSocketGroup _vscan
LocalSocketMode 660
User _vscan
MaxRecursion 12

The package installer created a _clamav user and group, but we run as the Amavis user.

SpamAssassin

Leave /etc/mail/spamassassin/local.cf at defaults. We set the score configuration, etc. in the Amavis config.

Set up Backups

#  mkdir -p /root/bin
#  touch /root/bin/backup.sh
#  u+x /root/bin/backup.sh
#  chmod u+x /root/bin/backup.sh

Edit the shell script:

#!/bin/sh
set -euf
host=$(hostname -s)
tar -C / -czf /var/backups/backup-"$host".tgz \
        etc \
        root \
        var/cron

Create a root crontab entry:

10  1   *   *   *   /root/bin/backup.sh

TODO: move backup tgz to a location where the enterprise backup job will see it.

Go!

Final /etc/mail/smtpd.conf:

pki mail.example.com certificate "/etc/ssl/example.com.crt"
pki mail.example.com key "/etc/ssl/private/example.com.key"

table aliases file:/etc/mail/aliases
table passwd-smtps file:/etc/mail/passwd-smtps
table relay-recipients file:/etc/mail/relay-recipients
table our-nets {10.0.0.0/16, 127.0.0.0/8}
table our-domains {example.com, example.net}

listen on all tls pki mail.example.com hostname mail.example.com
listen on lo0 port 10025 tag AMAVIS
listen on all port 587 tls-require pki mail.example.com auth <passwd-smtps> hostname mail.example.com

accept form local for local alias <aliases> deliver to mbox  # Send local mail according to aliases.
reject from ! source <our-nets> for ! domain <our-domains>  # No open relay!
reject from ! source <our-nets> sender "@example.com" for any
reject from any for domain <our-domains> recipient ! <relay-recipients>
accept tagged AMAVIS from source <our-nets> for any relay pki mail.example.com
accept tagged AMAVIS from any for domain <our-domains> relay pki mail.example.com
accept from any for any relay via smtp://127.0.0.1:10024  # Send to Amavis.

We added the reject rule (and the file /etc/mail/relay-recipients) to only relay mail for some internal addresses

Enable all services, and start them:

#  rcctl enable freshclam clamd spamassassin amavisd smtpd
#  rcctl start freshclam clamd spamassassin amavisd smtpd

Outlooks and Gmail don’t play well with Graylisting!

spamd uses a tuple of sending server IP, sender email address, and recipient email to greylist. Unfortunately, some sender have big pools of machines, so the retry send may come from a different IP than the one listed in the spamdb tuple.

A kind soul wrote a shell script to grab SPF records from such senders, and feed them to PF.

Install the spf_fetch utility:

--- nostromo ~ $  mkdir repo
--- nostromo ~ $  cd repo/
--- nostromo repo $  git clone https://github.com/akpoff/spf_fetch
--- nostromo repo $  cd spf_fetch/
--- nostromo spf_fetch $  doas make install
cp spf_fetch     "/usr/local/bin/"
cp spf_update_pf "/usr/local/bin/"
cp experimental/spf_mta_capture "/usr/local/bin/"
cp spf_fetch.1     "/usr/local/man/man1/"
cp spf_update_pf.1 "/usr/local/man/man1/"
cp experimental/spf_mta_capture.1 "/usr/local/man/man1/"

Define a list of common/safe (but greylist-breaking) senders:

--- nostromo ~ $  doas vim /etc/mail/common_domains
amazonses.com
aol.com
att.net
charter.net
comcast.net
cox.net
email.com
gmail.com
googlemail.com
google.com
hotmail.com
icloud.com
mac.com
me.com
mail.com
msn.com
live.com
outlook.com
sbcglobal.net
verizon.net
yahoo.com

Hook into PF:

--- nostromo ~ $  doas vim /etc/pf.conf
table <spamd-white> persist
table <nospamd> persist file "/etc/mail/nospamd"
table <common_white> persist file "/etc/mail/common_domains_ips"
set block-policy drop
set skip on { lo }
block drop in quick from urpf-failed to any
block drop log all
pass in on egress inet proto tcp from any to any port smtp divert-to 127.0.0.1 port spamd
pass in log on egress proto tcp from <spamd-white> to any port smtp
pass in log on egress proto tcp from <nospamd> to any port smtp
pass in log on egress proto tcp from <common_white> to any port smtp
pass in inet proto tcp from any to any port { 22 443 587 }
pass in inet proto icmp from 10.0.0.0/24 to self
pass out on egress proto { tcp udp icmp icmp6 } all modulate state
pass out log on egress proto tcp to any port smtp
--- nostromo ~ $  doas touch /etc/mail/common_domains_ips
--- nostromo ~ $  doas pfctl -F all -f /etc/pf.conf

Note that the table name spamd-white is hard-coded. See spamd(8).

Give it a whirl:

--- nostromo ~ $  doas spf_update_pf
247 addresses added.
--- nostromo ~ $  doas pfctl -T show -t common_white
17.36.0.0/16
17.41.0.0/16
17.58.0.0/16
17.110.0.0/15
17.111.110.0/23
...

New smtpd.conf Grammar

As of mid-2018, a new grammar for smtpd.conf is on the horizon.

The big change splits rules into an “action” line and a “match” line.

Although I have yet to test this, our updated smtpd.conf will look something like:

pki mail.example.com cert "/etc/ssl/example.com.crt"
pki mail.example.com key "/etc/ssl/private/example.com.key"

table aliases file:/etc/mail/aliases
table passwd-smtps file:/etc/mail/passwd-smtps
table relay-recipients file:/etc/mail/relay-recipients
table our-nets {10.0.0.0/16, 127.0.0.0/8}
table our-domains {example.com, example.net}

listen on all tls pki mail.example.com hostname mail.example.com
listen on lo0 port 10025 tag AMAVIS
listen on all port 587 tls-require pki mail.example.com auth <passwd-smtps> hostname mail.example.com

action "local" mbox alias <aliases>
action "relay" relay
action "amavis" relay host smtp://127.0.0.1:10024

match from local for local action "local"
match from ! src <our-nets> for ! domain <our-domains> reject
match from ! src <our-nets> sender "@example.com" reject
match for domain <our-domains> recipient ! <relay-recipients> reject
match tag AMAVIS from src <our-nets> action "relay"
match tag AMAVIS for domain <our-domains> action "relay"
match for any action "amavis"

References

mail flow diagram