paulgorman.org/technical

Systemd journalctl

The linux systemd init system comes with journalctl a queriable binary system log.

Configuration

First, check the timezone on the system:

$ timedatectl status

If necessary, change the time zone:

# timedatectl set-timezone 'America/Detroit'

The config file resides at /etc/systemd/journald.conf. See the man page journald.conf(5). If Storage=auto (the default?), journald saves data to /run/log/journal/ unless /var/log/journal/ exists. Since /run/ is typically a tmpfs, the journal data effectively does not persist across reboots. We persist the journal data like:

# mkdir /var/log/journal
# chown root:systemd-journal /var/log/journal
# chmod 2755 /var/log/journal

To restart journald, which we must do after making any config changes:

# systemctl restart systemd-journald

If necessary, add log-reading users to the systemd-journald group:

# usermod -a -G systemd-journald paul

Find disk used by journals:

$ journalctl --disk-usage

Shrink the journal by removing old entries until it reaches a specified size:

$ sudo journalctl --vacuum-size=1G

Shrink the journal by removing entries older than a given time:

$ sudo journalctl --vacuum-time=1years

These and other limits may be configured in /etc/systemd/journald.conf.

Viewing Logs

If called without arguments, journalctl spits out all its entries, from oldest to newest.

$ journalctl

When comparing logs from various time zones, consider the --utc flag.

$ journalctl --utc

Scope results to the last boot (or a previous boot):

$ journalctl -b
$ journalctl -b -1
$ journalctl --list-boots

Or confine returned log entries based on date/time:

$ journalctl --since "2017-01-10" --until "2017-01-18 02:30"
$ journalctl --since yesterday
$ journalctl --since 10:00 --until "1 hour ago"

The -p flag shows one or a range of log levels (following SYSLOG(3)). Show kernel messages with severity of emergency, alert, critical, error, or warning:

$ journalctl -k -p 0..4

Filter for a particular systemd unit/service:

$ journalctl -u fancontrol

Filter for a particular PID or user ID:

$ journalctl _PID=1660
$ journalctl _UID=1001

See SYSTEMD.JOURNAL-FIELDS(7) for additional filterable fields.

Three flags of note:

Pager

By default, journalctl pages its output, trying well-known pagers (less, more, etc.). To stop it defaulting to that:

export SYSTEMD_PAGER=cat

Or, to stop it paging for one invocation:

$ journalctl --no-pager

Remote Logging