awk

awk(1)

awk is a pattern scanning and processing language.

awk processes column-oriented data, and (like sed) works well in a command pipeline.

The -f flag sets us specify a file containing awk commands, rather than specifying the on the command line.

Examples

awk '{ print $3 }' foo.txt prints the third field of every line.

awk '{ print $2, $5 }' foo.txt prints the second and fifth fields of every line.

($0 is the entire line, so $1 is the first column.)

By default, awk splits on whitespace. We can specify a delimiter with the -F flag:

awk -F: '{ print $6 }' /etc/passwd

We can do arithmetic:

awk '{ print ($4 + $7) / 2 }' foo.txt

awk can conditionally act based on matching:

awk -F: '/nobody/ { print $1, $7 }' /etc/passwd

Besides the print actions, awk actions include:

All variables are global.

awk also has built-in variables:

Here we use pattern matching, variable assingment, and multiple commands:

awk -F: 'BEGIN { OFS=":" }; $1 ~ /^root$/ { home = $6 }; $1 ~ /^backup$/ { print $1, $2, $3, $4, $5, home, $7 }' /etc/passwd

(BEGIN is always executed before any records are examined, and END after all records.)

Take a file containing a list of domain names (ignoring comments prepended with “#”), and output lines like local-zone: "example.com" refuse:

% awk '{ if(substr($1, 0, 1) != "#") print "local-zone: \""$1"\" refuse" }' /tmp/blacklist.acl > /var/unbound/conf.d/blacklist.conf

awk has a number of built-in functions. awk understands two data types — numeric and string — and functions operate on one of the two.

Numeric functions include:

String functions include:

See awk(1) for more functions.

It is possible to define our own awk functions like function foo(a, b) { ... }.