awk === awk(1) awk is a pattern scanning and processing language. awk processes column-oriented data, and (like sed) works well in a command pipeline. The -f flag sets us specify a file containing awk commands, rather than specifying the on the command line. ## Examples ## `awk '{ print $3 }' foo.txt` prints the third field of every line. `awk '{ print $2, $5 }' foo.txt` prints the second and fifth fields of every line. ($0 is the entire line, so $1 is the first column.) By default, awk splits on whitespace. We can specify a delimiter with the -F flag: awk -F: '{ print $6 }' /etc/passwd We can do arithmetic: awk '{ print ($4 + $7) / 2 }' foo.txt awk can conditionally act based on matching: awk -F: '/nobody/ { print $1, $7 }' /etc/passwd ## More Actions and Variables ## Besides the `print` actions, awk actions include: - `exit` to end awk processing - `next` to skip to the next record - variable assignment, like `x=$1` All variables are global. awk also has built-in variables: - $NF is the last field. - NF is the number of fields. - NR is the number of records (i.e. line number). - FS is the input field separator. - OFS is the output field separator. - RS is the record separator (newline by default). - ORS is the output record separator (also newline by default). - FILENAME is the file being read. Here we use pattern matching, variable assingment, and multiple commands: awk -F: 'BEGIN { OFS=":" }; $1 ~ /^root$/ { home = $6 }; $1 ~ /^backup$/ { print $1, $2, $3, $4, $5, home, $7 }' /etc/passwd (`BEGIN` is always executed before any records are examined, and `END` after all records.) Take a file containing a list of domain names (ignoring comments prepended with "#"), and output lines like `local-zone: "example.com" refuse`: % awk '{ if(substr($1, 0, 1) != "#") print "local-zone: \""$1"\" refuse" }' /tmp/blacklist.acl > /var/unbound/conf.d/blacklist.conf ## Functions ## awk has a number of built-in functions. awk understands two data types --- numeric and string --- and functions operate on one of the two. Numeric functions include: - exp(expr) exponent - int(expr) truncate to integer - rand() random number between 0 and 1 String functions include: - asort(array) sort array - index(array, str) return index of string in array - length(str) length of string - split(str, array) split string to array - substr(str, i[, n]) substring between i and n See awk(1) for more functions. It is possible to define our own awk functions like `function foo(a, b) { ... }`. ## Links ## http://www.grymoire.com/Unix/Awk.html http://ferd.ca/awk-in-20-minutes.html https://www.gnu.org/software/gawk/manual/