paulgorman.org/technical

Bash Shell Scripting

Make shell scripts executable (chmod u+x myscript). The first line of the script must be a shebang interpreter directive pointing to the absolute path of bash. For extra safety, set these options:

#!/bin/bash
set -euf -o pipefail
echo "Hello, world!"

Comments

#!/bin/bash
set -euf -o pipefail

#
# Hash marks precede comments.
#

echo "Hello, world!"

Variables, Quotation Marks, Interpolation & Substitution

Follow the convention of ALL CAPS for environment variables (PAGER, EDITOR, etc.) and internal shell variables (SHELL, BASH_VERSION, etc). Use lower case for other variable names to avoid accidentally overwriting environment or internal shell variables.

$ foo=bar
$ echo "The value of 'foo' is $foo."
The value of 'foo' is bar.
$ myarray=(red blue green); for c in ${myarray[@]}; do echo $c; done
red
blue
green

N.b. = functions as either assignment or equality test, depending on the context. A == may/should be used when testing equality.

Double-quoted things (“foo $bar”) are treated as a single argument or string, regardless of internal whitespace. Variables inside double quotes are interpolated.

Single-quoted things (‘foo $bar’) are also treated as a single argument or string, but variables are not interpolated.

Don’t confuse single quotes with backticks. Backticks replace a command with the output of the sub-command. However, use $() instead of backticks. $() does the same thing as backticks, is less ambiguous, and can be nested.

$ echo $(date)
Fri Jan 11 15:10:00 EST 2013

Bash does arithmetic expansion for expressions between $(( and )) (but only for integers!):

$ echo $((2*2))
4

By default, Bash variables are global. Use the local keyword to confine the variable to the scope of the enclosing function.

#!/bin/bash
set -euf -o pipefail
x=globalvalue
function myfunc {
        local x=localvalue
        echo "$x"
}
myfunc # -> "localvalue"
echo "$x" # -> "globalvalue"

Some Built-in Variables

Arrays

http://www.tldp.org/LDP/abs/html/arrays.html


area[11]=23
area[13]=37
area[51]=UFOs

echo ${area[11]}    #  {curly brackets} needed.

#  Array members need not be consecutive or contiguous.
#  Some members of the array can be left uninitialized.

array=( zero one two three four five )
echo ${array[0]}       #  zero

array2=( [0]="first element" [1]="second element" [3]="fourth element" )
#            ^     ^       ^     ^      ^       ^     ^      ^       ^
# Quoting permits embedding whitespace within individual array elements.

echo ${array2[0]}      # first element
echo ${array2[1]}      # second element

Iterate over an array:

#!/usr/bin/env bash
# Quick check of HTTP status codes for various URL's.
urls=(
        http://test-nonexistant.example.com/
        http://www1.example.com/
        https://www1.example.com/
        http://www.example.net/
        https://www.example.net/
        http://m.example.com/
        https://m.example.com/
)
for u in "${urls[@]}"
do
        echo -n "$u "
        curl -i --silent --show-error --connect-timeout 3 "$u" | grep '^HTTP'
done

Conditionals & Loops

The double-square-braces are a Bash-ism that offer some additional safety over tests using single square braces. See test(1).

For a list of the tests (e.g. -e) search the bash(1) man page for “conditional expressions”.

if [[ -e "fileexists.txt" ]]; then
    echo "The file exists.";
elif [[ $a = $b ]]; then
    echo "They're equal.";
elif [[ -s "nonemptyfile.txt" ]]; then
    echo "A non-empty file exists.";
else
    echo "The file does not exist.";
fi


for f in ~/tmp/*;
do
    if [[ -d $f ]]; then
        echo "$f is a directory.";
    fi;
done


n=0;
while [[ $n -ne 42 ]];
do
    echo $n;
    n=$(( $n + 1 ));
done


case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    *)
        echo $"Usage: $0 {start|stop}"
        exit 1
esac

Functions and Includes

#!/bin/bash
set -euf -o pipefail
function quit {
	exit
}
function e {
	echo $1
}
e Hello
e World
quit
echo "This never runs"

Include another file with the dot operator, just like in vanilla sh:

. myOtherFile

Redirection

# Stdout to file
ls -l > ls-l.txt

# Stderr to file
grep foo * 2> grep-errors.txt

# Stdout to stderr
grep foo * 1>&2 grep-errors.txt

# Stderr to stdout
grep foo * 2>&1 grep-errors.txt

# Stderr and stdout to file
rm -f $(find /tmp -name core) &> /dev/null

Compound Commands

{ echo "Hello" ; cat 1.txt 2.txt ; } > all.txt
( echo "Hello" ; cat 1.txt 2.txt ; ) > all.txt

Curly braces or parentheses create a list of commands. Curly braces execute the list in the current shell environment. Parentheses execute the list in a subshell.

A newline can be used in place of a semicolon in most cases.

Set “Compound Commands” in bash(1).

Getting User Input

Use select to make a simple multi-select menu:

#!/bin/bash
options="Hello Quit"
select opt in $options; do
	if [[ "$opt" = "Quit" ]]; then
		echo done
		exit
	elif [[ "$opt" = "Hello" ]]; then
		echo Hello World
	else
		clear
		echo bad option
	fi
done

Use read to get a value from user input:

#!/bin/bash
set -euf -o pipefail
echo "Please enter your name"
read name
echo "Hello, $name"

Or, access command-line arguments like $1, $2, etc.

Special Characters

The colon acts as a “no-op” and evaluates to “true”.

if [ "$T1" = "$T2" ]
then
      :
else
      echo "Nope"
fi

Commands in parentheses execute in a sub-shell. Variables in the sub-shell are not visible outside it.

a=123
( a=321; )

echo "a = $a"   # a = 123

The braces-and-dots notation expands some character classes.

echo {a..z} # a b c d e f g h i j k l m n o p q r s t u v w x y z

echo {0..3} # 0 1 2 3

Code blocks grouped as braces act somewhat like anonymous functions. They don’t create an isolated namespace for variables, but they do allow redirection of their collective output.

{
	echo "one"
	echo "two"
	echo "three"
} > /tmp/123.txt

Empty curly braces act as a placeholder for text.

ls . | xargs -i -t cp ./{} $1

Equal-tilde acts as the regular expression match operator inside double-square-braces.

#!/bin/bash
input=$1
if [[ "$input" =~ "[0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9]" ]]
#                 ^ NOTE: Quoting not necessary, as of version 3.2 of Bash.
# NNN-NN-NNNN (where each N is a digit).
then
  echo "Social Security number."
else
  echo "Not a Social Security number!"
fi

Linter

A linter is available for Bash scripts.

# apt-get install shellcheck
$ shellcheck myscript.sh

Random

Bash includes a pseudo-random number generator that returns values between zero and 32,767 (i.e., 2^16 / 2 - 1) Don’t use it for anything where true randomness is important, like cryptography.

$ echo $RANDOM
29725
$ echo $RANDOM
14630
$ bash -c 'RANDOM=600; echo $RANDOM $RANDOM'
24424 9510
$ bash -c 'RANDOM=600; echo $RANDOM $RANDOM'
24424 9510
$ echo $((1 + RANDOM % 10))
10
$ echo $((1 + RANDOM % 10))
2
$ echo $((1 + RANDOM % 100))
24
$ echo $((1 + RANDOM % 100))
74

Further Reading