Bash Shell Scripting ============================================================================ Make shell scripts executable (chmod u+x `myscript`). The first line of the script must be a shebang interpreter directive pointing to the absolute path of bash. For extra safety, set these options: - `-e` exits immediately on an error. - `-u` "nounset" writes an error instead of executing a command with an unset parameter. - `-f` "noglob" turns off expansion of file name patterns. - `-o pipefail` returns a failure if _any_ of the commands in a pipeline fail (without which, the command only returns a failure if the last stage of the pipeline fails). ``` #!/bin/bash set -euf -o pipefail echo "Hello, world!" ``` Comments ---------------------------------------------------------------------------- ``` #!/bin/bash set -euf -o pipefail # # Hash marks precede comments. # echo "Hello, world!" ``` Variables, Quotation Marks, Interpolation & Substitution ---------------------------------------------------------------------------- Follow the convention of ALL CAPS for environment variables (`PAGER`, `EDITOR`, etc.) and internal shell variables (`SHELL`, `BASH_VERSION`, etc). Use lower case for other variable names to avoid accidentally overwriting environment or internal shell variables. ``` $ foo=bar $ echo "The value of 'foo' is $foo." The value of 'foo' is bar. $ myarray=(red blue green); for c in ${myarray[@]}; do echo $c; done red blue green ``` N.b. `=` functions as either assignment or equality test, depending on the context. A `==` may/should be used when testing equality. Double-quoted things ("foo $bar") are treated as a single argument or string, regardless of internal whitespace. Variables inside double quotes are interpolated. Single-quoted things ('foo $bar') are also treated as a single argument or string, but variables are not interpolated. Don't confuse single quotes with backticks. Backticks replace a command with the output of the sub-command. However, use `$()` instead of backticks. `$()` does the same thing as backticks, is less ambiguous, and can be nested. ``` $ echo $(date) Fri Jan 11 15:10:00 EST 2013 ``` Bash does arithmetic expansion for expressions between `$((` and `))` (but only for integers!): ``` $ echo $((2*2)) 4 ``` By default, Bash variables are global. Use the `local` keyword to confine the variable to the scope of the enclosing function. ``` #!/bin/bash set -euf -o pipefail x=globalvalue function myfunc { local x=localvalue echo "$x" } myfunc # -> "localvalue" echo "$x" # -> "globalvalue" ``` Some Built-in Variables ---------------------------------------------------------------------------- - `$0` Name of the shell script - `$-` Flags passed to the script - `$1-$N` Arguments passed from the command line - `$#` The number/quantity of command line arguments ($1-$N) - `$*` Command line arguments $1-$N - `"$@"` Command line arguments $1-$N, individually quoted - `$?` Return value of last command. Success returns zero, error returns non-zero - `$$` PID of this script Arrays ---------------------------------------------------------------------------- http://www.tldp.org/LDP/abs/html/arrays.html ``` area[11]=23 area[13]=37 area[51]=UFOs echo ${area[11]} # {curly brackets} needed. # Array members need not be consecutive or contiguous. # Some members of the array can be left uninitialized. array=( zero one two three four five ) echo ${array[0]} # zero array2=( [0]="first element" [1]="second element" [3]="fourth element" ) # ^ ^ ^ ^ ^ ^ ^ ^ ^ # Quoting permits embedding whitespace within individual array elements. echo ${array2[0]} # first element echo ${array2[1]} # second element ``` Iterate over an array: ``` #!/usr/bin/env bash # Quick check of HTTP status codes for various URL's. urls=( http://test-nonexistant.example.com/ http://www1.example.com/ https://www1.example.com/ http://www.example.net/ https://www.example.net/ http://m.example.com/ https://m.example.com/ ) for u in "${urls[@]}" do echo -n "$u " curl -i --silent --show-error --connect-timeout 3 "$u" | grep '^HTTP' done ``` Conditionals & Loops ---------------------------------------------------------------------------- The double-square-braces are a Bash-ism that offer some additional safety over tests using single square braces. See `test(1)`. For a list of the tests (e.g. `-e`) search the `bash(1)` man page for "conditional expressions". ``` if [[ -e "fileexists.txt" ]]; then echo "The file exists."; elif [[ $a = $b ]]; then echo "They're equal."; elif [[ -s "nonemptyfile.txt" ]]; then echo "A non-empty file exists."; else echo "The file does not exist."; fi for f in ~/tmp/*; do if [[ -d $f ]]; then echo "$f is a directory."; fi; done n=0; while [[ $n -ne 42 ]]; do echo $n; n=$(( $n + 1 )); done case "$1" in start) start ;; stop) stop ;; *) echo $"Usage: $0 {start|stop}" exit 1 esac ``` Functions and Includes ---------------------------------------------------------------------------- ``` #!/bin/bash set -euf -o pipefail function quit { exit } function e { echo $1 } e Hello e World quit echo "This never runs" ``` Include another file with the dot operator, just like in vanilla `sh`: ``` . myOtherFile ``` Redirection ---------------------------------------------------------------------------- ``` # Stdout to file ls -l > ls-l.txt # Stderr to file grep foo * 2> grep-errors.txt # Stdout to stderr grep foo * 1>&2 grep-errors.txt # Stderr to stdout grep foo * 2>&1 grep-errors.txt # Stderr and stdout to file rm -f $(find /tmp -name core) &> /dev/null ``` Compound Commands ---------------------------------------------------------------------------- ``` { echo "Hello" ; cat 1.txt 2.txt ; } > all.txt ( echo "Hello" ; cat 1.txt 2.txt ; ) > all.txt ``` Curly braces or parentheses create a list of commands. Curly braces execute the list in the current shell environment. Parentheses execute the list in a subshell. A newline can be used in place of a semicolon in most cases. Set "Compound Commands" in `bash(1)`. Getting User Input ---------------------------------------------------------------------------- Use `select` to make a simple multi-select menu: ``` #!/bin/bash options="Hello Quit" select opt in $options; do if [[ "$opt" = "Quit" ]]; then echo done exit elif [[ "$opt" = "Hello" ]]; then echo Hello World else clear echo bad option fi done ``` Use read to get a value from user input: ``` #!/bin/bash set -euf -o pipefail echo "Please enter your name" read name echo "Hello, $name" ``` Or, access command-line arguments like `$1`, `$2`, etc. Special Characters ---------------------------------------------------------------------------- The colon acts as a "no-op" and evaluates to "true". ``` if [ "$T1" = "$T2" ] then : else echo "Nope" fi ``` Commands in parentheses execute in a sub-shell. Variables in the sub-shell are not visible outside it. ``` a=123 ( a=321; ) echo "a = $a" # a = 123 ``` The braces-and-dots notation expands some character classes. ``` echo {a..z} # a b c d e f g h i j k l m n o p q r s t u v w x y z echo {0..3} # 0 1 2 3 ``` Code blocks grouped as braces act somewhat like anonymous functions. They don't create an isolated namespace for variables, but they do allow redirection of their collective output. ``` { echo "one" echo "two" echo "three" } > /tmp/123.txt ``` Empty curly braces act as a placeholder for text. ``` ls . | xargs -i -t cp ./{} $1 ``` Equal-tilde acts as the regular expression match operator inside double-square-braces. ``` #!/bin/bash input=$1 if [[ "$input" =~ "[0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9]" ]] # ^ NOTE: Quoting not necessary, as of version 3.2 of Bash. # NNN-NN-NNNN (where each N is a digit). then echo "Social Security number." else echo "Not a Social Security number!" fi ``` Linter ---------------------------------------------------------------------------- A linter is available for Bash scripts. ``` # apt-get install shellcheck $ shellcheck myscript.sh ``` - https://www.shellcheck.net/ - https://github.com/koalaman/shellcheck Random ---------------------------------------------------------------------------- Bash includes a pseudo-random number generator that returns values between zero and 32,767 (i.e., `2^16 / 2 - 1`) Don't use it for anything where true randomness is important, like cryptography. ``` $ echo $RANDOM 29725 $ echo $RANDOM 14630 $ bash -c 'RANDOM=600; echo $RANDOM $RANDOM' 24424 9510 $ bash -c 'RANDOM=600; echo $RANDOM $RANDOM' 24424 9510 $ echo $((1 + RANDOM % 10)) 10 $ echo $((1 + RANDOM % 10)) 2 $ echo $((1 + RANDOM % 100)) 24 $ echo $((1 + RANDOM % 100)) 74 ``` Further Reading ---------------------------------------------------------------------------- - Bash Guide for Beginners (TLDP) http://tldp.org/LDP/Bash-Beginners-Guide/html/index.html - Advanced Bash Scripting Guide (TLDP) http://tldp.org/LDP/abs/html/index.html - Bash by example (IBM) http://www.ibm.com/developerworks/linux/library/l-bash2/index.html - GNU Bash Reference http://www.gnu.org/software/bash/manual/bashref.html