GNU Parallel lets us run shell commands in parallel.
# apt-get install parallel
$ mkdir -p ~/.parallel && touch ~/.parallel/will-cite
Parallel accepts a list of arguments, which we provide after ::::
$ parallel echo ::: alpha bravo charlie
alpha
bravo
charlie
If we supply multiple lists of arguments, parallel combines them:
$ parallel echo ::: a b ::: c d
a c
a d
b c
b d
The output order my vary.
The tool provides a number of replacement string options. The default string {} represents the input:
$ parallel echo {} ::: /tmp
/tmp
(i.e. {} is replaced by a line of input, STDIN by default or else the arguments following :::.)
The replacement string {/} removes everything up to and including the last forward slash:
$ parallel echo {/} ::: /tmp/test.txt
test.txt
If you want to return the path only, use the {//} string:
$ parallel echo {//} ::: /tmp/test.txt
/tmp
The string {.} removes any filename extension:
$ parallel echo {.} ::: /tmp/test.txt
/tmp/test
See the man page for more substitution and argument options (e.g. :::: to specify input read from files).
To run the same command multiple times for the same argument:
$ seq 5 | parallel -n0 echo "Hello!"
Test with --dry-run:
$ parallel --dry-run -k sleep {}\; echo {} ::: 5 2 1 4 3
sleep 5; echo 5
sleep 2; echo 2
sleep 1; echo 1
sleep 4; echo 4
sleep 3; echo 3
(-k forces the output order to match the arguments; I’m not certain that implies any guarantee about the order of execution.)