# GNU Parallel # GNU Parallel lets us run shell commands in parallel. # apt-get install parallel $ mkdir -p ~/.parallel && touch ~/.parallel/will-cite `Parallel` accepts a list of arguments, which we provide after `:::`: $ parallel echo ::: alpha bravo charlie alpha bravo charlie If we supply multiple lists of arguments, `parallel` combines them: $ parallel echo ::: a b ::: c d a c a d b c b d The output order my vary. The tool provides a number of replacement string options. The default string `{}` represents the input: $ parallel echo {} ::: /tmp /tmp (i.e. {} is replaced by a line of input, STDIN by default or else the arguments following `:::`.) The replacement string `{/}` removes everything up to and including the last forward slash: $ parallel echo {/} ::: /tmp/test.txt test.txt If you want to return the path only, use the `{//}` string: $ parallel echo {//} ::: /tmp/test.txt /tmp The string `{.}` removes any filename extension: $ parallel echo {.} ::: /tmp/test.txt /tmp/test See the man page for more substitution and argument options (e.g. `::::` to specify input read from files). To run the same command multiple times for the same argument: $ seq 5 | parallel -n0 echo "Hello!" Test with `--dry-run`: $ parallel --dry-run -k sleep {}\; echo {} ::: 5 2 1 4 3 sleep 5; echo 5 sleep 2; echo 2 sleep 1; echo 1 sleep 4; echo 4 sleep 3; echo 3 (`-k` forces the output order to match the arguments; I'm not certain that implies any guarantee about the order of execution.) ## References ## - https://www.gnu.org/software/parallel/man.html - https://www.gnu.org/software/parallel/parallel_tutorial.html - http://www.shakthimaan.com/posts/2014/11/27/gnu-parallel/news.html - https://www.msi.umn.edu/support/faq/how-can-i-use-gnu-parallel-run-lot-commands-parallel - https://www.gnu.org/software/parallel/