Bash scripting: lists and random things
Imagine you have an HPC cluster with a hundred compute nodes named node001-node100. The two commands below will help you generate a list of node names – either all name on one line or one name per line. Such a list can be useful as input in a “for” or “while” loops.
Node names on one line:
eval "echo node{`seq -w -s ',' 1 100`}"Example:
[root@wil-dev-l04]# eval "echo node{`seq -w -s ',' 1 100`}"
node001 node002 node003 ... node100Node name one per line:
for i in `seq -w 1 100` ; do echo node$i ; done
Example:
[root@wil-dev-l04]# for i in `seq -w 1 100` ; do echo node$i ; done node001 node002 node003 ... node100
While we are on the subject of generating tediouly long lists of sequential things, here’s a quick example of generating every usable IP address for 192.168.0.1/24 subnet
for a in `seq 192 192`
do
for b in `seq 168 168`
do
for c in `seq 0 255`
do
for d in `seq 1 254`
do
echo "${a}.${b}.${c}.${d}"
done
done
done
doneExample:
[root@wil-dev-l04]# for a in `seq 192 192`; do for b in `seq 168 168`; do for c in `seq 0 255`; do for d in `seq 1 254`; do echo "${a}.${b}.${c}.${d}"; done; done; done; done
192.168.0.1
192.168.0.2
192.168.0.3
...
192.168.255.254You can also use shell brace expansion like so:
echo -e 192.168.{0..255}.{1..254}'\n'This is simpler than the loop, however, this method should be avoided for very large ranges.
Here’s pointless but fun scripting exercise. First, we generate a directory structure going three levels deep and looking something like this:
level1_{1..10}
---level2_{1..10}
---level3_{1..10}
-file{1..10}So in the end we have 1111 directories and 10000 files.
target_dir="/tmp/source" ; mkdir -p "${target_dir}"
for i in `seq 1 10`
do
mkdir -p "${target_dir}/level1_${i}"
for j in `seq 1 10`
do
mkdir -p "${target_dir}/level1_${i}/level2_${j}"
for k in `seq 1 10`
do
mkdir -p "${target_dir}/level1_${i}/level2_${j}/level3_${k}"
for l in `seq 1 10`
do
touch "${target_dir}/level1_${i}/level2_${j}/level3_${k}/file_level3_${k}_${l}.file"
done
done
done
doneNext step is to configure an array of some common commands
set -a commands commands[0]='tar cvf archive_`expr $RANDOM % 1000`.tar' commands[1]="gzip -fr" commands[2]="/bin/rm -r" commands[3]="bzip2"
Now we run a “find” and for every file or directory execute a random command from the array
find "${target_dir}" | while read line ; do eval "${commands[$(echo `expr $RANDOM % 4`)]} `echo ${line}`" ; doneAfter this whole random mess is done, what we end up with is very different from the original 1111 directories and 10000 files:
[root@wil-dev-l04]# find . -type d | wc -l 530 [root@wil-dev-l04]# find . -type f | wc -l 4552 [root@wil-dev-l04]# find . -type f -name "*.tar" | wc -l 934 [root@wil-dev-l04]# find . -type f -name "*.gz" | wc -l 2562 [root@wil-dev-l04]# find . -type f -name "*.bz2" | wc -l 1056
What is the point of this? No point, just an exercise in randomness to sharpen our scripting skills.


In the IP sequencing script, {c} should be ${c}
Very useful scripts on this site… thank you!