Empty Loops in Bash

One of the very common things I do in bash/sh is look over item in a folder. However, bash and sh behavior when the directory is empty can be unexpected. For example, let’s say I want to loop over all the items in my web folder:

#!/bin/bash

for file in /Users/Shared/web/*; do	
	echo "doing something with ${file}"	
done

When you run it, does exactly what you think:

doing something with /Users/Shared/web/1111.dmg
doing something with /Users/Shared/web/Install_macOS_10.15.5-19F96.dmg
doing something with /Users/Shared/web/authorized_keys
doing something with /Users/Shared/web/bs.dmg
doing something with /Users/Shared/web/test.sh
doing something with /Users/Shared/web/test2.sh

But what happens when the path doesn’t exist or the folder is empty? Taking all the items out of web and running the script again:

doing something with /Users/Shared/web/*

Not really what I was expecting. I was expecting the loop to be skipped since web is empty, but it seems to be passing a single item into the loop with the value of “/Users/Shared/web/*”.

What happens if I want to actually do something with the item:

for file in /Users/Shared/web/*; do
	echo "doing something with ${file}"
	chmod 755 "${file}"
done

This works as expected if there are files and folders in the folder I am looking in, but what if /Users/Shared/web is empty? Things go poorly:

doing something with /Users/Shared/web/*
chown: /Users/Shared/web/*: No such file or directory

The behavior gets even stranger if I try to use that path and it succeeds in weird ways — for instance, if I have a loop that checks for files and does one thing/a folder if it does another thing:

#!/bin/bash

for file in /Users/Shared/web/*; do
	if [ -f "${file}" ]; then
		echo "file, so skipping"
	else
		echo "making subdirectory in ${file}"
		mkdir -p "${file}/subdir"
	fi
done

If the web folder is empty, “/Users/Shared/web/*” gets passed in. Since it is not a file, it assumes it is a folder and tries to create it:

Not what I was expecting at all! In this case, I was checking for the existence of the item (is it a file?) but making the assumption that if it isn’t a file and it exists, then it must be a folder.

One way to fix this is to verify that $file exists and is the explicit type before doing anything with it. However, I know it exists since the loop is iterating over known items. That always felt very wrong to me. Typically, in other languages, if you try to iterate over an empty array, it will just not enter the loop. I want that behavior in bash/sh.

Luckily, there is a shell option to set this behavior:

shopt -s nullglob

If you set this before the loop, it just skips the loop if it is empty or if the path doesn’t exist:

#!/bin/bash

shopt -s nullglob 

for file in /Users/Shared/web/*; do
	if [ -f "${file}" ]; then
		echo "file, so skipping"
	else
		echo "making subdirectory in ${file}"
		mkdir -p "${file}/subdir"
	fi
done

Now the loop is skipped if /Users/Shared/web is empty or doesn’t exist. I definitely should have checked if an item that isn’t file is a directory, but the shopt -s nullglob helps make the behavior more like what I was expecting.