I do not like the Bash Strict Mode

Published: Tue 27 July 2021

In software.

Ever seen this command?

set -euo pipefail

That is the "Unofficial Bash Strict Mode". It's a set of options that's meant to make programming in bash more predictable. It is beloved by bloggers and internet commenters everywhere.

I don't like it.

First, a disclaimer

I work on what is arguably a competitor to bash (and zsh, and ksh, and elvish and others). I work on fish. Fish doesn't have any of these options, and we're not going to add them in this form. I would argue that the reason we're not going to add them like this is because I don't like them, not that I don't like them because fish doesn't have them, but you'll have to decide for yourself.

Okay, so why don't I like it?

The Bash Strict Mode is a set of options that, I believe:

  • Don't reduce the number of failures
  • Don't make failure more predictable
  • Don't meaningfully reduce the amount of work you need to do

I believe that it's easier to write a working script without them. In fact, reading what I think is the original blog post that introduced the idea to the world, it really reads like "Enable this so you can then think of always doing these things correctly so you don't have to do these other things correctly".

So, let's get in to it.

What is it?

The "Bash Strict Mode" consists of three options [1]:

  • set -e - aka set -o errexit
  • set -u - aka set -o nounset
  • set -o pipefail

errexit makes the shell exit when any command exits with an unchecked falsey status. This means that this:

if false; then
    echo first
fi

echo second
false
echo third

prints just "second", and exits unceremoniously with a status of 1 (the status that false returned).

nounset makes expanding unset variables an error, except for when used with default values. So

echo ${unset:-UNSET}
echo This variable is also unset $unsettwo

prints "UNSET" and returns 1. It also prints an error explaing which variable was unset where.

pipefail makes the shell consider a pipe as failed if any one of the commands in it failed.

So now

grep nonexistentfile | sort

counts as failed, because the grep failed, even though the sort didn't.

Where is the problem?

Let's go from best to worst, shall we?

nounset is probably the most tasteful of the bunch. It's... it's okay. It prints a proper error helping you find the issue, so any mistakes can be easily found. The opt-in to unset variables is simple enough - ${var:-defaultvalue} is okay, and often what you'd want anyway.

However, it still aborts the script at seemingly random times. Just exiting is not a good failure mode! And having to remember to opt-in isn't a fantastic trade for having to remember to spell correctly or set variables before. I'm not sure if this simplifies matters or not.

pipefail is awkward. It's entirely context dependent whether the failure of earlier things should count or not. If my last command is a grep, yet it returns true because it matched something, then chances are the entire pipe is working correctly, even if something in the middle, technically, failed.

There is an additional problem here: Some tools close the read-end of the pipe once they are done with it, and by default that means the process at the write-end gets sent a SIGPIPE signal, which by default is fatal and causes a false status to happen.

So, the canonical example:

somecmd | head -n1

may or may not fail, depending on whether somecmd hangs around long enough to be killed, and whether or not it intercepted an entirely benign signal.

errexit is the worst. One problem with it is an extension of the problem with SIGPIPE - often, having a non-zero status is not a critical error. grep returns 1 if it didn't match anything. expr [2] returns 1 if the result of a computation was 0.

The ecosystem doesn't really have the idea that a status other than zero is only reserved for the absolute worst conditions, and the way bash works incentivices that thinking! Using the status to signal various interesting conditions means you can use the command in if conditions, instead of looking at the output, which can often be tedious.

Another big issue is that the rules for when and how errexit is in effect aren't always clear and change from time to time. And because old bash versions are being kept alive by old distributions and macOS [3], that means if you want your script to run everywhere it needs to run under all the rules. Which entirely explodes your test matrix!

The BashFAQ has a nice example:

#!/usr/bin/env bash
set -e
i=0
((i++))
echo "i is $i"

works in Bash <= 4.0, but fails in Bash since 4.1. The ((i++)) block returns false because $i was 0 before the post-increment, and that is enough to trigger an abort of the script, under the new, expanded, rules! The script crashed because a variable happened to be 0 at one point.

Combine this with pipefail, and now you have scripts exiting for hard to understand reasons, unless you remember to sprinkle || true at the correct places (and you know what the correct places are).

The trade-off here is, in my humble opinion, simply not worth it. You've exchanged one set of things you need to remember with another, except the new thing requires more typing and fails under arguably more mysterious circumstances.

> The correct answer to every exercise is actually "because set -e is crap".

True.

But wait, there's more!

So we have one option that's meh, one option that's not great and one that is awful.

But there's more in the Original post:

IFS=$'\n\t'

This changes the Input Field Separator to just a newline and a tab, from the default of space, newline and tab.

That's a great idea. It means that command substitutions ($(these)) aren't split on spaces anymore, only newlines (and tabs, which I'd also remove). This works nicely, because many many many unix utilities already operate on lines! grep matches each line, sed edits each line, find outputs files one per line [4].

I don't encounter things where splitting on spaces is correct all that often - for the most part it's pkg-config, and there I'd argue that the output is awkward! It prints what amounts to arguments to your compiler, on a line so it looks like a line to paste into your terminal. It should just be separate lines and it would keep on working for basically everyone (except for people who set $IFS to be just spaces, I guess?). [5]

So, in conclusion: I don't like the bash strict mode, except for that $IFS thing.

[1]This is because of option grouping - -euo is the same as -e -u -o.
[2]Not that you should use expr, but it works as an example.
[3]Oh how I hate macOS.
[4]Filenames are allowed to include newlines, so this is inherently an ambiguous format. Personally, I never see newlines in filenames, and I would want them to just be outlawed outright. There have been efforts in this area, e.g. David Wheeler's essay Fixing Unix/Linux/POSIX Filenames.
[5]I know this because splitting only on newlines by default is exactly what fish does, and I honestly feel that it's much more natural.

social