Last Updated: October 19, 2022
·
46.56K
· vaneyckt

Safer bash scripts with 'set -euxo pipefail'

Preface: This article was originally posted here on my personal blog.

Often times developers go about writing bash scripts the same as writing code in a higher-level language. This is a big mistake as higher-level languages offer safeguards that are not present in bash scripts by default. For example, a Ruby script will throw an error when trying to read from an uninitialized variable, whereas a bash script won't. In this article, we'll look at how we can improve on this.

The bash shell comes with several builtin commands for modifying the behavior of the shell itself. We are particularly interested in the set builtin, as this command has several options that will help us write safer scripts. I hope to convince you that it's a really good idea to add set -euxo pipefail to the beginning of all your future bash scripts.

set -e

The -e option will cause a bash script to exit immediately when a command fails. This is generally a vast improvement upon the default behavior where the script just ignores the failing command and continues with the next line. This option is also smart enough to not react on failing commands that are part of conditional statements. Moreover, you can append a command with || true for those rare cases where you don't want a failing command to trigger an immediate exit.

Before

#!/bin/bash

# 'foo' is a non-existing command
foo
echo "bar"

# output
# ------
# line 4: foo: command not found
# bar

After

#!/bin/bash
set -e

# 'foo' is a non-existing command
foo
echo "bar"

# output
# ------
# line 5: foo: command not found

Prevent immediate exit

#!/bin/bash
set -e

# 'foo' is a non-existing command
foo || true
echo "bar"

# output
# ------
# line 5: foo: command not found
# bar

set -o pipefail

The bash shell normally only looks at the exit code of the last command of a pipeline. This behavior is not ideal as it causes the -e option to only be able to act on the exit code of a pipeline's last command. This is where -o pipefail comes in. This particular option sets the exit code of a pipeline to that of the rightmost command to exit with a non-zero status, or zero if all commands of the pipeline exit successfully.

Before

#!/bin/bash
set -e

# 'foo' is a non-existing command
foo | echo "a"
echo "bar"

# output
# ------
# a
# line 5: foo: command not found
# bar

After

#!/bin/bash
set -eo pipefail

# 'foo' is a non-existing command
foo | echo "a"
echo "bar"

# output
# ------
# a
# line 5: foo: command not found

set -u

This option causes the bash shell to treat unset variables as an error and exit immediately. This brings us much closer to the behavior of higher-level languages.

Before

#!/bin/bash
set -eo pipefail

echo $a
echo "bar"

# output
# ------
#
# bar

After

#!/bin/bash
set -euo pipefail

echo $a
echo "bar"

# output
# ------
# line 5: a: unbound variable

set -x

The -x option causes bash to print each command before executing it. This can be of great help when you have to try and debug a bash script failure through its logs. Note that arguments get expanded before a command gets printed. This causes our logs to display the actual argument values at the time of execution!

#!/bin/bash
set -euxo pipefail

a=5
echo $a
echo "bar"

# output
# ------
# + a=5
# + echo 5
# 5
# + echo bar
# bar

And that's it. I hope this post showed you why using set -euxo pipefail is such a good idea. If you have any other options you want to suggest, then please let me know and I'll be happy to add them to this list.

6 Responses
Add your response

I'd like to suggest adding shopt -s inherit_errexit.
Without it, command substitution does not inherit the set -e.
Pipefail is inherited though in any case.
Without that shopt line, foo=$(gits --version; true); echo $foo will fail and print an error, but will not make the script fail.
With the option, the script is aborted on that error.
Strangely echo $(gits --version; true) will not make the script fail either way, but I guess this is a bug.

over 1 year ago ·

I'd like to complete -o pipefail. In your example there is a further property of this option invisible.
Let's look at:

Before

$ foo | echo "a"
$ echo $?
$ echo bar

a

bash: foo: Kommando nicht gefunden.

0

bar

After

set -o pipefail
$ foo | echo "a"
$ echo $?
$ echo bar

a

bash: foo: Kommando nicht gefunden.

127

bar

See manpage bash:
"The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all com mands exit successfully."

over 1 year ago ·

@kernrad how is that different or new?
He didn't show it with the return code but with "bar" not being printed.
But as far as I can see you just repeat what the entry said.
Did I miss something?

over 1 year ago ·

It's different by use of the returncode. Without -o pipefail we will get the returncode of echo "a", which is 0. By use of -o pipefail it is the returncode of foo which is not found and therefore returns 127.

over 1 year ago ·

Which is explicitly mentione in the post

This particular option sets the exit code of a pipeline to that of the rightmost command to exit with a non-zero status, or zero if all commands of the pipeline exit successfully.

over 1 year ago ·

Well, you are right.
I only wanted to complete the example, nothing more, for this property is used for error exit but not directly visible. The article is correct.

over 1 year ago ·