set -o posix

Robustness in shell scripts is tricky business. I discovered today that in a script I have running in automation, some shell options were not propagating to subshells, leading to silent failures. Consider the following:

#!/bin/bash
echo "--- Before set -e"
set -o | grep errexit
set -e
echo "--- After set -e"
set -o | grep errexit
foo=`set -o | grep errexit`
echo "--- Subshell"
echo $foo 

This outputs the following:

--- Before set -e
errexit        	off
--- After set -e
errexit        	on
--- Subshell
errexit off

It turns out the magic needed to propagate the errexit option to the subshell is with use of the little-known ‘posix’ option. With that little tweak….

#!/bin/bash
set -o posix
echo "--- Before set -e"
set -o | grep errexit
set -e
echo "--- After set -e"
set -o | grep errexit
foo=`set -o | grep errexit`
echo "--- Subshell"
echo $foo 

… we get the desired output:

--- Before set -e
errexit        	off
--- After set -e
errexit        	on
--- Subshell
errexit on
Advertisements
This entry was posted in Uncategorized and tagged . Bookmark the permalink.

7 Responses to set -o posix

  1. lopiuh says:

    Didn’t know, thanks, will use it.

    Unfortunately it stays problematic to rely on errexit. You have to remenber when it gets deactivated (!, if, while, …)

    if you rely in functions, you must “hope” your function is not used in such a context e.g. “if function” as far as there is no way to have errexit staying on within such a construct.

    see http://mywiki.wooledge.org/BashFAQ/105

  2. lopiuh says:

    Ya, Better Than Nothing™ 😉 is my position as well in that case.

    Unfortunately that does mean for me, I don’t use returncodes in functions anymore. If you use errexit in main, you have to catch the returncode of the function (if, ||, &&) But doing so you deactivate errexit in the function. So I use a global variable as returncode and just do a if [ gbl_returncade = “xxx” ], so that there is no code executed within the if-clause.

    But then: Subshells: Loops, pipes (!) are executed in a subshell.
    https://stackoverflow.com/questions/3085518/variables-value-gets-lost-in-subshell
    You can read variables from the caller within the subshell, but if you set global variables in the subshell (e.g. after the pipe) these variable contentes / changes are not (!) visible to the caller shell. So you have to use files or similar approaches to pass information between functions within one and the same script only because you did a loop. Doing so you have to care about locking or pid-relative information_transfer_files in order not to overwrite the value for other instances (if you run several instances of your script at the same time).

    Do you know any alternative to posix shells? python, ruby and so on are not so easy doing jobs the shell is made for (glue language for piping and so on).

    Maybe perl?

    Cheers

    lopiuh

    • Tirath says:

      I have no recent experience with Perl, so I don’t know how appropriate it would be. This in itself is a factor against it because I think my teammates are also not fluent with Perl, so there would be a productivity penalty.

      Shell scripting is just too damn convenient for prototyping things of unknown future value. It is true that they become fragile and very hard to maintain beyond a certain scale, but lately I’ve been dealing with that by writing the gnarly bits as Python helper programs.

      At the moment I have quite a bit of technical debt in the form of shell scripts that were never written with automation in mind, but now they’re behaving well – as far as I know anyway 😉 – and where it is sensible to do so, new features get written up as Python helper programs. For the most part the shell scripts just shuffle data from one Python helper to another. Obviously it’s not that simple or I could easily get rid of all the shell, but that’s the approach for adding new features into a base of shell scripts that has obviously outgrown it’s… shell (pardon the pun).

      And if I ever run out of things to do I can gradually move more pre-existing shell functionality into Python, and eventually be shell-free. Starting off in shell was still the right thing to do because I would never have gotten anything at all if I had to spend more time on it than I had – the project was of unknown value (and unsanctioned by PM) when I started on it.

  3. lopiuh says:

    Tirath,

    thanks for sharing your thoughts. Have a great time, nice to “meet” you on your blog 😉

    lopiuh

  4. lopiuh says:

    arghh, realy want to use -o posix but it turns of Process substitution

    https://www.gnu.org/software/bash/manual/html_node/Bash-POSIX-Mode.html#Bash-POSIX-Mode

    its such a mess 😦

    ls -al | tee >(tail -n 1) for example won’t work anymore

    Greetings

    • Tirath says:

      Ah yes you’re quite right, I encountered that just a few weeks ago in fact in a script snippet that I got from someone else.

      I agree – “mess” is the right word to describe the situation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s