Discussions: Lobsters, /r/programming, /r/commandline. Translations: Chinese (中文).
I launched GoatCounter: privacy-aware web statistics.
You can now support my work.

Shell scripting is great. It is amazingly simple to create something very useful. Even a simple no-brainer command such as:

# Official way of naming Go-related things:
$ grep -i ^go /usr/share/dict/* | cut -d: -f2 | sort -R | head -n1

Takes several lines of code and a lot more brainpower in many programming languages. For example in Ruby:

puts(Dir['/usr/share/dict/*-english'].map do |f|
    .select { |l| l[0..1].downcase == 'go' }

The Ruby version isn’t that long, or even especially complicated. But the shell script version was so simple that I didn’t even need to actually test it to make sure it is correct, whereas I did have to test the Ruby version to ensure I didn’t make a mistake. It’s also twice as long and looks a lot more dense.

This is why people use shell scripts, it’s so easy to make something useful. Here’s is another example:

curl https://nl.wikipedia.org/wiki/Lijst_van_Nederlandse_gemeenten |
	grep '^<li><a href=' |
	sed -r 's|<li><a href="/wiki/.+" title=".+">(.+)</a>.*</li>|\1|' |
	grep -Ev '(^Tabel van|^Lijst van|Nederland)'

This gets a list of all Dutch municipalities. I actually wrote this as a quick one-shot script to populate a database years ago, but it still works fine today, and it took me a minimum of effort to make it. Doing this in e.g. Ruby would take a lot more effort.

But there’s a downside, as your script grows it will become increasingly harder to maintain, but you also don’t really want to rewrite it to something else, as you’ve already spent so much time on the shell script version.

This is what I call ‘the shell script trap’, which is a special case of the sunk cost fallacy.

And many scripts do grow beyond their original intended size, and often you will spend a lot more time than you should on “fixing that one bug”, or “adding just one small feature”. Rinse, repeat.

If you had written it in Python or Ruby or another similar language from the start, you would have spent some more time writing the original version, but would have spent much less time maintaining it, while almost certainly having fewer bugs.

Take my packman.vim script for example. It started out as a simple for loop over all directories and a git pull and has grown from there. At about 200 lines it’s hardly the most complex script, but had I written it in Go as I originally planned then it would have been much easier to add support for printing out the status or cloning new repos from a config file. It would also be almost trivial to add support for parallel clones, which is hard (though not impossible) to do correct in a shell script. In hindsight, I would have saved time, and gotten a better result to boot.

I regret writing most shell scripts I’ve written for similar reasons, and my 2018 new year’s pledge will be to not write any more.

Appendix: the problems

And to be clear, shell scripting does come with some real limitation. Some examples:

I launched GoatCounter: privacy-aware web statistics.
You can now support my work.