Shell Scripts Are Executable Documentation

2021-01-24

This post is part of "flushing the blog queue", mentioned in the January Blog Roadmap. It links to stories and comments making the same point in different ways:

Shell scripts are a unique form executable documentation. No other mechanism reflects exactly what you're supposed to type on the command line!

Table of Contents

Becoming Shell Literate Lets You Share Knowledge

How to Join a Team and Learn a Codebase

Automating Software Releases

Caveats

Scripts Need a Known Environment

Isn't Shell an Ugly Language?

Becoming Shell Literate Lets You Share Knowledge

Comment from mooreds:

One thing the author didn't cover is how you can share reified knowledge when you write shell scripts... That is really really powerful, because you can not only share it with others but also with your future self.

Summary: Instead of writing docs with shell commands, I invert that and write shell scripts with comments. I show two real examples:

Using the Zulip API for semi-automation of a blog post: https://github.com/oilshell/oil/blob/master/services/zulip.sh
Figuring out how to use uftrace for the first time: https://github.com/oilshell/oil/blob/master/benchmarks/uftrace.sh

These scripts are not pretty, but I can clean them up when it's worth it. The key point is to record the information in a low-effort way.

My other comments in this thread:

Tips on Iteratively Developing Pipelines: Use head / shuf, and use tmux.
QSN and Solutions to the Framing Problem: Delimiting, escaping, and length prefixing.
Command vs. Expression Mode in Oil. Oil takes some pains to look like shell + Python syntactically, and users seem to like it.

How to Join a Team and Learn a Codebase

Summary: The first thing I do when working on an unfamiliar project is write a shell script that records everything I did.

Example: Hacking on Kernighan's awk 5 years ago: https://github.com/andychu/bwk/blob/master/run.sh

With 30 seconds of inspection and a minute of fiddling, I can pick up a project after years.

Another meme: Never remember a port number!

Automating Software Releases

It's worth repeating a comment I mentioned in this February 2020 post:

Summary: Oil's release process is a big shell script, developed gradually: https://github.com/oilshell/oil/blob/master/devtools/release.sh. It produces the /release/$VERSION/ tree.

Caveats

Scripts Need a Known Environment

Using shell scripts as executable documention works best if everyone is on a similar operating system. This is often the case in an engineering team at a company, but it's not true in open source.

I've run into this problem on the Oil project itself! Contributors may have difficulty getting started because shell scripts rely on programs/paths that only exist on certain distros.

I hope to solve this problem within the Oil project itself, and then expose the solution to users.

https://github.com/oilshell/oil/issues/513: Nix and Guix.
- And Nix Flakes
https://github.com/oilshell/oil/issues/803: "Rootless" containers.

Isn't Shell an Ugly Language?

If you think so, take a look at the Oil language!