blog |

Four More Posts in "Shell: The Good Parts"

2020-02-19 (Last updated 2022-03-24)

This is an outline of the most important series in this blog. As I wrote on Hacker News, I started this project nearly 4 years ago with a plan:

  1. Explain the problems with shell, motivating the need for a new one. I tagged these posts #shell-the-bad-parts and #why-a-new-shell.
  2. Write a new but compatible shell called Oil.
  3. Explain the good parts of shell with #shell-the-good-parts. Circulate these posts and let people try the good parts without the bad parts.

Unfortunately there are only two posts in that series, written over 3 years ago. Step 2 is taking longer than expected!

So while I'm chipping away at Oil's implementation, here are the main ideas. These ideas should be useful for experienced shell users, and I hope to explain them more carefully to non-shell users later in 2020.

Thanks to reader enriquto for reminding me to write this.

Table of Contents
Semi-automation With Scripts
The $0-Dispatch Pattern Solves Three Important Problems
Web Sites Are Naturally Made With Shell Scripts
The Alternative Shell Challenge
What's Next?

Each section below describes an article I'd like to write. The heading is the proposed title, and the body has the main ideas and important URLs.

Semi-automation With Scripts

I used to call these scripts, but that name shares a prefix with the common doc/ directory, creating a speed bump for autocompletion.

2/26 update: I'm not the only one who advocates this pattern:

March 2022 Update:

Github has a related pattern called Scripts to Rule Them All (2015):

Being able to get from git clone to an up-and-running project in a development environment is imperative for fast, reliable contributions.

The $0-Dispatch Pattern Solves Three Important Problems

  1. A shell script is a simple way of documenting project-specific dev tools. That is, if you have a text file that describes a bunch of shell commands (e.g. a "playbook"), you can turn it into a shell script:

  2. The pattern also lets you expose shell functions to tools like xargs / find -exec, and sudo / chroot.

  3. The $0-dispatch pattern solves the ignored errexit problem.

Related points:

Web Sites Are Naturally Made With Shell Scripts

An HTTP daemon is such a simple thing that a simple shell script will often suffice.

Tim Berners-Lee, inventor of the World Wide Web

The Unix philosophy of combining heterogeneous tools remains useful and powerful today. Building sites with tools rather than libraries leads to efficient, flexible, and loosely coupled code.

I built this website with classic Unix tools and my own tools written in Python. The main() is always in shell, not Python.

Some direct analogies between HTML and shell:

The Alternative Shell Challenge

I've noticed that some alternative shells on the wiki don't have all the good parts of shell. This is often because they don't directly control the file descriptor table of a process, which is done with low-level system calls like dup2() and fcntl().

That is, they aren't Unix shells in the classic sense.

A couple years ago, I wrote a challenge for alternative shells (and Perl). How do you express this simple program?

f() {
  echo ---
  ls /
  echo ---

f > out.txt
f | wc -l

It's very related to the Git Log in HTML post, which adds proper escaping.

print-html-listing() {
  local dir="$1"
  echo '<html>'
  echo '  <body>'
  ls "$dir" | escape-html
  echo '  </body>'
  echo '</html>'

print-html-listing > listing.txt

I recently learned that Elvish does well on this challenge with a "virtual" file descriptor table. However, it doesn't have the POSIX semantics of "copying" the shell's memory with fork().

I'd like to do a survey of other shells. Let me know if you'd like to help test an alternative shell or scripting language.

What's Next?

This post is two levels deep on the "blog stack". After releasing Oil 0.8.pre1, I drafted an announcement:

  1. Oil 0.8.pre1: Contributions and Metrics

But it needed context, so I started writing:

  1. February Brain Dump: with links to Hacker News comments and Zulip threads.

This post summarized just one comment, mentioned in the first paragraph. The next two posts will pop the stack, summarizing more comments and describing the latest release.