Home

The Long Slog Through a Shell: Intro

I'm going to to write about my recent work on the OSH runtime. The last few posts explain the context for this work:

• Working Toward an OSH Release. I wanted to release the shell this summer, but it wasn't clear how much work was left before it would be useful. Attempting to run real shell programs is a good way to see what's left.

• How I Use Tests. I introduced the test-driven style of working, with 4 kinds of test. I started with coarse gold tests, then nail down behavior with spec tests, and then sometimes I need fine-grained unit tests. The gold tests helped me choose what features to implement.

• How I Plan to Use Tests: Transforming OSH. I warned readers again that OSH will be too big and too slow upon initial release. The good news is that the same tests will help us address this problem.

• OSH Runs Real Shell Programs -- After fixing the hardest bug ever, OSH can run a real program I didn't write! This is the end of the journey; now I'll go back to the beginning.

Where We Were Six Weeks Ago

I worked on OPy and the OVM build system for several weeks, including most of April. Thus, as of mid-May, the runtime was basically as I described in late March, in Measuring Progress With Tests.

That is, there were around 375 tests passing, but I hadn't run any significant shell programs with OSH.

So I started hacking away, using gold tests and spec tests as a guide.

I'll describe this journey in the next few posts. The work fell into these categories:

• Implementing language features
  • e.g. background jobs like echo hi | wc -l &.
• Implementing shell builtins
  • e.g. shift, read, set, etc.
• Implementing options to the shell itself
  • nounset, errexit, etc.
• Fixing bugs
  • A tricky one involving the concurrent processes of a pipeline and backkground jobs
• Implementing new components
  • two reusable flag parsers for builtins.
• Redesign of Major subsystems
  • processes and redirects, which let you implement pipelines, subshells, and background jobs)
  • the "memory model", which stores variables.

And of course:

• Tests to cover all this behavior.

This inspired some higher level thoughts on:

• Static vs. Dynamic Parsing.
  • Converting OSH to Oil vs. executing OSH. The former requires more static parsing.
  • To replace bash, we must statically parse it!