Home

Roadmap #5

2017-10-03

I reviewed roadmap #4 over two months ago. The project completely changed directions: instead of rewriting OSH in C++ with custom code generators, I shipped the Python prototype of OSH!

Since then, I've made second release, but I haven't written a new roadmap yet.

Table of Contents

Oil or OSH?

One reason I haven't done that is that I've changed my mind about what to work on. After the first release, I was burnt out by the long slog through a shell, so I decided to focus on Oil instead.

I also promised that I would "re-sell" Oil. I want to focus on the message, which is:

Oil is your upgrade path from bash. It's the only language that bash can be automatically converted to.

I did some work Oil language design and documentation, and I fixed the auto-conversion tool, which had acquired some bugs.

But by then I was actually ready to work on OSH again!

Meta-Goal: Users

The meta-goal is to have some real users, whether they're for Oil or OSH. Right now OSH is closer, so I'm working on that.

So here are some concrete things that I think will help make this happen. These could all change based on user feedback, so feel free to leave comments.

1. Run More Real Programs

I explained this new direction on lobste.rs and /r/ProgrammingLanguages.

I've been playing with Alpine Linux and I like it. abuild is a 2500-line shell script that builds Alpine packages, and I believe it's a good use case for motivating work on OSH.

A primary goal of Oil is to treat shell as a real programming language. Linux distros are major software projects that do this — they stretch the limits of shell. (And recall that some of the first scripts I parsed were related to Linux distros: Aboriginal Linux and debootstrap.)

Here's a rough plan to make OSH rock solid:

  1. I just implemented the getopts builtin, so abuild -h works under OSH. It revealed the speed problem, which got me thinking again about optimizing OSH.

Then:

  1. Build a package with abuild running under OSH. Unusually, it's written in busybox ash, so this is easier than running arbitrary bash .
  2. Build packages with /bin/sh as OSH.
  3. Build packages with /bin/bash as OSH. It should be easy to find a packages that builds with bash and not POSIX shell.

Then I could move onto runtime. e.g. booting an entire Alpine Linux system with /bin/sh as OSH.

2. Test Cases in Various Shell Dialects

There is a similar progression with shell dialects:

  1. I ran my own bash scripts with OSH, including most of the scripts in the Oil git repo. I use a fairly restricted dialect of bash, so this was easy.
  2. I ran Python's configure script with OSH. autoconf generates scripts that use a subset of POSIX shell.
  3. abuild is written in ash, which is a supset of POSIX shell.
  4. After I get abuild running, I can run other people's bash scripts. debootstrap would be a big challenge.

Future blog topic: I should write An Evaluation of Alpine Linux. Let me know if you don't understand why I chose it, or if you think another project would help OSH get better faster.

Further Down the Road

I'm itching to slim down the Python VM and write a bytecode interpreter for OSH, but I want to make sure OSH is complete and correct before optimizing things.

So logically I have to get OSH correct before making it small and fast. It is abysmally slow though, which annoys me.

These things probably won't be done in the next 3 months. But I am itching to work on them.

I will just give a little status update, and current thoughts.

Help Wanted

If you want an interactive shell, you should tell me.

Also, the idea of an in-person Hackathon crossed my mind. If I can get 3 people to show up and contribute 20 lines of code each it could be worth it.

If this soudns appealing e-mail me.