The Long Slog: Running Programs I Didn't Write


Here I start to run real programs.


Flags Parsing

- this went into a huge flags parsing digression.  TWO flags parsers.
- case that convinced me: set -o pipefail vs. read -t1.0
  - another special case: echo


Python configure.

SUCCESS! I can now run Python's configure script.

This is sort of cheating since configure scripts are meant to be extremely portable and use none of bash's features. But I'm willing to go with it to get a release out.

Python config.status

Another candidate: && || bugs

configure scripts use a lot of awk, here docs, and MULTIPLE LEVELS of code genration.

configure.ac is an autoconf/m4 script that generates a configure script, which generates a shell script config.status, which generates Makefile.pre and pyconfig.h. What a huge mess!

TODO: Try to run QEMU's hand-written script?

Env variables: 1. At startup, all environment variables are turned into global shell variables with the 'export' flag set. Previously, I was doing things lazily: the $HOME lookup fall back on getenv(). But this doesn't work because there is this eparate bit of export state.

  1. you can mutate BOTH THE VALUE AND THE EXPORT BIT (with export, export -n) (dash and mksh don't let you manipulate the export bit)

  2. you can export stuff in ANY SCOPE

Other issues:

static vs. dynamic parsing of export. Switched to dynamic because of export $1 in aboriginal. Still struggling.

unset builtin -- this could be parsed statically too. unset a[1] works! Just like delete in Python

Allowed implementing LHS -- ++a[0], etc. Needed the LValue concept for this.

7/1: Going back to descriptor bug.

HARDEST BUG EVER. - reading through generated config.status is really annoying. it uses every weird part of hsell. - I can't log!!! Because stderr is getting redirected. - Opening a separate disk file causes some weird dup2() error I don't understand.

RUNTIME.ASDL and OSH.ASDL need to be closed! Because of my stupid metaprogramming.


/dev/urandom still open -- a problem with using Python interpreter.

I don't even really understand what happened still! Gah!

It took me WAY TO LONG to look at /proc/$PID/fd/ of an OSH process! It has symlinks to open files.

Hard-coded descriptors are an anti-pattern! That means the parent process cannot have any open files! But I guess that's what CLO_EXEC means. I learned that the hard way -- through hours of debugging!

next feature:

eval set a b

-- I'm not joining the arguments!

No gratituous incompatibilities. I made this "fixup" decision along time ago, when I had not come to terms with how inelegant shell/bash can be. Now I am in "copying bugs" mode.

Updated Roadmap

  1. build: finish configure and install. Linux-only for now. At least two distros! Ubuntu, test on Alpine Linux which should be sufficiently different. GNU make (-j) and bash required?
  2. Error messages. Pipefail demo. Which one failed! This is good for showing off.
  3. Run a real program. configure. DONE.
  4. Spec tests: failures below 100. Almost there. Probably just implement LHS array?

Later, maybe OSH 0.2

  1. POSIX build environment. Raspian? BSD? make -j or bash?
  2. More polish of error messages.
  3. work on debootstrap / toybox
  4. No particular goal here; make it organic. Might rise about 100 if we implement failing tests.