Home

X2016/11/26: Blog Roadmap

These are low priority. E-mail me if you want feedback?

top posts: about oil language design. Lots of elements to address:

name a version of the language: oil0? or oil.A

• already touched on arrays
• redirect syntax
• proc vs func
• value semantics

I've written about the parser, but there are interesting things about the runtime architecture as well:

• The shell uses very few system calls. And how do redirects work?
• Wehn a shell construct must run in a subshell, e.g. find | while.
• Quadratic algorithm found in ${var#prefix*}, etc.

Shell Runtime WTFs -- Wat-like - arith recursive evaluation rule - the crazy -eq rule that D360 pointed out. Conjecture: str2int function is used all over the place as an accident.

• Tool Intros
  • what is re2c? - dot his after i've implemented it or at least published the old one

Parser Documentation

• re2c, "pre-structuring" -- better to do this when it's implemented

• 13 or more lexer modes, and lexer hint mechanism. The tool [re2c][] and the underlying algorithm.

CODE VISUALIZATION (i.e. significant work)

• Diagram of 4 parsers and data flow, probably as a graphviz file.

• UML diagram -- I think this is actually useful

  • 4 parsers, 4 executors
  • what members they hold

• File/class/function browser: extract doc strings, as I did for a previous project (poly2). Maybe use epydoc.

• RECAP facts about oil implementation:

  • arith / infix used in 9 different places
  • arith / bool languages overlap
  • there are 4 fully recursive languages, plus regex for bash. There are 2 or 3 sublanguages: glob/extended glob, brace expansion
  • each of them has both a parser and an executor/evaluator

  • there are 2 parsing algorithms used: recursive descent and TDOP - DONE

  • stats

  • there are 219+ Id/tokens (get exact number)
  • there are 7 Node base classes? ABCPRVW... (this may change)
    • how many concrete classes are there?
  • 13 lexical states

• what are lexer hints? - Right paren means 6 things (or more)

• Surfacing docs

  • which ones go first? cheat sheet?

• Good videos on programming language design

  • Clang AST, refactoring tools. Notes I've taken.
  • golang videos? At least it's a blog post.

• Every Single language has a language-specific shell -- discussion with MacInnis, author of Oh shell.

  • none of them have caught on
  • this might be better as a wiki page, or maybe just use Wikipedia

• Overview of Academic Papers. Categories:

  • "Classic" system shells: rc, es
  • Choc thesis has the best overview
  • Academic "workflow" languages for big data pre-big-data
  • Push: dataflow shell
  • Cyber command thingy
  • language-specific shells: caml/schaml. Not as interesting.

• Overview of other shells: elvish, ngs, oh, xonsh -- not a language

• Shell Implementation Comparison:

  • lines of code
  • lexer algorithm
  • parser algorithm
  • strace
  • honestly most of this is low priority.

Code Gen Process (for oil and for other shells)

HN Posts: - Meta-language Challenge - Why parsing is hard -- response to jblow - memory model one -- started a wiki page about this - there shouldn't be a single language and a single OS (this one was sort of angry though). Big misconceptions.

Big Ideas and Software Philosophy

• Distributed software architecture -- static and dynamic architecture / build time vs runtime.
• Unix philosophy and the REST philosophy
• As mentioned: generative vs. recognition-based methods for parsing
• Richard Gabriel's essay on feedback systems
  • logs vs flags -- feedback translated into model params

Alternative shells / Historical trivia

I've collected interesting tidbits in my research, but blogging about them is probably the last priority. It doesn't contribute to the project getting done :)