blog | oilshell.org

BayLISA Presentation Materials

2019-01-18

These are my notes for last night's BayLISA presentation on Oil.

I gave many live demos, so these notes are missing the essence of the talk. But I expect to give a recorded talk covering this same material in the future.

The talk lasted for over 90 minutes, and there was a lively audience with many questions and useful feedback. Thanks to Marc Rovner and the BayLISA board for inviting me to speak!

I prepared too much material, so I skipped everything below the Design and Implementation section. I now realize that's an entirely separate talk.


The main point of the talk is that I'd like experienced users to run their shell scripts with OSH and send me feedback. Please:

I just released OSH 0.6.pre12, which you can download here:

Thank you!

Table of Contents
Audience Background
Intro
About Me
What the Project Is: OSH and Oil
Takeaways for this Talk
Motivation
Slogans to Explain the Project
A Story About a 30-Year-Old Security Problem
Quiz on POSIX Shell
Project Status and History
Demos
Static Parsing and Error Messages
Running Scripts from Linux Distros
Running Interactive Completion Scripts
OSH to Oil translation
JSON Crash Dump
Development Process (testing)
Design and Implementation
Line Counts
Implementation Style / Metaprogramming
Architecture: The Parser as a Library
Open Problems
Criteria for the OSH Language Definition (analogy to POSIX)
Recap
Q&A
Future: Oil Language

Audience Background

I was excited to speak in front of an experienced UNIX audience.

Intro

About Me

Work:

Languages I Use:

What the Project Is: OSH and Oil

Takeaways for this Talk

Motivation

Slogans to Explain the Project

If Tcl does become the "standard scripting language", users will curse it for years — the way people curse Fortran, MSDOS, Unix shell syntax, and other de facto standards they feel stuck with.

Richard Stallman on Usenet, 1994

A second problem with using a Bourne shell compatible language is that field splitting and file name generation are done on every command word. In purely string processing applications, this is not the desired default [...] With ksh it is possible to disable field splitting and/or file name generation on a per function basis, which makes it possible to eliminate this common source of errors.

David Korn in a Usenix paper on ksh, 1994

... nobody really knows what the Bourne shell’s grammar is. Even examination of the source code is little help. The parser is implemented by recursive descent, but the routines corresponding to the syntactic categories all have a flag argument that subtly changes their operation depending on the context.

Tom Duff in a paper on Plan 9's rc shell, 1991

A Story About a 30-Year-Old Security Problem

Suppose you run the following script as root. How many people think the user who supplies the number should be able to delete your hard drive?

# print-incremented.sh
x=$(cat user-supplied-number.txt)
echo $(( x + 1 ))

Related blog posts (from October 2016):

Audit: What do shell scripts use arithmetic for?

Quiz on POSIX Shell

This section was meant to motivate POSIX shell as a bad language — not just bash. But I skipped it because motivation was already well understood.

Project Status and History

Summary:

Details:

Other facts:

Demos

Static Parsing and Error Messages

Notes in retrospect: The audience liked this section, but there was some confusion and I could have done a crisper demo, and explained it more clearly.

Running Scripts from Linux Distros

Success with Aboriginal, Alpine, and Debian Linux

Running Interactive Completion Scripts

OSH to Oil translation

JSON Crash Dump

Development Process (testing)

Design and Implementation

As noted, I skipped everything below in the name of time. It's a separate presentation.

Line Counts

OSH runs many real bash scripts, but it's much smaller than bash.

Caveats:

Implementation Style / Metaprogramming

Oil uses domain-specific languages to "compress" the code:

Summary: All the code in Oil is processed by our own compilers.

Benefits of metaprogramming:

Architecture: The Parser as a Library

See doc/architecture-notes.md

The OSH front end is written in a particular style:

This style enables easy language composition.

Open Problems

Criteria for the OSH Language Definition (analogy to POSIX)

Recap

Q&A

General areas:

Future: Oil Language

This is future work, so I saved it for last.

Slogan: