Warning: Work in progress! Leave feedback on Zulip or Github if you'd like this doc to be updated.

The Oil Language from 10,000 Feet

This document describes the Oil language. See What is Oil? for the larger context.

It also discusses future work.

Table of Contents
Oil Retains These Shell Concepts
Syntactic Concepts
Static Parsing
Parse Options to Take Over @, (), {}, set, and maybe =
Command vs. Expression Mode
Sigils, Sigil Pairs, and Lexer Modes
Syntax
The Expression Language Is Mostly Python
Word Language: Inline Function Calls, Static Printf, Formatters
Homogeneous Arrays
New Keywords: var, set, do, func, return
Dict Literals Look like JavaScript
String Literals may be r or c
Docstrings and Multiline Commands
Runtime Semantics
shopt -s simple-word-eval Does Static Word Evalation
Scope and Namespaces
Functions are like Python/JavaScript
Data Types Are Mostly Python
Special Variables
Shell-Like Builtins
Builtins Accept Long Options
Changed: echo
New: use, push, repr
Builtins Can Take Ruby-Like Blocks
cd, env, and shopt Have Their Own Stack
wait and fork builtins Replace () and & Syntax
each { } Runs Processes in Parallel and Replaces xargs
More Use Cases for Blocks
Configuration Files
Awk Dialect
Make Dialect
Flag Parsing to replace getopts
Unit Tests
Builtin Functions From Python, C
Textual Protocols / Interchange Formats
JSON
TSV2
You Can Influence the Design
How to Give Good Feedback
Deferred Features
Implementation Status
Appendix: Why an Upgrade?

Oil Retains These Shell Concepts

Oil retains them all. A big difference is that keywords rather than builtins are used for assignment.

Syntactic Concepts

Static Parsing

Like Python, JS. See Oil language definition.

Parse Options to Take Over @, (), {}, set, and maybe =

Another concept is parsing modes.

shopt -s all:oil  # most important thing, turns on many options

if ( ) {
}

echo @array

set x = 1
builtin set -o errexit

equals:

x = 1
equivalent to 
const x = 1

This is for Oil as a configuration language.

Command vs. Expression Mode

echo hi

Expression mode in three places:

echo @array
myprog --flag=$f(x, y)
var z = f(x+1, y)

Control FLow:

if grep foo if (foo) {} # replaces if [ -n $foo ]; then

while

for

switch/case -- for translation to C++ like mycpp match/case

Sigils, Sigil Pairs, and Lexer Modes

Sigils:

Sigil Pairs:

Table example:

var people = @{
  name      age:Int
  bob       10_000
  'andy c'  15_000
}
var people = {name: @(bob 'andy c'), age: @[10_000 15_000]}

Sigil pairs often change the lexer mode.

Syntax

The Expression Language Is Mostly Python

Word Language: Inline Function Calls, Static Printf, Formatters

echo ${myfloat %.3f}

Formatters (like template languages)

echo ${myfloat|json}

Homogeneous Arrays

New Keywords: var, set, do, func, return

var x = 1

auto for autovivification:

auto dict['key'] += 1

maybe const. I want that to be compile-time though.

return has to be a keyword and not a builtin because it can take arbitrary data types, e.g.

return {name: 'bob', age: 15}

Dict Literals Look like JavaScript

Later: Data Frames.

String Literals may be r or c

backslashes

Docstrings and Multiline Commands

##

%%%

Runtime Semantics

shopt -s simple-word-eval Does Static Word Evalation

Mentioned in the last post. This is big!

Scope and Namespaces

The use builtin will provide scope.

Functions are like Python/JavaScript

But take simplified keyword args like Julia.

Data Types Are Mostly Python

Special Variables

@ARGV and ARGV

Shell-Like Builtins

Assignment builtins local, declare, export

Builtins Accept Long Options

TODO: Link HN thread

Changed: echo

New: use, push, repr

Builtins Can Take Ruby-Like Blocks

NOTE: Haven't decided if blocks have parameters or now? Maybe just magic variables.

cd ~/src { echo $PWD } replaces pushd/popd pairs

Shell options have a stack (modernish also has this):

shopt -s errexit {
}

Env is now a builtin and takes a block

env FOO=bar {
}

fork builtin replaces the weird & terminator:

{ echo 1; sleep 1; echo 2; } &

fork { echo 1; sleep 1; echo 2 }

wait builtin replaces ( ), because ( ) is going

cd, env, and shopt Have Their Own Stack

Like Python's context manager.

wait and fork builtins Replace () and & Syntax

each { } Runs Processes in Parallel and Replaces xargs

each func

each { }

More Use Cases for Blocks

Configuration Files

Evaluates to JSON (like YAML and TOML):

server foo {
  port = 80
}

And can also be serialized as command line flags.

Replaces anti-patterns:

Awk Dialect

BEGIN {
  end
}

when x {
}

Make Dialect

rule foo.c : foo.bar {
  cc -o out @srcs
}

Flag Parsing to replace getopts

Probably use a block format. Compare with Python's optparse.o

See issue.

Unit Tests

Haven't decided on this yet.

check {
}

Builtin Functions From Python, C

Python and shell both borrow from C. Oil borrows from Python and C.

Textual Protocols / Interchange Formats

JSON

TSV2

text over binary

You Can Influence the Design

How to Give Good Feedback

Deferred Features

Implementation Status

Appendix: Why an Upgrade?

Hejlsberg quote?


Generated on Mon Feb 17 16:48:51 PST 2020