Warning: Work in progress! Leave feedback on Zulip or Github if you'd like this doc to be updated.

Oil Help

This doc describes every aspect of Oil and OSH briefly, and is indexed by keywords.

The help builtin prints portions of it.

You can also navigate this doc with the help index.

Table of Contents
Overview
Usage
Lexing
Oil Lexing
Command Language
Commands
Conditional
Iteration
Control Flow
Grouping
Concurrency
Redirects
Other Command
Oil Keywords
Coil Keywords
Assigning Variables
Operators
Compound Data
Builtins
Oil Keywords
Oil Expression Language
Functions
Literals
Regexes
Word Language
Quotes
Substitutions
Var Ops
Oil Word
Other Shell Sublanguages
Arithmetic
Boolean
Patterns
Brace Expand
History
Builtin Commands
I/O
Run Code
Set Options
Working Dir
Completion
Shell Process
Child Process
External
Introspection
Word Lookup
Interactive
Oil Builtins
Shell Options
Errors
Globbing
Debugging
Interactive
Other Option
strict:all
oil:basic
oil:all
Environment Variables
Shell Options
Other Env
Oil Paths
Special Variables
Special
POSIX Special
Other Special
Oil Special
Platform
Call Stack
Tracing
Process State
Process Stack
Shell State
Completion
Functions
Other Special
Plugins and Hooks
Signals
Traps
Words
Completion
Other Plugin
Oil Libraries
Collections
Pattern
String
Better Syntax
Arrays
Assoc Arrays
Block
libc
Hashing

Overview

Usage

This section describes how to use the Oil binary.

bin/osh Usage

Usage: osh [OPTION]... SCRIPT [ARG]...
       osh [OPTION]... -c COMMAND [ARG]...

The command line accepted by bin/osh is compatible with /bin/sh and bash.

osh -c 'echo hi'
osh myscript.sh
echo 'echo hi' | osh

It also has a few enhancements:

osh -n -c 'hello'                    # pretty-print the AST
osh --ast-format text -n -c 'hello'  # print it full

osh accepts POSIX sh flags, with these additions:

-n parse the program but don't execute it. Print the AST. --ast-format what format the AST should be in

bin/oil Usage

Usage: oil  [OPTION]... SCRIPT [ARG]...
       oil [OPTION]... -c COMMAND [ARG]...

bin/oil is the same as bin/osh with a the oil:all option group set. So bin/oil also accepts shell flags.

oil -c 'echo hi'
oil myscript.oil
echo 'echo hi' | oil

App Bundle Usage

Usage: oil.ovm MAIN_NAME [ARG]...
       MAIN_NAME [ARG]...

oil.ovm behaves like busybox. If it's invoked through a symlink, e.g. 'osh', then it behaves like that binary. Otherwise the binary name can be passed as the first argument, e.g.:

oil.ovm osh -c 'echo hi'

Configuring the Shell

If the --rcfile flag is specified, that file will be executed on startup. Otherwise:

Pass --rcfile /dev/null to disable this behavior.

Startup Files

History is read?

Lexing

comments

A comment starts with # and goes until the end of the line.

echo hi  # print a greeting

line-continuation

A backslash \ at the end of a line continues the line without executing it:

ls /usr/bin \
   /usr/lib \
   ~/src        # A single command split over three lines

Oil Lexing

single-command

The %%% prefix Starts a Single Command Over Multiple Lines (?)

This special lexer mode has several use cases:

Long command lines without trailing \

%%% chromium-browser
    --no-proxy-server
    # comments allowed
    --incognito

Long pipelines or and-or chains without trailing \

%%% find .
    # exclude tests
  | grep -v '_test.py'
  | xargs wc -l
  | sort -n

%%% ls /
 && ls /bin
 && ls /lib
 || error "oops"

docstring

TODO

Command Language

Commands

simple-command

Commands are composed of words. The first word may by the name of a shell builtin, an Oil proc / shell "function", an external command, or an alias:

echo hi               # a shell builtin doesn't start a process
ls /usr/bin ~/src     # starts a new process
myproc "hello $name"
myshellfunc "hello $name"
myalias -l

Redirects are also allowed in any part of the command:

echo 'to stderr' >&2
echo >&2 'to stderr'

echo 'to file' > out.txt
echo > out.txt 'to file'

semicolon ;

Run two commands in sequence like this:

echo one; echo two

or this:

echo one
echo two

Conditional

case

Match a string against a series of glob patterns. Execute code in the section below the matching pattern.

path='foo.py'
case "$path" in
  *.py)
    echo 'python'
    ;;
  *.sh)
    echo 'shell'
    ;;
esac

if

Test if a command exited with status zero (true). If so, execute the corresponding block of code.

Shell:

if test -d foo; then
  echo 'foo is a directory'
elif test -f foo; then
  echo 'foo is a file'
else
  echo 'neither'
fi

Oil:

if test -d foo {
  echo 'foo is a directory'
} elif test -f foo {
  echo 'foo is a file'
} else {
  echo 'neither'
}

true

Do nothing and return status 0.

if true; then
  echo hello
fi

false

Do nothing and return status 1.

if false; then
  echo 'not reached'
else
  echo hello
fi

Iteration

for

For loops iterate over words.

Oil style:

var mystr = 'one'
var myarray = @(two three)

for i in $mystr @myarray *.py; done
  echo $i
done

Shell style:

mystr='one'
myarray=(two three)

for i in "mystr" "${myarray[@]}" *.py; done
  echo $i
done

Both fragments output 3 lines and then Python files on remaining lines.

Control Flow

Grouping

Concurrency

Redirects

redir-file

Three variants of redirecting stdout:

echo foo > out.txt    # write to a file
echo foo >> out.txt   # append to a file
echo foo >| out.txt   # clobber the file even if set -o noclobber

Redirect stdin:

cat < in.txt

redir-desc

Redirect to a file descriptor:

echo 'to stderr' >&2

here-doc

TODO: unbalanced HTML if we use <<?

cat <<EOF
here doc with $double ${quoted} substitution
EOF

myfunc() {
        cat <<-EOF
        here doc with one tab leading tab stripped
        EOF
}

cat <<< 'here string'

Other Command

dparen ((

time

time [-p] pipeline

Measures the time taken by a command / pipeline. It uses the getrusage() function from libc.

Note that time is a KEYWORD, not a builtin!

Oil Keywords

const

Initializes a constant name to the Oil expression on the right.

const c = 'mystr'        # equivalent to readonly c=mystr
const pat = / digit+ /   # an eggex, with no shell equivalent

It's either a global constant or scoped to the current function.

var

Initializes a name to the Oil expression on the right.

var s = 'mystr'        # equivalent to declare s=mystr
var pat = / digit+ /   # an eggex, with no shell equivalent

It's either a global or scoped to the current function.

setvar

Like shell's x=1, setvar x = 1 either:

It's meant for interactive use and to easily convert existing shell scripts.

New Oil programs should use set, setglobal, or setref instead of setvar.

set

A shorter name for setlocal in the Oil language. Requires shopt -s parse_set, because otherwise it would conflict with the set builtin. Use builtin set -- 1 2 3 to get the builtin, or shopt -o to change options.

setlocal

Mutates an existing variable in the current scope. If it doesn't exist, the shell exits with a fatal error.

setglobal

Mutates a global variable. If it doesn't exist, the shell exits with a fatal error.

setref

Mutates a variable through a named reference. TODO: Show example.

Coil Keywords

Assigning Variables

Operators

Compound Data

array

Array literals in shell accept any sequence of words, just like a command does:

ls $mystr "$@" *.py

# Put it in an array
a=(ls $mystr "$@" *.py)

In Oil, use oil-array.

Builtins

Oil Keywords

Oil Expression Language

Functions

proc-decl

func-call

Literals

oil-string

Oil strings appear in expression contexts, and look like shell strings:

var s = 'foo'
var double = "hello $world and $(hostname)"

However, strings with backslashes are forced to specify whether they're raw strings or C-style strings:

var s = 'line\n'    # parse error: ambiguous

var s = c'line\n'   # C-style string
var s = $'line\n'   # also accepted for compatibility

var s = r'[a-z]\n'  # raw strings are useful for regexes (not eggexes)

oil-array

Like shell arays, Oil arrays accept anything that can appear in a command:

ls $mystr @ARGV *.py

# Put it in an array
var a = @(ls $mystr @ARGV *.py)

Regexes

Word Language

Quotes

quotes

Also see oil-string.

Substitutions

com-sub

Evaluates to the stdout of a command. If a trailing newline is returned, it's stripped:

$ hostname
example.com

$ x=$(hostname)
$ echo $x
example.com

var-sub

Evaluates to the value of a variable:

$ x=X
$ echo $x ${x}
X X

arith-sub

Shell has C-style arithmetic:

$ echo $(( 1 + 2*3 ))
7

tilde-sub

Used as a shortcut for a user's home directory:

~/src     # my home dir
~bob/src  # user bob's home dir

Var Ops

op-format

${x@P} evaluates x as a prompt string, e.g. the string that would be printed if PS1=$x.

Oil Word

Other Shell Sublanguages

Arithmetic

Boolean

Patterns

Brace Expand

History

Builtin Commands

I/O

These builtins take input and output. They're often used with redirects.

read

read FLAGS* VAR*

Read a line from stdin, split it into tokens with the $IFS algorithm, and assign the tokens to the given variables. When no VARs are given, assign to $REPLY.

Flags:

-a ARRAY  assign the tokens to elements of this array
-d CHAR   use DELIM as delimiter, instead of newline
-n NUM    read up to NUM characters, respecting delimiters
-p STR    print the string PROMPT before reading input
-r        raw mode: don't let backslashes escape characters
-s        silent: do not echo input coming from a terminal
-t NUM    time out and fail after TIME seconds
          -t 0 returns whether any input is available
-u FD     read from file descriptor FD instead of 0 (stdin)

echo

echo FLAGS* ARG*

Prints ARGs to stdout, separated by a space, and terminated by a newline.

Flags:

-e  enable interpretation of backslash escapes
-n  omit the trailing newline

Backslash sequences recognized by -e:

\\         backslash
\a         (unimplemented ?) alert (BEL)
\b         backspace
\c         stop processing remaining input
\e         escape next character
\f         form feed, equivalent to \n + 4 spaces
\n         new line
\r         carriage return, returns to the beggining of the line
\t         horizontal tab
\v         vertical tab
\0NNN      print character specified as an octal value with 1 to 3 octal
           digits 
\xHH       print character specified as an hexadecimal value with 1 to 2
           hex digits
\uHHHH     Unicode character specified as an hexadecimal value with 1 to
           4 hex digits 
\UHHHHHHHH Unicode character specified as an hexadecimal value with 1 to
           8 hex digits

readarray

Alias for mapfile

mapfile

The mapfile builtin reads lines from stdin into an array.

mapfile FLAGS* ARRAY*

ARRAY: Where the read lines will be stored. If no variable is specified, MAPFILE will be used.

FLAGS: -t delete the delimiter from each line

Run Code

source

The source builtin executes a script, which is run in the context of the running shell and can modify local variables (as opposed to running in a subshell, where it can't)

source SCRIPT ARGS*

SCRIPT: Path to the script that you want to run.

ARGS: One or more arguments for the script.

eval

The eval builtin constructs a string from the arguments, then runs the string as a command. It is very powerful, but must be used with care, as it can introduce security problems (see https://mywiki.wooledge.org/BashFAQ/048)

eval ARGS+

ARGS: One or more arguments that will be used to construct a command.

EXAMPLES: a='echo '; b='Hello, '; c='World!' eval $a $b $c This will create the string "echo Hello, World!" and run it.

trap

The trap builtin runs commands after a signal is received.

trap FLAGS* CMD SIGNALS

FLAGS: -l Lists all signals and their signal number -p Prints a list of the installed signal handlers

CMD: Command to be run. If the command is the null string, the signal is ignored.

SIGNALS: List of signals to be caught.

Set Options

set

The set builtin modifies the shell's configuration and behavior.

shopt

The shopt builtin configures the shell.

Working Dir

cd

The cd builtin changes the working directory.

cd FLAGS* DIR

DIR: The path where you want to move. If not specified, variable HOME is used. Path "~" is also an alias for the variable HOME. If the specified path is "-", variable OLDPWD will be used, which is set by the shell to the previous working directory.

FLAGS: -L Follow symbolic links, working directory will be the one pointed by the symbolic link (default behavior). -P Don't follow symbolic links, working directory will be the symbolic link.

pwd

The pwd builtin prints the current working directory.

pwd FLAGS

FLAGS: -L If present in the path, follow symbolic links (default behavior) -P Don't follow symbolic links, print the link instead of the reference.

pushd

The pushd builtin adds a directory to the directory stack, and changes the working directory to it. This is typically used along with popd and dirs.

pushd DIR

DIR: File system directory where you want to move and which you want to stack.

popd

The popd builtin removes a directory from the directory stack, and changes the working directory to it. Typically used along with pushd and dirs.

popd

dirs

The dirs builtin shows the contents of the directory stack. Typically used along with pushd and popd.

dirs FLAGS*

FLAGS: -c Removes all the directories from the stack. -l Show the directory stack, but with the real path instead of ~. -p Show the directory stack, but formatted as one line per entry. -v Like -p, but numbering each line.

Completion

complete

The complete builtin registers completion policies for different commands.

compgen

The compgen builtin generates completion candidates inside a user-defined completion function.

It can also be used in scripts, i.e. outside a completion function.

compopt

The compopt builtin changes completion options inside a user-defined completion function.

compadjust

The compadjust builtin adjusts COMP_ARGV according to specified delimiters, and optionally set variables cur, prev, words (an array), and cword. May also set 'split'.

This is an OSH extension that makes it easier to run the bash-completion project.

Shell Process

exec

The exec builin replaces the running shell with the specified command.

exec CMD ARGS*

CMD ARGS: The command to be run and its arguments.

umask

The umask builtin sets the mask that determines the permissions for new files and directories. The mask is substracted from 666 for files and 777 for directories.

umask MODE?

MODE: Mask to be used, in octal mode. If none is specified, the current mask will be shown.

times

The times builtin shows the user and system time used by the shell and all its children processes.

times

Child Process

jobs

The jobs builtin shows all jobs running in the shell and its status.

jobs

wait

The wait builtin waits for the death of a job or a process and returns its exit status.

wait FLAGS* PID|JOB*

FLAGS: -n Wait for the next job termination.

PID|JOB: The PID of the process, or the job number, which you want to wait for, or a job is specified as '%jobnumber'. If none is specified, wait will wait for all the active child processes.

ampersand

The '&' builtin runs a command in the background as a job, and immediately returns the control to the shell. The PID of the command is recorded in the $! variable.

CMD &

CMD: The command to be run.

fg

The fg builtin returns a job running in the background to the foreground.

fg JOB?

JOB: Job ID to be moved to the foreground. If none is specified, the latest job is chosen.

External

kill

TODO

enable

Bash has this, but OSH won't implement it.

Introspection

help

help index           # list all help topics
help index GROUP...  # list help topics in the given groups
help TOPIC           # show help on a given topic
help osh-usage       # same as osh --help
help oil-usage       # same as oil --help

View on the web: http://www.oilshell.org/$VERSION/doc/osh-quick-ref.html

Word Lookup

Interactive

Oil Builtins

repr

Displays the internal representation of a cell. (Cells are locations for values like strings and arrays.)

Shell Options

Errors

Globbing

nullglob

Normally, when no files match a glob, the glob itself is returned:

$ echo L *.py R  # no Python files in this dir
L *.py R

With nullglob on, the glob expands to no arguments:

shopt -s nullglob
$ echo L *.py R
L R

(This option is in GNU bash as well.)

dashglob

Do globs return results that start with -? It's on by default in bin/osh, but off when Oil is enabled.

Turning it off prevents a command like rm * from being confused by a file called -rf.

$ touch -- myfile -rf

$ echo *
-rf myfile

$ shopt -u dashglob
$ echo *
myfile

Debugging

Interactive

Other Option

strict:all

strict_tilde

Failed tilde expansions cause hard errors (like zsh) rather than silently evaluating to ~ or ~bad.

strict_word_eval

TODO

strict_nameref

When strict_nameref is set, undefined references produce fatal errors:

declare -n ref
echo $ref  # fatal error, not empty string
ref=x      # fatal error instead of decaying to non-reference

References that don't contain variables also produce hard errors:

declare -n ref='not a var'
echo $ref  # fatal
ref=x      # fatal

parse_ignored

For compatibility, Oil will parse some constructs it doesn't execute, like:

return 0 2>&1  # redirect on control flow

When this option is disabled, that statement is a syntax error.

oil:basic

simple_word_eval

TODO:

oil:all

Environment Variables

Shell Options

SHELLOPTS

For the 'set' builtin.

BASHOPTS

For the 'shopt' builtin.

Other Env

HOME

$HOME is used for:

  1. ~ expansion
  2. ~ abbreviation in the UI (the dirs builtin, \W in $PS1).

Note: The shell doesn't set $HOME. According to POSIX, the program that invokes the login shell sets it based on /etc/passwd.

PATH

A colon-separated string that's used to find executables to run.

IFS

Used for word splitting. And the builtin split() function.

Oil Paths

Special Variables

Special

POSIX Special

Other Special

Oil Special

Platform

Call Stack

Tracing

Process State

Process Stack

Shell State

Completion

COMP_WORDS

An array of words, split by : and = for compatibility with bash. New completion scripts should use COMP_ARGV instead.

COMP_CWORD

Discouraged; for compatibility with bash.

COMP_LINE

Discouraged; for compatibility with bash.

COMP_POINT

Discouraged; for compatibility with bash.

COMPREPLY

User-defined completion functions should Fill this array with candidates. It is cleared on every completion request.

COMP_ARGV

An array of partial command arguments to complete. Preferred over COMP_WORDS. The compadjust builtin uses this variable.

(An OSH extension to bash.)

Functions

Other Special

Plugins and Hooks

Signals

Traps

Words

PS1

First line of a prompt.

PS2

Second line of a prompt.

PS3

For the 'select' builtin (unimplemented).

PS4

For 'set -o xtrace'. The leading character is special.

Completion

Other Plugin

Oil Libraries

Collections

len()

copy():

var d = {name: value}

var alias = d  # illegal, because it can create ownership problems
               # reference cycles
var new = copy(d)  # valid

Pattern

String

Better Syntax

These functions give better syntax to existing shell constructs.

Arrays

Assoc Arrays

Block

libc

strftime()

Useful for logging callbacks. NOTE: bash has this with the obscure printf '%(...)' syntax.

Hashing


Generated on Tue Aug 18 10:26:21 PDT 2020