NOTE: This document is a work in progress!

OSH Quick Reference

Version 0.7.pre3

- Below is a list of topics, organized into [Sections].
- The X prefix means "unimplemented".
- HTML version: https://www.oilshell.org/release/0.7.pre3/doc/osh-quick-ref.html

INTRO
  [Overview]      overview   osh-vs-oil   command-vs-expr
  [Usage]         bundle-usage   osh-usage   oil-usage   config   startup
                  line-editing   prompt
  [Lexing]        comments #   line-continuation \
  [Oil Lexing]    single-command %%%   docstring ###

COMMAND LANGUAGE
  [Commands]      simple-command   semicolon ;
  [Conditional]   case   if   true   false   colon :
                  bang !   and &&   or ||   dbracket [[
  [Iteration]     while   until   for   for-expr-sh ((
  [Control Flow]  break   continue   return   exit
  [Grouping]      function   block {   subshell (
  [Concurrency]   pipe   |   X |&
                  ampersand &
  [Redirects]     redir-file  >  >>  >|  <  <>   X &>
                  redir-desc  >&  <&
                  here-doc    <<  <<-  <<<
  [Other]         dparen ((   time   X coproc   X select
  [Oil Keywords]  func   do   pass   fs
                  (Change parsing so they can't be builtins)

ASSIGNING VARIABLES
  [Operators]     assign        str='xyz'
                  append        str+='abc'
  [Compound Data] array         array=(a b c)   array[1]=B   "${a[@]}"
                  assoc         assoc=(['a']=1 ['b']=2)   assoc['x']=b
  [Builtins]      local   readonly   export   unset   shift
                  declare   typeset   X let
  [Oil Keywords]  var   setvar   auto

OIL EXPRESSION LANGUAGE
X [Data Types]    Str           r'\'   c"\n"   multiline r""" c'''
                  Bool          true false
                  Int           1_000_000
                  Float         3.14
                  Array<>       @[ls -l]  @[T F F]  @[1 2 3]  @[1.5 2.5] 
                  Dict<>        {name: 'oil'}  {['name']: 'oil'}  {name}
                  List          [1, 'str', false]  (for JSON compatibility)
                  Null          null
                  Regex         $/ d+ /
                  Func          func(x) { return x+1 }
X [Comprehension] Array, List, Dict
X [Operators]     unary         -a
                  binary        a+b   base^exp  a xor b  p div q
                  ternary       a if cond else b
                  subscript     a[b, c]   a[start:end]
                  chain         pass a => f(y, z) => var new
                  getattr       d->key is like d['key'] or d.key
                  scope-attr    module::name
X [Functions]     func-decl     func inc(a Int, b Int = 1) Int { ... }
                  cmd-call      call f(x, y); echo $foo
                  word-call     echo $strfunc(x, y) @arrayfunc(z)

WORD LANGUAGE
  [Quotes]        quotes        'abc'  $'\n'  "$var"
  [Substitutions] com-sub       $(command)   `command`
                  var-sub       ${var}
                  arith-sub     $((1 + 2))  $[1 + 2]
                  tilde-sub     ~/src
                  proc-sub      diff <(sort L.txt) <(sort R.txt)
  [Special Vars]  special-vars  $@  $*  $#     $?  $-     $$  $!
  [Var Ops]       op-test       ${x:-default}  
                  op-unary      ${x%%suffix}  etc.
                  op-str        ${x//y/z}
                  op-slice      ${a[@]:0:1}
                  op-format     ${x@P}
X [Oil Word]      oil-printf    ${x %.3f}
                  oil-format    ${x|html}

OTHER SHELL SUBLANGUAGES
  [Arithmetic]    arith-context Where legacy arithmetic is allowed
                  num-literals  0xFF  0755  etc.
                  math          1 + 2*3
                  arith-logical !a && b
                  bitwise       ~a ^ b
                  arith-assign  a *= 2
  [Boolean]       dbracket      [[ vs. the test builtin
                  bool-expr       [[ ! $x && $y || $z ]]
                                test ! $x -a $y -o $z
                  bool-infix    [[ $a -nt $b ]]  [[ $x == $y ]]
                  bool-path     [[ -d /etc ]]
                  bool-str      [[ -z '' ]]
                  bool-other    [[ -o errexit ]]
  [Patterns]      glob          *.py
                  extglob       @(*.py|*.sh)
                  regex         [[ foo =~ [a-z]+ ]]
  [Brace Expand]  braces        {alice,bob}@example.com
  [History]       histsub       !$  !!  !n

BUILTIN COMMANDS
  [I/O]           read   echo 
                  X readarray   X mapfile
  [Run Code]      source .   eval   trap
  [Set Options]   set   shopt
  [Working Dir]   cd   pwd   pushd   popd   dirs
  [Completion]    complete   compgen   compopt   compadjust
  [Shell Process] exec   X logout 
                  umask   X ulimit   X times
  [Child Process] jobs   wait   ampersand &
                  fg   X bg   X disown 
  [External]      test [   printf   getopts   X kill
  [Introspection] help   hash   type   X caller
  [Word Lookup]   command   builtin
  [Interactive]   alias   unalias   history   X fc   X bind
X [Unsupported]   enable
X [Oil Builtins]  log   die              common functions
                  dirname   basename     simple optimization
                  cd   shopt   env       compatible, and takes a block
                  fork   wait            replaces & and (), takes a block
                  use                    source with namespace, file-relative 
                  push                   sugar for 'do array.push( @(a b) )'

SHELL OPTIONS
  [Errors]        nounset   errexit   pipefail
  [Globbing]      noglob   failglob   nullglob
  [Debugging]     xtrace   X verbose   X extdebug
  [Interactive]   emacs   vi
  [Other]         X noclobber
  [OSH Strict]    all:strict             All options starting with 'strict-'
                  strict-argv            No empty argv
                  strict-array           Arrays don't decay to strings
                  strict-arith           Fatal parse errors (on by default)
                  strict-errexit         Fail earlier, at command sub 
                  strict-control-flow    Do we need this?  Special builtins?
                  strict-word-eval       Expose unicode and slicing errors
                  X strict-tilde         Tilde subst can result in error
                  X strict-backslash     Parse the sublanguage more strictly
                  X strict-glob          Parse the sublanguage more strictly
                  X strict-eval-builtin  Single arg
                  X strict-trap          Function name only
X [Oil Semantics] all:oil                All options below, plus POSIX and bash
                  static-word-eval       No splitting, static globbing
                  longopts               test -file, shopt -unset, etc.
                  oil-echo               No -e or -n, one arg per line
                  oil-test-builtin       Only file tests (no strings), remove [
X [Oil Syntax]    all:oil-parse
                  oil-parse-at           echo @array @arrayfunc(x, y)
                  oil-parse-brace        cd ~/src { ... }
                  oil-parse-equals       x = 's' (for cleaner config snippets)
                  oil-parse-paren        if (x > 0) ...
                  oil-parse-set          instead of setvar

ENVIRONMENT VARIABLES
  [Shell Options] SHELLOPTS   X BASHOPTS
  [Other Vars]    HOME   PATH   IFS
  [Oil Paths]     ?builtins   ?completion_plugins   ?coprocesses

SPECIAL VARIABLES
  [Oil]           ARGV
X [Platform]      HOSTNAME   OSTYPE   BASH_VERSION   @BASH_VERSINFO
  [Call Stack]    @BASH_SOURCE   @FUNCNAME   @BASH_LINENO   
                  X BASH_ARGV   X BASH_ARGC
  [Tracing]       LINENO   SOURCE_NAME
  [Process State] X BASHPID   X PPID   UID   EUID   
X [Process Stack] BASH_SUBSHELL   SHLVL
X [Shell State]   BASH_CMDS   @DIRSTACK
  [Completion]    @COMP_WORDS   COMP_CWORD   COMP_LINE   COMP_POINT
                  COMP_WORDBREAKS   @COMPREPLY   X COMP_KEY   
                  X COMP_TYPE   COMP_ARGV

  [cd]            PWD   OLDPWD   X CDPATH
  [getopts]       OPTIND   OPTARG   X OPTERR
  [read]          REPLY   IFS
  [Functions]     X RANDOM   X SECONDS
  [Other]         BASH_REMATCH   @PIPESTATUS

PLUGINS AND HOOKS
  [Signals]       SIGTERM   X SIGINT   X SIGABRT   SIG...
  [Traps]         EXIT   X ERR   X DEBUG   X RETURN
  [Words]         PS1   X PS2   X PS3   PS4
  [Completion]    complete
  [Other]         X command_not_found   PROMPT_COMMAND

OIL EXTENSIONS

X [awk]           BEGIN   END   when
X [make]          rule (builtin)
X [xargs]         each (builtin)
X [find]          fs

OIL LIBRARIES

X [Builtin Funcs] join()   split()    $IFS algorithm or awk algorithm
                  read(n)             better than read -n, no short reads?
                  write()             replaces echo -n
                  posix::read()       raw bindings?
                  strftime()
X [getopts]       ?
X [Testing]       check ?
X [Data Formats]  json   csv   tsv2   struct (binary)
X [Hash Functions]








Introduction


Overview


overview


OSH is a shell.


Usage


Usage of the Oil App Bundle

Usage: oil.ovm MAIN_NAME [ARG]...
       MAIN_NAME [ARG]...

oil.ovm behaves like busybox.  If it's invoked through a symlink, e.g. 'osh',
then it behaves like that binary.  Otherwise the binary name can be passed as
the first argument, e.g.:

    oil.ovm osh -c 'echo hi'

Usage of the OSH Binary

Usage: osh [OPTION]... SCRIPT [ARG]...
       osh [OPTION]... -c COMMAND [ARG]...

osh accepts POSIX sh flags, with the following differences:

  -n             only validate the syntax.  Also prints the AST.
  --show-ast     print the AST in addition to executing.
  --ast-format   what format the AST should be in


Usage of the Oil Binary

Usage: oil [OPTION]... SCRIPT [ARG]...
       oil [OPTION]... -c COMMAND [ARG]...

TODO: any changes?

Configuration Files


If the --rcfile flag is specified, osh and oil will source that on startup.

Otherwise they source ~/.config/oil/oshrc and ~/.config/oil/oilrc,
respectively.

To disable startup files, pass --rcfile /dev/null.

Shell Startup


Line Editing


Oil currently has support for building against GNU readline.

Customizing the Prompt String


OSH supports bash-compatible $PS1 syntax.

Lexing


Oil Lexing


The % Prefix Starts a Single Command Over Multiple Lines


This special lexer mode has several use cases:

Long command lines without trailing \

    % chromium-browser
      --no-proxy-server
      # comments allowed
      --incognito

Long pipelines or and-or chains without trailing \ 

    % find .
    # exclude tests
    | grep -v '_test.py'
    | xargs wc -l
    | sort -n

    %  ls /
    && ls /bin
    && ls /lib
    || error "oops"

Using {} for brace expansion, rather than the start of a block:

    % echo {alice,bob}@example.com
    %
    echo next   # blank line or bare % required to end the command

NOTE: This should be valid without % :

    ls *.[ch]

Using Oil syntax at an OSH shell prompt:

    $   echo hi >&2    # POSIX sh syntax
    $ % echo hi > !2   # Oil syntax

Command Language


The command language is specified by the POSIX shell grammar.

Commands


Simple Commands

Simple commands are separated by words:
    ls /

Redirects can also appear anywhere
    echo hi 1>&2

Semi-colon ;

;  -- separate statements

Conditional Constructs


case



if


true false colon :


bang !


and && or ||


dbracket [[

For conditionals.

Iteration Constructs


while until


for for-expr


Grouping Constructs


function


block


subshell


Concurrency


pipe


ampersand &


Redirects


redir-file


redir-desc


here-doc


Other Commands


dparen ((


time


coproc



Assigning Variables


Assignment Keywords


Assignment Operators


Compound Data Structures



Word Language


Quotes


Substitutions


Special Variables


Operations on Variables


Variable Formatting Operators


${x@P} evaluates x as a prompt string, e.g. the string that would be printed if
PS1=$x.

Other Shell Sublanguages


Arithmetic


Boolean


Patterns


Brace Expansion


Builtin Commands


OSH aims to have almost all of the builtins that bash does.  Here they are,
divided into sections.

I/O Builtins


These builtins take input and output.  They are often used with redirects[1].

[1] help redirects

read

Usage: read -p 

Or maybe get rid of #END -- it can just go until the next # command.  It's a
little bit like the spec tests honestly.  Can copy sh_specpy

Run Code

source .   eval

Set Shell Options

set   X shopt

Builtins - Working Dir

cd   pwd   pushd   popd   dirs

Completion


complete


Register completion policies for different commands.

compgen


Generate completion candidates inside a user-defined completion function.

compopt


Change completion options inside a user-defined completion function.

compadjust


Adjust COMP_ARGV according to specified delimiters, and optionally set
variables cur, prev, words (an array), and cword.  May also set 'split'.

This is an OSH extension that makes it easier to run the bash-completion
project.

Shell Process Control

exec   exit   X logout 
umask   X ulimit   X trap   X times

Child Process Control

jobs   wait   ampersand &
X fg   X bg   X disown 

Builtins That Introspect


help

Usage:
  help <topic>   -- show help on a given topic
  help toc       -- list help topics
  help osh-usage -- same as osh --help
  help oil-usage -- same as oil --help

View on the web: http://www.oilshell.org/$VERSION/doc/osh-quick-ref.html

hash


caller


type


Builtins That Are Like External Commands


External: bash has builtins that replace these external commands, but OSH
doesn't)

getopt


use /usr/bin/getopt

kill


bash accepts job control syntax

enable


Bash has this, but OSH won't implement it.

Shell Options



Parsing Options


Execution Options


Options Only in OSH


Environment Variables


Variables That Set Shell Options


SHELLOPTS


For the 'set' builtin.

BASHOPTS


For the 'shopt' builtin.

Other Variables


HOME


$HOME is used for:

1. ~ expansion 
2. ~ abbreviation in the UI (the dirs builtin, \W in $PS1).

Note: The shell doesn't set $HOME.  According to POSIX, the program that
invokes the login shell sets it based on /etc/passwd.

PATH


Used for finding executables to run.

IFS


Used for word splitting.

Special Variables


Platform


Call Stack


Tracing


Process State


Process Stack


Shell State


Completion


COMP_WORDS


An array of words, split by : and = for compatibility with bash.  New
completion scripts should use COMP_ARGV instead.

COMP_CWORD


Discouraged; for compatibility with bash.

COMP_LINE


Discouraged; for compatibility with bash.

COMP_POINT


Discouraged; for compatibility with bash.

COMPREPLY


User-defined completion functions should Fill this array with candidates.  It
is cleared on every completion request.

COMP_ARGV


An array of partial command arguments to complete.  Preferred over COMP_WORDS.
The compadjust builtin uses this variable.

Plugins and Hooks


Signals


Traps


Words


PS1


First line of a prompt.

PS2


Second line of a prompt.

PS3


For the 'select' builtin (unimplemented).

PS4


For 'set -o xtrace'.  The leading character is special.

Oil Extensions


Oil Libraries


Builtins Procs


Builtins Funcs


strftime()


Useful for logging callbacks.  NOTE: bash has this with the obscure 
printf '%(...)' syntax.

Generated on Fri Aug 9 11:43:28 PDT 2019