OSH User Manual

OSH is a Unix shell designed to run existing shell scripts. More precisely, it's

  1. POSIX-compatible
  2. Has features from GNU Bash, the most commonly used shell.

It's designed to be "stricter" than other shells. To avoid programs that don't behave as intended,

  1. It produces more errors.
  2. It produces them earlier — at parse time, if possible.

"Batch" programs are most likely to run unmodified under OSH. Interactive programs like .bashrc and bash completion scripts may require small changes.

This manual covers the differences between OSH and other shells. It leaves the details of each construct to the help builtin and the Quick Reference (Warning: both are incomplete). It also doesn't cover the Oil language, which is a newer part of the Oil project.

Existing educational materials for the Unix shell apply to OSH, because they generally don't teach the quirks that OSH disallows. For example, much of the information and advice in BashGuide can be used without worrying about which shell you're using. See the end of this manual for more resources.

Table of Contents
Downloading OSH
Setup
Startup Files
Global Execution Options
shopt -s strict:all is Recommended
List of Options
OSH Has Four errexit Options (while Bash Has Two)
Quirk 1: the Shell Sometimes Disables And Restores errexit
Quirk 2: x=$(false) is inconsitent with local x=$(false)
Additional errexit options
Example
Recap/Summary
Features Unique to OSH
Dumping the AST
OSH_HIJACK_SHEBANG
--debug-file
Crash Dumps
Completion API
Exit Codes
Unicode
Program Encoding
Data Encoding
Bugs
Links

Downloading OSH

The releases page links to source tarballs for every release. It also links to the documentation tree, which includes this manual.

Setup

After running the instructions in INSTALL, run:

mkdir -p ~/.config/oil

Startup Files

On startup, the interactive shell sources only ~/.config/oil/oshrc.

Other shells have a confusing initialization sequence involving many files (original). It's very hard to tell when and if /etc/profile, ~/.bashrc, ~/.bash_profile, etc. are executed.

OSH intentionally avoids this. If you want those files, simply source them in your oshrc.

I describe my own oshrc file on the wiki: How To Test OSH.

Global Execution Options

All Unix shells have global options that affect execution. There are two kinds:

List them all with set -o and shopt -p. Other than syntax, there's no essential difference between the two kinds.

shopt -s strict:all is Recommended

OSH adds more options on top of those provided by POSIX and bash. It has a shortcut shopt -s strict:all which turns on many options at once:

If you want your script to be portable to other shells, use this line:

shopt -s strict:all 2>/dev/null || true  # suppress errors

You can also turn individual options on or off:

shopt -s strict_array  # Set this option.  I want more fatal errors.
shopt -u strict_array  # Unset it.  Ignore errors and keep executing.

List of Options

strict_arith. Strings that don't look like integers cause a fatal error in arithmetic expressions.

strict_argv. Empty argv arrays are disallowed, since there's no practical use for them. For example, the second statement in x=''; $x results in a fatal error.

strict_array. No implicit conversions between string an array. In other words, turning this on gives you a "real" array type.

strict_control_flow. break and continue outside of a loop are fatal errors.

strict_eval_builtin. The eval builtin takes exactly one argument. It doesn't concatenate its arguments with a space, or accept zero arguments.

strict_word_eval. More word evaluation errors are fatal.

See the Oil manual for options that fundamentally change the shell language, e.g. those categorized under shopt -s oil:all.

OSH Has Four errexit Options (while Bash Has Two)

The complex behavior of these global execution options requires extra attention in this manual.

But you don't need to understand all the details. Simply choose between:

# Turn on four errexit options.  I don't run this script with other shells.
shopt -s oil:all

and

# Turn on three errexit options.  I run this script with other shells.
shopt -s strict:all

Quirk 1: the Shell Sometimes Disables And Restores errexit

Here's some background for understanding the additional errexit options described below.

In all Unix shells, the errexit check is disabled in these situations:

  1. The condition of the if, while, and until constructs
  2. A command/pipeline prefixed by !
  3. Every clause in || and && except the last.

Now consider this situation:

  1. errexit is on
  2. The shell disables it one of those three situations
  3. While disabled, the user touches it with set -o errexit (or +o to turn it off).

Surprising behavior: Unix shells ignore the set builtin for awhile, delaying its execution until after the temporary disablement.

Quirk 2: x=$(false) is inconsitent with local x=$(false)

Background: In shell, local is a builtin rather than a keyword, which means local foo=$(false) behaves differently than than foo=$(false).

Additional errexit options

OSH aims to fix the many quirks of errexit. It has this bash-compatible option:

And two more options:

Example

When both inherit_errexit and more_errexit are on, this code

echo 0; echo $(touch one; false; touch two); echo 3

will print 0 and touch the file one.

  1. The command sub aborts at false (`inherrit_errexit), and
  2. The parent process aborts after the command sub fails (more_errexit).

Recap/Summary

Good articles on errexit:

Features Unique to OSH

Dumping the AST

The -n flag tells OSH to parse the program rather than executing it. By default, it prints an abbreviated abstract syntax tree:

$ bin/osh -n -c 'ls | wc -l'
(command.Pipeline children:[(C {(ls)}) (C {(wc)} {(-l)})] negated:F)

You can also ask for the full text format:

$ bin/osh -n --ast-format text -c 'ls | wc -l'
(command.Pipeline
  children: [
    (command.Simple
      words: [
        (word.Compound
          parts: [(word_part.Literal
                   token:(token id:Lit_Chars val:ls span_id:0))]
        )
      ]
    )
    (command.Simple
      words: [
        (word.Compound
          parts: [(word_part.Literal
                   token:(token id:Lit_Chars val:wc span_id:4))]
        )
        (word.Compound
          parts: [(word_part.Literal
                   token:(token id:Lit_Chars val:-l span_id:6))]
        )
      ]
    )
  ]
  negated: F
  spids: [2]
)

This format is subject to change. It's there for debugging the parser, but sophisticated users may use it to interpret tricky shell programs without running them.

OSH_HIJACK_SHEBANG

This environment variable can be set to the path of a shell. Before OSH executes a program, it will inspect the shebang line to see if it looks like a shell script. If it does, it will use this shell instead of the one specified in the shebang line.

For example, suppose you have myscript.sh:

#!/bin/sh
# myscript.sh

./otherscript.sh --flag ...

and otherscript.sh:

#!/bin/sh
# otherscript.sh

echo 'hello world'

Then you can run myscript.sh like this:

OSH_HIJACK_SHEBANG=osh osh myscript.sh

and otherscript.sh will be executed with OSH rather than the /bin/sh.

Note that osh appears twice in that command line: once for the initial run, and once for all recursive runs.

(This is an environment variable rather than a flag because it needs to be inherited.)

--debug-file

Print internal debug logs to this file. It's useful to make it a FIFO:

mkfifo _tmp/debug
osh --debug-file _tmp/debug

Then run this in another window to see logs as you type:

cat _tmp/debug

Related:

Crash Dumps

This is implemented, but a JSON library isn't in the release build.

Completion API

The completion API is modeled after the bash completion API

However, an incompatibility is that it deals with argv entries and not command strings.

OSH moves the responsibility for quoting into the shell. Completion plugins should not do it.

Exit Codes

Unicode

Program Encoding

Shell programs should be encoded in UTF-8 (or its ASCII subset). Unicode characters can be encoded directly in the source:

echo 'μ'

or denoted in ASCII with C-escaped strings, i.e. $'':

echo $'[\u03bc]'

(This construct is preferred over echo -e because it's statically parsed.)

Data Encoding

Strings in OSH are arbitrary sequences of bytes. Caveats:

The GNU iconv program converts text from one encoding to another.

Also see Notes on Unicode in Shell.

Bugs

Links

External:


Generated on Sat Jul 18 02:28:24 PDT 2020