Index of /preview/doc

      Name                    Last modified      Size  Description
Parent Directory - command-vs-expressio..> 2024-03-01 11:33 7.7K deprecations.html 2020-08-19 16:43 9.3K eggex.html 2022-05-06 11:28 28K error-handling.html 2022-05-03 20:07 35K hay.html 2022-06-16 21:36 28K help-index.html 2020-08-18 10:27 31K help.html 2020-08-18 10:26 36K idioms.html 2022-05-03 19:58 33K io-builtins.html 2020-08-19 00:07 4.5K j8-notation.html 2024-03-01 11:34 24K json.html 2021-12-02 20:37 6.6K language-design.html 2020-10-06 18:16 10K language-influences...> 2020-10-08 00:05 9.1K oil-builtins.html 2020-08-20 23:25 6.0K oil-expressions.html 2020-08-20 11:58 17K oil-language-tour.html 2022-05-03 19:59 46K oil-proc-func-block...> 2020-08-20 11:58 8.4K parser-architecture...> 2024-03-01 11:35 14K pretty-printing.html 2024-03-06 10:50 33K qsn.html 2021-01-19 11:44 15K simple-word-eval.html 2020-08-19 16:43 17K syntactic-concepts.html 2020-10-09 11:59 11K syntax-feelings.html 2020-10-08 00:05 13K upgrade-breakage.html 2022-06-11 13:16 9.8K variables.html 2021-05-07 23:42 19K xtrace.html 2021-01-19 11:43 8.5K
Oils Source Code

Oils Source Code

Build Status Contribute with Gitpod

Oils is our upgrade path from bash to a better language and runtime!

(The project was slightly renamed in March 2023, so there are still old references to "Oil". Feel free to send pull requests with corrections!)

Oils 2023 FAQ / Why Create a New Unix Shell?

It's written in Python, so the code is short and easy to change. But we automatically translate it to C++ with custom tools, to make it fast and small. The deployed executable doesn't depend on Python.

This README is at the root of the git repo.

Table of Contents
Quick Start on Linux
Dev Build vs. Release Build
Important: We Accept Small Contributions!
I aim for 24 hour response time
Repository Structure
A Collection of Interpreters
DSLs / Code Generators
Native Code and Build System
Several Kinds of Tests
Dev Tools and Scripts
Temp Dirs
Build Dependencies in ../oil_DEPS
Build System for End Users version.
Doc Sources
More info


Quick Start on Linux

After following the instructions on the Contributing page, you'll have a Python program that you can quickly run and change! Try it interactively:

bash$ bin/osh

osh$ name=world
osh$ echo "hello $name"
hello world

Let us know if any of these things don't work! The continuous build tests them at every commit.

Dev Build vs. Release Build

Again, note that the developer build is very different from the release tarball. The Contributing page describes this difference in detail.

The release tarballs are linked from the home page. (Developer builds don't work on OS X, so use the release tarballs on OS X.)

Important: We Accept Small Contributions!

Oils is full of many ideas, which may be intimidating at first.

But the bar to contribution is very low. It's basically a medium size Python program with many tests, and many programmers know how to change such programs. It's great for prototyping.

I aim for 24 hour response time

Please feel free to ping andychu on Zulip or Github if you're waiting for a pull request review! (or to ask questions)

Usually I can respond in 24 hours. I might be traveling, in which case I'll respond with something like I hope to look at this by Tuesday.

I might have also missed your Github message, so it doesn't hurt to ping me.

Thank you for the contributions!


The Wiki has many developer docs. Feel free to edit them. If you make a major change, let us know on Zulip!

There are also READMEs in some subdirectories, like opy/ and mycpp/.

If you're confused, the best thing to do is to ask on Zulip and someone should produce a pointer and/or improve the docs.

Docs for end users are linked from each release page.

Repository Structure

Try this to show a summary of what's in the repo and their line counts:

$ metrics/ overview

(Other functions in this file may be useful as well.)

A Collection of Interpreters

Oils is naturally structured as a set of mutually recursive parsers and evaluators. These interpreters are specified at a high-level: with regular languages, Zephyr ASDL, and a statically-typed subset of Python.

bin/              # Main entry points like bin/osh (source in bin/
frontend/         # Input and lexing common to OSH and YSH
osh/              # OSH parsers and evaluators (cmd, word, sh_expr)
ysh/              # YSH parser and evaluator
data_lang/        # Languages based on JSON
builtin/          # Builtin commands and functions
core/             # Other code shared between OSH and YSH
pyext/            # Python extension modules, e.g. libc.c
pylib/            # Borrowed from the Python standard library.
tools/            # User-facing tools, e.g. the osh2oil translator

DSLs / Code Generators

Here are the tools that transform that high-level code to efficient code:

asdl/             # ASDL implementation, derived from CPython
pgen2/            # Parser Generator, borrowed from CPython
mycpp/            # Experimental translator from typed Python to C++.
                  # Depends on MyPy.  See mycpp/
pea/              # Perhaps a cleaner version of mycpp
opy/              # Python compiler in Python (mycpp/ will replace it)

Native Code and Build System

We have native code to support both the dev build (running under CPython) and the oils-for-unix build (pure C++):   # Generates

build/            # High level build   # invoked by
  oil-defs/       # Files that define our slice of CPython.           # For development builds, running CPython
cpp/              # C++ code which complements the mycpp translation
mycpp/            # Runtime for the translator

prebuilt/         # Prebuilt files committed to git, instead of in _gen/

Python-2.7.13/    # For the slow Python build

# Temp dirs (see below)

Several Kinds of Tests

Unit tests are named and live next to

test/             # Test automation
  gold/           # Gold Test cases        # shell spec test framework         # Types of test runner: spec, unit, gold, wild  
spec/             # Spec test cases
  bin/            # tools used in many spec tests
  testdata/       # scripts for specific test cases
  stateful/       # Tests that use pexpect

Dev Tools and Scripts

We use a lot of automation to improve the dev process. It's largely written in shell, of course!

benchmarks/       # Benchmarks should be run on multiple machines.
metrics/          # Metrics don't change between machines (e.g. code size)
client/           # Demonstration of OSH as a headless server.
deps/             # Dev dependencies and Docker images
devtools/         # For Oils developers (not end users)      # The (large) release process.
  services/       # talk to cloud services
demo/             # Demonstrations of bash/shell features.  Could be
                  # moved to tests/ if automated.
  old/            # A junk drawer.
web/              # HTML/JS/CSS for tests and tools
soil/             # Multi-cloud continuous build (e.g. sourcehut, Github)

Temp Dirs

Directories that begin with _ are not stored in git. The dev tools above create and use these dirs.

_bin/             # Native executables are put here
_build/           # Temporary build files
_cache/           # Dev dependency tarballs
_devbuild/        # Generated Python code, etc.
_gen/             # Generated C++ code that mirrors the repo
_release/         # Source release tarballs are put here
  VERSION/        # Published at$VERSION/
    web/          # Static files, copy of $REPO_ROOT/web
_test/            # Unit tests, mycpp examples
_tmp/             # Output of other test suites; temp files

Build Dependencies in ../oil_DEPS

These tools are built from shell scripts in soil/. The oil_DEPS dir is "parallel" to Oils because it works better with container bind mounds.

  re2c/           # to build the lexer
  cmark/          # for building docs
  spec-bin/       # shells to run spec tests against
  mypy/           # MyPy repo
  mycpp-venv/     # MyPy binaries deps in a VirtualEnv

  py3/            # for mycpp and pea/
  cpython-full/   # for bootstrapping Oils-CPython

Build System for End Users version.

These files make the slow "Oils Python" build, which is very different than the developer build of Oils.


These files are for the C++ oils-for-unix tarball (in progress):


Doc Sources

doc/              # A mix of docs
doctools/         # Tools that use lazylex/ to transform Markdown/HTML
lazylex/          # An HTML lexer which doctools/ builds upon.         # This page, which is For Oils developers

LICENSE.txt       # For end users

More info

There are README files in many subdirectories, like mycpp/

Generated on Fri, 01 Mar 2024 14:33:46 -0500