blog | oilshell.org

Oil 0.8.pre3 - A Line Editor and a Continuous Build

2020-03-25

This is the latest release of Oil, a Unix shell:

Please try it on your shell scripts and report bugs! To build and run it, follow the instructions in INSTALL.txt. The wiki has more tips on How To Test OSH.

This release has two major themes:

  1. Features and fixes toward running the Bash Line Editor (ble.sh). I learned of this fascinating program a few weeks ago.
  2. Behind-the-scenes work to continuously build and test Oil. We now have travis-ci.oilshell.org, and I'm excited about offloading more work onto it.
Table of Contents
Tie-Ins
Project Scope
Serendipity With ble.sh
Shell: The Good Parts
More Shell Usage
New in This Release
Code Changes
Documentation
Zulip Threads
What's Next?
Appendix A: Selected Release Metrics
Appendix B: Closed Issues

Tie-Ins

The two themes strongly relate to prior blog topics. Before diving into details, here is some higher-level context.

Project Scope

Shell is both an user interface and a programming language, and I've addressed this issue a few times.

There's no reason a shell can't do well in both respects — it's just a lot of work. For reference, bash itself is ~142K lines of code. Its companion GNU readline, which provides a minimal experienced compared to say fish, is ~34K lines of code.

Serendipity With ble.sh

A few weeks ago, Danilo Spinella pointed out the Bash Line Editor project, which I'll refer to as ble.sh from now on.

It gives you a fish-like interactive experience in bash. The astonishing thing is that it's written in bash. I'm no stranger to big shell scripts, both batch and interactive, but this surprised me!

I'll write more about ble.sh in the future, but here are some links:

I'm not sure if Oil will be able to run ble.sh, but this release is major progress toward doing so.

If it eventually does, that will show an unexpected benefit of Oil's compatibility with bash, which has been a tremendous effort.

Shell: The Good Parts

The other major theme is the continuous build at travis-ci.oilshell.org.

Its source code lives in the new services/ directory, and I started calling it "Toil". Besides being a useful tool, its implementation demonstrates techniques that I want to write about under #shell-the-good-parts:

In short, I use the Unix philosophy of reusing programs written in different languages as shell "libraries".

More Shell Usage

New in This Release

Now that we've reviewed how Oil's infrastructure uses the Unix philosophy, here are the detail of this release.

Code Changes

This section summarizes the full changelog, starting with external contributors:

Travis A. Everett fixed spec tests to make them more portable, e.g. to a Nix environment.

Koichi Murase, author of ble.sh, made many changes necessary to run it.

In addition to many other changes, I made the following changes related to ble.sh:

Other:

Documentation

The docs are still in their early stages, but I've updated a few of them:

On the Wiki:

Zulip Threads

Reminder: There are threads on oilshell.zulipchat.com about almost every topic in this post, and more:

What's Next?

If you're a bash user who wants a fish-like experience, please try ble.sh and let me know what you think. It took me less than 20 seconds to install and try. The instructions are at akinomyoga/ble.sh.

Appendix A: Selected Release Metrics

Let's compare this release with the previous one, version 0.8.pre2, which I reported detailed metrics on earlier this month.

Tests

The features and bug fixes for ble.sh caused many more spec tests to pass. And we have more failing spec tests to guide future work.

A few more Oil spec tests fail because I made the test framework stricter. If there's a Python traceback on stderr, the test fails no matter what. This surfaced a few bugs which I quickly fixed, but others remain.

Lines of Code

I'm happy that all the new features fit in ~400 new significant lines of code:

... and ~600 new physical lines of code:

Benchmarks

The translation to C++ still works, but it didn't change in this release. There was no meaningful change in parser performance:

Or the size of the binary:

Appendix B: Closed Issues

#674 enforce upper limit on hard-coded file descriptors
#664 error message for $(( a )) doesn't have location
#662 trap doesn't accept / ignore --
#661 declare -A dict=() not allowed in Oil
#659 set -o vi and emacs should be have differently while interactive
#650 Implement printf %*s and %.*s
#637 'type' says control flow like 'return' is a builtin, but it's not in Oil
#620 disallowing redirects on control flow too strict by default (Run ble.sh)
#537 Implement `test -v` and `test -k`
#357 implement printf %b to do backslash escaping
#356 Implement read -d and other options