Why Sponsor Oils? | blog | oilshell.org
I'm writing this FAQ for an upcoming AMA on Reddit's /r/linux, but I expect it to be useful for all readers. (AMA stands for "ask me anything"; it's like a group interview.)
Update: AMA: I spent 3 years creating a new bash-compatible shell called Oil
Because Oil is a complement to "metal"! Shell is an essential lubricant for software systems.
Programs in C, C++, Java, Go, Rust, etc. are analogous to "metal". I also use shell to glue together Python, R, and JavaScript.
Because oil is a short word, like cat, awk, or sed.
The name often reminds people of the company "Shell Oil". Of course, it has nothing to do with that.
I expect associations to change over time. Python creator Guido van Rossum was fighting "snakes" for a long time, but now the name stands on its own.
8/2020 update: "Oil" may also remind you that the upgrade process is relatively smooth! It runs your existing shell scripts.
The very first post on the blog has some color on this. I started the first serious of version Oil in C++, but progress was too slow.
Python let me implement a large portion of bash in five to seven times less code. Small programs are easier to write and maintain, and they have fewer bugs.
Current line counts [1]:
| Physical Lines | Significant Lines | |
| bash | 142K | 101K |
| osh | 26K | 14K |
| Approximate ratio | 5.5x | 7.2x |
Squeezing a backward-compatible shell into 14K lines of code (excluding whitespace and comments) is a great result! It gives me confidence that new features won't bloat the shell.
To be fair, a big chunk of the Python interpreter sits below that 14K lines. But I didn't have to write that code. And I think of it as having an 87K line "code budget" to work with (101K minus 14K).
The main downside of using Python is that the OSH interpreter is too slow. But now that OSH can run thousands of lines of bash scripts, speeding it up is the top priority. I aim to do this without rewriting the whole program by hand, using "metaprogramming" and DSLs.
2020 Update: This work is looking good, although it's not done. See:
See the FAQ in Building Oil with the OPy Bytecode Compiler.
OSH is ready for early adopters to test. I started using it myself in February.
At the moment, there are a couple reasons to use it:
(1) As a development aid while writing shell scripts.
.bashrc and
completion scripts may require minor patches.strict-argv). This February
release announcement shows that example, and there are more
that need to be documented. You can think of it as a
ShellCheck that also has knowledge of what
happens at runtime.(2) To support a new project to replace bash. The wiki page Shell Programs That Run Under OSH is editable and I would love your help in making it longer.
I hope to get feedback from the AMA for a more featureful 0.7.0 release. Although I'll still be happy if 0.7.0 improves only in speed.
oil-native
shows how we'll optimize Oil./release/$VERSION/doc/ tree, but it's still missing content.If you're interested in helping with any of these things, feel free to join oilshell.zulipchat.com.
I described a new, legacy-free shell language over 2 years ago, but I haven't worked on it much since then.
Instead, I worked on running existing bash scripts because I think that's more important. There are many alternative shells, but I don't think they will replace bash because of its strong network effect.
I still believe that many shell features hinge on a better language, including:
In the last post, I mentioned that I made some progress on Oil's expression language.
With the caveat that I just started implementing it, the two sections below
give you a flavor for the language. In short, the syntax and semantics
resemble Python and JavaScript. You'll be able to use the var keyword
and then write a "normal looking" expression. There are also some elements of
Ruby, Perl, and Awk.
The following POSIX shell code tests if a number is between 5 and 10:
n=$(( 3 + 4 )) if [ 5 -le $n ] && [ $n -le 10 ]; then echo "$n is between 5 and 10" fi
# Output: 7 is between 5 and 10
I'd guess that 50-90% of shell users couldn't write that correctly the first time. Reading it afterward is an additional challenge.
Here's how you do it in the ksh dialect that bash supports:
(( n = 3 + 4 )) if (( 5 <= n && n <= 10 )); then echo "$n is between 5 and 10" fi
# Output: 7 is between 5 and 10
This syntax is OK (ignoring some extra parentheses). But the semantics are bad in at least three ways.
n doesn't look like an integer? What happens when n is 0x0A,
0123, x, or x+8?n is $(echo pwned), or a[$(echo 42 >pwned.txt)]?I didn't have time to blog about the security issue, but the gist of it is that
there's a hidden eval every time you use shell arithmetic.
A statement as simple as echo $((x + 1)) leads to the evaluation of arbitrary
code in x! An adversary that controls x also controls your system.
Oil will have more familiar syntax and simpler, safer semantics:
var n = 3 + 4 # 'var' for real ints, floats, dicts, etc.
if (5 <= n and n <= 10) { # 'and' in expressions, && in commands
echo "$n is between 5 and 10"
}
# Output: 7 is between 5 and 10
That's all I want to say about the Oil language right now. That's because I don't know if I'll be able to finish it!
If you're interested in a saner and more powerful shell language, the last post has some notes on how to get involved. And feel free to ask questions on oilshell.zulipchat.com.
The home page links to the latest release. The wiki page How To Test OSH has details.
I wrote about this in the last post. Feedback is appreciated!
I wrote the last FAQ in January 2018. It's by far the most popular page on this site.
Other blog posts with answers are labeled #FAQ and #why-a-new-shell.
[1] The line-counts/run.sh script in the oilshell/blog-code repo counts lines of bash code.
The pydeps-src-only function in metrics/tarball.sh counts line of Oil
code. It reports slightly higher numbers than the src and osh-cloc
metrics, which are published with every release.