blog | oilshell.org
Whenever I announce a new Oil release, some readers are understandably confused.
This post, which I'll update periodically, explains the project's motivation from several perspectives. Because Unix shell is an old and successful technology, it serves many purposes and its users are diverse.
Before explaining why I created Oil, let's review what it is. As I wrote in Roadmap #5:
Oil is your upgrade path from bash. It's the only language that shell / bash can be automatically translated to.
It's important to understand the difference between OSH and Oil:
Without making promises about future work, here's what Oil offers:
If you're still confused after reading the rest of this post, leave a comment. I'll answer your question, and probably update this page with this answer.
This section paraphrases questions I've received and summarizes the answers. In most cases, I link to the original comment thread, which you can read for details.
zsh? What's your angle?
Oil is taking shell seriously as a programming language, rather than treating it as a text-based UI that can be abused to write programs.
To see why this is valuable, consider these two groups of shell users:
Oil is aimed at group 2. If you're in group 1, there's admittedly no reason to use it right now.
However, group 2 writes scripts for group 1 to use! So I believe the benefits of Oil will eventually bubble up.
In other words, I'm building a solid foundation for a few more decades of shell usage.
It's important to be compatible with existing code. You might not personally use shell as a programming language, but all Unix users still rely on big shell programs. It's often used at build time, but it's still used at runtime too, e.g. on embedded Linux devices.
Some of this code is old, but much of it is new. It's not a small amount of code, either. Examples:
Original question and answer (reddit.com)
There are three problems with that:
That's a reasonable choice in some circumstances. If everyone on your dev team knows Python, maintaining a shell script can be more costly than maintaining a Python script.
However, Python and Ruby aren't good shell replacements in general. Shell is a domain-specific language for dealing with concurrent processes and the file system. But Python and Ruby have too much abstraction over these concepts, sometimes in the name of portability (e.g. to Windows). They hide what's really going on.
I encountered a nice blog post, Replacing Shell Scripts with Python, which, in my opinion, inadvertently proves the opposite point. The Python version is more verbose and difficult to write.
It's true that Perl is closer to shell than Python and Ruby are. For example,
perl -pie idiom can take the place of
sed. However, Perl is
also not an acceptable shell:
my_shell_func 2> err.txt, where
my_shell_funccan invoke both functions and external commands?
grep(), but the real
grepis better for many problems.
However, it's true that, in some respects, Oil is retreading the same ground as Perl. But Oil is more faithful to shell, and its syntax uses fewer punctuation characters. In other words, it's less like "line noise".
You might be angry because you had to maintain a nasty shell script written by a coworker.
If that's the case, you should be helping Oil succeed! The only way to "kill bash" is to:
This is analogous to how Facebook is moving away from PHP by developing a similar, but cleaner, language called Hack.
Perl, Python, and Ruby have all existed for over 25 years, but they haven't replaced shell. New shell scripts are being written every day.
I've seen this suggestion a lot, and there are entire books devoted to it. If your script is small, it may be a reasonable goal.
For bigger programs, limiting yourself to POSIX is not just inconvenient, it's also a ill-defined and virtually untestable concept. Evidence:
dashand the Debian Policy on Shell Scripts both allow non-POSIX features, like
local. Local variables are essential for writing maintainable shell scripts, but POSIX doesn't mention them.
Most shells parse assignment builtins differently than other builtins, but the POSIX shell grammar has no notion of an assignment. This issue surfaced as early as 2010, but as of 2018, it's not in a published spec.
In other words, POSIX is incomplete and out of date. (However, I've discovered that shells are highly conformant with respect to things the standard does specify.)
As of 2018, I believe that OSH is a "better POSIX". POSIX is a descriptive specification and not a normative one. That means that it's an observation of how popular shells like ksh and bash happened to behave at a certain time. In other words, it's a compromise.
Similarly, OSH is based on extensive testing of the behavior of bash,
dash, mksh, zsh, and busybox
ash. That is, it uses the
same philosophy as POSIX, but it specifies more of the language. Roughly
speaking, spec tests are an executable specification.
It's indeed difficult, but after prototyping the OSH-to-Oil translator, I believe it can be done.
Oil isn't yet implemented, and documenting it is a large project. But here is a glimpse:
set -o strict-arrayetc.
Many of these posts are tagged #oil-language. Also see #shell-the-good-parts.
(1) Roadmap #5: Why OSH and Why Oil (October 2017). Linux distributions are heavy users of shell as a programming language, so OSH is initially focused on that use case.
(2) Project Goals (January 2017). Ilya Sher asked me why I created Oil, so I wrote this post and started a Wiki Page. I describe some more ambitious ideas, like creating a distributed shell. As of January 2018, I'm still struggling to replace bash, so those ideas are on the back burner!
Ilya also wrote a post explaining his shell project in FAQ form: Why Next Generation Shell?
I agree with all of his answers. Oil and NGS have largely the same motivations. The main difference between them is that Oil is compatible with POSIX shell and bash, but NGS isn't.
Leave a comment if there is something you don't understand, and I'll answer and possibly update the FAQ.
Future blog posts can explain Oil in different ways:
OSH has a modular architecture:
libckernel interface, for a Lua-like embeddable interpreter. The
processmodules are cleanly separated.