Why Sponsor Oils? | blog | oilshell.org

Winter Status Update

2023-11-30 (Last updated 2023-12-01)

I just released Oils 0.19.0, and there's a lot to talk about!

It has more than 2 months of deep work on YSH, fundamental performance improvements, and features/fixes based on user feedback. The announcement will be long and detailed, so I'm writing this post first.

I want to give readers a sense of what's going on, and to organize my thoughts about the project.

Table of Contents
Last Two Releases
YSH Design (June 2023)
Upcoming Blog Posts
Interactive Shell Screenshots
Oils 0.19.0 - Dicts, Procs, and Funcs
Please Review These Docs
Blog Backlog: YSH Design
Third NLnet Grant Received!
Proposed Milestones
We Can Use Help
Appendix: More Blog Backlog


Let's review what's happened this year.

Last Two Releases

Oils 0.19.0 continues in the same direction as:

  1. Oils 0.17.0 - YSH Is Becoming Real (August)

  2. Oils 0.18.0 - Progress on All Fronts (September)

To repeat, we're going for the "whole enchilada" - a general purpose, reflective language, with structured data types, and data serialization.

It turns out that you need that much of a language to express basic things like a flag parsing library (a form of serialization) and a test framework (reflection). I've learned more than a few things about language design, which I hope to write about.

YSH Design (June 2023)

I wrote five posts tagged #ysh in June:

  1. Reviewing YSH - 7 language features, 3 arguments for structured data, ...
  2. Sketches of YSH Features - 14 use cases for blocks/reflection. Just updated to sixteen.
  3. Narrow Waists Can Be Interior or Exterior: PyObject vs. Unix Files
  4. How to Create a UTF-16 Surrogate Pair by Hand, with Python
  5. Oils Is Exterior-First (Code, Text, and Structured Data)

The main takeaways are:

I wouldn't say YSH has fundamentally changed since it was introduced in 2019, but we've learned a lot through iteration. The steady trickle of feedback helps.

Upcoming Blog Posts

Interactive Shell Screenshots

I said I would publish screencasts over 2 months ago, in the Oils 0.18.0 announcement. Pictures are worth more than words, but to summarize, we have both:

  1. Extreme compatibility, like POSIX job control.
  2. The forward-looking headless shell protocol, which no other shell has.

Oils 0.19.0 - Dicts, Procs, and Funcs

We want YSH to be stable, so this release has breaking changes, as Oils 0.16.0 - Breaking Renames and YSH did in June. It's better to break things earlier, rather than later.

On the other hand, OSH is very stable. It may feel precarious because OSH and YSH are in the same binary, but our tests are excellent at catching regressions, and our code is short, without many nooks and crannies. I recall only one or two regressions in the ~7 year history of the project.

This release announcement will be long and detailed. There are a few design issues I'd like feedback on.

Please Review These Docs

I've been keeping our docs up-to-date, and writing new ones. New docs:

I updated these docs, which may give sophisticated users a sense of YSH:

Minor updates:

Please review these docs, and send us feedback on Github or Zulip: Where To Send Feedback.

We still need to make a big push on the nascent Oils reference.

Blog Backlog: YSH Design

I drafted five backlog blog posts based on the #blog-ideas Zulip channel, similar to the series I published in Winter 2021.

I probably won't publish them all, but the one on YSH Design is the most important. It should help users understand the Oils 0.19.0 release. The draft starts out like this:

In short, the language is converging. But there are still open design issues, which I mention in the next section.

Third NLnet Grant Received!

I'm happy to say that we got our third grant from NLnet last week!

For context, the first one was awarded in April 2022, and the second one around February 2023. Pictures of Working Garbage Collector describes a big milestone that required multiple people.

These grants, and the 6 contributors we've paid, are the reason that YSH is now feasible. Remember that back in 2020, I said Oils would be based on strings only (but in a principled way). I've since learned that such a design can't reach our goals.

I'd like to write a blog post How are Programming Languages Funded?. I learned the hard way that creating a language takes a lot of effort :-)

Related discussion: The Economics of Programming Languages from Strange Loop 2023.

Proposed Milestones

Here's a rough summary of what I think we should do for the third grant. We brainstormed about this on #oil-discuss > Third Grant Milestones.

  1. Data Languages: JSON and our J8 Notation upgrade.
  2. Reflection for parseArgs() (flags) and describe (testing)
  3. Implement builtin YSH functions and methods
  4. Documentation.

Performance is under control, so I left it out of this list. But there's important (and fun) performance work left:

We Can Use Help

As mentioned, the project wouldn't be possible without contributors. And we need 1 or 2 more people for the third grant!

Here are two signs you could be a good contributor:

  1. Our wiki page Where Contributors Have Problems doesn't scare you off.
  2. You log into https://oilshell.zulipchat.com, and it seems interesting (if overwhelming).

As mentioned, I'm a bit self-conscious that there are so many long and deeply interlinked threads, mostly by me. (What really happened is that Zulip replaced my personal wiki.)

But it does seem to produce results in the end. And we do get feedback that leads to YSH improvements, which you'll see in the 0.19.0 announcement.

Also, contributors suggested we use Github milestones to organize the project. I think that's a good idea, and it seems easy to do.

Overall, I don't feel I've been doing a great job growing the project, since I've been heads down in design and implementation. But nevertheless we picked up a new contributor recently (Ellen Potter), so perhaps we're not too far from where we want to be.


This post summarized what's happened for the last couple months, and what I've been thinking about. To recap, here are some things you can do:

  1. Review the docs linked above, and send feedback.
  2. Help us find contributors! If you don't have time to contribute yourself, please let others know about Oils.
  3. Try OSH and YSH, and send feedback.

Up next:

Please ask questions in the comments!

Appendix: More Blog Backlog

Here are some other posts I'd like to write.

Oils Implementation

These Zulip threads could be blog posts, to help new contributors:

More Language Design

The main thing I'd like to talk about is Perlis-Thompson language design problems.

In particular, I claim the famous "function coloring" async vs. sync problem is a Perlis-Thompson problem, and so is our func-proc problem.

I noticed that sudo apt-get install moreutils vs. isutf8 is another problem. In a distributed shell, having both packages and executables is could be superfluous.


Many of these slogans are based on online discussions, e.g. #comments:

Coding Experiments

In addition to working on YSH, I've done a few coding experiments that may show up later in the project: