Why Sponsor Oils? | blog | oilshell.org
I just released Oils 0.19.0, and there's a lot to talk about!
It has more than 2 months of deep work on YSH, fundamental performance improvements, and features/fixes based on user feedback. The announcement will be long and detailed, so I'm writing this post first.
I want to give readers a sense of what's going on, and to organize my thoughts about the project.
Let's review what's happened this year.
Oils 0.19.0 continues in the same direction as:
Oils 0.17.0 - YSH Is Becoming Real (August)
Oils 0.18.0 - Progress on All Fronts (September)
To repeat, we're going for the "whole enchilada" - a general purpose, reflective language, with structured data types, and data serialization.
It turns out that you need that much of a language to express basic things like a flag parsing library (a form of serialization) and a test framework (reflection). I've learned more than a few things about language design, which I hope to write about.
I wrote five posts tagged #ysh in June:
The main takeaways are:
I wouldn't say YSH has fundamentally changed since it was introduced in 2019, but we've learned a lot through iteration. The steady trickle of feedback helps.
I said I would publish screencasts over 2 months ago, in the Oils 0.18.0 announcement. Pictures are worth more than words, but to summarize, we have both:
We want YSH to be stable, so this release has breaking changes, as Oils 0.16.0 - Breaking Renames and YSH did in June. It's better to break things earlier, rather than later.
On the other hand, OSH is very stable. It may feel precarious because OSH and YSH are in the same binary, but our tests are excellent at catching regressions, and our code is short, without many nooks and crannies. I recall only one or two regressions in the ~7 year history of the project.
This release announcement will be long and detailed. There are a few design issues I'd like feedback on.
I've been keeping our docs up-to-date, and writing new ones. New docs:
I updated these docs, which may give sophisticated users a sense of YSH:
Minor updates:
Please review these docs, and send us feedback on Github or Zulip: Where To Send Feedback.
We still need to make a big push on the nascent Oils reference.
I drafted five backlog blog posts based on the #blog-ideas Zulip channel, similar to the series I published in Winter 2021.
I probably won't publish them all, but the one on YSH Design is the most important. It should help users understand the Oils 0.19.0 release. The draft starts out like this:
Experiences to Explain YSH
dir() function. I reach back 20 years to the way I learned Python, in 2003. I learned more by experimenting than reading docs. This relates to both our design for reflection in YSH, and to our strategy for docs.
dir() is part of PyObject*, the narrow waist mentioned above! Narrow waists have many implications.cp myfile /tmp/ looks a bit like C's memcpy(&dest, src, 10)? It doesn't look like var y = f(x). This relates strongly to procs vs. funcs, and to the new value.Place mechanism, which I'll explain later.Overview of YSH. These posts will make the language more concrete.
var constsetvar setglobalcall =proc funcIn short, the language is converging. But there are still open design issues, which I mention in the next section.
I'm happy to say that we got our third grant from NLnet last week!
For context, the first one was awarded in April 2022, and the second one around February 2023. Pictures of Working Garbage Collector describes a big milestone that required multiple people.
These grants, and the 6 contributors we've paid, are the reason that YSH is now feasible. Remember that back in 2020, I said Oils would be based on strings only (but in a principled way). I've since learned that such a design can't reach our goals.
I'd like to write a blog post How are Programming Languages Funded?. I learned the hard way that creating a language takes a lot of effort :-)
Related discussion: The Economics of Programming Languages from Strange Loop 2023.
Here's a rough summary of what I think we should do for the third grant. We brainstormed about this on #oil-discuss > Third Grant Milestones.
parseArgs() (flags) and describe (testing)
argparse-like "builder" objects, and modules/namespaces. More on this later.dir() in Python is a key reflection mechanism which makes the language consistent and learnable. It's a mechanism that shell, awk, and make all lack.strftime()Str->find(), Dict->erase(), IO->glob(), etc.Performance is under control, so I left it out of this list. But there's important (and fun) performance work left:
value.Str, and I think they'll work in our current codebase. I don't think we'll have to rewrite mycpp, which has always been a bit of a sore spot.As mentioned, the project wouldn't be possible without contributors. And we need 1 or 2 more people for the third grant!
Here are two signs you could be a good contributor:
As mentioned, I'm a bit self-conscious that there are so many long and deeply interlinked threads, mostly by me. (What really happened is that Zulip replaced my personal wiki.)
But it does seem to produce results in the end. And we do get feedback that leads to YSH improvements, which you'll see in the 0.19.0 announcement.
Also, contributors suggested we use Github milestones to organize the project. I think that's a good idea, and it seems easy to do.
Overall, I don't feel I've been doing a great job growing the project, since I've been heads down in design and implementation. But nevertheless we picked up a new contributor recently (Ellen Potter), so perhaps we're not too far from where we want to be.
This post summarized what's happened for the last couple months, and what I've been thinking about. To recap, here are some things you can do:
Up next:
Please ask questions in the comments!
Here are some other posts I'd like to write.
These Zulip threads could be blog posts, to help new contributors:
The main thing I'd like to talk about is Perlis-Thompson language design problems.
In particular, I claim the famous "function coloring" async vs. sync problem is a Perlis-Thompson problem, and so is our func-proc problem.
I noticed that sudo apt-get install moreutils vs. isutf8 is another problem. In a distributed shell, having both packages and executables is could be superfluous.
Many of these slogans are based on online discussions, e.g. #comments:
In addition to working on YSH, I've done a few coding experiments that may show up later in the project:
micro-syntax is in the Oils 0.19.0 release! Another great experience with re2c.