blog | oilshell.org

Blog Roadmap for January 2021

2021-01-05

A year ago, I wrote several posts to take stock of the project, starting with these two:

They still feel useful after year: I didn't write every post, but I ended up addressing almost every issue.

So it's time to do it again. What's happened recently in the Oil project, and what are our plans for 2021?

Table of Contents
Summary for Casual Readers
Flushing the Blog Queue
Selected Recap of 2020
Good News
Plans for Oil in 2021
More Writing
Conclusion
Appendix: Deferred Blog Topics
Unix-y Web Services I Wrote
Experiences With Various Languages

Summary for Casual Readers

I described the biggest accomplishment of the year in October, in Four Features That Justify a New Unix Shell. Summary: the OSH language is mature, with a compelling feature set, and the Oil language is taking a nice shape.

But the interpreter needs to be faster, and we're speeding it up by translating statically typed Python to C++. The translation relies on a runtime library that includes a garbage collector.

Unfortunately, you could say that writing and debugging the collector kicked my ass in December. I spent much of the month knee deep in segfaults, GDB plugins, and CLion.

I did manage to fix many bugs, and I addressed overlooked issues like vtable pointers in object headers. And I still believe the design is right. However, I grew tired of the work and took a break.

As a result, Oil didn't fully achieve its goals for 2020. But I'm still happy with the progress we made.

Flushing the Blog Queue

My "break" was writing some fun tools in PHP, Python, and JavaScript that will help with blogging. I'm going to take these tools for a spin before returning to work on the garbage collector.

Long-time readers know that I make many comments about the project on Hacker News, Reddit, and lobste.rs. Explaining the project ends up shaping it.

Some of these comments are naturally blog posts, so I keep track of them on the #blog-ideas stream on Zulip (and previously #oil-documentation).

But rather than write full blog posts, I'll link and summarize the comments. I feel like this style of dialogue could be more effective at communicating subtle ideas, and it takes less time. I divided them into these categories:

  1. Comments About Parsing: Theory vs. Practice. I want to preserve a few popular comments under the #parsing tag.
  2. Comments About Shell and Oil's Design. These comments provide color on the Oil project. There is also some shell history, philosophy, and trivia.
  3. Comments About Languages, Distributed Systems, and Software Architecture. I hinted at the relation between shell in distributed systems at the end of this November post. Around that time, I also chatted with Michael Greenberg and the authors of PaSh about some related and ambitious topics.
  4. Comments About C++, mycpp, and Tea. Oil has an unusual implementation strategy, and I wrote several comments to justify it.

Writing these summaries should take a week or two. I hope that some of the "big ideas" will attract a new audience to the project.

Selected Recap of 2020

Let's review a few posts from 2020 before talking about 2021.

Good News

I can see that many people are downloading Oil, and the blog is more popular than ever. Each new post has many readers, and people continue to read what I wrote years ago.

But I can use more feedback! Let me know if OSH works on your shell programs, and let me know what you think of the Oil language. (Thanks to Danilo Spinella for recent comments.)

See Where To Send Feedback. There's also a new #oil-help stream on Zulip.

Plans for Oil in 2021

To be honest, I haven't figured out what 2021 will look like. After I flush the blog queue, I should write about project plans. Explaining the project to others often helps me figure out what to do.

Off the top of my head, here are the top priorities:

More Writing

Conclusion

This post outlined what I want to write about in January. I hope to get most of it done, but no promises. The first post in this series will be Comments About Parsing: Theory vs. Practice.

Appendix: Deferred Blog Topics

A few interesting side projects have made their appearance behind the scenes. I mention them here so you know they exist and can questions, but I don't have plans to write about them.

Unix-y Web Services I Wrote

Three tools that help with blogging and web publishing:

  1. wwz. A FastCGI program to serve static files out of a .zip file. I wrote this in Python in 2017, and it's used extensively on every release page.
  2. picdir. Context: I want to make the blog more visual, but that's difficult in Vim. I'm now using the StackEdit Markdown editor, which provides realtime preview.
  3. hashdiv. JavaScript web scraping and an associated Flask server. It's immature, but will help with the next few blog posts.

Experiences With Various Languages

I tend to write code in useful languages, not simple or elegant ones. This is a rich source of ideas for language design, both positive and negative.