Why Sponsor Oils? | blog | oilshell.org

Shell Scripts, Audio, Images, and 3D Graphics

Yesterday's post gave different views on Shell Scripts As Executable Documentation. I tagged it #shell-the-good-parts and #comments.

This post has the same two tags. It shows examples of a multiprocess architecture for multimedia:

1. One process generates data in a specified format. This is often the code you write.
2. Another process renders, plays, or interprets that data. This is often the code you reuse.

I would call this reuse by versionless protocols, as opposed to reuse by fragile libraries. (Related: my comment about textual protocols, which I'll elaborate on in the future.)

Generators and Renderers

Audio With Raw Samples

Summary: you can generate music by piping raw samples into programs like mplayer or ffplay.

Images With netpbm

With the Netpbm file formats, it’s trivial to output pixels using nothing but text based IO

In this case, the protocol / generated format is text rather than binary.

Raytracing with PBRT Scene Descriptions

Are the above techniques silly hacks? Isn't the better solution to use a "proper" library?

No, you can render professional quality computer graphics with this architecture. My 2018 Recurse Center project taught me this:

• A Raytraced Video Made With Shell Scripts (August 2018)

I wanted to work with graphics rather than text, but I still ended up making use of shell. Why?

1. The PBRT renderer uses a text file to describe scene graphs. (PBRT is a high quality textbook and C++ implementation of physically-based 3D rendering.)
   • Example: scenes/contemporary-bathroom/bathroom-demo.pbrt
2. I wanted to distribute the video frame rendering over 4 multicore machines located in the NYC space. Shell is the easiest way to do this, and it saved us hours of time.

Links:

• Hyperdodecahedron in a Bathroom. A video of the end result of our project. A 4D polytope can be represented with an animation of 3D slices, with time as the 4th dimension.
• Lobste.rs thread on my post, with history about Debian and Pixar. I believe that Pixar was an early user of Linux because buying an OS license for hundreds or thousands of machines was expensive.
• Swallowing the elephant (part 3). An article by PBRT author Matt Pharr about parsing speed as a bottleneck. This is a disadvantage of text, which is why #parsing is one of my long-term interests.
• Pixar's Renderman Protocol. This is more binary than PBRT's format, but it's still has a polyglot multiprocess flavor. If you have experience with it, leave a comment!

The Web Uses This Architecture

I touched on the relation between Unix and the web in this February 2020 episode of #shell-the-good-parts. Summary:

• You can easily turn a directory on your file system into a website. You just need a server like Apache or nginx.
  • The web adds the useful notion of URLs and hyperlinks.
• A CGI script is a command line tool that generates a web page / HTTP response.

Now that we've seen some Unix-y examples of audio, images, and 3D graphics, we can add to the analogy:

• A web server is a program that can generate not just text, but also audio, images, and video. A web browser is a renderer for such versionless protocols.

In a sense, you're reusing the web browser to provide a rich experience to visitors.

Importantly, you can write the web server in any language, which means that this Unix- and web-style of multiprocess programming enables language diversity.

This is good news to designers of nascent languages, e.g. on /r/ProgrammingLanguages! You don't have to write a "binding" to do fun things with a language. You can print data to stdout or write a web server.

Appendix: Raytracing in Pure Bash

An esoteric raytracer in CMake made the rounds recently. I suggested that this program can also be expressed in shell, since shell naturally supports multiple processes for parallel rendering, just as we did with PBRT.

I got a couple great responses!

1. https://github.com/aneeshdurg/bash-raytracer from Aneesh Durg. A working raytracer in bash. I ran this on my machine and it worked!
2. The start of another raytracer and two nice articles from Tristan Le Guern:
   • Fixed point arithmetic in pure shell. Shell only has integers, but raytracing naturally uses floating point numbers.
   • Calculating the square root of a number in pure shell. To be honest, I don't understand this yet, but it's an important piece of the puzzle! Aneesh's code also has an approximation of sqrt(), in math.sh.

Important: These pieces of code are different than the ones above! They are more "esoteric" because doing math in shell is slow.

However, I hope to add floating point to the Oil language in the near future, as well as a richer notion of functions. Along with Oil's more familiar syntax, raytracing in shell might not be esoteric!

Ad: I need help on Oil! Just running OSH on existing shell scripts is useful, and there are many other ways to help.