OVM will be a Slice of the CPython VM


Today's post is a status update on OVM, the virtual machine that OSH and Oil will run on.


Earlier this month, I listed six problems with using CPython to implement shell. The OSH interpreter is currently ~12K lines of Python.

In Cobbling Together a Python Interpreter, I described a potential solution to these problems. Inspired by tinypy, I would assemble a Python interpreter to run OSH and Oil. I would reuse 8K - 9K lines of Python for the front end, and write an estimated 3K - 5K lines of C++ from scratch for the runtime.

Yesterday, I described a successful integration of the front end components.

Writing the VM is achievable, but it's not clear how long it will take. The biggest risk is probably that I've never written a garbage collector. Garbage collectors are hard to debug, and I've observed that each one is a unique research project.

This post describes an alternative plan:

  1. Using the CPython VM as the basis for OVM.
  2. The experiments I've done to validate this approach.
  3. How it addresses the six problems with the Python interpreter.

(Aside: A few people on Lobsters doubted the original OPy plan, and I concede that they had a point. But they didn't suggest a better solution. Neither rewriting all the code in C/C++ nor shipping OSH as a Python program are good solutions. Leave a comment if you're unclear about this.)

Demo Code

In oil/cpython-slice on Github, there are shell scripts that build a stripped-down CPython 2.7 VM. It's messy, but it proves the concept:

I plan to further strip down CPython, guided by these analysis tools:

How the CPython Slice Addresses the Six Problems

Here I address the six problems in order of most-solved to least-solved:


Rather than writing my own VM, I'm forking the CPython interpreter, which is the minimum amount of work to address the six problems.

Essentially, my strategy is to ship the prototype. I mentioned in the first post that I wrote ~3K lines of C++ to start OSH, but realized I would never finish at that rate. The Python code turned into the whole project.

I'm not sad about that, because I need leverage to complete not only OSH, but Oil with its Awk and Make functionality.

I believe there are no remaining obstacles to an OSH 0.1 release. There's just work! I don't know when it will be done, but I'll lay out the criteria for a release in a future post.

Deferred Topics

Potential future topics: