blog | oilshell.org

OSH Future Work

Objective: get people thinking about helping?

Turn this into a bug. Help wanted.

Differences Between OPy and Python

• See the README.md

• unicode, print() syntax because OPy copmiler doesn't support __future__ or -*- coding -*-

• Bugs

[opy][] OVM

. ==> md-blog-tag opy

Future Work

OVM Changes

• ASDL as already mentioned

• Unify pickle and marshal

  • pickle is used for compiled pgen2 grammar
  • marshal is used for .pyc bytecode

  (text is used for ASDL)

Small tasks

• Release OPy itself under OPy. For forking the language.
• Removing unused bytecodes from ceval.c
• Making a bytecode histogram to understand what the difference is
• Remove multiple inheritance. I believe this is used for some kind of visitors.
  • OPy is more static than Python.

transforming - various notes. compiler bigger, and runtime smaller.

• Optimzations

  • bytecode is kinda big? nearly 1MB. Same size as bash binary alone, and then we have our native binary.
  • removing all the names from bytecode proper (const pool), and instead pointing to GLOBAL debug info, that can be compressed.
    • This is more like a C program.
  • Python-level dead-code elimination. Once we have static resolution of names we can do this.

• Optimizations of the Compiler itself, not the generated code

  • Is there a way to generate the AST directly from the parse process?
  • transformer.py is 1578 lines of code to transform the parse tree to an AST.
    • why not just make these semantic actions? They already look like them.
    • Yeah I think I should be able to automate it? We'll have to see.

• Bigger tasks:

  • Use def chains -- A graph-based representation!

    • Find unused code! Remove unused code!
    • I don't think Python has this.

  • Type Inference for better code.

  • Classes should be statically defined.

  • TODO: Link to the appendix

  • CHange the instruction set of CPython. Add bytecodes.

  • work

• Bug fix

  • Making OPy bytecode deterministic
    • PYTHONHASHSEED
  • Making CPython bytecode deterministic

• ASDL Convergence

  • requires OVM changes

Two Virtual Machines: for OSH and for OPy

• Shell
  • value semantics, no references, copying everywhere
  • no garbage collection
  • dynamic scoping
  • shell functions only take strings (not arrays or integers), and only return integers (the status code).
• Python
  • reference semantics
  • garbage collection
  • lexical scoping and closures
  • Python functions

1. Shell Virtual Machine. Should I write one in C?

2. Python virtual machine.

3. Write a OSH to OVM compiler. This will make shell scripts faster than bash. But it means that the parser will still be in Python, and we'll still need the Python interpreter.

4. Write an OPy to OVM compiler. This makes

Help?

If you've worked on a VM before.

Help me make it faster?

• Reimplementing find / xargs in GNU.
  • front end / back end separatiron, so I can use the back end for the Oil language.
  • the front end should be able to just run the Oil codebase.

Conclusion

This is an experiment. I have no idea how long it will take to make OSH small and fast.

I hope it will be by the end of 2018, but I can see the work stretching out beyond that.

But I think OSH should be usable even if it's a bit bigger and slower than bash.

This release also has a preview of oilc. Oilc is a little like 'go' -- a suite of tools.

Should

oshc deps

oshc to-oil

oilc to-osh ?

opyc to-oil? Hm interesting idea.