| category | total lines | num files |
| OSH (and common libraries) | 36,899 | 77 |
| Oil Language (and Tea) | 4,843 | 11 |
| Code Borrowed from Python's stdlib | 215 | 4 |
| Spec Tests | 34,273 | 128 |
| Gold Tests | 952 | 20 |
| Language Unit Tests | 7,225 | 42 |
| Other Unit Tests | 921 | 11 |
| mycpp Translator | 4,294 | 10 |
| mycpp Test Data | 2,182 | 25 |
| Zephyr ASDL | 2,904 | 10 |
| pgen2 Parser Generator | 1,717 | 8 |
| Other Code Generators | 2,062 | 8 |
| Build Automation | 5,091 | 35 |
| Test Automation | 13,099 | 43 |
| Release Automation | 2,121 | 5 |
| Benchmarks | 5,634 | 33 |
| Metrics | 1,398 | 8 |
| Generated Python Code | 46,570 | 27 |
| Tools | 1,523 | 4 |
| Doc Tools | 2,173 | 9 |
| Web | 1,058 | 5 |
The README for oilshell/oil has another overview of the repository.
This is the input to the translators, written in statically-typed Python. Note that bash is at least 140K lines of code, and OSH implements a large part of bash and more.
bin/oil.py 360 core/alloc.py 133 core/ansi.py 17 core/completion.py 1,261 core/comp_ui.py 583 core/dev.py 597 core/error.py 183 core/executor.py 552 core/main_loop.py 359 core/optview.py 57 core/oven.py 573 core/process.py 1,500 core/pyerror.py 58 core/pyos.py 337 core/pyutil.py 247 core/runtime.asdl 193 core/shell_native.py 491 core/shell.py 695 core/state.py 2,131 core/ui.py 381 core/util.py 93 core/vm.py 209 frontend/args.py 673 frontend/builtin_def.py 155 frontend/consts.py 321 frontend/flag_def.py 318 frontend/flag_spec.py 380 frontend/id_kind_def.py 638 frontend/lexer_def.py 1,012 frontend/lexer.py 330 frontend/location.py 94 frontend/match.py 257 frontend/option_def.py 389 frontend/parse_lib.py 458 frontend/py_reader.py 81 frontend/reader.py 136 frontend/signal_def.py 43 frontend/syntax_abbrev.py 144 frontend/syntax.asdl 576 frontend/typed_args.py 63 frontend/types.asdl 40 mycpp/mylib.pyi 72 native/fanos.c 294 native/fastlex.c 239 native/fastlex.pyi 13 native/libc.c 402 native/libc.pyi 11 native/posix_.pyi 211 osh/arith_parse.py 184 osh/bool_parse.py 269 osh/bool_stat.py 136 osh/braces.py 498 osh/builtin_assign.py 514 osh/builtin_bracket.py 280 osh/builtin_comp.py 518 osh/builtin_lib.py 94 osh/builtin_meta.py 403 osh/builtin_misc.py 844 osh/builtin_printf.py 477 osh/builtin_process.py 549 osh/builtin_pure.py 670 osh/cmd_eval.py 1,772 osh/cmd_parse.py 2,311 osh/glob_.py 498 osh/history.py 184 osh/prompt.py 300 osh/sh_expr_eval.py 1,047 osh/split.py 305 osh/string_ops.py 454 osh/tdop.py 333 osh/word_compile.py 257 osh/word_eval.py 2,297 osh/word_parse.py 1,838 osh/word_.py 817 qsn_/qsn_native.py 75 qsn_/qsn.py 615 36,899 lines in 77 files
oil_lang/builtin_oil.py 338 oil_lang/expr_eval.py 806 oil_lang/expr_parse.py 335 oil_lang/expr_to_ast.py 1,536 oil_lang/funcs_builtin.py 440 oil_lang/grammar.pgen2 615 oil_lang/objects.py 290 oil_lang/regex_translate.py 262 tea/tea_eval.py 122 tea/tea_main.py 99 4,843 lines in 11 files
pylib/cgi.py 19 pylib/os_path.py 169 pylib/path_stat.py 27 215 lines in 4 files
A comprehensive test suite that compares OSH against other shells. If OSH passes these tests in BOTH Python and C++, it means that the translation works.
spec/alias.test.sh 536 spec/append.test.sh 291 spec/arith-context.test.sh 81 spec/arith.test.sh 631 spec/array-compat.test.sh 96 spec/array.test.sh 637 spec/assign-deferred.test.sh 105 spec/assign-dialects.test.sh 59 spec/assign-extended.test.sh 810 spec/assign.test.sh 671 spec/assoc.test.sh 631 spec/assoc-zsh.test.sh 53 spec/background.test.sh 154 spec/ble-features.test.sh 644 spec/ble-idioms.test.sh 168 spec/blog1.test.sh 95 spec/blog2.test.sh 49 spec/blog-other1.test.sh 55 spec/brace-expansion.test.sh 441 spec/bugs.test.sh 103 spec/builtin-bash.test.sh 346 spec/builtin-bracket.test.sh 564 spec/builtin-completion.test.sh 414 spec/builtin-dirs.test.sh 177 spec/builtin-eval-source.test.sh 249 spec/builtin-getopts.test.sh 318 spec/builtin-io.test.sh 751 spec/builtin-printf.test.sh 831 spec/builtins2.test.sh 233 spec/builtin-special.test.sh 111 spec/builtins.test.sh 420 spec/builtin-times.test.sh 15 spec/builtin-trap.test.sh 406 spec/builtin-vars.test.sh 710 spec/case_.test.sh 165 spec/command-parsing.test.sh 63 spec/command-sub.test.sh 272 spec/command_.test.sh 192 spec/comments.test.sh 11 spec/dbracket.test.sh 436 spec/dparen.test.sh 202 spec/empty-bodies.test.sh 24 spec/errexit-oil.test.sh 675 spec/errexit.test.sh 392 spec/exit-status.test.sh 238 spec/explore-parsing.test.sh 42 spec/extglob-files.test.sh 391 spec/extglob-match.test.sh 400 spec/for-expr.test.sh 92 spec/func-parsing.test.sh 105 spec/glob.test.sh 396 spec/here-doc.test.sh 365 spec/if_.test.sh 56 spec/interactive.test.sh 216 spec/introspect.test.sh 237 spec/let.test.sh 23 spec/loop.test.sh 397 spec/nameref.test.sh 642 spec/nix-idioms.test.sh 151 spec/nocasematch-match.test.sh 69 spec/oil-array.test.sh 86 spec/oil-assign.test.sh 326 spec/oil-bin.test.sh 17 spec/oil-blocks.test.sh 163 spec/oil-builtin-pp.test.sh 46 spec/oil-builtin-process.test.sh 85 spec/oil-builtins.test.sh 554 spec/oil-builtin-try.test.sh 140 spec/oil-command-sub.test.sh 108 spec/oil-config.test.sh 100 spec/oil-demo.test.sh 59 spec/oil-expr-sub.test.sh 40 spec/oil-expr.test.sh 866 spec/oil-funcs-builtin.test.sh 229 spec/oil-interactive.test.sh 15 spec/oil-json.test.sh 119 spec/oil-keywords.test.sh 37 spec/oil-multiline.test.sh 179 spec/oil-options.test.sh 795 spec/oil-proc.test.sh 252 spec/oil-regex.test.sh 689 spec/oil-scope.test.sh 748 spec/oil-slice-range.test.sh 195 spec/oil-special-vars.test.sh 66 spec/oil-string.test.sh 344 spec/oil-tuple.test.sh 44 spec/oil-user-feedback.test.sh 209 spec/oil-var-sub.test.sh 45 spec/oil-word-eval.test.sh 81 spec/oil-xtrace.test.sh 477 spec/osh-only.test.sh 78 spec/parse-errors.test.sh 192 spec/pipeline.test.sh 166 spec/posix.test.sh 146 spec/process-sub.test.sh 205 spec/prompt.test.sh 254 spec/quote.test.sh 248 spec/redirect.test.sh 654 spec/regex.test.sh 344 spec/serialize.test.sh 229 spec/shell-grammar.test.sh 205 spec/sh-func.test.sh 150 spec/sh-options.test.sh 694 spec/sh-usage.test.sh 90 spec/smoke.test.sh 123 spec/strict-options.test.sh 249 spec/subshell.test.sh 23 spec/tea-func.test.sh 288 spec/tilde.test.sh 172 spec/toysh-posix.test.sh 347 spec/toysh.test.sh 144 spec/type-compat.test.sh 80 spec/var-num.test.sh 39 spec/var-op-bash.test.sh 264 spec/var-op-len.test.sh 219 spec/var-op-patsub.test.sh 291 spec/var-op-slice.test.sh 315 spec/var-op-strip.test.sh 357 spec/var-op-test.test.sh 339 spec/var-ref.test.sh 485 spec/vars-bash.test.sh 27 spec/vars-special.test.sh 456 spec/var-sub-quote.test.sh 312 spec/var-sub.test.sh 74 spec/word-eval.test.sh 62 spec/word-split.test.sh 387 spec/xtrace.test.sh 344 34,273 lines in 128 files
Another suite that tests shells "from the outside". Instead of making explicit assertions, we verify that OSH behaves like bash.
test/gold/abuild.sh 9 test/gold/and-or.sh 27 test/gold/comments.sh 22 test/gold/complex-here-docs.sh 192 test/gold/configure-bug.sh 13 test/gold/de1.sh 24 test/gold/declare.sh 28 test/gold/dollar-sq.sh 42 test/gold/echo-e.sh 36 test/gold/errexit-confusion.sh 139 test/gold/export.sh 65 test/gold/glob.sh 26 test/gold/lineno.sh 23 test/gold/nix.sh 32 test/gold/readlink.sh 61 test/gold/readonly.sh 24 test/gold/scope.sh 126 test/gold/strip-op-char-class.sh 6 test/gold/word-eval.sh 57 952 lines in 20 files
core/alloc_test.py 60 core/completion_test.py 798 core/comp_ui_test.py 179 core/process_test.py 196 core/pyutil_test.py 19 core/state_test.py 316 core/ui_test.py 32 core/util_test.py 25 frontend/args_test.py 232 frontend/id_kind_def_test.py 97 frontend/lexer_def_test.py 436 frontend/lexer_gen_test.py 51 frontend/lexer_test.py 45 frontend/match_test.py 52 frontend/reader_test.py 49 native/fanos_test.py 246 native/fastlex_test.py 98 native/libc_test.py 304 native/line_input_test.py 27 native/posix_test.py 184 oil_lang/builtin_oil_test.py 135 oil_lang/expr_parse_test.py 144 oil_lang/objects_test.py 20 osh/arith_parse_test.py 179 osh/bool_parse_test.py 122 osh/braces_test.py 187 osh/builtin_bracket_test.py 29 osh/builtin_comp_test.py 19 osh/builtin_lib_test.py 93 osh/builtin_misc_test.py 48 osh/cmd_eval_test.py 74 osh/cmd_parse_test.py 1,264 osh/glob_test.py 163 osh/history_test.py 124 osh/prompt_test.py 54 osh/split_test.py 169 osh/string_ops_test.py 133 osh/word_compile_test.py 16 osh/word_eval_test.py 144 osh/word_parse_test.py 561 osh/word_test.py 101 7,225 lines in 42 files
asdl/format_test.py 47 asdl/front_end_test.py 141 asdl/gen_python_test.py 153 asdl/typed_arith_parse_test.py 227 build/app_deps_test.py 27 pylib/os_path_test.py 59 pylib/path_stat_test.py 24 test/sh_spec_test.py 135 test/wild_report_test.py 29 tools/osh2oil_test.py 79 921 lines in 11 files
This prototype uses the MyPy frontend to translate statically-typed Python to C++. The generated code calls a small runtime which implements things like List[T], Dict[K, V], and Python's len().
mycpp/const_pass.py 526 mycpp/cppgen_pass.py 2,602 mycpp/crash.py 37 mycpp/debug_pass.py 459 mycpp/format_strings.py 110 mycpp/mycpp_main.py 346 mycpp/mylib.py 148 mycpp/pass_state.py 54 mycpp/util.py 12 4,294 lines in 10 files
Small Python examples that translate to C++, compile, and run.
mycpp/examples/asdl_generated.py 62 mycpp/examples/cartesian.py 75 mycpp/examples/cgi.py 59 mycpp/examples/classes.py 141 mycpp/examples/containers.py 162 mycpp/examples/control_flow.py 110 mycpp/examples/escape.py 67 mycpp/examples/fib_iter.py 54 mycpp/examples/fib_recursive.py 49 mycpp/examples/files.py 53 mycpp/examples/length.py 106 mycpp/examples/lexer_main.py 48 mycpp/examples/loops.py 177 mycpp/examples/modules.py 78 mycpp/examples/parse.py 232 mycpp/examples/pgen2_demo.py 106 mycpp/examples/scoped_resource.py 176 mycpp/examples/strings.py 61 mycpp/examples/test_cast.py 59 mycpp/examples/test_conditional.py 67 mycpp/examples/test_hoist.py 61 mycpp/examples/test_switch.py 42 mycpp/examples/tuple_return_value.py 53 mycpp/examples/varargs.py 84 2,182 lines in 25 files
A DSL for algebraic data types, borrowed from Python. Oil is the most strongly typed Bourne shell implementation!
asdl/asdl_.py 253 asdl/format.py 494 asdl/front_end.py 488 asdl/gen_cpp.py 622 asdl/gen_python.py 529 asdl/pybase.py 63 asdl/runtime.py 41 asdl/tool.py 280 asdl/visitor.py 134 2,904 lines in 10 files
An LL(1) parser generator used to parse Oil expressions. Also borrowed from CPython.
pgen2/driver.py 101 pgen2/grammar.py 222 pgen2/parse.py 209 pgen2/pgen.py 428 pgen2/pnode.py 29 pgen2/tokenize.py 579 pgen2/token.py 149 1,717 lines in 8 files
In order to make Oil statically typed, we had to abandon Python reflection and use C++ source code generation instead. The lexer, flag definitions, and constants can be easily compiled to C++.
core/optview_gen.py 89 frontend/consts_gen.py 551 frontend/flag_gen.py 485 frontend/lexer_gen.py 495 frontend/option_gen.py 115 oil_lang/grammar_gen.py 226 osh/arith_parse_gen.py 101 2,062 lines in 8 files
build/actions.sh 220 build/app_deps.py 152 build/c_module_srcs.py 90 build/c_module_toc.py 51 build/codegen.sh 140 build/common.sh 69 build/compile.sh 421 build/cpython_defs.py 504 build/cpython-defs.sh 347 build/dev.sh 442 build/dev-shell.sh 42 build/doc.sh 402 build/hello.mk 51 build/make_zip.py 53 build/mycpp.sh 316 build/native_graph.py 104 build/native.sh 120 build/native-steps.sh 185 build/oil.mk 60 build/opy.mk 49 build/prepare.sh 27 build/setup_fanos.py 11 build/setup_fastlex.py 18 build/setup_line_input.py 13 build/setup_posix.py 19 build/setup.py 11 build/slice-2020.sh 121 build/test.sh 92 build/translate.sh 184 build/with-clang.sh 8 configure 395 install 87 Makefile 199 portable-rules.mk 88 5,091 lines in 35 files
test/alpine.sh 277 test/arena.sh 65 test/ble.sh 51 test/common.sh 135 test/cpp-unit.sh 81 test/gentoo.sh 83 test/gold.sh 200 test/interactive.py 581 test/interactive.sh 76 test/lint.sh 331 test/nix.sh 24 test/oil-runtime-errors.sh 124 test/opyc.sh 142 test/opy.sh 121 test/osh2oil.sh 1,364 test/oshc-deps.sh 103 test/osh-usage.sh 205 test/other.sh 34 test/parse-errors.sh 1,224 test/py3_parse.py 64 test/py3_parse.sh 37 test/report.R 54 test/runtime-errors.sh 1,039 test/shebang.sh 26 test/sh_spec.py 1,265 test/smoosh_import.py 31 test/smoosh.sh 213 test/spec-alpine.sh 151 test/spec-bin.sh 242 test/spec-common.sh 49 test/spec-cpp.sh 369 test/spec_lib.py 173 test/spec-runner.sh 412 test/spec.sh 1,071 test/stdlib.sh 32 test/syscall.py 219 test/syscall.sh 415 test/unit.sh 248 test/vagrant.sh 54 test/wild_report.py 802 test/wild-runner.sh 193 test/wild.sh 719 13,099 lines in 43 files
devtools/release-native.sh 101 devtools/release-note.sh 242 devtools/release.sh 979 devtools/release-version.sh 799 2,121 lines in 5 files
benchmarks/alloclog.py 41 benchmarks/alloclog.sh 95 benchmarks/auto.sh 175 benchmarks/awk-python.sh 97 benchmarks/builtin-io.sh 60 benchmarks/common.R 89 benchmarks/common.sh 97 benchmarks/compute.sh 524 benchmarks/fake_libc.py 13 benchmarks/gperftools.sh 60 benchmarks/id.sh 400 benchmarks/__init__.py 0 benchmarks/micro.sh 45 benchmarks/mycpp.sh 79 benchmarks/osh-parser.sh 452 benchmarks/osh-runtime.sh 364 benchmarks/ovm-build.sh 469 benchmarks/parse-help.sh 59 benchmarks/perf.sh 165 benchmarks/pypy.sh 42 benchmarks/pytrace.py 92 benchmarks/pytrace.sh 184 benchmarks/report.R 707 benchmarks/report.sh 127 benchmarks/startup.sh 165 benchmarks/systemtap.sh 75 benchmarks/time_.py 193 benchmarks/time-test.sh 275 benchmarks/uftrace_plugin.py 95 benchmarks/uftrace.sh 140 benchmarks/virtual_memory.py 106 benchmarks/vm-baseline.sh 149 5,634 lines in 33 files
metrics/bytecode.R 388 metrics/bytecode.sh 142 metrics/cpython-defs.R 72 metrics/native-code.R 88 metrics/native-code.sh 119 metrics/source-code.sh 457 metrics/tarball.sh 132 1,398 lines in 8 files
For the Python App Bundle.
_devbuild/gen/arg_types.py 261 _devbuild/gen/demo_lib_asdl.py 201 _devbuild/gen/find_asdl.py 935 _devbuild/gen/find_nt.h 13 _devbuild/gen/find_nt.py 10 _devbuild/gen/grammar_nt.h 93 _devbuild/gen/grammar_nt.py 90 _devbuild/gen/help_.py 209 _devbuild/gen/hnode_asdl_debug.py 6 _devbuild/gen/hnode_asdl.py 102 _devbuild/gen/id.h 382 _devbuild/gen/id_kind_asdl.py 864 _devbuild/gen/id_kind.py 492 _devbuild/gen/__init__.py 0 _devbuild/gen/option_asdl.py 354 _devbuild/gen/osh-lex.h 29,060 _devbuild/gen/osh-types.h 32 _devbuild/gen/runtime_asdl_debug.py 37 _devbuild/gen/runtime_asdl.py 2,211 _devbuild/gen/shared_variant_asdl.py 326 _devbuild/gen/syntax_asdl_debug.py 169 _devbuild/gen/syntax_asdl.py 9,087 _devbuild/gen/typed_arith_asdl.py 576 _devbuild/gen/typed_demo_asdl.py 955 _devbuild/gen/types_asdl_debug.py 3 _devbuild/gen/types_asdl.py 102 46,570 lines in 27 files
tools/deps.py 152 tools/osh2oil.py 1,343 tools/readlink.py 28 1,523 lines in 4 files
doctools/cmark.py 376 doctools/doc_html.py 106 doctools/html_head.py 85 doctools/html_lib.py 20 doctools/make_help.py 465 doctools/oil_doc.py 554 doctools/split_doc.py 151 lazylex/html.py 416 2,173 lines in 9 files
web/ajax.js 149 web/osh-to-oil.js 79 web/table/csv2html.py 397 web/table/table-sort.js 433 1,058 lines in 5 files