| category | total lines | num files |
| OSH (and common libraries) | 37,322 | 76 |
| Oil Language (and Tea) | 4,831 | 12 |
| Code Borrowed from Python's stdlib | 419 | 5 |
| Spec Tests | 36,910 | 143 |
| Gold Tests | 2,725 | 26 |
| Language Unit Tests | 7,234 | 42 |
| Other Unit Tests | 924 | 11 |
| mycpp Translator | 4,326 | 10 |
| mycpp Test Data | 2,190 | 25 |
| Zephyr ASDL | 2,905 | 10 |
| pgen2 Parser Generator | 1,717 | 8 |
| Other Code Generators | 2,070 | 8 |
| Build Automation | 5,110 | 33 |
| Test Automation | 13,498 | 44 |
| Release Automation | 2,324 | 7 |
| Benchmarks | 5,628 | 33 |
| Metrics | 1,418 | 8 |
| Generated Python Code | 41,634 | 27 |
| Tools | 1,523 | 4 |
| Doc Tools | 2,381 | 10 |
| Web | 1,058 | 5 |
The README for oilshell/oil has another overview of the repository.
This is the input to the translators, written in statically-typed Python. Note that bash is at least 140K lines of code, and OSH implements a large part of bash and more.
bin/oil.py 355 core/alloc.py 166 core/ansi.py 17 core/completion.py 1,261 core/comp_ui.py 585 core/dev.py 597 core/error.py 198 core/executor.py 568 core/main_loop.py 361 core/optview.py 57 core/process.py 1,498 core/pyerror.py 58 core/pyos.py 384 core/pyutil.py 247 core/runtime.asdl 194 core/shell_native.py 500 core/shell.py 716 core/state.py 2,385 core/ui.py 386 core/util.py 93 core/vm.py 213 frontend/args.py 673 frontend/builtin_def.py 164 frontend/consts.py 342 frontend/flag_def.py 328 frontend/flag_spec.py 391 frontend/id_kind_def.py 638 frontend/lexer_def.py 1,012 frontend/lexer.py 330 frontend/location.py 99 frontend/match.py 254 frontend/option_def.py 382 frontend/parse_lib.py 477 frontend/py_reader.py 81 frontend/reader.py 141 frontend/signal_def.py 43 frontend/syntax_abbrev.py 144 frontend/syntax.asdl 582 frontend/typed_args.py 70 frontend/types.asdl 41 mycpp/mylib.pyi 74 native/fanos.c 294 native/fastlex.c 223 native/fastlex.pyi 13 native/libc.c 402 native/libc.pyi 11 native/posix_.pyi 211 osh/arith_parse.py 184 osh/bool_parse.py 269 osh/bool_stat.py 136 osh/braces.py 498 osh/builtin_assign.py 514 osh/builtin_bracket.py 280 osh/builtin_comp.py 518 osh/builtin_lib.py 94 osh/builtin_meta.py 436 osh/builtin_misc.py 853 osh/builtin_printf.py 477 osh/builtin_process.py 555 osh/builtin_pure.py 960 osh/cmd_eval.py 1,858 osh/cmd_parse.py 2,411 osh/glob_.py 498 osh/history.py 184 osh/prompt.py 300 osh/sh_expr_eval.py 1,047 osh/split.py 305 osh/string_ops.py 454 osh/tdop.py 333 osh/word_compile.py 257 osh/word_eval.py 2,297 osh/word_parse.py 1,838 osh/word_.py 817 qsn_/qsn_native.py 75 qsn_/qsn.py 615 37,322 lines in 76 files
oil_lang/builtin_oil.py 364 oil_lang/expr_eval.py 797 oil_lang/expr_parse.py 335 oil_lang/expr_to_ast.py 1,536 oil_lang/funcs_builtin.py 415 oil_lang/funcs.py 127 oil_lang/grammar.pgen2 615 oil_lang/objects.py 144 oil_lang/regex_translate.py 262 tea/tea_eval.py 137 tea/tea_main.py 99 4,831 lines in 12 files
pylib/cgi.py 19 pylib/collections_.py 204 pylib/os_path.py 169 pylib/path_stat.py 27 419 lines in 5 files
A comprehensive test suite that compares OSH against other shells. If OSH passes these tests in BOTH Python and C++, it means that the translation works.
spec/alias.test.sh 536 spec/append.test.sh 291 spec/arith-context.test.sh 81 spec/arith.test.sh 631 spec/array-compat.test.sh 96 spec/array.test.sh 637 spec/assign-deferred.test.sh 105 spec/assign-dialects.test.sh 59 spec/assign-extended.test.sh 810 spec/assign.test.sh 672 spec/assoc.test.sh 631 spec/assoc-zsh.test.sh 53 spec/background.test.sh 182 spec/ble-features.test.sh 644 spec/ble-idioms.test.sh 168 spec/blog1.test.sh 95 spec/blog2.test.sh 49 spec/blog-other1.test.sh 55 spec/brace-expansion.test.sh 441 spec/bugs.test.sh 155 spec/builtin-bash.test.sh 346 spec/builtin-bracket.test.sh 564 spec/builtin-completion.test.sh 415 spec/builtin-dirs.test.sh 205 spec/builtin-eval-source.test.sh 249 spec/builtin-getopts.test.sh 318 spec/builtin-io.test.sh 751 spec/builtin-printf.test.sh 831 spec/builtins2.test.sh 233 spec/builtin-special.test.sh 111 spec/builtins.test.sh 420 spec/builtin-times.test.sh 15 spec/builtin-trap.test.sh 410 spec/builtin-vars.test.sh 710 spec/case_.test.sh 165 spec/command-parsing.test.sh 63 spec/command-sub.test.sh 272 spec/command_.test.sh 192 spec/comments.test.sh 11 spec/dbracket.test.sh 436 spec/dparen.test.sh 202 spec/empty-bodies.test.sh 24 spec/errexit-oil.test.sh 755 spec/errexit.test.sh 393 spec/exit-status.test.sh 238 spec/explore-parsing.test.sh 42 spec/extglob-files.test.sh 391 spec/extglob-match.test.sh 400 spec/fatal-errors.test.sh 186 spec/for-expr.test.sh 92 spec/func-parsing.test.sh 105 spec/glob.test.sh 396 spec/hay-isolation.test.sh 169 spec/hay-meta.test.sh 200 spec/hay.test.sh 645 spec/here-doc.test.sh 365 spec/if_.test.sh 56 spec/interactive.test.sh 216 spec/introspect.test.sh 237 spec/let.test.sh 23 spec/loop.test.sh 408 spec/nameref.test.sh 642 spec/nix-idioms.test.sh 151 spec/nocasematch-match.test.sh 69 spec/oil-array.test.sh 38 spec/oil-assign.test.sh 359 spec/oil-bin.test.sh 17 spec/oil-blocks.test.sh 377 spec/oil-bugs.test.sh 36 spec/oil-builtin-argparse.test.sh 35 spec/oil-builtin-describe.test.sh 38 spec/oil-builtin-error.test.sh 424 spec/oil-builtin-pp.test.sh 47 spec/oil-builtin-process.test.sh 85 spec/oil-builtin-shopt.test.sh 179 spec/oil-builtins.test.sh 559 spec/oil-case.test.sh 27 spec/oil-command-sub.test.sh 109 spec/oil-demo.test.sh 59 spec/oil-expr-sub.test.sh 40 spec/oil-expr.test.sh 881 spec/oil-for.test.sh 173 spec/oil-funcs-builtin.test.sh 195 spec/oil-funcs-external.test.sh 87 spec/oil-interactive.test.sh 15 spec/oil-json.test.sh 121 spec/oil-keywords.test.sh 37 spec/oil-multiline.test.sh 179 spec/oil-options-assign.test.sh 103 spec/oil-options.test.sh 781 spec/oil-proc.test.sh 270 spec/oil-regex.test.sh 689 spec/oil-reserved.test.sh 46 spec/oil-scope.test.sh 764 spec/oil-slice-range.test.sh 162 spec/oil-special-vars.test.sh 48 spec/oil-string.test.sh 344 spec/oil-tuple.test.sh 44 spec/oil-usage.test.sh 22 spec/oil-user-feedback.test.sh 209 spec/oil-var-sub.test.sh 45 spec/oil-with-sh.test.sh 77 spec/oil-word-eval.test.sh 81 spec/oil-xtrace.test.sh 477 spec/osh-only.test.sh 78 spec/parse-errors.test.sh 192 spec/pipeline.test.sh 166 spec/posix.test.sh 153 spec/process-sub.test.sh 205 spec/prompt.test.sh 254 spec/quote.test.sh 248 spec/redirect.test.sh 654 spec/regex.test.sh 344 spec/serialize.test.sh 229 spec/shell-grammar.test.sh 205 spec/sh-func.test.sh 150 spec/sh-options.test.sh 694 spec/sh-usage.test.sh 90 spec/smoke.test.sh 123 spec/strict-options.test.sh 249 spec/subshell.test.sh 23 spec/tea-func.test.sh 288 spec/tilde.test.sh 172 spec/TODO-deprecate.test.sh 14 spec/toysh-posix.test.sh 347 spec/toysh.test.sh 144 spec/type-compat.test.sh 80 spec/var-num.test.sh 39 spec/var-op-bash.test.sh 309 spec/var-op-len.test.sh 219 spec/var-op-patsub.test.sh 291 spec/var-op-slice.test.sh 315 spec/var-op-strip.test.sh 357 spec/var-op-test.test.sh 339 spec/var-ref.test.sh 485 spec/vars-bash.test.sh 27 spec/vars-special.test.sh 456 spec/var-sub-quote.test.sh 312 spec/var-sub.test.sh 74 spec/word-eval.test.sh 62 spec/word-split.test.sh 387 spec/xtrace.test.sh 344 36,910 lines in 143 files
Another suite that tests shells "from the outside". Instead of making explicit assertions, we verify that OSH behaves like bash.
test/gold/abuild.sh 9 test/gold/and-or.sh 27 test/gold/big-here-doc.sh 1,531 test/gold/case-in-subshell.sh 37 test/gold/char-class.sh 33 test/gold/command-sub-2.sh 12 test/gold/command-sub.sh 39 test/gold/comments.sh 22 test/gold/complex-here-docs.sh 192 test/gold/configure-bug.sh 13 test/gold/de1.sh 24 test/gold/declare.sh 28 test/gold/dollar-sq.sh 42 test/gold/echo-e.sh 36 test/gold/errexit-confusion.sh 139 test/gold/errexit.sh 121 test/gold/export.sh 65 test/gold/glob.sh 26 test/gold/lineno.sh 23 test/gold/nix.sh 32 test/gold/readlink.sh 61 test/gold/readonly.sh 24 test/gold/scope.sh 126 test/gold/strip-op-char-class.sh 6 test/gold/word-eval.sh 57 2,725 lines in 26 files
core/alloc_test.py 60 core/completion_test.py 798 core/comp_ui_test.py 179 core/process_test.py 196 core/pyutil_test.py 19 core/state_test.py 316 core/ui_test.py 32 core/util_test.py 25 frontend/args_test.py 232 frontend/id_kind_def_test.py 97 frontend/lexer_def_test.py 436 frontend/lexer_gen_test.py 51 frontend/lexer_test.py 45 frontend/match_test.py 56 frontend/reader_test.py 49 native/fanos_test.py 246 native/fastlex_test.py 83 native/libc_test.py 304 native/line_input_test.py 27 native/posix_test.py 184 oil_lang/builtin_oil_test.py 157 oil_lang/expr_parse_test.py 144 oil_lang/objects_test.py 19 osh/arith_parse_test.py 179 osh/bool_parse_test.py 122 osh/braces_test.py 187 osh/builtin_bracket_test.py 29 osh/builtin_comp_test.py 19 osh/builtin_lib_test.py 93 osh/builtin_misc_test.py 48 osh/cmd_eval_test.py 74 osh/cmd_parse_test.py 1,263 osh/glob_test.py 163 osh/history_test.py 124 osh/prompt_test.py 54 osh/split_test.py 169 osh/string_ops_test.py 133 osh/word_compile_test.py 16 osh/word_eval_test.py 144 osh/word_parse_test.py 561 osh/word_test.py 101 7,234 lines in 42 files
asdl/format_test.py 47 asdl/front_end_test.py 141 asdl/gen_python_test.py 153 asdl/typed_arith_parse_test.py 227 build/app_deps_test.py 27 pylib/os_path_test.py 59 pylib/path_stat_test.py 24 test/sh_spec_test.py 138 test/wild_report_test.py 29 tools/osh2oil_test.py 79 924 lines in 11 files
This prototype uses the MyPy frontend to translate statically-typed Python to C++. The generated code calls a small runtime which implements things like List[T], Dict[K, V], and Python's len().
mycpp/const_pass.py 526 mycpp/cppgen_pass.py 2,622 mycpp/crash.py 37 mycpp/debug_pass.py 459 mycpp/format_strings.py 110 mycpp/mycpp_main.py 346 mycpp/mylib.py 160 mycpp/pass_state.py 54 mycpp/util.py 12 4,326 lines in 10 files
Small Python examples that translate to C++, compile, and run.
mycpp/examples/asdl_generated.py 62 mycpp/examples/cartesian.py 75 mycpp/examples/cgi.py 59 mycpp/examples/classes.py 141 mycpp/examples/containers.py 170 mycpp/examples/control_flow.py 110 mycpp/examples/escape.py 67 mycpp/examples/fib_iter.py 54 mycpp/examples/fib_recursive.py 49 mycpp/examples/files.py 53 mycpp/examples/length.py 106 mycpp/examples/lexer_main.py 48 mycpp/examples/loops.py 177 mycpp/examples/modules.py 78 mycpp/examples/parse.py 232 mycpp/examples/pgen2_demo.py 106 mycpp/examples/scoped_resource.py 176 mycpp/examples/strings.py 61 mycpp/examples/test_cast.py 59 mycpp/examples/test_conditional.py 67 mycpp/examples/test_hoist.py 61 mycpp/examples/test_switch.py 42 mycpp/examples/tuple_return_value.py 53 mycpp/examples/varargs.py 84 2,190 lines in 25 files
A DSL for algebraic data types, borrowed from Python. Oil is the most strongly typed Bourne shell implementation!
asdl/asdl_.py 253 asdl/format.py 494 asdl/front_end.py 488 asdl/gen_cpp.py 622 asdl/gen_python.py 528 asdl/pybase.py 63 asdl/runtime.py 41 asdl/tool.py 282 asdl/visitor.py 134 2,905 lines in 10 files
An LL(1) parser generator used to parse Oil expressions. Also borrowed from CPython.
pgen2/driver.py 101 pgen2/grammar.py 222 pgen2/parse.py 209 pgen2/pgen.py 428 pgen2/pnode.py 29 pgen2/tokenize.py 579 pgen2/token.py 149 1,717 lines in 8 files
In order to make Oil statically typed, we had to abandon Python reflection and use C++ source code generation instead. The lexer, flag definitions, and constants can be easily compiled to C++.
core/optview_gen.py 89 frontend/consts_gen.py 564 frontend/flag_gen.py 485 frontend/lexer_gen.py 490 frontend/option_gen.py 115 oil_lang/grammar_gen.py 226 osh/arith_parse_gen.py 101 2,070 lines in 8 files
build/actions.sh 223 build/app_deps.py 152 build/c_module_srcs.py 90 build/c_module_toc.py 51 build/codegen.sh 143 build/common.sh 63 build/compiler-demo.sh 106 build/compile.sh 436 build/cpython_defs.py 515 build/cpython-defs.sh 349 build/dev.sh 460 build/dev-shell.sh 42 build/doc.sh 416 build/hello.mk 51 build/make_zip.py 53 build/native_graph.py 368 build/native.sh 110 build/native-steps.sh 226 build/oil.mk 60 build/opy.mk 49 build/prepare.sh 30 build/setup_fanos.py 11 build/setup_fastlex.py 18 build/setup_line_input.py 13 build/setup_posix.py 19 build/setup.py 11 build/test.sh 92 build/translate.sh 184 configure 395 install 87 Makefile 199 portable-rules.mk 88 5,110 lines in 33 files
test/alpine.sh 277 test/arena.sh 60 test/ble.sh 51 test/common.sh 118 test/common-test.sh 22 test/cpp-unit.sh 81 test/gentoo.sh 83 test/gold.sh 163 test/group-session.sh 127 test/lint.sh 334 test/nix.sh 24 test/oil-runtime-errors.sh 233 test/opyc.sh 122 test/osh2oil.sh 1,370 test/oshc-deps.sh 104 test/osh-usage.sh 213 test/other.sh 34 test/parse-errors.sh 1,466 test/py_stdlib.sh 32 test/report.R 54 test/runtime-errors.sh 1,111 test/shebang.sh 26 test/shell-vs-shell.sh 314 test/sh_spec.py 1,267 test/signal-state.sh 96 test/smoosh_import.py 31 test/smoosh.sh 213 test/spec-alpine.sh 151 test/spec-bin.sh 246 test/spec-common.sh 51 test/spec-cpp.sh 382 test/spec_lib.py 188 test/spec-runner.sh 411 test/spec.sh 1,145 test/stateful.sh 216 test/syscall.py 219 test/syscall.sh 415 test/tsv-lib.sh 37 test/unit.sh 247 test/vagrant.sh 54 test/wild_report.py 802 test/wild-runner.sh 189 test/wild.sh 719 13,498 lines in 44 files
devtools/release-history.sh 207 devtools/release-native.sh 103 devtools/release-note.sh 225 devtools/release-patch.sh 20 devtools/release.sh 925 devtools/release-version.sh 844 2,324 lines in 7 files
benchmarks/alloclog.py 41 benchmarks/alloclog.sh 93 benchmarks/auto.sh 172 benchmarks/awk-python.sh 97 benchmarks/builtin-io.sh 60 benchmarks/common.R 89 benchmarks/common.sh 80 benchmarks/compute.sh 531 benchmarks/fake_libc.py 13 benchmarks/gperftools.sh 60 benchmarks/id.sh 403 benchmarks/__init__.py 0 benchmarks/micro.sh 45 benchmarks/mycpp.sh 83 benchmarks/osh-parser.sh 455 benchmarks/osh-runtime.sh 364 benchmarks/ovm-build.sh 480 benchmarks/parse-help.sh 59 benchmarks/perf.sh 165 benchmarks/pypy.sh 42 benchmarks/pytrace.py 92 benchmarks/pytrace.sh 184 benchmarks/report.R 707 benchmarks/report.sh 127 benchmarks/startup.sh 165 benchmarks/systemtap.sh 75 benchmarks/time_.py 193 benchmarks/time-test.sh 260 benchmarks/uftrace_plugin.py 95 benchmarks/uftrace.sh 143 benchmarks/virtual_memory.py 106 benchmarks/vm-baseline.sh 149 5,628 lines in 33 files
metrics/bytecode.R 388 metrics/bytecode.sh 142 metrics/cpython-defs.R 72 metrics/native-code.R 88 metrics/native-code.sh 133 metrics/source-code.sh 463 metrics/tarball.sh 132 1,418 lines in 8 files
For the Python App Bundle.
_devbuild/gen/arg_types.py 265 _devbuild/gen/demo_lib_asdl.py 202 _devbuild/gen/find_asdl.py 936 _devbuild/gen/find_nt.h 13 _devbuild/gen/find_nt.py 10 _devbuild/gen/grammar_nt.h 93 _devbuild/gen/grammar_nt.py 90 _devbuild/gen/help_.py 220 _devbuild/gen/hnode_asdl_debug.py 6 _devbuild/gen/hnode_asdl.py 103 _devbuild/gen/id.h 382 _devbuild/gen/id_kind_asdl.py 864 _devbuild/gen/id_kind.py 492 _devbuild/gen/__init__.py 0 _devbuild/gen/option_asdl.py 372 _devbuild/gen/osh-lex.h 24,024 _devbuild/gen/osh-types.h 36 _devbuild/gen/runtime_asdl_debug.py 37 _devbuild/gen/runtime_asdl.py 2,211 _devbuild/gen/shared_variant_asdl.py 327 _devbuild/gen/syntax_asdl_debug.py 169 _devbuild/gen/syntax_asdl.py 9,125 _devbuild/gen/typed_arith_asdl.py 577 _devbuild/gen/typed_demo_asdl.py 956 _devbuild/gen/types_asdl_debug.py 3 _devbuild/gen/types_asdl.py 121 41,634 lines in 27 files
tools/deps.py 152 tools/osh2oil.py 1,343 tools/readlink.py 28 1,523 lines in 4 files
doctools/cmark.py 376 doctools/doc_html.py 107 doctools/html_head.py 85 doctools/html_lib.py 20 doctools/make_help.py 465 doctools/oil_doc.py 619 doctools/spelling.py 142 doctools/split_doc.py 151 lazylex/html.py 416 2,381 lines in 10 files
web/ajax.js 149 web/osh-to-oil.js 79 web/table/csv2html.py 397 web/table/table-sort.js 433 1,058 lines in 5 files