OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,265.5 2,928.6
mksh 111,674 427.4 959.8
bash 118,863 250.2 632.3
oil-native 118,863 189.3 620.3
zsh 118,863 32.1 67.3
osh-cpython 118,863 3.5 10.1
osh-ovm 118,863 3.1 9.1

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 14 2 6 155 290 212 9 0.6 1,679 t4014-format-patch.sh
flanders 16 4 14 41 963 667 17 1.0 1,733 functions
flanders 20 NA NA 78 943 701 21 1.0 2,029 configure-helper.sh
flanders 15 4 18 63 1,357 1,013 24 1.6 2,512 abuild
flanders 18 3 11 146 987 743 20 1.1 2,698 Build.sh
flanders 15 4 5 206 280 245 9 0.6 3,123 t9300-fast-import.sh
flanders 42 NA NA 209 2,657 2,161 46 1.1 5,160 test-cmd-util.sh
flanders 47 14 27 237 3,231 2,485 52 1.1 9,661 ltmain.sh
flanders 84 20 52 591 8,617 5,643 113 1.3 20,489 configure
flanders 203 37 129 1,982 19,331 19,809 318 1.6 69,779 configure-coreutils
lisa 3 1 2 66 99 92 2 0.7 1,679 t4014-format-patch.sh
lisa 4 1 3 22 289 293 5 1.2 1,733 functions
lisa 7 NA NA 40 316 292 5 0.7 2,029 configure-helper.sh
lisa 8 2 4 35 485 403 7 0.9 2,512 abuild
lisa 4 1 3 60 328 300 5 1.2 2,698 Build.sh
lisa 5 1 2 69 107 113 3 0.6 3,123 t9300-fast-import.sh
lisa 17 NA NA 109 952 864 14 0.8 5,160 test-cmd-util.sh
lisa 22 4 13 136 1,091 983 17 0.8 9,661 ltmain.sh
lisa 38 8 22 290 2,427 2,176 36 1.0 20,489 configure
lisa 80 20 68 940 7,009 6,238 98 1.2 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 117.5 809.2 293.9 10.9 5.8 7.9 180.8 1,679 t4014-format-patch.sh
flanders 105.5 430.9 122.9 42.7 1.8 2.6 104.1 1,733 functions
flanders 100.8 NA NA 26.1 2.2 2.9 97.7 2,029 configure-helper.sh
flanders 165.0 575.1 140.3 39.8 1.9 2.5 103.1 2,512 abuild
flanders 149.0 789.1 241.3 18.4 2.7 3.6 136.4 2,698 Build.sh
flanders 201.7 888.5 622.5 15.2 11.1 12.8 366.5 3,123 t9300-fast-import.sh
flanders 121.8 NA NA 24.7 1.9 2.4 112.3 5,160 test-cmd-util.sh
flanders 206.6 700.6 357.2 40.7 3.0 3.9 184.5 9,661 ltmain.sh
flanders 244.8 1,043.8 396.8 34.7 2.4 3.6 181.7 20,489 configure
flanders 344.4 1,864.5 542.1 35.2 3.6 3.5 219.7 69,779 configure-coreutils
lisa 539.0 2,412.4 936.9 25.4 17.0 18.3 758.4 1,679 t4014-format-patch.sh
lisa 425.1 1,156.1 655.0 80.6 6.0 5.9 348.1 1,733 functions
lisa 273.7 NA NA 51.3 6.4 6.9 413.4 2,029 configure-helper.sh
lisa 330.7 1,351.3 626.4 72.1 5.2 6.2 376.9 2,512 abuild
lisa 642.1 2,149.8 807.3 44.6 8.2 9.0 528.5 2,698 Build.sh
lisa 648.7 3,794.7 1,974.1 45.5 29.3 27.6 1,149.0 3,123 t9300-fast-import.sh
lisa 301.4 NA NA 47.5 5.4 6.0 369.3 5,160 test-cmd-util.sh
lisa 448.4 2,599.8 753.5 71.0 8.9 9.8 575.2 9,661 ltmain.sh
lisa 540.6 2,632.2 912.1 70.7 8.4 9.4 561.9 20,489 configure
lisa 870.0 3,403.9 1,030.9 74.2 10.0 11.2 713.6 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 3.1 1.6 1.7 3.7 13.4 16.3 5.2 1,679 t4014-format-patch.sh
flanders 3.1 1.7 1.8 3.8 16.0 19.1 7.3 1,733 functions
flanders 3.2 NA NA 3.6 16.5 19.8 7.7 2,029 configure-helper.sh
flanders 3.1 1.6 1.8 3.8 18.1 21.3 9.3 2,512 abuild
flanders 3.1 1.7 1.8 3.6 16.5 20.1 8.2 2,698 Build.sh
flanders 3.1 1.7 1.7 3.7 13.5 16.7 5.5 3,123 t9300-fast-import.sh
flanders 3.3 NA NA 3.7 26.7 30.6 16.5 5,160 test-cmd-util.sh
flanders 4.2 2.2 2.8 4.2 28.4 32.6 18.6 9,661 ltmain.sh
flanders 3.9 2.3 2.4 4.0 48.4 54.0 36.1 20,489 configure
flanders 3.5 1.7 1.9 3.7 110.9 120.5 97.1 69,779 configure-coreutils
lisa 3.2 1.5 1.7 3.7 13.4 16.4 5.2 1,679 t4014-format-patch.sh
lisa 3.2 1.7 1.8 3.6 15.8 19.2 7.2 1,733 functions
lisa 3.3 NA NA 3.6 16.3 19.6 7.7 2,029 configure-helper.sh
lisa 3.2 1.5 1.8 3.6 17.9 21.3 9.1 2,512 abuild
lisa 3.2 1.6 1.8 3.6 16.7 20.0 8.0 2,698 Build.sh
lisa 3.3 1.6 1.7 3.7 13.6 16.7 5.4 3,123 t9300-fast-import.sh
lisa 3.3 NA NA 3.6 26.7 30.6 16.5 5,160 test-cmd-util.sh
lisa 4.3 2.1 2.8 4.2 28.3 32.5 18.6 9,661 ltmain.sh
lisa 3.9 2.2 2.4 4.0 48.5 54.0 36.0 20,489 configure
lisa 3.5 1.7 1.9 3.7 110.8 120.6 97.0 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 13.0 25.8 1,679 t4014-format-patch.sh
lisa 13.3 26.1 1,679 t4014-format-patch.sh
flanders 16.0 28.7 1,733 functions
lisa 15.7 28.7 1,733 functions
flanders 16.5 29.1 2,029 configure-helper.sh
lisa 16.2 29.1 2,029 configure-helper.sh
flanders 18.1 30.9 2,512 abuild
lisa 17.9 30.7 2,512 abuild
flanders 16.5 29.5 2,698 Build.sh
lisa 16.7 29.5 2,698 Build.sh
flanders 13.5 26.2 3,123 t9300-fast-import.sh
lisa 13.6 26.5 3,123 t9300-fast-import.sh
flanders 26.7 39.5 5,160 test-cmd-util.sh
lisa 26.7 39.4 5,160 test-cmd-util.sh
flanders 28.4 41.2 9,661 ltmain.sh
lisa 28.3 41.2 9,661 ltmain.sh
flanders 48.4 61.4 20,489 configure
lisa 48.5 61.4 20,489 configure
flanders 110.9 123.6 69,779 configure-coreutils
lisa 110.8 123.9 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-26f3fb5b
osh-ovm osh-20bfc770
oil-native osh_eval.opt.stripped-76be85f8
host label host id
flanders flanders-59823a5d
lisa lisa-59823a5d

Raw Data

filename
flanders.2021-01-22__23-48-39.times.csv
lisa.2021-01-22__23-54-10.times.csv