OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,199.5 2,543.8
mksh 111,674 384.2 985.6
oil-native 118,863 261.6 703.7
bash 118,863 251.6 603.7
zsh 118,863 27.6 62.9
osh-cpython 118,863 2.6 10.3
osh-ovm 118,863 2.6 8.8

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 10 3 6 162 282 271 8 0.7 1,679 t4014-format-patch.sh
flanders 16 5 13 45 965 863 17 1.1 1,733 functions
flanders 17 NA NA 84 1,012 992 17 1.0 2,029 configure-helper.sh
flanders 21 5 17 68 1,607 1,765 20 1.0 2,512 abuild
flanders 14 5 15 156 1,439 1,324 16 1.1 2,698 Build.sh
flanders 16 5 9 204 435 434 8 0.5 3,123 t9300-fast-import.sh
flanders 43 NA NA 226 3,905 5,264 35 0.8 5,160 test-cmd-util.sh
flanders 48 12 32 264 3,799 5,962 40 0.8 9,661 ltmain.sh
flanders 86 21 56 637 8,359 9,964 81 0.9 20,489 configure
flanders 203 37 143 2,454 23,060 18,879 213 1.1 69,779 configure-coreutils
lisa 5 1 2 65 92 91 4 0.7 1,679 t4014-format-patch.sh
lisa 5 2 4 22 311 270 6 1.1 1,733 functions
lisa 7 NA NA 49 302 305 5 0.6 2,029 configure-helper.sh
lisa 6 2 4 34 463 406 6 1.0 2,512 abuild
lisa 5 2 4 60 330 289 5 1.0 2,698 Build.sh
lisa 5 1 3 78 105 104 3 0.7 3,123 t9300-fast-import.sh
lisa 19 NA NA 118 978 853 14 0.7 5,160 test-cmd-util.sh
lisa 19 4 11 138 1,209 950 14 0.7 9,661 ltmain.sh
lisa 40 9 23 318 2,500 2,096 31 0.8 20,489 configure
lisa 84 22 62 1,006 7,157 6,156 82 1.0 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 161.4 541.6 305.3 10.3 6.0 6.2 223.9 1,679 t4014-format-patch.sh
flanders 109.7 376.7 129.3 38.3 1.8 2.0 103.8 1,733 functions
flanders 120.8 NA NA 24.3 2.0 2.0 117.3 2,029 configure-helper.sh
flanders 121.4 512.7 149.5 36.7 1.6 1.4 126.9 2,512 abuild
flanders 194.1 529.0 181.1 17.3 1.9 2.0 170.8 2,698 Build.sh
flanders 200.2 650.6 347.0 15.3 7.2 7.2 376.3 3,123 t9300-fast-import.sh
flanders 119.4 NA NA 22.8 1.3 1.0 149.1 5,160 test-cmd-util.sh
flanders 203.0 785.4 301.0 36.6 2.5 1.6 239.7 9,661 ltmain.sh
flanders 239.1 961.9 365.9 32.2 2.5 2.1 253.6 20,489 configure
flanders 344.1 1,885.9 488.0 28.4 3.0 3.7 327.3 69,779 configure-coreutils
lisa 310.9 1,199.3 763.2 25.8 18.2 18.5 453.8 1,679 t4014-format-patch.sh
lisa 327.0 912.1 481.4 78.4 5.6 6.4 298.8 1,733 functions
lisa 277.9 NA NA 41.2 6.7 6.6 441.1 2,029 configure-helper.sh
lisa 425.8 1,256.0 558.2 73.0 5.4 6.2 425.8 2,512 abuild
lisa 509.1 1,420.0 749.4 44.6 8.2 9.3 509.1 2,698 Build.sh
lisa 624.6 2,230.7 1,076.9 40.2 29.8 30.2 946.4 3,123 t9300-fast-import.sh
lisa 268.8 NA NA 43.9 5.3 6.0 363.4 5,160 test-cmd-util.sh
lisa 503.2 2,146.9 862.6 69.8 8.0 10.2 700.1 9,661 ltmain.sh
lisa 505.9 2,302.1 879.4 64.4 8.2 9.8 665.2 20,489 configure
lisa 832.7 3,186.3 1,125.5 69.4 9.8 11.3 856.2 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 6.5 6.4 6.6 6.6 13.0 16.0 6.3 1,679 t4014-format-patch.sh
flanders 6.6 6.4 6.4 6.4 15.5 18.8 7.2 1,733 functions
flanders 6.6 NA NA 6.4 16.0 19.4 7.5 2,029 configure-helper.sh
flanders 6.6 6.6 6.4 6.4 17.7 21.0 8.9 2,512 abuild
flanders 6.5 6.4 6.4 6.6 16.4 19.6 7.9 2,698 Build.sh
flanders 6.4 6.4 6.5 6.5 13.2 16.3 6.1 3,123 t9300-fast-import.sh
flanders 6.5 NA NA 6.4 26.2 30.2 16.1 5,160 test-cmd-util.sh
flanders 6.4 6.6 6.5 6.5 27.9 31.9 18.3 9,661 ltmain.sh
flanders 6.4 6.4 6.6 6.6 48.1 53.5 35.7 20,489 configure
flanders 6.4 6.6 6.6 6.5 110.0 120.0 96.5 69,779 configure-coreutils
lisa 6.5 6.5 6.6 6.6 12.9 15.8 6.3 1,679 t4014-format-patch.sh
lisa 6.6 6.6 6.5 6.6 15.4 18.7 7.1 1,733 functions
lisa 6.6 NA NA 6.7 16.0 19.4 7.5 2,029 configure-helper.sh
lisa 6.7 6.7 6.5 6.5 17.5 21.0 8.9 2,512 abuild
lisa 6.5 6.6 6.5 6.5 16.3 19.6 7.9 2,698 Build.sh
lisa 6.7 6.5 6.7 6.5 13.2 16.5 6.3 3,123 t9300-fast-import.sh
lisa 6.7 NA NA 6.7 26.3 30.3 16.2 5,160 test-cmd-util.sh
lisa 6.5 6.7 6.5 6.5 28.1 31.9 18.2 9,661 ltmain.sh
lisa 6.5 6.6 6.6 6.5 48.0 53.4 35.5 20,489 configure
lisa 6.5 6.5 6.7 6.6 110.0 119.9 96.4 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 12.8 25.7 1,679 t4014-format-patch.sh
lisa 12.7 25.7 1,679 t4014-format-patch.sh
flanders 15.5 28.4 1,733 functions
lisa 15.4 28.4 1,733 functions
flanders 16.0 28.7 2,029 configure-helper.sh
lisa 16.0 28.8 2,029 configure-helper.sh
flanders 17.7 30.5 2,512 abuild
lisa 17.5 30.4 2,512 abuild
flanders 16.3 29.1 2,698 Build.sh
lisa 16.2 29.1 2,698 Build.sh
flanders 13.0 26.1 3,123 t9300-fast-import.sh
lisa 13.2 26.1 3,123 t9300-fast-import.sh
flanders 26.2 39.0 5,160 test-cmd-util.sh
lisa 26.2 39.1 5,160 test-cmd-util.sh
flanders 27.8 40.6 9,661 ltmain.sh
lisa 27.8 40.6 9,661 ltmain.sh
flanders 48.1 61.0 20,489 configure
lisa 48.0 61.0 20,489 configure
flanders 110.0 122.8 69,779 configure-coreutils
lisa 110.0 122.8 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-ef5e7265
osh-ovm osh-b4e29904
oil-native osh_eval.opt.stripped-70218822
host label host id
flanders flanders-889e300d
lisa lisa-6c96a848

Raw Data

filename
flanders.2020-10-08__12-49-33.times.csv
lisa.2020-10-08__13-14-01.times.csv