OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,229.9 2,684.5
mksh 111,674 425.3 1,009.7
oil-native 118,863 283.3 768.8
bash 118,863 249.2 665.5
zsh 118,863 30.8 70.0
osh-cpython 118,863 3.1 10.7
osh-ovm 118,863 3.0 9.6

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 13 3 6 162 246 223 10 0.8 1,679 t4014-format-patch.sh
flanders 14 4 13 42 862 713 14 1.0 1,733 functions
flanders 20 NA NA 81 905 909 14 0.7 2,029 configure-helper.sh
flanders 18 6 17 62 1,199 1,357 20 1.1 2,512 abuild
flanders 18 5 15 151 863 987 14 0.8 2,698 Build.sh
flanders 18 4 7 199 257 318 8 0.4 3,123 t9300-fast-import.sh
flanders 38 NA NA 205 2,600 3,809 30 0.8 5,160 test-cmd-util.sh
flanders 49 14 26 236 3,382 3,895 41 0.8 9,661 ltmain.sh
flanders 83 21 53 628 8,620 6,842 75 0.9 20,489 configure
flanders 204 34 125 2,098 20,554 19,053 194 0.9 69,779 configure-coreutils
lisa 3 1 2 67 82 84 2 0.7 1,679 t4014-format-patch.sh
lisa 5 2 3 22 268 254 4 0.9 1,733 functions
lisa 6 NA NA 41 295 268 5 1.0 2,029 configure-helper.sh
lisa 6 2 5 32 415 378 7 1.1 2,512 abuild
lisa 5 2 5 57 298 283 5 1.0 2,698 Build.sh
lisa 4 1 2 69 90 96 3 0.7 3,123 t9300-fast-import.sh
lisa 16 NA NA 107 874 797 12 0.7 5,160 test-cmd-util.sh
lisa 17 4 14 125 1,040 916 14 0.8 9,661 ltmain.sh
lisa 34 8 21 290 2,280 2,017 29 0.8 20,489 configure
lisa 82 21 58 890 6,760 6,006 73 0.9 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 130.2 559.7 284.6 10.4 6.8 7.5 171.3 1,679 t4014-format-patch.sh
flanders 124.7 456.1 137.5 40.8 2.0 2.4 128.4 1,733 functions
flanders 99.5 NA NA 25.0 2.2 2.2 148.1 2,029 configure-helper.sh
flanders 141.9 411.8 144.4 40.5 2.1 1.9 123.7 2,512 abuild
flanders 145.8 550.6 182.3 17.9 3.1 2.7 186.1 2,698 Build.sh
flanders 169.7 780.8 427.8 15.7 12.1 9.8 395.3 3,123 t9300-fast-import.sh
flanders 134.0 NA NA 25.2 2.0 1.4 169.2 5,160 test-cmd-util.sh
flanders 195.6 705.2 373.0 40.9 2.9 2.5 236.8 9,661 ltmain.sh
flanders 247.5 980.3 383.7 32.6 2.4 3.0 273.9 20,489 configure
flanders 341.4 2,028.5 556.9 33.3 3.4 3.7 360.2 69,779 configure-coreutils
lisa 493.8 1,291.5 883.7 25.0 20.4 19.9 671.6 1,679 t4014-format-patch.sh
lisa 361.0 962.8 577.7 80.6 6.5 6.8 412.6 1,733 functions
lisa 368.9 NA NA 49.7 6.9 7.6 375.7 2,029 configure-helper.sh
lisa 411.8 1,322.1 474.0 79.7 6.1 6.6 358.9 2,512 abuild
lisa 550.6 1,498.9 499.6 47.2 9.0 9.5 574.0 2,698 Build.sh
lisa 694.0 2,230.7 1,643.7 45.5 34.9 32.6 946.4 3,123 t9300-fast-import.sh
lisa 318.5 NA NA 48.4 5.9 6.5 441.0 5,160 test-cmd-util.sh
lisa 568.3 2,415.2 690.1 77.3 9.3 10.5 710.4 9,661 ltmain.sh
lisa 602.6 2,529.5 975.7 70.8 9.0 10.2 711.4 20,489 configure
lisa 848.9 3,276.0 1,201.0 78.4 10.3 11.6 950.7 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 6.4 6.6 6.6 6.6 13.1 16.1 6.3 1,679 t4014-format-patch.sh
flanders 6.4 6.4 6.6 6.4 15.5 18.9 7.2 1,733 functions
flanders 6.4 NA NA 6.5 16.1 19.5 7.4 2,029 configure-helper.sh
flanders 6.6 6.6 6.4 6.4 17.8 21.0 9.0 2,512 abuild
flanders 6.6 6.6 6.4 6.4 16.5 19.7 7.9 2,698 Build.sh
flanders 6.4 6.4 6.4 6.6 13.4 16.8 6.2 3,123 t9300-fast-import.sh
flanders 6.4 NA NA 6.4 26.4 30.4 16.2 5,160 test-cmd-util.sh
flanders 6.5 6.6 6.6 6.5 28.0 32.1 18.3 9,661 ltmain.sh
flanders 6.6 6.6 6.6 6.6 48.1 53.7 35.7 20,489 configure
flanders 6.5 6.5 6.4 6.5 110.5 120.1 96.5 69,779 configure-coreutils
lisa 6.5 6.5 6.5 6.7 13.0 16.0 6.2 1,679 t4014-format-patch.sh
lisa 6.6 6.6 6.7 6.5 15.7 19.0 7.1 1,733 functions
lisa 6.7 NA NA 6.6 16.2 19.4 7.5 2,029 configure-helper.sh
lisa 6.5 6.7 6.7 6.7 17.6 21.1 8.9 2,512 abuild
lisa 6.7 6.7 6.7 6.5 16.4 19.7 7.9 2,698 Build.sh
lisa 6.7 6.6 6.5 6.7 13.3 16.5 6.3 3,123 t9300-fast-import.sh
lisa 6.6 NA NA 6.5 26.4 30.4 16.2 5,160 test-cmd-util.sh
lisa 6.6 6.5 6.7 6.7 28.2 32.5 18.2 9,661 ltmain.sh
lisa 6.5 6.7 6.5 6.5 48.1 53.7 35.6 20,489 configure
lisa 6.5 6.6 6.7 6.5 110.4 120.3 96.6 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 12.9 25.7 1,679 t4014-format-patch.sh
lisa 13.0 25.7 1,679 t4014-format-patch.sh
flanders 15.4 28.4 1,733 functions
lisa 15.7 28.4 1,733 functions
flanders 16.1 29.0 2,029 configure-helper.sh
lisa 16.2 29.1 2,029 configure-helper.sh
flanders 17.8 30.5 2,512 abuild
lisa 17.5 30.4 2,512 abuild
flanders 16.5 29.2 2,698 Build.sh
lisa 16.4 29.4 2,698 Build.sh
flanders 13.4 26.1 3,123 t9300-fast-import.sh
lisa 13.2 26.1 3,123 t9300-fast-import.sh
flanders 26.4 39.1 5,160 test-cmd-util.sh
lisa 26.3 39.1 5,160 test-cmd-util.sh
flanders 28.0 40.9 9,661 ltmain.sh
lisa 28.1 40.9 9,661 ltmain.sh
flanders 48.1 61.0 20,489 configure
lisa 48.0 61.0 20,489 configure
flanders 110.5 123.1 69,779 configure-coreutils
lisa 110.4 123.4 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-8ecab9e7
osh-ovm osh-47967044
oil-native osh_eval.opt.stripped-d335486a
host label host id
flanders flanders-1f35df1a
lisa lisa-1f35df1a

Raw Data

filename
flanders.2020-10-22__23-57-59.times.csv
lisa.2020-10-22__23-31-08.times.csv