OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,141.9 2,697.4
mksh 111,674 405.4 1,010.6
oil-native 118,863 270.0 779.9
bash 118,863 233.3 638.0
zsh 118,863 31.9 68.7
osh-cpython 118,863 3.1 11.0
osh-ovm 118,863 2.7 9.9

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 11 3 7 163 277 203 8 0.7 1,679 t4014-format-patch.sh
flanders 17 5 10 47 942 632 15 0.8 1,733 functions
flanders 21 NA NA 84 903 726 18 0.8 2,029 configure-helper.sh
flanders 14 5 12 71 1,280 1,028 21 1.5 2,512 abuild
flanders 16 6 11 146 940 756 15 1.0 2,698 Build.sh
flanders 18 4 9 200 287 254 10 0.6 3,123 t9300-fast-import.sh
flanders 47 NA NA 207 3,246 2,612 36 0.8 5,160 test-cmd-util.sh
flanders 48 14 32 240 3,313 3,647 45 0.9 9,661 ltmain.sh
flanders 91 21 54 584 7,424 8,118 76 0.8 20,489 configure
flanders 226 40 142 1,986 25,343 20,888 198 0.9 69,779 configure-coreutils
lisa 4 1 3 56 84 86 3 0.7 1,679 t4014-format-patch.sh
lisa 4 2 3 23 282 264 5 1.1 1,733 functions
lisa 6 NA NA 38 279 271 4 0.7 2,029 configure-helper.sh
lisa 6 2 4 33 407 371 7 1.2 2,512 abuild
lisa 7 2 3 56 301 269 5 0.8 2,698 Build.sh
lisa 5 1 2 71 92 99 3 0.6 3,123 t9300-fast-import.sh
lisa 19 NA NA 110 882 779 11 0.6 5,160 test-cmd-util.sh
lisa 20 5 13 126 1,006 897 15 0.8 9,661 ltmain.sh
lisa 35 9 24 297 2,231 1,979 27 0.8 20,489 configure
lisa 82 19 58 920 6,456 5,814 73 0.9 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 148.6 508.8 250.6 10.3 6.1 8.3 215.3 1,679 t4014-format-patch.sh
flanders 99.6 327.0 175.1 37.0 1.8 2.7 117.9 1,733 functions
flanders 96.2 NA NA 24.3 2.2 2.8 114.0 2,029 configure-helper.sh
flanders 183.4 502.4 202.6 35.3 2.0 2.4 121.9 2,512 abuild
flanders 172.9 435.2 254.5 18.5 2.9 3.6 176.3 2,698 Build.sh
flanders 174.5 821.8 363.1 15.6 10.9 12.3 312.3 3,123 t9300-fast-import.sh
flanders 109.1 NA NA 25.0 1.6 2.0 144.5 5,160 test-cmd-util.sh
flanders 200.9 710.4 300.0 40.3 2.9 2.6 215.2 9,661 ltmain.sh
flanders 225.2 989.8 382.3 35.1 2.8 2.5 271.0 20,489 configure
flanders 308.8 1,748.8 493.1 35.1 2.8 3.3 352.8 69,779 configure-coreutils
lisa 466.4 1,199.3 645.8 29.8 20.1 19.5 645.8 1,679 t4014-format-patch.sh
lisa 433.2 912.1 509.7 75.0 6.1 6.6 376.7 1,733 functions
lisa 312.2 NA NA 52.7 7.3 7.5 471.9 2,029 configure-helper.sh
lisa 448.6 1,196.2 570.9 75.2 6.2 6.8 358.9 2,512 abuild
lisa 396.8 1,498.9 817.6 48.2 9.0 10.0 499.6 2,698 Build.sh
lisa 637.3 2,230.7 1,419.5 43.9 34.1 31.6 1,156.7 3,123 t9300-fast-import.sh
lisa 273.0 NA NA 47.1 5.8 6.6 486.8 5,160 test-cmd-util.sh
lisa 495.4 1,971.6 731.9 77.0 9.6 10.8 648.4 9,661 ltmain.sh
lisa 587.1 2,276.6 871.9 69.0 9.2 10.4 756.1 20,489 configure
lisa 855.1 3,692.0 1,205.2 75.8 10.8 12.0 953.3 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 6.6 6.5 6.6 6.4 12.9 15.9 6.1 1,679 t4014-format-patch.sh
flanders 6.5 6.5 6.4 6.4 15.5 18.6 6.8 1,733 functions
flanders 6.6 NA NA 6.5 16.0 19.1 7.1 2,029 configure-helper.sh
flanders 6.5 6.6 6.4 6.4 17.4 20.7 8.5 2,512 abuild
flanders 6.4 6.5 6.5 6.4 16.3 19.6 7.5 2,698 Build.sh
flanders 6.4 6.4 6.4 6.4 13.1 16.0 6.3 3,123 t9300-fast-import.sh
flanders 6.6 NA NA 6.6 26.1 30.1 15.3 5,160 test-cmd-util.sh
flanders 6.6 6.6 6.6 6.4 27.8 31.9 17.3 9,661 ltmain.sh
flanders 6.6 6.4 6.6 6.6 48.1 53.3 33.5 20,489 configure
flanders 6.5 6.6 6.5 6.4 109.9 119.7 90.4 69,779 configure-coreutils
lisa 6.6 6.7 6.5 6.5 12.8 15.9 6.2 1,679 t4014-format-patch.sh
lisa 6.5 6.5 6.5 6.6 15.2 18.6 6.8 1,733 functions
lisa 6.7 NA NA 6.7 15.9 19.0 7.1 2,029 configure-helper.sh
lisa 6.7 6.5 6.5 6.5 17.4 20.7 8.5 2,512 abuild
lisa 6.7 6.7 6.7 6.7 16.3 19.5 7.5 2,698 Build.sh
lisa 6.7 6.7 6.5 6.6 13.0 16.3 6.2 3,123 t9300-fast-import.sh
lisa 6.7 NA NA 6.5 26.2 30.1 15.4 5,160 test-cmd-util.sh
lisa 6.5 6.7 6.7 6.6 27.8 31.9 17.3 9,661 ltmain.sh
lisa 6.7 6.5 6.6 6.7 47.9 53.3 33.4 20,489 configure
lisa 6.5 6.6 6.5 6.5 109.8 119.5 90.5 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 12.5 25.5 1,679 t4014-format-patch.sh
lisa 12.6 25.5 1,679 t4014-format-patch.sh
flanders 15.5 28.2 1,733 functions
lisa 15.2 28.2 1,733 functions
flanders 16.0 28.6 2,029 configure-helper.sh
lisa 15.9 28.6 2,029 configure-helper.sh
flanders 17.4 30.2 2,512 abuild
lisa 17.4 30.4 2,512 abuild
flanders 16.3 29.0 2,698 Build.sh
lisa 16.3 29.0 2,698 Build.sh
flanders 13.1 25.9 3,123 t9300-fast-import.sh
lisa 12.9 26.0 3,123 t9300-fast-import.sh
flanders 26.0 38.9 5,160 test-cmd-util.sh
lisa 26.2 38.9 5,160 test-cmd-util.sh
flanders 27.8 40.4 9,661 ltmain.sh
lisa 27.7 40.5 9,661 ltmain.sh
flanders 47.9 60.8 20,489 configure
lisa 47.9 60.8 20,489 configure
flanders 109.9 122.7 69,779 configure-coreutils
lisa 109.8 122.7 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-367b6013
osh-ovm osh-42c9a6ae
oil-native osh_eval.opt.stripped-f3882163
host label host id
flanders flanders-6c96a848
lisa lisa-cc016cb1

Raw Data

filename
flanders.2020-09-03__10-04-37.times.csv
lisa.2020-09-03__10-14-06.times.csv