OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,437.6 2,773.2
mksh 111,674 437.6 961.1
bash 118,863 251.8 663.8
oil-native 118,863 202.3 624.2
zsh 118,863 31.6 68.0
osh-cpython 118,863 3.5 10.3
osh-ovm 118,863 2.7 9.2

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 16 2 5 157 289 210 11 0.7 1,679 t4014-format-patch.sh
flanders 16 4 11 42 917 663 19 1.2 1,733 functions
flanders 19 NA NA 79 930 742 23 1.2 2,029 configure-helper.sh
flanders 20 4 17 65 1,333 1,000 24 1.2 2,512 abuild
flanders 17 4 9 154 977 770 20 1.2 2,698 Build.sh
flanders 14 2 7 209 333 267 10 0.7 3,123 t9300-fast-import.sh
flanders 41 NA NA 207 3,260 2,323 48 1.2 5,160 test-cmd-util.sh
flanders 47 9 28 239 3,346 2,484 50 1.1 9,661 ltmain.sh
flanders 85 16 54 586 7,668 5,549 103 1.2 20,489 configure
flanders 199 36 125 2,019 24,409 19,750 279 1.4 69,779 configure-coreutils
lisa 3 1 2 59 85 93 2 0.9 1,679 t4014-format-patch.sh
lisa 4 1 3 24 300 257 5 1.2 1,733 functions
lisa 5 NA NA 39 317 289 6 1.2 2,029 configure-helper.sh
lisa 6 1 4 32 439 401 6 1.0 2,512 abuild
lisa 4 1 3 57 308 291 5 1.2 2,698 Build.sh
lisa 4 1 1 79 96 104 2 0.6 3,123 t9300-fast-import.sh
lisa 15 NA NA 109 934 837 15 1.0 5,160 test-cmd-util.sh
lisa 20 4 12 127 1,101 977 19 1.0 9,661 ltmain.sh
lisa 34 10 24 303 2,334 2,142 36 1.0 20,489 configure
lisa 84 21 68 917 6,940 6,111 93 1.1 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 105.6 819.8 308.7 10.7 5.8 8.0 153.4 1,679 t4014-format-patch.sh
flanders 111.3 478.1 158.9 41.3 1.9 2.6 90.4 1,733 functions
flanders 108.5 NA NA 25.6 2.2 2.7 90.0 2,029 configure-helper.sh
flanders 123.8 604.6 150.7 38.8 1.9 2.5 104.9 2,512 abuild
flanders 161.5 631.3 314.4 17.6 2.8 3.5 133.1 2,698 Build.sh
flanders 228.1 1,321.1 448.6 14.9 9.4 11.7 307.0 3,123 t9300-fast-import.sh
flanders 127.2 NA NA 24.9 1.6 2.2 106.4 5,160 test-cmd-util.sh
flanders 204.2 1,068.0 349.8 40.5 2.9 3.9 193.7 9,661 ltmain.sh
flanders 242.1 1,245.6 380.8 34.9 2.7 3.7 198.1 20,489 configure
flanders 351.3 1,953.6 557.2 34.6 2.9 3.5 250.2 69,779 configure-coreutils
lisa 593.1 1,529.1 1,018.2 28.5 19.7 18.1 674.0 1,679 t4014-format-patch.sh
lisa 432.5 1,479.9 656.4 73.0 5.8 6.7 360.6 1,733 functions
lisa 408.2 NA NA 51.6 6.4 7.0 344.6 2,029 configure-helper.sh
lisa 403.5 1,706.5 659.1 77.4 5.7 6.3 392.7 2,512 abuild
lisa 615.1 2,227.9 1,014.7 47.0 8.8 9.3 526.7 2,698 Build.sh
lisa 716.8 3,928.3 2,115.9 39.6 32.5 30.1 1,258.8 3,123 t9300-fast-import.sh
lisa 351.7 NA NA 47.2 5.5 6.2 336.3 5,160 test-cmd-util.sh
lisa 489.2 2,688.8 821.3 76.1 8.8 9.9 510.2 9,661 ltmain.sh
lisa 597.1 2,027.8 854.5 67.6 8.8 9.6 572.9 20,489 configure
lisa 835.1 3,350.7 1,023.0 76.1 10.1 11.4 748.8 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 3.2 1.7 1.7 3.7 13.4 16.3 5.3 1,679 t4014-format-patch.sh
flanders 3.1 1.7 1.7 3.7 16.0 19.2 7.3 1,733 functions
flanders 3.2 NA NA 3.7 16.5 19.7 7.8 2,029 configure-helper.sh
flanders 3.2 1.7 1.8 3.7 18.1 21.3 9.2 2,512 abuild
flanders 3.2 1.7 1.7 3.7 16.7 20.0 8.2 2,698 Build.sh
flanders 3.0 1.6 1.7 3.7 13.6 16.8 5.5 3,123 t9300-fast-import.sh
flanders 3.3 NA NA 3.7 26.6 30.7 16.6 5,160 test-cmd-util.sh
flanders 4.3 2.2 2.8 4.2 28.5 32.5 18.6 9,661 ltmain.sh
flanders 4.0 2.2 2.4 4.1 48.5 53.9 36.0 20,489 configure
flanders 3.4 1.9 1.8 3.8 110.9 120.5 97.0 69,779 configure-coreutils
lisa 3.1 1.7 1.8 3.7 13.4 16.4 5.2 1,679 t4014-format-patch.sh
lisa 3.2 1.7 1.7 3.6 15.9 19.1 7.2 1,733 functions
lisa 3.2 NA NA 3.6 16.5 19.8 7.7 2,029 configure-helper.sh
lisa 3.2 1.5 1.9 3.7 17.9 21.2 9.2 2,512 abuild
lisa 3.3 1.5 1.8 3.7 16.7 20.0 8.1 2,698 Build.sh
lisa 3.1 1.6 1.7 3.6 13.6 16.7 5.5 3,123 t9300-fast-import.sh
lisa 3.4 NA NA 3.6 26.5 30.7 16.5 5,160 test-cmd-util.sh
lisa 4.3 2.1 2.8 4.2 28.5 32.4 18.6 9,661 ltmain.sh
lisa 4.0 2.2 2.3 3.9 48.4 53.9 35.9 20,489 configure
lisa 3.5 1.8 1.9 3.7 111.0 120.6 97.1 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 13.1 26.1 1,679 t4014-format-patch.sh
lisa 13.3 26.1 1,679 t4014-format-patch.sh
flanders 15.9 28.7 1,733 functions
lisa 15.9 28.7 1,733 functions
flanders 16.5 29.1 2,029 configure-helper.sh
lisa 16.4 29.1 2,029 configure-helper.sh
flanders 18.1 30.9 2,512 abuild
lisa 17.9 30.7 2,512 abuild
flanders 16.7 29.5 2,698 Build.sh
lisa 16.7 29.5 2,698 Build.sh
flanders 13.6 26.2 3,123 t9300-fast-import.sh
lisa 13.4 26.2 3,123 t9300-fast-import.sh
flanders 26.6 39.5 5,160 test-cmd-util.sh
lisa 26.5 39.4 5,160 test-cmd-util.sh
flanders 28.5 41.2 9,661 ltmain.sh
lisa 28.5 41.2 9,661 ltmain.sh
flanders 48.5 61.4 20,489 configure
lisa 48.4 61.4 20,489 configure
flanders 110.9 123.6 69,779 configure-coreutils
lisa 111.0 123.9 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-fdccbdcd
osh-ovm osh-0d3ffea9
oil-native osh_eval.opt.stripped-9c917933
host label host id
flanders flanders-f9790abf
lisa lisa-f9790abf

Raw Data

filename
flanders.2021-03-18__15-01-01.times.csv
lisa.2021-03-18__14-57-16.times.csv