OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 956.9 2,538.0
mksh 111,674 293.2 891.3
oil-native 118,863 233.5 635.0
bash 118,863 159.7 553.6
zsh 118,863 24.4 63.5
osh-cpython 118,863 2.6 9.4
osh-ovm 118,863 2.0 8.5

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 17 4 8 187 296 337 9 0.5 1,679 t4014-format-patch.sh
flanders 19 5 14 66 1,084 1,106 17 0.9 1,733 functions
flanders 26 NA NA 120 1,242 1,141 16 0.6 2,029 configure-helper.sh
flanders 28 7 18 95 1,802 1,950 20 0.7 2,512 abuild
flanders 24 6 15 186 1,711 1,745 18 0.8 2,698 Build.sh
flanders 24 5 9 232 564 545 9 0.4 3,123 t9300-fast-import.sh
flanders 72 NA NA 298 4,926 4,285 41 0.6 5,160 test-cmd-util.sh
flanders 80 14 38 326 6,058 4,075 43 0.5 9,661 ltmain.sh
flanders 130 23 76 790 10,669 7,169 90 0.7 20,489 configure
flanders 324 52 203 2,564 31,991 23,622 245 0.8 69,779 configure-coreutils
lisa 4 2 2 62 96 104 10 2.7 1,679 t4014-format-patch.sh
lisa 4 2 4 23 314 288 5 1.2 1,733 functions
lisa 6 NA NA 41 340 307 5 0.8 2,029 configure-helper.sh
lisa 6 2 5 34 490 456 8 1.3 2,512 abuild
lisa 5 2 4 59 341 330 6 1.1 2,698 Build.sh
lisa 6 2 3 76 106 128 3 0.5 3,123 t9300-fast-import.sh
lisa 19 NA NA 122 1,107 922 14 0.8 5,160 test-cmd-util.sh
lisa 26 5 13 136 1,218 1,053 15 0.6 9,661 ltmain.sh
lisa 36 9 24 314 2,526 2,316 35 1.0 20,489 configure
lisa 102 21 71 1,005 7,462 6,802 86 0.8 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 97.6 373.1 218.1 9.0 5.7 5.0 193.0 1,679 t4014-format-patch.sh
flanders 91.2 320.9 123.8 26.3 1.6 1.6 103.2 1,733 functions
flanders 77.4 NA NA 16.9 1.6 1.8 125.2 2,029 configure-helper.sh
flanders 91.3 374.9 139.6 26.4 1.4 1.3 124.4 2,512 abuild
flanders 113.8 421.6 179.9 14.5 1.6 1.5 148.2 2,698 Build.sh
flanders 130.7 650.6 343.2 13.5 5.5 5.7 343.2 3,123 t9300-fast-import.sh
flanders 71.2 NA NA 17.3 1.0 1.2 125.5 5,160 test-cmd-util.sh
flanders 121.5 675.6 250.9 29.7 1.6 2.4 222.6 9,661 ltmain.sh
flanders 157.1 887.0 271.4 25.9 1.9 2.9 226.4 20,489 configure
flanders 215.2 1,354.9 343.6 27.2 2.2 3.0 285.0 69,779 configure-coreutils
lisa 453.8 1,119.3 799.5 27.3 17.4 16.2 169.6 1,679 t4014-format-patch.sh
lisa 393.9 753.5 481.4 75.3 5.5 6.0 333.3 1,733 functions
lisa 327.3 NA NA 49.4 6.0 6.6 414.1 2,029 configure-helper.sh
lisa 398.7 1,141.8 512.7 73.2 5.1 5.5 314.0 2,512 abuild
lisa 518.8 1,420.0 710.0 45.8 7.9 8.2 490.5 2,698 Build.sh
lisa 520.5 2,082.0 1,156.7 40.8 29.5 24.4 1,007.4 3,123 t9300-fast-import.sh
lisa 274.5 NA NA 42.4 4.7 5.6 363.4 5,160 test-cmd-util.sh
lisa 378.9 2,012.7 760.7 71.2 7.9 9.2 648.4 9,661 ltmain.sh
lisa 562.9 2,227.1 850.2 65.3 8.1 8.8 585.4 20,489 configure
lisa 682.8 3,387.3 977.3 69.4 9.4 10.3 806.7 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 6.5 6.4 6.5 6.5 13.0 16.5 6.1 1,679 t4014-format-patch.sh
flanders 6.4 6.4 6.6 6.4 15.8 18.9 7.2 1,733 functions
flanders 6.5 NA NA 6.5 16.3 19.5 7.5 2,029 configure-helper.sh
flanders 6.5 6.4 6.5 6.4 17.8 21.1 9.0 2,512 abuild
flanders 6.5 6.6 6.6 6.6 16.7 19.8 7.9 2,698 Build.sh
flanders 6.5 6.4 6.6 6.6 13.6 16.7 6.1 3,123 t9300-fast-import.sh
flanders 6.6 NA NA 6.6 26.6 30.4 16.3 5,160 test-cmd-util.sh
flanders 6.5 6.5 6.6 6.5 28.3 32.3 18.4 9,661 ltmain.sh
flanders 6.4 6.4 6.5 6.4 48.3 53.8 35.7 20,489 configure
flanders 6.5 6.6 6.4 6.6 110.7 120.5 96.9 69,779 configure-coreutils
lisa 6.5 6.7 6.6 6.7 12.9 16.4 6.3 1,679 t4014-format-patch.sh
lisa 6.6 6.6 6.7 6.6 15.8 18.9 7.2 1,733 functions
lisa 6.7 NA NA 6.7 16.4 19.6 7.6 2,029 configure-helper.sh
lisa 6.5 6.6 6.5 6.5 17.7 21.2 9.0 2,512 abuild
lisa 6.6 6.5 6.6 6.6 16.6 19.9 7.8 2,698 Build.sh
lisa 6.5 6.7 6.5 6.7 13.4 16.6 6.3 3,123 t9300-fast-import.sh
lisa 6.7 NA NA 6.5 26.5 30.5 16.3 5,160 test-cmd-util.sh
lisa 6.6 6.6 6.6 6.5 28.2 32.3 18.3 9,661 ltmain.sh
lisa 6.7 6.7 6.5 6.5 48.3 54.0 35.7 20,489 configure
lisa 6.5 6.6 6.7 6.7 110.8 120.5 96.9 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 13.0 25.8 1,679 t4014-format-patch.sh
lisa 12.9 25.8 1,679 t4014-format-patch.sh
flanders 15.7 28.4 1,733 functions
lisa 15.8 28.4 1,733 functions
flanders 16.3 29.2 2,029 configure-helper.sh
lisa 16.4 29.1 2,029 configure-helper.sh
flanders 17.7 30.4 2,512 abuild
lisa 17.7 30.7 2,512 abuild
flanders 16.6 29.5 2,698 Build.sh
lisa 16.6 29.5 2,698 Build.sh
flanders 13.6 26.2 3,123 t9300-fast-import.sh
lisa 13.4 26.2 3,123 t9300-fast-import.sh
flanders 26.6 39.1 5,160 test-cmd-util.sh
lisa 26.4 39.1 5,160 test-cmd-util.sh
flanders 28.3 40.9 9,661 ltmain.sh
lisa 28.2 40.9 9,661 ltmain.sh
flanders 48.3 61.3 20,489 configure
lisa 48.3 61.3 20,489 configure
flanders 110.7 123.6 69,779 configure-coreutils
lisa 110.8 123.6 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-8eab2174
osh-ovm osh-9b23ea09
oil-native osh_eval.opt.stripped-d8c4bb51
host label host id
flanders flanders-1f35df1a
lisa lisa-1f35df1a

Raw Data

filename
flanders.2020-11-06__15-12-26.times.csv
lisa.2020-11-06__14-02-33.times.csv