OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,276.1 2,787.1
mksh 111,674 428.4 992.9
bash 118,863 249.2 640.1
oil-native 118,863 172.6 557.3
zsh 118,863 32.1 67.3
osh-cpython 118,863 3.5 10.2
osh-ovm 118,863 3.0 9.2

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 10 3 5 167 233 211 7 0.7 1,679 t4014-format-patch.sh
flanders 16 3 12 46 829 667 19 1.2 1,733 functions
flanders 16 NA NA 81 925 710 18 1.1 2,029 configure-helper.sh
flanders 20 4 17 61 1,349 1,015 24 1.2 2,512 abuild
flanders 18 5 13 147 973 734 17 0.9 2,698 Build.sh
flanders 18 3 8 204 282 264 9 0.5 3,123 t9300-fast-import.sh
flanders 43 NA NA 210 2,616 2,379 50 1.2 5,160 test-cmd-util.sh
flanders 45 15 29 243 3,233 3,130 63 1.4 9,661 ltmain.sh
flanders 86 20 47 586 8,516 6,871 126 1.5 20,489 configure
flanders 204 35 129 1,955 20,731 18,143 355 1.7 69,779 configure-coreutils
lisa 3 1 2 62 97 101 2 0.9 1,679 t4014-format-patch.sh
lisa 3 1 3 23 287 266 9 2.5 1,733 functions
lisa 7 NA NA 40 313 289 5 0.7 2,029 configure-helper.sh
lisa 8 2 4 32 449 398 8 1.0 2,512 abuild
lisa 6 1 3 64 348 292 8 1.2 2,698 Build.sh
lisa 4 1 2 74 96 98 3 0.6 3,123 t9300-fast-import.sh
lisa 17 NA NA 121 929 846 20 1.2 5,160 test-cmd-util.sh
lisa 18 4 11 124 1,066 962 20 1.1 9,661 ltmain.sh
lisa 36 8 24 300 2,412 2,193 34 0.9 20,489 configure
lisa 82 22 64 926 6,937 6,201 105 1.3 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 170.4 543.0 335.3 10.1 7.2 8.0 234.8 1,679 t4014-format-patch.sh
flanders 110.0 674.6 139.4 38.0 2.1 2.6 93.3 1,733 functions
flanders 123.1 NA NA 25.1 2.2 2.9 110.5 2,029 configure-helper.sh
flanders 124.3 578.7 150.7 41.3 1.9 2.5 104.1 2,512 abuild
flanders 146.8 586.0 209.8 18.4 2.8 3.7 162.8 2,698 Build.sh
flanders 177.8 939.8 399.6 15.3 11.1 11.8 329.4 3,123 t9300-fast-import.sh
flanders 119.0 NA NA 24.6 2.0 2.2 103.3 5,160 test-cmd-util.sh
flanders 214.4 641.2 330.1 39.7 3.0 3.1 153.9 9,661 ltmain.sh
flanders 237.1 1,025.3 432.5 35.0 2.4 3.0 162.2 20,489 configure
flanders 342.3 2,020.6 539.7 35.7 3.4 3.8 196.4 69,779 configure-coreutils
lisa 596.0 1,689.1 1,039.0 27.0 17.3 16.6 685.0 1,679 t4014-format-patch.sh
lisa 507.5 1,603.1 658.7 76.7 6.0 6.5 202.2 1,733 functions
lisa 280.5 NA NA 50.5 6.5 7.0 398.5 2,029 configure-helper.sh
lisa 326.8 1,659.2 667.9 77.8 5.6 6.3 316.6 2,512 abuild
lisa 442.2 2,061.1 966.0 42.4 7.7 9.2 354.2 2,698 Build.sh
lisa 708.8 3,532.8 1,612.3 42.3 32.5 31.8 1,200.2 3,123 t9300-fast-import.sh
lisa 304.5 NA NA 42.5 5.6 6.1 261.7 5,160 test-cmd-util.sh
lisa 524.3 2,628.8 860.7 77.7 9.1 10.0 471.9 9,661 ltmain.sh
lisa 565.5 2,468.9 850.9 68.2 8.5 9.3 606.9 20,489 configure
lisa 846.6 3,127.4 1,083.0 75.3 10.1 11.3 664.0 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 3.1 1.6 1.7 3.7 13.2 16.3 5.2 1,679 t4014-format-patch.sh
flanders 3.1 1.7 1.7 3.7 15.9 19.2 7.3 1,733 functions
flanders 3.1 NA NA 3.7 16.4 19.6 7.7 2,029 configure-helper.sh
flanders 3.1 1.8 1.8 3.7 17.9 21.1 9.2 2,512 abuild
flanders 3.2 1.7 1.8 3.8 16.6 20.0 8.2 2,698 Build.sh
flanders 3.1 1.6 1.7 3.7 13.5 16.6 5.5 3,123 t9300-fast-import.sh
flanders 3.3 NA NA 3.7 26.6 30.4 16.6 5,160 test-cmd-util.sh
flanders 4.2 2.2 2.8 4.2 28.3 32.4 18.6 9,661 ltmain.sh
flanders 4.0 2.3 2.3 4.1 48.5 53.9 35.9 20,489 configure
flanders 3.5 1.9 1.8 3.8 110.8 120.2 97.1 69,779 configure-coreutils
lisa 3.2 1.5 1.7 3.7 13.0 16.2 5.1 1,679 t4014-format-patch.sh
lisa 3.2 1.6 1.7 3.7 15.8 19.1 7.3 1,733 functions
lisa 3.3 NA NA 3.7 16.4 19.7 7.6 2,029 configure-helper.sh
lisa 3.3 1.6 1.9 3.6 17.9 21.2 9.2 2,512 abuild
lisa 3.2 1.6 1.7 3.6 16.6 20.2 8.0 2,698 Build.sh
lisa 3.2 1.6 1.7 3.7 13.4 16.7 5.5 3,123 t9300-fast-import.sh
lisa 3.4 NA NA 3.7 26.7 30.4 16.5 5,160 test-cmd-util.sh
lisa 4.3 2.1 2.8 4.2 28.4 32.4 18.6 9,661 ltmain.sh
lisa 4.0 2.1 2.4 3.9 48.4 53.9 35.9 20,489 configure
lisa 3.5 1.7 2.0 3.7 110.8 120.5 97.0 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 13.2 25.8 1,679 t4014-format-patch.sh
lisa 13.0 25.8 1,679 t4014-format-patch.sh
flanders 15.9 28.7 1,733 functions
lisa 15.7 28.7 1,733 functions
flanders 16.4 29.1 2,029 configure-helper.sh
lisa 16.3 29.1 2,029 configure-helper.sh
flanders 17.9 30.7 2,512 abuild
lisa 17.8 30.9 2,512 abuild
flanders 16.6 29.5 2,698 Build.sh
lisa 16.6 29.5 2,698 Build.sh
flanders 13.5 26.2 3,123 t9300-fast-import.sh
lisa 13.3 26.2 3,123 t9300-fast-import.sh
flanders 26.6 39.4 5,160 test-cmd-util.sh
lisa 26.7 39.4 5,160 test-cmd-util.sh
flanders 28.3 40.9 9,661 ltmain.sh
lisa 28.4 40.9 9,661 ltmain.sh
flanders 48.5 61.3 20,489 configure
lisa 48.4 61.3 20,489 configure
flanders 110.8 123.6 69,779 configure-coreutils
lisa 110.8 123.6 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-836f915d
osh-ovm osh-46375dfa
oil-native osh_eval.opt.stripped-ebeb7305
host label host id
flanders flanders-9d0956ac
lisa lisa-9d0956ac

Raw Data

filename
flanders.2021-01-13__19-07-32.times.csv
lisa.2021-01-13__18-59-10.times.csv