OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,188.0 2,538.0
mksh 111,674 431.8 988.3
oil-native 118,863 262.4 718.6
bash 118,863 250.6 658.5
zsh 118,863 30.4 68.1
osh-cpython 118,863 3.0 10.8
osh-ovm 118,863 2.7 9.6

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 11 4 6 158 275 218 8 0.8 1,679 t4014-format-patch.sh
flanders 16 5 12 44 908 723 18 1.1 1,733 functions
flanders 20 NA NA 78 888 919 15 0.8 2,029 configure-helper.sh
flanders 21 6 13 66 1,286 1,303 20 0.9 2,512 abuild
flanders 18 5 10 152 924 958 16 0.9 2,698 Build.sh
flanders 14 5 9 199 281 309 10 0.7 3,123 t9300-fast-import.sh
flanders 41 NA NA 215 3,176 2,490 35 0.9 5,160 test-cmd-util.sh
flanders 44 15 31 262 3,323 3,011 45 1.0 9,661 ltmain.sh
flanders 82 17 53 628 8,486 8,229 84 1.0 20,489 configure
flanders 208 37 125 2,103 23,992 21,077 202 1.0 69,779 configure-coreutils
lisa 4 2 2 61 88 90 8 2.4 1,679 t4014-format-patch.sh
lisa 4 2 3 23 286 264 4 0.9 1,733 functions
lisa 6 NA NA 38 290 264 5 0.8 2,029 configure-helper.sh
lisa 6 2 4 32 422 374 6 1.1 2,512 abuild
lisa 6 2 4 58 302 278 5 0.8 2,698 Build.sh
lisa 5 2 3 74 95 94 3 0.6 3,123 t9300-fast-import.sh
lisa 15 NA NA 112 901 808 15 1.0 5,160 test-cmd-util.sh
lisa 18 4 11 126 1,033 937 14 0.8 9,661 ltmain.sh
lisa 34 10 22 291 2,250 2,070 28 0.8 20,489 configure
lisa 83 21 63 930 6,778 5,849 77 0.9 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 158.4 419.8 305.3 10.7 6.1 7.7 197.5 1,679 t4014-format-patch.sh
flanders 111.8 361.0 138.6 39.1 1.9 2.4 98.5 1,733 functions
flanders 103.5 NA NA 26.0 2.3 2.2 131.8 2,029 configure-helper.sh
flanders 118.5 411.8 199.4 38.2 2.0 1.9 127.5 2,512 abuild
flanders 148.2 518.8 275.3 17.8 2.9 2.8 170.8 2,698 Build.sh
flanders 221.5 664.5 363.1 15.7 11.1 10.1 318.7 3,123 t9300-fast-import.sh
flanders 124.6 NA NA 24.0 1.6 2.1 146.2 5,160 test-cmd-util.sh
flanders 218.6 639.8 312.7 36.8 2.9 3.2 214.7 9,661 ltmain.sh
flanders 250.5 1,177.5 383.7 32.6 2.4 2.5 243.3 20,489 configure
flanders 335.8 1,901.3 556.9 33.2 2.9 3.3 346.1 69,779 configure-coreutils
lisa 466.4 1,119.3 839.5 27.4 19.0 18.6 197.5 1,679 t4014-format-patch.sh
lisa 385.1 1,019.4 525.2 75.0 6.1 6.6 422.7 1,733 functions
lisa 338.2 NA NA 53.7 7.0 7.7 431.7 2,029 configure-helper.sh
lisa 425.8 1,256.0 570.9 77.3 6.0 6.7 398.7 2,512 abuild
lisa 465.2 1,498.9 658.0 46.4 8.9 9.7 550.6 2,698 Build.sh
lisa 650.6 2,082.0 1,156.7 42.4 32.7 33.3 1,156.7 3,123 t9300-fast-import.sh
lisa 339.5 NA NA 45.9 5.7 6.4 346.3 5,160 test-cmd-util.sh
lisa 539.7 2,195.7 855.0 76.8 9.4 10.3 666.3 9,661 ltmain.sh
lisa 609.8 1,989.2 918.8 70.4 9.1 9.9 726.6 20,489 configure
lisa 838.7 3,354.8 1,109.4 75.0 10.3 11.9 911.0 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 6.5 6.4 6.5 6.5 12.8 15.7 6.2 1,679 t4014-format-patch.sh
flanders 6.6 6.5 6.5 6.6 15.5 18.7 7.1 1,733 functions
flanders 6.5 NA NA 6.6 15.8 19.2 7.5 2,029 configure-helper.sh
flanders 6.6 6.5 6.4 6.5 17.4 20.8 8.9 2,512 abuild
flanders 6.6 6.4 6.6 6.5 16.3 19.6 7.8 2,698 Build.sh
flanders 6.6 6.4 6.6 6.5 13.2 16.2 6.1 3,123 t9300-fast-import.sh
flanders 6.5 NA NA 6.5 26.2 30.0 16.1 5,160 test-cmd-util.sh
flanders 6.6 6.4 6.4 6.6 27.8 31.9 18.1 9,661 ltmain.sh
flanders 6.4 6.5 6.4 6.6 47.9 53.6 35.6 20,489 configure
flanders 6.6 6.6 6.6 6.5 109.8 119.7 96.4 69,779 configure-coreutils
lisa 6.6 6.5 6.7 6.7 12.8 15.8 6.3 1,679 t4014-format-patch.sh
lisa 6.7 6.5 6.6 6.7 15.3 18.5 6.9 1,733 functions
lisa 6.7 NA NA 6.7 15.8 19.1 7.4 2,029 configure-helper.sh
lisa 6.7 6.5 6.5 6.5 17.4 20.7 8.9 2,512 abuild
lisa 6.7 6.6 6.5 6.5 16.1 19.5 7.9 2,698 Build.sh
lisa 6.6 6.7 6.7 6.6 13.1 16.2 6.2 3,123 t9300-fast-import.sh
lisa 6.7 NA NA 6.6 26.0 30.1 16.2 5,160 test-cmd-util.sh
lisa 6.5 6.7 6.5 6.7 27.7 31.6 18.1 9,661 ltmain.sh
lisa 6.6 6.6 6.7 6.5 47.8 53.4 35.5 20,489 configure
lisa 6.5 6.6 6.5 6.7 109.9 119.5 96.4 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 12.6 25.5 1,679 t4014-format-patch.sh
lisa 12.6 25.5 1,679 t4014-format-patch.sh
flanders 15.5 28.2 1,733 functions
lisa 15.3 28.2 1,733 functions
flanders 15.7 28.6 2,029 configure-helper.sh
lisa 15.8 28.6 2,029 configure-helper.sh
flanders 17.4 30.2 2,512 abuild
lisa 17.4 30.4 2,512 abuild
flanders 16.3 29.0 2,698 Build.sh
lisa 16.0 29.0 2,698 Build.sh
flanders 13.2 25.9 3,123 t9300-fast-import.sh
lisa 13.1 25.9 3,123 t9300-fast-import.sh
flanders 26.2 38.9 5,160 test-cmd-util.sh
lisa 26.0 39.0 5,160 test-cmd-util.sh
flanders 27.8 40.4 9,661 ltmain.sh
lisa 27.7 40.5 9,661 ltmain.sh
flanders 47.9 60.8 20,489 configure
lisa 47.8 60.8 20,489 configure
flanders 109.8 122.7 69,779 configure-coreutils
lisa 109.9 122.7 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-c334f523
osh-ovm osh-c64742d0
oil-native osh_eval.opt.stripped-81acfae2
host label host id
flanders flanders-889e300d
lisa lisa-6c96a848

Raw Data

filename
flanders.2020-09-28__00-05-52.times.csv
lisa.2020-09-27__23-51-48.times.csv