OSH Parser Performance

We run $sh -n $file for various files under various shells. This means that shell startup time is included in the elapsed time measurements, but long files are chosen to minimize its effect.

Average Parsing Rate, Measured on Two Machines (lines/ms)

shell label total lines host flanders host lisa
dash 111,674 1,255.4 2,747.1
mksh 111,674 403.3 981.4
bash 118,863 246.9 645.1
oil-native 118,863 222.0 592.5
zsh 118,863 28.6 66.2
osh-cpython 118,863 3.5 9.8
osh-ovm 118,863 2.8 9.1

Parse Time Breakdown by File

Elasped Time in milliseconds

host label bash dash mksh zsh osh-ovm osh-cpython oil-native osh to bash ratio num lines filename
flanders 15 2 4 161 248 293 8 0.5 1,679 t4014-format-patch.sh
flanders 12 4 12 48 894 951 17 1.4 1,733 functions
flanders 15 NA NA 86 937 998 16 1.0 2,029 configure-helper.sh
flanders 18 5 17 68 1,354 1,306 20 1.1 2,512 abuild
flanders 18 4 14 146 1,150 913 17 0.9 2,698 Build.sh
flanders 17 3 6 201 380 295 12 0.7 3,123 t9300-fast-import.sh
flanders 44 NA NA 229 3,697 2,563 43 1.0 5,160 test-cmd-util.sh
flanders 50 16 26 260 3,713 2,767 45 0.9 9,661 ltmain.sh
flanders 85 18 53 632 6,834 6,301 94 1.1 20,489 configure
flanders 207 36 145 2,327 22,755 17,742 263 1.3 69,779 configure-coreutils
lisa 3 1 1 60 88 90 2 0.8 1,679 t4014-format-patch.sh
lisa 3 1 3 20 295 280 5 1.4 1,733 functions
lisa 5 NA NA 42 301 289 6 1.1 2,029 configure-helper.sh
lisa 5 1 5 34 458 423 8 1.5 2,512 abuild
lisa 4 1 3 63 319 315 7 1.5 2,698 Build.sh
lisa 4 1 2 71 99 102 2 0.6 3,123 t9300-fast-import.sh
lisa 15 NA NA 113 935 882 18 1.2 5,160 test-cmd-util.sh
lisa 19 5 11 138 1,086 1,015 18 0.9 9,661 ltmain.sh
lisa 41 8 24 295 2,420 2,179 40 1.0 20,489 configure
lisa 84 22 64 960 7,002 6,508 95 1.1 69,779 configure-coreutils

Parsing Rate in lines/millisecond

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 113.1 737.4 373.3 10.4 6.8 5.7 221.7 1,679 t4014-format-patch.sh
flanders 142.5 389.3 143.2 36.3 1.9 1.8 103.9 1,733 functions
flanders 131.7 NA NA 23.5 2.2 2.0 130.0 2,029 configure-helper.sh
flanders 139.1 475.3 150.3 37.0 1.9 1.9 124.5 2,512 abuild
flanders 150.5 609.0 190.8 18.5 2.3 3.0 158.8 2,698 Build.sh
flanders 181.3 1,117.0 533.0 15.5 8.2 10.6 255.3 3,123 t9300-fast-import.sh
flanders 116.5 NA NA 22.6 1.4 2.0 120.3 5,160 test-cmd-util.sh
flanders 194.6 615.0 369.4 37.1 2.6 3.5 212.4 9,661 ltmain.sh
flanders 240.4 1,112.9 389.7 32.4 3.0 3.3 217.1 20,489 configure
flanders 337.7 1,960.3 481.7 30.0 3.1 3.9 264.9 69,779 configure-coreutils
lisa 532.5 2,072.8 1,270.0 27.8 19.2 18.7 683.6 1,679 t4014-format-patch.sh
lisa 498.7 1,324.9 615.6 86.6 5.9 6.2 347.1 1,733 functions
lisa 388.4 NA NA 47.9 6.7 7.0 349.7 2,029 configure-helper.sh
lisa 489.1 1,898.7 475.3 74.2 5.5 5.9 320.7 2,512 abuild
lisa 621.4 2,362.5 859.8 42.8 8.4 8.6 414.8 2,698 Build.sh
lisa 717.9 3,687.1 1,880.2 44.2 31.6 30.7 1,250.2 3,123 t9300-fast-import.sh
lisa 346.4 NA NA 45.7 5.5 5.8 291.1 5,160 test-cmd-util.sh
lisa 509.7 1,815.6 846.3 70.2 8.9 9.5 550.7 9,661 ltmain.sh
lisa 500.8 2,691.3 842.6 69.5 8.5 9.4 508.7 20,489 configure
lisa 832.7 3,130.9 1,093.0 72.6 10.0 10.7 734.8 69,779 configure-coreutils

Memory Usage (Max Resident Set Size in MB)

Note that Oil uses a different algorithm than POSIX shells. It builds an AST in memory rather than just validating the code line-by-line.

host label bash dash mksh zsh osh-ovm osh-cpython oil-native num lines filename
flanders 3.0 1.7 1.7 3.7 13.1 16.4 5.2 1,679 t4014-format-patch.sh
flanders 3.1 1.6 1.8 3.6 15.6 18.7 7.2 1,733 functions
flanders 3.1 NA NA 3.7 16.3 19.3 7.7 2,029 configure-helper.sh
flanders 3.1 1.8 1.8 3.7 17.6 20.9 9.2 2,512 abuild
flanders 3.2 1.8 1.7 3.7 16.6 19.7 8.0 2,698 Build.sh
flanders 3.0 1.6 1.6 3.6 13.5 16.7 5.6 3,123 t9300-fast-import.sh
flanders 3.2 NA NA 3.7 26.6 30.5 16.4 5,160 test-cmd-util.sh
flanders 4.2 2.1 2.8 4.1 28.3 32.3 18.5 9,661 ltmain.sh
flanders 3.8 2.2 2.3 4.1 48.3 53.7 35.8 20,489 configure
flanders 3.3 1.9 1.8 3.8 110.7 120.5 96.9 69,779 configure-coreutils
lisa 3.1 1.6 1.7 3.7 13.0 16.4 5.2 1,679 t4014-format-patch.sh
lisa 3.1 1.7 1.8 3.6 15.8 19.0 7.2 1,733 functions
lisa 3.3 NA NA 3.6 16.2 19.6 7.7 2,029 configure-helper.sh
lisa 3.1 1.7 1.8 3.6 17.8 21.2 9.0 2,512 abuild
lisa 3.2 1.7 1.9 3.7 16.6 19.8 8.0 2,698 Build.sh
lisa 3.2 1.5 1.7 3.7 13.4 16.6 5.5 3,123 t9300-fast-import.sh
lisa 3.3 NA NA 3.7 26.4 30.5 16.4 5,160 test-cmd-util.sh
lisa 4.3 2.1 2.8 4.1 28.1 32.3 18.5 9,661 ltmain.sh
lisa 4.0 2.1 2.4 4.0 48.3 53.8 35.8 20,489 configure
lisa 3.5 1.8 1.9 3.6 110.7 120.5 96.9 69,779 configure-coreutils

Old Memory Usage Metric

Running under osh-ovm. Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

host VmRSS MB VmPeak MB num lines filename
flanders 13.0 25.8 1,679 t4014-format-patch.sh
lisa 13.0 25.8 1,679 t4014-format-patch.sh
flanders 15.4 28.4 1,733 functions
lisa 15.8 28.4 1,733 functions
flanders 16.2 29.2 2,029 configure-helper.sh
lisa 16.2 29.1 2,029 configure-helper.sh
flanders 17.5 30.4 2,512 abuild
lisa 17.8 30.7 2,512 abuild
flanders 16.5 29.5 2,698 Build.sh
lisa 16.6 29.5 2,698 Build.sh
flanders 13.5 26.2 3,123 t9300-fast-import.sh
lisa 13.4 26.2 3,123 t9300-fast-import.sh
flanders 26.6 39.1 5,160 test-cmd-util.sh
lisa 26.4 39.1 5,160 test-cmd-util.sh
flanders 28.3 40.9 9,661 ltmain.sh
lisa 28.1 40.9 9,661 ltmain.sh
flanders 48.3 61.3 20,489 configure
lisa 48.3 61.3 20,489 configure
flanders 110.7 123.6 69,779 configure-coreutils
lisa 110.7 123.6 69,779 configure-coreutils

Shell and Host Details

shell label shell id
bash bash-d685a43c
dash dash-2887d9e6
mksh mksh-ae7d165b
zsh zsh-ca7b4198
osh-cpython osh-684d7f70
osh-ovm osh-5e6b21a7
oil-native osh_eval.opt.stripped-45df3fa6
host label host id
flanders flanders-e67e6fdb
lisa lisa-1f35df1a

Raw Data

filename
flanders.2020-11-18__13-59-48.times.csv
lisa.2020-11-18__14-23-35.times.csv