OSH Runtime Performance

Source code: oil/benchmarks/osh-runtime.sh

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself. Parse time is included.

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 38 18 838 62 22.1 1.6
abuild-print-help lenny 15 8 541 29 35.8 1.9
configure.cpython broome 26,783 26,845 63,394 37,229 2.4 1.4
configure.cpython lenny 14,904 14,774 33,784 23,590 2.3 1.6
configure.ocaml broome 6,116 6,035 7,759 6,317 1.3 1.0
configure.ocaml lenny 3,928 3,885 5,063 4,101 1.3 1.0
configure.tcc broome 91 81 335 104 3.7 1.1
configure.tcc lenny 45 36 185 72 4.1 1.6
configure.yash broome 3,078 3,026 4,657 3,262 1.5 1.1
configure.yash lenny 1,663 1,664 2,383 1,909 1.4 1.1
hello-world broome 2 1 68 4 31.0 1.9
hello-world lenny 1 1 55 2 63.9 1.9

Memory Usage (Max Resident Set Size in MB)

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 4.9 3.9 24.7 13.9 5.1 2.9
abuild-print-help lenny 4.9 4.0 24.6 13.9 5.0 2.8
configure.cpython broome 27.2 27.4 50.1 42.6 1.8 1.6
configure.cpython lenny 27.5 27.2 49.9 47.0 1.8 1.7
configure.ocaml broome 21.3 21.3 21.3 21.2 1.0 1.0
configure.ocaml lenny 21.4 21.4 21.2 21.3 1.0 1.0
configure.tcc broome 20.6 20.5 20.5 20.5 1.0 1.0
configure.tcc lenny 20.6 20.4 20.4 20.6 1.0 1.0
configure.yash broome 24.6 24.6 24.8 24.6 1.0 1.0
configure.yash lenny 24.7 24.7 24.6 24.6 1.0 1.0
hello-world broome 3.3 1.7 16.8 5.1 5.1 1.5
hello-world lenny 3.4 1.8 16.7 5.2 5.0 1.6

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num gc points num gc done gc threshold num growths
hello-world broome 4 0 0 0.1 5.1 970 970 0 1 0 100,000 0
abuild-print-help broome 62 0 0 8.5 13.9 178,545 94,507 76,959 170 2 153,918 1
configure.cpython broome 37,229 0 0 425.0 42.6 4,027,359 249,255 196,818 4,397 58 348,056 3
configure.ocaml broome 6,317 0 0 20.7 21.2 240,819 71,720 19,958 383 2 100,000 0
configure.tcc broome 104 0 0 3.6 20.5 46,022 46,022 0 90 0 100,000 0
configure.yash broome 3,262 0 0 28.3 24.6 282,994 24,676 18,519 365 3 100,000 0
hello-world lenny 2 0 0 0.1 5.2 1,142 1,142 0 1 0 100,000 0
abuild-print-help lenny 29 0 0 8.5 13.9 178,717 94,623 77,072 170 2 154,144 1
configure.cpython lenny 23,590 0 0 430.5 47.0 4,037,394 218,336 202,670 4,402 59 361,204 3
configure.ocaml lenny 4,101 0 0 21.2 21.3 241,537 73,838 19,880 383 2 100,000 0
configure.tcc lenny 72 0 0 3.5 20.6 46,198 46,198 0 90 0 100,000 0
configure.yash lenny 1,909 0 0 28.3 24.6 283,023 24,798 18,632 365 3 100,000 0

Details of All Tasks

task id host name workload elapsed ms user ms sys ms max rss MB shell label
0 broome hello-world 2 2 0 3.3 bash
1 broome abuild-print-help 38 35 5 4.9 bash
2 broome configure.cpython 26,783 19,511 8,122 27.2 bash
3 broome configure.ocaml 6,116 3,774 1,422 21.3 bash
4 broome configure.tcc 91 66 26 20.6 bash
5 broome configure.yash 3,078 2,101 991 24.6 bash
6 broome hello-world 1 1 0 1.7 dash
7 broome abuild-print-help 18 17 2 3.9 dash
8 broome configure.cpython 26,845 19,574 8,182 27.4 dash
9 broome configure.ocaml 6,035 3,574 1,500 21.3 dash
10 broome configure.tcc 81 58 24 20.5 dash
11 broome configure.yash 3,026 2,134 895 24.6 dash
12 broome hello-world 68 44 23 16.8 osh-cpython
13 broome abuild-print-help 838 742 102 24.7 osh-cpython
14 broome configure.cpython 63,394 41,988 25,765 50.1 osh-cpython
15 broome configure.ocaml 7,759 4,973 2,179 21.3 osh-cpython
16 broome configure.tcc 335 290 52 20.5 osh-cpython
17 broome configure.yash 4,657 3,462 1,314 24.8 osh-cpython
18 broome hello-world 4 0 4 5.1 osh-native
19 broome abuild-print-help 62 55 9 13.9 osh-native
20 broome configure.cpython 37,229 24,118 14,849 42.6 osh-native
21 broome configure.ocaml 6,317 3,794 1,624 21.2 osh-native
22 broome configure.tcc 104 60 46 20.5 osh-native
23 broome configure.yash 3,262 2,256 1,052 24.6 osh-native
0 lenny hello-world 1 0 1 3.4 bash
1 lenny abuild-print-help 15 10 5 4.9 bash
2 lenny configure.cpython 14,904 11,223 4,111 27.5 bash
3 lenny configure.ocaml 3,928 2,207 773 21.4 bash
4 lenny configure.tcc 45 39 5 20.6 bash
5 lenny configure.yash 1,663 1,277 397 24.7 bash
6 lenny hello-world 1 0 1 1.8 dash
7 lenny abuild-print-help 8 7 1 4.0 dash
8 lenny configure.cpython 14,774 11,332 3,930 27.2 dash
9 lenny configure.ocaml 3,885 2,039 892 21.4 dash
10 lenny configure.tcc 36 33 4 20.4 dash
11 lenny configure.yash 1,664 1,152 522 24.7 dash
12 lenny hello-world 55 40 16 16.7 osh-cpython
13 lenny abuild-print-help 541 511 33 24.6 osh-cpython
14 lenny configure.cpython 33,784 23,684 12,314 49.9 osh-cpython
15 lenny configure.ocaml 5,063 3,025 1,201 21.2 osh-cpython
16 lenny configure.tcc 185 168 20 20.4 osh-cpython
17 lenny configure.yash 2,383 1,940 506 24.6 osh-cpython
18 lenny hello-world 2 2 0 5.2 osh-native
19 lenny abuild-print-help 29 18 12 13.9 osh-native
20 lenny configure.cpython 23,590 16,551 8,032 47.0 osh-native
21 lenny configure.ocaml 4,101 2,286 894 21.3 osh-native
22 lenny configure.tcc 72 56 18 20.6 osh-native
23 lenny configure.yash 1,909 1,393 546 24.6 osh-native

Shell and Host Details

shell label shell id
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
osh-cpython osh-5edf5d16
osh-ovm osh-ecd4f2dc
osh-native osh-f7cd6d82
python2 python2-e809d907
host label host id
broome broome-7a69aabc
lenny lenny-76afa3e0