OSH Runtime Performance

Source code: oil/benchmarks/osh-runtime.sh

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself. Parse time is included.

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 38 19 865 60 22.5 1.6
abuild-print-help lenny 20 8 765 33 39.0 1.7
configure.cpython broome 26,787 27,026 64,357 35,181 2.4 1.3
configure.cpython lenny 16,202 16,137 37,627 22,521 2.3 1.4
configure.ocaml broome 6,156 6,041 7,962 6,259 1.3 1.0
configure.ocaml lenny 4,286 3,845 5,380 3,900 1.3 0.9
configure.tcc broome 91 79 361 97 4.0 1.1
configure.tcc lenny 51 44 275 52 5.4 1.0
configure.yash broome 3,090 3,011 4,724 3,215 1.5 1.0
configure.yash lenny 1,585 1,609 3,445 1,888 2.2 1.2
hello-world broome 2 1 70 4 32.8 1.9
hello-world lenny 1 1 54 3 57.3 3.0

Memory Usage (Max Resident Set Size in MB)

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 4.9 3.9 24.7 11.2 5.0 2.3
abuild-print-help lenny 5.0 3.9 24.8 11.1 4.9 2.2
configure.cpython broome 27.4 27.3 57.2 38.3 2.1 1.4
configure.cpython lenny 27.4 27.3 57.1 41.3 2.1 1.5
configure.ocaml broome 21.3 21.2 22.2 21.3 1.0 1.0
configure.ocaml lenny 21.3 21.3 22.1 21.3 1.0 1.0
configure.tcc broome 20.4 20.5 20.4 20.5 1.0 1.0
configure.tcc lenny 20.4 20.7 20.4 20.5 1.0 1.0
configure.yash broome 24.6 24.8 24.6 24.8 1.0 1.0
configure.yash lenny 24.8 24.5 24.8 24.8 1.0 1.0
hello-world broome 3.1 1.7 16.9 6.2 5.4 2.0
hello-world lenny 3.4 1.7 17.1 6.3 5.1 1.9

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num gc points num gc done gc threshold num growths
hello-world broome 4 0 0 0.1 6.2 950 950 0 2 0 50,000 0
abuild-print-help broome 60 0 0 4.8 11.2 141,324 90,488 66,628 477 3 133,256 2
configure.cpython broome 35,181 0 0 105.5 38.3 3,697,536 435,618 261,806 38,796 42 487,576 5
configure.ocaml broome 6,259 0 0 6.5 21.3 211,294 36,226 34,149 1,927 7 50,000 0
configure.tcc broome 97 0 0 1.3 20.5 39,002 39,002 0 329 0 50,000 0
configure.yash broome 3,215 0 0 7.6 24.8 259,444 36,819 24,745 2,608 7 50,000 0
hello-world lenny 3 0 0 0.1 6.3 1,097 1,097 0 2 0 50,000 0
abuild-print-help lenny 33 0 0 4.8 11.1 141,474 90,607 66,741 477 3 133,482 2
configure.cpython lenny 22,521 0 0 111.6 41.3 3,720,640 440,985 260,890 39,097 43 486,972 5
configure.ocaml lenny 3,900 0 0 7.0 21.3 212,023 37,288 34,034 1,927 7 50,000 0
configure.tcc lenny 52 0 0 1.3 20.5 39,178 39,178 0 329 0 50,000 0
configure.yash lenny 1,888 0 0 7.9 24.8 259,838 38,400 24,878 2,606 7 50,000 0

Details of All Tasks

task id host name workload elapsed ms user ms sys ms max rss MB shell label
0 broome hello-world 2 2 0 3.1 bash
1 broome abuild-print-help 38 33 7 4.9 bash
2 broome configure.cpython 26,787 19,337 8,315 27.4 bash
3 broome configure.ocaml 6,156 3,705 1,529 21.3 bash
4 broome configure.tcc 91 54 38 20.4 bash
5 broome configure.yash 3,090 2,290 814 24.6 bash
6 broome hello-world 1 1 0 1.7 dash
7 broome abuild-print-help 19 15 4 3.9 dash
8 broome configure.cpython 27,026 19,681 8,247 27.3 dash
9 broome configure.ocaml 6,041 3,602 1,477 21.2 dash
10 broome configure.tcc 79 49 30 20.5 dash
11 broome configure.yash 3,011 2,159 856 24.8 dash
12 broome hello-world 70 66 4 16.9 osh-cpython
13 broome abuild-print-help 865 808 64 24.7 osh-cpython
14 broome configure.cpython 64,357 42,430 26,444 57.2 osh-cpython
15 broome configure.ocaml 7,962 5,183 2,198 22.2 osh-cpython
16 broome configure.tcc 361 306 63 20.4 osh-cpython
17 broome configure.yash 4,724 3,496 1,347 24.6 osh-cpython
18 broome hello-world 4 0 4 6.2 osh-native
19 broome abuild-print-help 60 41 22 11.2 osh-native
20 broome configure.cpython 35,181 23,566 13,329 38.3 osh-native
21 broome configure.ocaml 6,259 3,822 1,524 21.3 osh-native
22 broome configure.tcc 97 70 29 20.5 osh-native
23 broome configure.yash 3,215 2,315 933 24.8 osh-native
0 lenny hello-world 1 1 0 3.4 bash
1 lenny abuild-print-help 20 11 9 5.0 bash
2 lenny configure.cpython 16,202 11,619 5,065 27.4 bash
3 lenny configure.ocaml 4,286 2,339 985 21.3 bash
4 lenny configure.tcc 51 20 31 20.4 bash
5 lenny configure.yash 1,585 1,134 465 24.8 bash
6 lenny hello-world 1 1 0 1.7 dash
7 lenny abuild-print-help 8 8 1 3.9 dash
8 lenny configure.cpython 16,137 11,944 4,692 27.3 dash
9 lenny configure.ocaml 3,845 1,981 895 21.3 dash
10 lenny configure.tcc 44 27 17 20.7 dash
11 lenny configure.yash 1,609 1,204 414 24.5 dash
12 lenny hello-world 54 42 11 17.1 osh-cpython
13 lenny abuild-print-help 765 727 43 24.8 osh-cpython
14 lenny configure.cpython 37,627 25,404 14,836 57.1 osh-cpython
15 lenny configure.ocaml 5,380 3,338 1,325 22.1 osh-cpython
16 lenny configure.tcc 275 216 63 20.4 osh-cpython
17 lenny configure.yash 3,445 2,551 987 24.8 osh-cpython
18 lenny hello-world 3 0 3 6.3 osh-native
19 lenny abuild-print-help 33 16 17 11.1 osh-native
20 lenny configure.cpython 22,521 15,185 8,295 41.3 osh-native
21 lenny configure.ocaml 3,900 2,091 859 21.3 osh-native
22 lenny configure.tcc 52 44 10 20.5 osh-native
23 lenny configure.yash 1,888 1,356 553 24.8 osh-native

Shell and Host Details

shell label shell id
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
osh-cpython osh-9d3a7f12
osh-ovm osh-39c84017
osh-native osh-7a6ca51a
python2 python2-e809d907
host label host id
broome broome-7a69aabc
lenny lenny-f88955fe