OSH Runtime Performance

Source code: oil/benchmarks/osh-runtime.sh

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself. Parse time is included.

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 37 17 828 53 22.2 1.4
abuild-print-help lenny 16 9 606 27 38.8 1.7
configure.cpython broome 26,931 26,968 60,212 32,147 2.2 1.2
configure.cpython lenny 15,596 15,185 33,662 19,276 2.2 1.2
configure.ocaml broome 6,037 6,004 7,851 6,280 1.3 1.0
configure.ocaml lenny 4,241 3,730 4,803 4,176 1.1 1.0
configure.tcc broome 83 78 358 102 4.3 1.2
configure.tcc lenny 49 44 215 74 4.4 1.5
configure.yash broome 3,032 2,986 4,648 3,151 1.5 1.0
configure.yash lenny 1,590 1,622 2,532 1,593 1.6 1.0
hello-world broome 2 1 68 5 30.4 2.4
hello-world lenny 1 1 46 2 55.9 2.5

Memory Usage (Max Resident Set Size in MB)

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 5.0 3.9 22.4 9.9 4.5 2.0
abuild-print-help lenny 4.9 3.9 22.4 9.7 4.5 2.0
configure.cpython broome 27.2 27.4 36.4 27.4 1.3 1.0
configure.cpython lenny 27.3 27.2 36.4 27.2 1.3 1.0
configure.ocaml broome 21.3 21.2 21.2 21.2 1.0 1.0
configure.ocaml lenny 21.3 21.2 21.2 21.3 1.0 1.0
configure.tcc broome 20.4 20.5 20.4 20.6 1.0 1.0
configure.tcc lenny 20.4 20.4 20.5 20.5 1.0 1.0
configure.yash broome 24.7 24.8 24.6 24.6 1.0 1.0
configure.yash lenny 24.7 24.6 24.6 24.8 1.0 1.0
hello-world broome 3.4 1.7 16.8 6.1 5.0 1.8
hello-world lenny 3.3 1.8 16.9 6.1 5.1 1.8

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool num gc points num gc done gc threshold num growths
hello-world broome 5 0 0 0.2 6.1 4,050 4,050 0 1,546 2,504 2 0 50,000 0
abuild-print-help broome 53 0 0 4.2 9.9 134,300 73,979 61,011 57,428 76,872 477 3 122,022 2
configure.cpython broome 32,147 0 0 54.5 27.4 3,372,440 108,453 82,014 910,692 2,461,748 38,796 101 118,962 2
configure.ocaml broome 6,280 0 0 4.6 21.2 195,447 25,166 10,410 70,704 124,743 1,927 4 50,000 0
configure.tcc broome 102 0 0 1.6 20.6 38,634 38,634 0 14,391 24,243 329 0 50,000 0
configure.yash broome 3,151 0 0 5.1 24.6 241,026 34,592 10,107 78,082 162,944 2,608 5 50,000 0
hello-world lenny 2 0 0 0.2 6.1 4,197 4,197 0 1,559 2,638 2 0 50,000 0
abuild-print-help lenny 27 0 0 4.2 9.7 134,450 74,098 61,124 57,445 77,005 477 3 122,248 2
configure.cpython lenny 19,276 0 0 60.3 27.2 3,393,110 108,631 82,096 920,136 2,472,974 39,097 103 119,136 2
configure.ocaml lenny 4,176 0 0 5.1 21.3 196,176 26,634 10,577 71,307 124,869 1,927 4 50,000 0
configure.tcc lenny 74 0 0 1.6 20.5 38,805 38,805 0 14,432 24,373 329 0 50,000 0
configure.yash lenny 1,593 0 0 5.4 24.8 241,586 35,450 10,209 78,519 163,067 2,608 5 50,000 0

Details of All Tasks

task id host name workload elapsed ms user ms sys ms max rss MB shell label
0 broome hello-world 2 0 2 3.4 bash
1 broome abuild-print-help 37 25 14 5.0 bash
2 broome configure.cpython 26,931 19,527 8,238 27.2 bash
3 broome configure.ocaml 6,037 3,577 1,537 21.3 bash
4 broome configure.tcc 83 60 24 20.4 bash
5 broome configure.yash 3,032 2,104 940 24.7 bash
6 broome hello-world 1 1 0 1.7 dash
7 broome abuild-print-help 17 15 3 3.9 dash
8 broome configure.cpython 26,968 19,618 8,293 27.4 dash
9 broome configure.ocaml 6,004 3,598 1,449 21.2 dash
10 broome configure.tcc 78 60 18 20.5 dash
11 broome configure.yash 2,986 2,149 844 24.8 dash
12 broome hello-world 68 46 22 16.8 osh-cpython
13 broome abuild-print-help 828 777 58 22.4 osh-cpython
14 broome configure.cpython 60,212 41,397 23,169 36.4 osh-cpython
15 broome configure.ocaml 7,851 5,065 2,195 21.2 osh-cpython
16 broome configure.tcc 358 271 95 20.4 osh-cpython
17 broome configure.yash 4,648 3,468 1,306 24.6 osh-cpython
18 broome hello-world 5 3 3 6.1 osh-native
19 broome abuild-print-help 53 39 15 9.9 osh-native
20 broome configure.cpython 32,147 22,757 10,601 27.4 osh-native
21 broome configure.ocaml 6,280 3,658 1,709 21.2 osh-native
22 broome configure.tcc 102 66 38 20.6 osh-native
23 broome configure.yash 3,151 2,231 952 24.6 osh-native
0 lenny hello-world 1 1 0 3.3 bash
1 lenny abuild-print-help 16 15 1 4.9 bash
2 lenny configure.cpython 15,596 11,797 4,262 27.3 bash
3 lenny configure.ocaml 4,241 2,396 907 21.3 bash
4 lenny configure.tcc 49 30 19 20.4 bash
5 lenny configure.yash 1,590 1,201 397 24.7 bash
6 lenny hello-world 1 1 0 1.8 dash
7 lenny abuild-print-help 9 9 0 3.9 dash
8 lenny configure.cpython 15,185 11,673 4,016 27.2 dash
9 lenny configure.ocaml 3,730 1,960 804 21.2 dash
10 lenny configure.tcc 44 22 23 20.4 dash
11 lenny configure.yash 1,622 1,216 416 24.6 dash
12 lenny hello-world 46 38 8 16.9 osh-cpython
13 lenny abuild-print-help 606 583 26 22.4 osh-cpython
14 lenny configure.cpython 33,662 24,472 11,499 36.4 osh-cpython
15 lenny configure.ocaml 4,803 2,926 1,066 21.2 osh-cpython
16 lenny configure.tcc 215 187 32 20.5 osh-cpython
17 lenny configure.yash 2,532 1,919 684 24.6 osh-cpython
18 lenny hello-world 2 0 2 6.1 osh-native
19 lenny abuild-print-help 27 24 3 9.7 osh-native
20 lenny configure.cpython 19,276 14,283 5,674 27.2 osh-native
21 lenny configure.ocaml 4,176 2,399 840 21.3 osh-native
22 lenny configure.tcc 74 50 25 20.5 osh-native
23 lenny configure.yash 1,593 1,187 428 24.8 osh-native

Shell and Host Details

shell label shell id
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
osh-cpython osh-66742cb0
osh-ovm osh-74ef0703
osh-native osh-f03c4b44
python2 python2-e809d907
host label host id
broome broome-76a68468
lenny lenny-098a8e52