OSH Runtime Performance

Source code: oil/benchmarks/osh-runtime.sh

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself. Parse time is included.

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 11 7 401 19 35.9 1.7
abuild-print-help lenny 15 9 560 20 36.8 1.3
configure.cpython hoover 11,294 11,254 27,178 14,007 2.4 1.2
configure.cpython lenny 15,339 15,062 31,577 18,529 2.1 1.2
configure.ocaml hoover 3,226 3,118 4,606 3,232 1.4 1.0
configure.ocaml lenny 4,037 3,967 4,299 4,116 1.1 1.0
configure.tcc hoover 36 32 162 43 4.5 1.2
configure.tcc lenny 53 42 184 69 3.5 1.3
configure.yash hoover 1,325 1,307 2,000 1,484 1.5 1.1
configure.yash lenny 1,671 1,567 2,448 1,715 1.5 1.0
hello-world hoover 1 1 34 2 35.0 2.5
hello-world lenny 1 1 57 2 68.8 2.4

Memory Usage (Max Resident Set Size in MB)

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 5.0 4.4 21.9 10.1 4.4 2.0
abuild-print-help lenny 4.9 4.1 22.6 9.9 4.6 2.0
configure.cpython hoover 33.5 33.4 35.4 33.8 1.1 1.0
configure.cpython lenny 27.3 27.3 36.2 27.4 1.3 1.0
configure.ocaml hoover 26.6 26.8 26.8 26.6 1.0 1.0
configure.ocaml lenny 21.3 21.2 21.2 21.3 1.0 1.0
configure.tcc hoover 26.5 26.4 26.5 26.3 1.0 1.0
configure.tcc lenny 20.5 20.5 20.5 20.5 1.0 1.0
configure.yash hoover 33.4 33.4 33.4 33.4 1.0 1.0
configure.yash lenny 24.7 24.6 24.6 24.6 1.0 1.0
hello-world hoover 3.4 1.7 16.4 6.5 4.8 1.9
hello-world lenny 3.4 1.7 17.1 6.3 5.1 1.9

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool num gc points num gc done gc threshold num growths
hello-world hoover 2 0 0 0.2 6.5 4,121 4,121 0 1,549 2,572 2 0 50,000 0
abuild-print-help hoover 19 0 0 4.2 10.1 128,912 72,996 61,969 56,294 72,618 477 3 123,938 2
configure.cpython hoover 14,007 0 0 44.8 33.8 2,320,136 92,650 78,234 703,739 1,616,397 37,336 71 112,958 2
configure.ocaml hoover 3,232 0 0 4.6 26.6 167,300 36,938 9,506 63,567 103,733 2,372 3 50,000 0
configure.tcc hoover 43 0 0 1.4 26.3 33,090 33,090 0 13,320 19,770 329 0 50,000 0
configure.yash hoover 1,484 0 0 4.2 33.4 153,721 30,072 9,764 57,848 95,873 2,608 3 50,000 0
hello-world lenny 2 0 0 0.2 6.3 4,013 4,013 0 1,543 2,470 2 0 50,000 0
abuild-print-help lenny 20 0 0 4.1 9.9 128,826 72,931 61,885 56,301 72,525 477 3 123,770 2
configure.cpython lenny 18,529 0 0 45.3 27.4 2,355,433 81,511 77,460 712,178 1,643,255 38,792 84 113,450 2
configure.ocaml lenny 4,116 0 0 4.0 21.3 138,114 49,894 7,881 58,959 79,155 1,927 2 50,000 0
configure.tcc lenny 69 0 0 1.4 20.5 33,047 33,047 0 13,357 19,690 329 0 50,000 0
configure.yash lenny 1,715 0 0 4.2 24.6 153,706 29,497 9,718 57,950 95,756 2,608 3 50,000 0

Details of All Tasks

task id host name workload elapsed ms user ms sys ms max rss MB shell label
0 hoover hello-world 1 0 1 3.4 bash
1 hoover abuild-print-help 11 4 7 5.0 bash
2 hoover configure.cpython 11,294 8,434 2,898 33.5 bash
3 hoover configure.ocaml 3,226 1,609 623 26.6 bash
4 hoover configure.tcc 36 25 11 26.5 bash
5 hoover configure.yash 1,325 946 372 33.4 bash
6 hoover hello-world 1 0 1 1.7 dash
7 hoover abuild-print-help 7 6 1 4.4 dash
8 hoover configure.cpython 11,254 8,306 3,008 33.4 dash
9 hoover configure.ocaml 3,118 1,535 565 26.8 dash
10 hoover configure.tcc 32 23 9 26.4 dash
11 hoover configure.yash 1,307 974 324 33.4 dash
12 hoover hello-world 34 17 17 16.4 osh-cpython
13 hoover abuild-print-help 401 378 25 21.9 osh-cpython
14 hoover configure.cpython 27,178 18,426 10,448 35.4 osh-cpython
15 hoover configure.ocaml 4,606 2,269 984 26.8 osh-cpython
16 hoover configure.tcc 162 134 23 26.5 osh-cpython
17 hoover configure.yash 2,000 1,454 583 33.4 osh-cpython
18 hoover hello-world 2 2 0 6.5 osh-native
19 hoover abuild-print-help 19 18 1 10.1 osh-native
20 hoover configure.cpython 14,007 10,006 4,291 33.8 osh-native
21 hoover configure.ocaml 3,232 1,535 704 26.6 osh-native
22 hoover configure.tcc 43 32 12 26.3 osh-native
23 hoover configure.yash 1,484 1,098 389 33.4 osh-native
0 lenny hello-world 1 1 0 3.4 bash
1 lenny abuild-print-help 15 10 6 4.9 bash
2 lenny configure.cpython 15,339 11,998 3,808 27.3 bash
3 lenny configure.ocaml 4,037 2,367 732 21.3 bash
4 lenny configure.tcc 53 44 10 20.5 bash
5 lenny configure.yash 1,671 1,250 439 24.7 bash
6 lenny hello-world 1 1 0 1.7 dash
7 lenny abuild-print-help 9 8 1 4.1 dash
8 lenny configure.cpython 15,062 11,672 3,893 27.3 dash
9 lenny configure.ocaml 3,967 2,218 796 21.2 dash
10 lenny configure.tcc 42 32 10 20.5 dash
11 lenny configure.yash 1,567 1,193 385 24.6 dash
12 lenny hello-world 57 53 4 17.1 osh-cpython
13 lenny abuild-print-help 560 534 29 22.6 osh-cpython
14 lenny configure.cpython 31,577 23,225 10,637 36.2 osh-cpython
15 lenny configure.ocaml 4,299 2,550 906 21.2 osh-cpython
16 lenny configure.tcc 184 169 19 20.5 osh-cpython
17 lenny configure.yash 2,448 1,940 578 24.6 osh-cpython
18 lenny hello-world 2 2 0 6.3 osh-native
19 lenny abuild-print-help 20 19 1 9.9 osh-native
20 lenny configure.cpython 18,529 14,119 5,131 27.4 osh-native
21 lenny configure.ocaml 4,116 2,424 750 21.3 osh-native
22 lenny configure.tcc 69 53 17 20.5 osh-native
23 lenny configure.yash 1,715 1,313 427 24.6 osh-native

Shell and Host Details

shell label shell id
bash bash-0d42a1ee
dash dash-d40a0412
mksh mksh-84e6599c
zsh zsh-25b8aadb
osh-cpython osh-c58c917a
osh-ovm osh-3ed2a1b7
osh-native osh-37d2a50c
python2 python2-e2e2dfb7
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
python2 python2-e809d907
host label host id
hoover hoover-94601b4f
lenny lenny-098a8e52