OSH Runtime Performance

Source code: oil/benchmarks/osh-runtime.sh

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself. Parse time is included.

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 13 7 396 18 31.6 1.5
abuild-print-help lenny 15 9 597 18 39.9 1.2
configure.cpython hoover 11,579 11,814 27,356 13,283 2.4 1.1
configure.cpython lenny 14,995 14,835 31,735 16,738 2.1 1.1
configure.ocaml hoover 3,345 3,242 4,016 3,295 1.2 1.0
configure.ocaml lenny 3,987 3,920 4,543 3,825 1.1 1.0
configure.tcc hoover 40 33 161 39 4.0 1.0
configure.tcc lenny 54 54 193 60 3.6 1.1
configure.yash hoover 1,367 1,274 1,996 1,351 1.5 1.0
configure.yash lenny 1,788 1,762 2,533 1,732 1.4 1.0
hello-world hoover 1 1 42 2 39.1 2.0
hello-world lenny 1 1 45 2 54.9 2.4

Memory Usage (Max Resident Set Size in MB)

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 5.1 4.3 22.2 10.4 4.3 2.0
abuild-print-help lenny 5.0 4.0 22.9 10.0 4.6 2.0
configure.cpython hoover 33.6 33.4 35.5 33.4 1.1 1.0
configure.cpython lenny 27.3 27.3 36.4 27.4 1.3 1.0
configure.ocaml hoover 26.7 26.6 26.7 26.6 1.0 1.0
configure.ocaml lenny 21.3 21.3 21.2 21.4 1.0 1.0
configure.tcc hoover 26.3 26.2 26.4 26.2 1.0 1.0
configure.tcc lenny 20.7 20.5 20.7 20.4 1.0 1.0
configure.yash hoover 33.3 33.2 33.4 33.3 1.0 1.0
configure.yash lenny 24.6 24.6 24.6 24.6 1.0 1.0
hello-world hoover 3.5 1.7 16.7 7.0 4.8 2.0
hello-world lenny 3.5 1.7 17.3 6.8 5.0 1.9

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world lenny 2 0 0 0.2 6.8 3,750 3,750 0 672 1,874 1,204 2 0 50,000 0
abuild-print-help lenny 18 0 0 3.4 10.0 126,732 70,415 63,370 4,771 63,730 58,231 477 3 126,740 2
configure.cpython lenny 16,738 0 0 22.1 27.4 2,357,918 106,378 74,151 105,466 1,462,141 790,311 38,285 84 113,370 2
configure.ocaml lenny 3,825 0 0 2.8 21.4 138,446 49,680 7,204 6,755 68,553 63,138 1,927 2 50,000 0
configure.tcc lenny 60 0 0 1.4 20.4 33,311 33,311 0 1,637 17,195 14,479 329 0 50,000 0
configure.yash lenny 1,732 0 0 2.9 24.6 152,900 28,054 9,309 7,444 82,225 63,231 2,608 3 50,000 0
hello-world hoover 2 0 0 0.2 7.0 3,860 3,860 0 672 1,957 1,231 2 0 50,000 0
abuild-print-help hoover 18 0 0 3.4 10.4 126,813 70,291 63,721 4,762 63,811 58,240 477 3 127,442 2
configure.cpython hoover 13,283 0 0 21.9 33.4 2,319,661 111,975 73,006 103,811 1,436,387 779,463 36,731 82 113,596 2
configure.ocaml hoover 3,295 0 0 3.1 26.6 167,402 36,751 8,833 7,759 91,386 68,257 2,372 3 50,000 0
configure.tcc hoover 39 0 0 1.4 26.2 33,406 33,406 0 1,610 17,331 14,465 329 0 50,000 0
configure.yash hoover 1,351 0 0 2.9 33.3 152,584 27,854 9,389 7,334 82,203 63,047 2,604 3 50,000 0

Details of All Tasks

task id host name workload elapsed ms user ms sys ms max rss MB shell label
0 lenny hello-world 1 1 0 3.5 bash
1 lenny abuild-print-help 15 15 0 5.0 bash
2 lenny configure.cpython 14,995 11,504 3,971 27.3 bash
3 lenny configure.ocaml 3,987 2,285 765 21.3 bash
4 lenny configure.tcc 54 38 17 20.7 bash
5 lenny configure.yash 1,788 1,329 476 24.6 bash
6 lenny hello-world 1 1 0 1.7 dash
7 lenny abuild-print-help 9 5 4 4.0 dash
8 lenny configure.cpython 14,835 11,488 3,852 27.3 dash
9 lenny configure.ocaml 3,920 2,255 693 21.3 dash
10 lenny configure.tcc 54 38 16 20.5 dash
11 lenny configure.yash 1,762 1,311 463 24.6 dash
12 lenny hello-world 45 41 4 17.3 osh-cpython
13 lenny abuild-print-help 597 575 25 22.9 osh-cpython
14 lenny configure.cpython 31,735 23,077 10,916 36.4 osh-cpython
15 lenny configure.ocaml 4,543 2,784 928 21.2 osh-cpython
16 lenny configure.tcc 193 176 21 20.7 osh-cpython
17 lenny configure.yash 2,533 1,948 656 24.6 osh-cpython
18 lenny hello-world 2 0 2 6.8 osh-native
19 lenny abuild-print-help 18 16 4 10.0 osh-native
20 lenny configure.cpython 16,738 12,864 4,449 27.4 osh-native
21 lenny configure.ocaml 3,825 2,172 712 21.4 osh-native
22 lenny configure.tcc 60 42 19 20.4 osh-native
23 lenny configure.yash 1,732 1,265 493 24.6 osh-native
0 hoover hello-world 1 1 0 3.5 bash
1 hoover abuild-print-help 13 9 4 5.1 bash
2 hoover configure.cpython 11,579 8,511 3,113 33.6 bash
3 hoover configure.ocaml 3,345 1,672 672 26.7 bash
4 hoover configure.tcc 40 28 11 26.3 bash
5 hoover configure.yash 1,367 930 414 33.3 bash
6 hoover hello-world 1 0 0 1.7 dash
7 hoover abuild-print-help 7 5 1 4.3 dash
8 hoover configure.cpython 11,814 8,740 3,161 33.4 dash
9 hoover configure.ocaml 3,242 1,627 607 26.6 dash
10 hoover configure.tcc 33 20 12 26.2 dash
11 hoover configure.yash 1,274 923 335 33.2 dash
12 hoover hello-world 42 38 4 16.7 osh-cpython
13 hoover abuild-print-help 396 383 16 22.2 osh-cpython
14 hoover configure.cpython 27,356 18,720 10,314 35.5 osh-cpython
15 hoover configure.ocaml 4,016 2,260 867 26.7 osh-cpython
16 hoover configure.tcc 161 137 25 26.4 osh-cpython
17 hoover configure.yash 1,996 1,471 562 33.4 osh-cpython
18 hoover hello-world 2 0 2 7.0 osh-native
19 hoover abuild-print-help 18 19 0 10.4 osh-native
20 hoover configure.cpython 13,283 9,794 3,714 33.4 osh-native
21 hoover configure.ocaml 3,295 1,683 633 26.6 osh-native
22 hoover configure.tcc 39 32 7 26.2 osh-native
23 hoover configure.yash 1,351 973 372 33.3 osh-native

Shell and Host Details

shell label shell id
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
osh-cpython osh-fe6227a2
osh-ovm osh-30efce40
osh-native osh-dfba53e5
python2 python2-e809d907
bash bash-0d42a1ee
dash dash-d40a0412
mksh mksh-84e6599c
zsh zsh-25b8aadb
python2 python2-e2e2dfb7
host label host id
lenny lenny-098a8e52
hoover hoover-94601b4f