OSH Runtime Performance

Source code: oil/benchmarks/osh-runtime.sh

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself. Parse time is included.

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 11 7 403 21 37.0 2.0
abuild-print-help lenny 14 7 571 21 40.0 1.4
configure.cpython hoover 11,449 11,330 27,064 13,533 2.4 1.2
configure.cpython lenny 15,083 15,413 32,091 17,277 2.1 1.1
configure.ocaml hoover 3,185 3,206 3,942 3,306 1.2 1.0
configure.ocaml lenny 3,940 3,998 4,158 4,161 1.1 1.1
configure.tcc hoover 34 33 164 42 4.8 1.2
configure.tcc lenny 53 63 184 58 3.5 1.1
configure.yash hoover 1,349 1,359 1,966 1,351 1.5 1.0
configure.yash lenny 1,667 1,490 2,580 1,712 1.5 1.0
hello-world hoover 1 1 33 2 36.1 2.4
hello-world lenny 1 0 60 2 83.9 3.0

Memory Usage (Max Resident Set Size in MB)

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 5.1 4.4 22.0 10.1 4.3 2.0
abuild-print-help lenny 4.9 4.0 22.6 9.7 4.6 2.0
configure.cpython hoover 33.5 33.4 35.4 33.4 1.1 1.0
configure.cpython lenny 27.4 27.2 36.2 27.2 1.3 1.0
configure.ocaml hoover 26.7 26.6 26.6 26.6 1.0 1.0
configure.ocaml lenny 21.4 21.3 21.3 21.3 1.0 1.0
configure.tcc hoover 26.3 26.3 26.2 26.2 1.0 1.0
configure.tcc lenny 20.6 20.4 20.6 20.5 1.0 1.0
configure.yash hoover 33.2 33.4 33.3 33.3 1.0 1.0
configure.yash lenny 24.6 24.8 24.7 24.7 1.0 1.0
hello-world hoover 3.5 1.8 16.3 6.8 4.7 1.9
hello-world lenny 3.3 1.6 17.2 6.4 5.3 2.0

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world hoover 2 0 0 0.3 6.8 4,587 4,587 0 662 2,548 1,377 2 0 50,000 0
abuild-print-help hoover 21 0 0 3.4 10.1 129,401 73,686 61,248 6,496 64,551 58,354 477 3 122,496 2
configure.cpython hoover 13,533 0 0 22.8 33.4 2,362,036 106,779 72,712 123,914 1,465,954 772,168 37,279 84 112,926 2
configure.ocaml hoover 3,306 0 0 3.1 26.6 169,407 41,010 7,496 8,884 92,707 67,816 2,372 3 50,000 0
configure.tcc hoover 42 0 0 1.4 26.2 34,442 34,442 0 1,938 17,964 14,540 329 0 50,000 0
configure.yash hoover 1,351 0 0 3.0 33.3 155,082 30,663 9,867 8,895 84,231 61,956 2,608 3 50,000 0
hello-world lenny 2 0 0 0.2 6.4 4,477 4,477 0 662 2,465 1,350 2 0 50,000 0
abuild-print-help lenny 21 0 0 3.4 9.7 129,307 73,012 61,963 6,499 64,469 58,339 477 3 123,926 2
configure.cpython lenny 17,277 0 0 23.2 27.2 2,398,078 107,414 73,482 126,087 1,489,468 782,523 38,792 73 112,828 2
configure.ocaml lenny 4,161 0 0 2.8 21.3 140,216 14,243 10,012 7,663 69,871 62,682 1,927 3 50,000 0
configure.tcc lenny 58 0 0 1.4 20.5 34,343 34,343 0 1,961 17,828 14,554 329 0 50,000 0
configure.yash lenny 1,712 0 0 3.0 24.7 155,062 30,429 9,785 8,969 84,081 62,012 2,608 3 50,000 0

Details of All Tasks

task id host name workload elapsed ms user ms sys ms max rss MB shell label
0 hoover hello-world 1 1 0 3.5 bash
1 hoover abuild-print-help 11 8 4 5.1 bash
2 hoover configure.cpython 11,449 8,418 3,075 33.5 bash
3 hoover configure.ocaml 3,185 1,576 619 26.7 bash
4 hoover configure.tcc 34 30 4 26.3 bash
5 hoover configure.yash 1,349 992 344 33.2 bash
6 hoover hello-world 1 1 0 1.8 dash
7 hoover abuild-print-help 7 4 3 4.4 dash
8 hoover configure.cpython 11,330 8,400 2,984 33.4 dash
9 hoover configure.ocaml 3,206 1,560 628 26.6 dash
10 hoover configure.tcc 33 20 13 26.3 dash
11 hoover configure.yash 1,359 936 358 33.4 dash
12 hoover hello-world 33 25 8 16.3 osh-cpython
13 hoover abuild-print-help 403 390 16 22.0 osh-cpython
14 hoover configure.cpython 27,064 18,402 10,216 35.4 osh-cpython
15 hoover configure.ocaml 3,942 2,184 867 26.6 osh-cpython
16 hoover configure.tcc 164 129 34 26.2 osh-cpython
17 hoover configure.yash 1,966 1,481 520 33.3 osh-cpython
18 hoover hello-world 2 2 0 6.8 osh-native
19 hoover abuild-print-help 21 21 0 10.1 osh-native
20 hoover configure.cpython 13,533 9,918 3,820 33.4 osh-native
21 hoover configure.ocaml 3,306 1,675 623 26.6 osh-native
22 hoover configure.tcc 42 37 5 26.2 osh-native
23 hoover configure.yash 1,351 981 367 33.3 osh-native
0 lenny hello-world 1 1 0 3.3 bash
1 lenny abuild-print-help 14 14 1 4.9 bash
2 lenny configure.cpython 15,083 11,603 3,957 27.4 bash
3 lenny configure.ocaml 3,940 2,202 797 21.4 bash
4 lenny configure.tcc 53 45 9 20.6 bash
5 lenny configure.yash 1,667 1,238 444 24.6 bash
6 lenny hello-world 0 0 0 1.6 dash
7 lenny abuild-print-help 7 7 0 4.0 dash
8 lenny configure.cpython 15,413 11,966 3,974 27.2 dash
9 lenny configure.ocaml 3,998 2,240 793 21.3 dash
10 lenny configure.tcc 63 43 20 20.4 dash
11 lenny configure.yash 1,490 1,133 368 24.8 dash
12 lenny hello-world 60 22 37 17.2 osh-cpython
13 lenny abuild-print-help 571 542 32 22.6 osh-cpython
14 lenny configure.cpython 32,091 23,331 11,049 36.2 osh-cpython
15 lenny configure.ocaml 4,158 2,450 864 21.3 osh-cpython
16 lenny configure.tcc 184 152 36 20.6 osh-cpython
17 lenny configure.yash 2,580 1,997 656 24.7 osh-cpython
18 lenny hello-world 2 0 2 6.4 osh-native
19 lenny abuild-print-help 21 21 0 9.7 osh-native
20 lenny configure.cpython 17,277 13,205 4,739 27.2 osh-native
21 lenny configure.ocaml 4,161 2,340 888 21.3 osh-native
22 lenny configure.tcc 58 40 20 20.5 osh-native
23 lenny configure.yash 1,712 1,297 441 24.7 osh-native

Shell and Host Details

shell label shell id
bash bash-0d42a1ee
dash dash-d40a0412
mksh mksh-84e6599c
zsh zsh-25b8aadb
osh-cpython osh-e9438750
osh-ovm osh-0a14ddc2
osh-native osh-4dab5cf8
python2 python2-e2e2dfb7
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
python2 python2-e809d907
host label host id
hoover hoover-94601b4f
lenny lenny-098a8e52