OSH Runtime Performance

Source code: oil/benchmarks/osh-runtime.sh

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself. Parse time is included.

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 39 18 851 59 22.0 1.5
abuild-print-help lenny 20 7 613 25 30.8 1.3
configure.cpython broome 26,922 27,012 65,606 33,851 2.4 1.3
configure.cpython lenny 14,648 15,228 37,192 20,097 2.5 1.4
configure.ocaml broome 6,194 6,033 7,913 6,292 1.3 1.0
configure.ocaml lenny 4,000 4,074 4,970 3,930 1.2 1.0
configure.tcc broome 92 79 359 96 3.9 1.1
configure.tcc lenny 58 53 193 52 3.3 0.9
configure.yash broome 3,104 3,014 4,681 3,205 1.5 1.0
configure.yash lenny 1,641 1,496 2,607 1,608 1.6 1.0
hello-world broome 2 1 68 6 31.5 2.9
hello-world lenny 11 1 48 2 4.4 0.2

Memory Usage (Max Resident Set Size in MB)

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help broome 5.0 3.9 24.7 11.1 5.0 2.2
abuild-print-help lenny 4.8 3.9 24.8 10.9 5.2 2.3
configure.cpython broome 27.2 27.1 58.6 34.8 2.2 1.3
configure.cpython lenny 27.5 27.2 58.4 37.9 2.1 1.4
configure.ocaml broome 21.4 21.3 22.5 21.3 1.0 1.0
configure.ocaml lenny 21.3 21.4 22.4 21.3 1.1 1.0
configure.tcc broome 20.5 20.4 20.5 20.5 1.0 1.0
configure.tcc lenny 20.4 20.4 20.4 20.5 1.0 1.0
configure.yash broome 24.6 24.5 24.8 24.8 1.0 1.0
configure.yash lenny 24.6 24.7 24.6 24.7 1.0 1.0
hello-world broome 3.4 1.7 17.2 6.6 5.0 1.9
hello-world lenny 3.2 1.7 17.1 6.5 5.3 2.0

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool num gc points num gc done gc threshold num growths
hello-world broome 6 0 0 0.2 6.6 3,940 3,940 0 1,484 2,456 2 0 50,000 0
abuild-print-help broome 59 0 0 5.0 11.1 143,921 93,820 67,432 59,514 84,407 477 3 134,864 2
configure.cpython broome 33,851 0 0 64.6 34.8 3,500,218 480,835 275,228 923,984 2,576,234 38,796 39 512,346 5
configure.ocaml broome 6,292 0 0 4.9 21.3 206,693 50,320 39,516 72,797 133,896 1,927 7 79,032 1
configure.tcc broome 96 0 0 1.7 20.5 40,775 40,775 0 14,820 25,955 329 0 50,000 0
configure.yash broome 3,205 0 0 5.3 24.8 251,422 45,187 27,405 78,878 172,544 2,608 7 50,000 0
hello-world lenny 2 0 0 0.2 6.5 4,120 4,120 0 1,499 2,621 2 0 50,000 0
abuild-print-help lenny 25 0 0 5.0 10.9 144,104 94,867 66,243 59,781 84,323 477 3 132,486 2
configure.cpython lenny 20,097 0 0 70.3 37.9 3,521,431 326,311 291,947 934,574 2,586,857 39,097 40 503,450 5
configure.ocaml lenny 3,930 0 0 5.5 21.3 207,271 50,252 39,448 73,508 133,763 1,927 7 78,896 1
configure.tcc lenny 52 0 0 1.7 20.5 40,979 40,979 0 14,917 26,062 329 0 50,000 0
configure.yash lenny 1,608 0 0 5.7 24.7 252,015 46,530 27,497 79,427 172,588 2,608 7 50,000 0

Details of All Tasks

task id host name workload elapsed ms user ms sys ms max rss MB shell label
0 broome hello-world 2 2 0 3.4 bash
1 broome abuild-print-help 39 29 11 5.0 bash
2 broome configure.cpython 26,922 19,580 8,243 27.2 bash
3 broome configure.ocaml 6,194 3,747 1,536 21.4 bash
4 broome configure.tcc 92 71 22 20.5 bash
5 broome configure.yash 3,104 2,271 851 24.6 bash
6 broome hello-world 1 1 0 1.7 dash
7 broome abuild-print-help 18 14 4 3.9 dash
8 broome configure.cpython 27,012 19,754 8,185 27.1 dash
9 broome configure.ocaml 6,033 3,610 1,464 21.3 dash
10 broome configure.tcc 79 60 19 20.4 dash
11 broome configure.yash 3,014 2,091 926 24.5 dash
12 broome hello-world 68 53 16 17.2 osh-cpython
13 broome abuild-print-help 851 813 45 24.7 osh-cpython
14 broome configure.cpython 65,606 42,546 27,751 58.6 osh-cpython
15 broome configure.ocaml 7,913 5,080 2,259 22.5 osh-cpython
16 broome configure.tcc 359 290 77 20.5 osh-cpython
17 broome configure.yash 4,681 3,403 1,402 24.8 osh-cpython
18 broome hello-world 6 0 6 6.6 osh-native
19 broome abuild-print-help 59 37 23 11.1 osh-native
20 broome configure.cpython 33,851 23,065 12,449 34.8 osh-native
21 broome configure.ocaml 6,292 3,751 1,630 21.3 osh-native
22 broome configure.tcc 96 78 20 20.5 osh-native
23 broome configure.yash 3,205 2,337 898 24.8 osh-native
0 lenny hello-world 11 1 0 3.2 bash
1 lenny abuild-print-help 20 19 1 4.8 bash
2 lenny configure.cpython 14,648 11,113 3,985 27.5 bash
3 lenny configure.ocaml 4,000 2,250 800 21.3 bash
4 lenny configure.tcc 58 31 27 20.4 bash
5 lenny configure.yash 1,641 1,278 379 24.6 bash
6 lenny hello-world 1 1 0 1.7 dash
7 lenny abuild-print-help 7 3 4 3.9 dash
8 lenny configure.cpython 15,228 11,670 4,067 27.2 dash
9 lenny configure.ocaml 4,074 2,231 886 21.4 dash
10 lenny configure.tcc 53 45 9 20.4 dash
11 lenny configure.yash 1,496 1,081 423 24.7 dash
12 lenny hello-world 48 36 12 17.1 osh-cpython
13 lenny abuild-print-help 613 597 19 24.8 osh-cpython
14 lenny configure.cpython 37,192 25,281 14,508 58.4 osh-cpython
15 lenny configure.ocaml 4,970 3,007 1,144 22.4 osh-cpython
16 lenny configure.tcc 193 144 52 20.4 osh-cpython
17 lenny configure.yash 2,607 1,970 710 24.6 osh-cpython
18 lenny hello-world 2 2 0 6.5 osh-native
19 lenny abuild-print-help 25 19 7 10.9 osh-native
20 lenny configure.cpython 20,097 14,570 6,381 37.9 osh-native
21 lenny configure.ocaml 3,930 2,205 775 21.3 osh-native
22 lenny configure.tcc 52 36 18 20.5 osh-native
23 lenny configure.yash 1,608 1,188 444 24.7 osh-native

Shell and Host Details

shell label shell id
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
osh-cpython osh-424d86f7
osh-ovm osh-613703d2
osh-native osh-3c290bd3
python2 python2-e809d907
host label host id
broome broome-7a69aabc
lenny lenny-5570ba51