OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 13 10 394 18 30.25 1.35
abuild-print-help lenny 23 15 535 29 23.11 1.26
bin-true hoover 639 230 1,842 900 2.88 1.41
bin-true lenny 762 449 1,702 448 2.23 0.59
configure.cpython hoover 11,226 11,221 25,408 12,172 2.26 1.08
configure.cpython lenny 14,574 15,154 30,999 16,001 2.13 1.10
configure.ocaml hoover 3,211 3,141 3,909 3,328 1.22 1.04
configure.ocaml lenny 4,036 4,033 4,426 4,270 1.10 1.06
configure.tcc hoover 35 30 153 40 4.36 1.15
configure.tcc lenny 57 45 210 47 3.70 0.83
configure.util-linux hoover 10,948 11,517 27,568 12,374 2.52 1.13
configure.util-linux lenny 15,218 16,368 34,285 16,853 2.25 1.11
configure.yash hoover 1,340 1,305 1,942 1,370 1.45 1.02
configure.yash lenny 1,572 1,686 2,648 1,674 1.68 1.07
hello-world hoover 1 1 38 2 38.11 2.09
hello-world lenny 1 1 60 2 73.75 2.43

Minor Page Faults

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 2,453 1,243 23,048 3,379 9.40 1.38
abuild-print-help lenny 2,528 1,169 23,074 3,086 9.13 1.22
bin-true hoover 103,422 51,468 771,168 142,134 7.46 1.37
bin-true lenny 97,255 67,227 774,588 124,084 7.96 1.28
configure.cpython hoover 1,883,950 1,885,723 5,939,897 2,068,018 3.15 1.10
configure.cpython lenny 2,183,734 2,198,551 6,288,999 2,283,777 2.88 1.05
configure.ocaml hoover 336,875 318,517 569,484 349,855 1.69 1.04
configure.ocaml lenny 406,129 389,900 641,537 419,788 1.58 1.03
configure.tcc hoover 5,190 4,336 22,205 6,033 4.28 1.16
configure.tcc lenny 6,328 5,578 23,323 6,894 3.69 1.09
configure.util-linux hoover 1,757,499 1,922,164 4,999,372 1,968,086 2.84 1.12
configure.util-linux lenny 2,123,171 2,275,775 5,573,702 2,327,903 2.63 1.10
configure.yash hoover 177,148 164,590 311,509 183,899 1.76 1.04
configure.yash lenny 231,618 221,900 365,796 235,826 1.58 1.02
hello-world hoover 161 89 3,536 397 21.96 2.47
hello-world lenny 133 71 3,401 334 25.57 2.51

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native py bash ratio native bash ratio
abuild-print-help hoover 5.1 4.4 21.9 10.0 4.31 1.98
abuild-print-help lenny 4.9 4.0 22.6 9.8 4.61 2.00
bin-true hoover 3.5 1.7 17.1 9.6 4.95 2.79
bin-true lenny 3.3 1.8 17.8 9.3 5.35 2.78
configure.cpython hoover 33.4 33.5 33.4 33.5 1.00 1.00
configure.cpython lenny 27.5 27.3 27.2 27.3 0.99 0.99
configure.ocaml hoover 26.6 26.6 26.6 26.7 1.00 1.00
configure.ocaml lenny 21.3 21.3 21.3 21.3 1.00 1.00
configure.tcc hoover 26.4 26.3 26.4 26.3 1.00 1.00
configure.tcc lenny 20.6 20.5 20.5 20.5 1.00 1.00
configure.util-linux hoover 37.5 37.6 37.5 37.3 1.00 1.00
configure.util-linux lenny 30.5 30.7 30.6 30.7 1.00 1.00
configure.yash hoover 33.4 33.3 33.3 33.3 1.00 1.00
configure.yash lenny 24.7 24.6 24.7 24.7 1.00 1.00
hello-world hoover 3.4 1.7 17.2 7.1 5.12 2.13
hello-world lenny 3.3 1.8 17.9 6.8 5.49 2.08

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world lenny 2 0 0 0.3 6.8 3,819 3,819 0 699 1,926 1,194 2 0 50,000 0
bin-true lenny 448 0 0 5.0 9.3 172,801 31,413 2,894 18,754 89,400 64,647 5,012 3 50,000 0
configure.cpython lenny 16,001 0 0 19.4 27.3 1,846,930 43,754 29,074 110,864 971,447 764,619 38,285 64 50,000 0
configure.util-linux lenny 16,853 0 0 25.8 30.7 2,833,688 58,447 67,715 139,259 1,417,506 1,276,923 47,756 43 135,430 1
configure.ocaml lenny 4,270 0 0 2.9 21.3 138,232 45,903 4,171 7,973 66,517 63,742 1,927 2 50,000 0
configure.tcc lenny 47 0 0 1.4 20.5 31,139 31,139 0 1,732 15,156 14,251 329 0 50,000 0
configure.yash lenny 1,674 0 0 2.9 24.7 143,911 11,929 6,611 8,580 73,636 61,695 2,608 3 50,000 0
abuild-print-help lenny 29 0 0 2.9 9.8 117,741 67,684 43,083 4,893 55,707 57,141 477 3 86,166 1
hello-world hoover 2 0 0 0.3 7.1 3,929 3,929 0 699 2,009 1,221 2 0 50,000 0
bin-true hoover 900 0 0 5.0 9.6 172,899 31,757 2,979 18,749 89,482 64,668 5,012 3 50,000 0
configure.cpython hoover 12,172 0 0 19.1 33.5 1,818,448 39,895 29,148 108,822 955,429 754,197 36,731 63 50,000 0
configure.util-linux hoover 12,374 0 0 24.9 37.3 2,756,688 124,962 67,816 133,032 1,373,901 1,249,755 43,778 41 135,632 1
configure.ocaml hoover 3,328 0 0 3.0 26.7 153,656 14,996 4,250 9,446 74,568 69,642 2,372 3 50,000 0
configure.tcc hoover 40 0 0 1.4 26.3 31,176 31,176 0 1,705 15,234 14,237 329 0 50,000 0
configure.yash hoover 1,370 0 0 2.9 33.3 143,583 11,783 6,699 8,468 73,600 61,515 2,604 3 50,000 0
abuild-print-help hoover 18 0 0 2.9 10.0 117,822 67,752 43,171 4,884 55,788 57,150 477 3 86,342 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 lenny abuild-print-help 23 19 5 4.9 bash
15 lenny abuild-print-help 15 5 11 4.0 dash
23 lenny abuild-print-help 535 499 38 22.6 osh-cpython
31 lenny abuild-print-help 29 24 6 9.8 osh-native
7 hoover abuild-print-help 13 13 0 5.1 bash
15 hoover abuild-print-help 10 5 3 4.4 dash
23 hoover abuild-print-help 394 359 37 21.9 osh-cpython
31 hoover abuild-print-help 18 17 0 10.0 osh-native
1 lenny bin-true 762 646 174 3.3 bash
9 lenny bin-true 449 344 112 1.8 dash
17 lenny bin-true 1,702 1,292 588 17.8 osh-cpython
25 lenny bin-true 448 341 111 9.3 osh-native
1 hoover bin-true 639 471 194 3.5 bash
9 hoover bin-true 230 204 25 1.7 dash
17 hoover bin-true 1,842 1,228 786 17.1 osh-cpython
25 hoover bin-true 900 593 304 9.6 osh-native
2 lenny configure.cpython 14,574 11,199 3,788 27.5 bash
10 lenny configure.cpython 15,154 11,751 3,915 27.3 dash
18 lenny configure.cpython 30,999 23,752 9,218 27.2 osh-cpython
26 lenny configure.cpython 16,001 12,150 4,204 27.3 osh-native
2 hoover configure.cpython 11,226 8,267 3,025 33.4 bash
10 hoover configure.cpython 11,221 8,235 3,051 33.5 dash
18 hoover configure.cpython 25,408 18,115 8,526 33.4 osh-cpython
26 hoover configure.cpython 12,172 8,921 3,213 33.5 osh-native
4 lenny configure.ocaml 4,036 2,304 783 21.3 bash
12 lenny configure.ocaml 4,033 2,225 828 21.3 dash
20 lenny configure.ocaml 4,426 2,694 908 21.3 osh-cpython
28 lenny configure.ocaml 4,270 2,319 1,025 21.3 osh-native
4 hoover configure.ocaml 3,211 1,614 595 26.6 bash
12 hoover configure.ocaml 3,141 1,500 629 26.6 dash
20 hoover configure.ocaml 3,909 2,176 832 26.6 osh-cpython
28 hoover configure.ocaml 3,328 1,699 627 26.7 osh-native
5 lenny configure.tcc 57 33 24 20.6 bash
13 lenny configure.tcc 45 31 15 20.5 dash
21 lenny configure.tcc 210 184 29 20.5 osh-cpython
29 lenny configure.tcc 47 39 8 20.5 osh-native
5 hoover configure.tcc 35 30 4 26.4 bash
13 hoover configure.tcc 30 28 2 26.3 dash
21 hoover configure.tcc 153 118 36 26.4 osh-cpython
29 hoover configure.tcc 40 38 2 26.3 osh-native
3 lenny configure.util-linux 15,218 11,764 4,178 30.5 bash
11 lenny configure.util-linux 16,368 12,420 4,882 30.7 dash
19 lenny configure.util-linux 34,285 27,967 7,449 30.6 osh-cpython
27 lenny configure.util-linux 16,853 12,345 4,890 30.7 osh-native
3 hoover configure.util-linux 10,948 8,235 2,889 37.5 bash
11 hoover configure.util-linux 11,517 8,452 3,215 37.6 dash
19 hoover configure.util-linux 27,568 20,710 7,409 37.5 osh-cpython
27 hoover configure.util-linux 12,374 8,831 3,517 37.3 osh-native
6 lenny configure.yash 1,572 1,221 366 24.7 bash
14 lenny configure.yash 1,686 1,332 367 24.6 dash
22 lenny configure.yash 2,648 1,995 702 24.7 osh-cpython
30 lenny configure.yash 1,674 1,268 419 24.7 osh-native
6 hoover configure.yash 1,340 911 412 33.4 bash
14 hoover configure.yash 1,305 1,027 263 33.3 dash
22 hoover configure.yash 1,942 1,445 505 33.3 osh-cpython
30 hoover configure.yash 1,370 958 395 33.3 osh-native
0 lenny hello-world 1 1 0 3.3 bash
8 lenny hello-world 1 0 0 1.8 dash
16 lenny hello-world 60 56 4 17.9 osh-cpython
24 lenny hello-world 2 2 0 6.8 osh-native
0 hoover hello-world 1 0 1 3.4 bash
8 hoover hello-world 1 1 0 1.7 dash
16 hoover hello-world 38 19 18 17.2 osh-cpython
24 hoover hello-world 2 2 0 7.1 osh-native

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 lenny abuild-print-help 2,528 0 0 0 16 0 45 1 bash
15 lenny abuild-print-help 1,169 0 0 0 8 0 39 0 dash
23 lenny abuild-print-help 23,074 0 0 0 8 0 42 2 osh-cpython
31 lenny abuild-print-help 3,086 0 0 0 16 0 38 0 osh-native
7 hoover abuild-print-help 2,453 0 0 128 8 0 39 0 bash
15 hoover abuild-print-help 1,243 0 0 0 8 0 38 1 dash
23 hoover abuild-print-help 23,048 0 0 0 8 0 40 9 osh-cpython
31 hoover abuild-print-help 3,379 0 0 0 16 0 33 2 osh-native
1 lenny bin-true 97,255 0 0 0 8 0 1,948 62 bash
9 lenny bin-true 67,227 0 0 0 8 0 1,664 174 dash
17 lenny bin-true 774,588 0 0 0 8 0 1,999 12 osh-cpython
25 lenny bin-true 124,084 0 0 0 16 0 1,881 363 osh-native
1 hoover bin-true 103,422 0 0 0 8 0 1,985 94 bash
9 hoover bin-true 51,468 0 0 0 8 0 2,870 42 dash
17 hoover bin-true 771,168 0 0 0 8 0 1,996 164 osh-cpython
25 hoover bin-true 142,134 0 0 0 16 0 1,987 193 osh-native
2 lenny configure.cpython 2,183,734 1 0 1,376 29,904 0 14,920 226 bash
10 lenny configure.cpython 2,198,551 0 0 0 29,968 0 14,944 229 dash
18 lenny configure.cpython 6,288,999 0 0 0 24,504 0 13,673 166 osh-cpython
26 lenny configure.cpython 2,283,777 0 0 0 24,512 0 13,705 208 osh-native
2 hoover configure.cpython 1,883,950 0 0 0 25,288 0 13,968 1,763 bash
10 hoover configure.cpython 1,885,723 2 0 0 25,288 0 14,010 1,547 dash
18 hoover configure.cpython 5,939,897 0 0 0 25,288 0 12,857 2,629 osh-cpython
26 hoover configure.cpython 2,068,018 1 0 128 25,296 0 12,849 1,496 osh-native
4 lenny configure.ocaml 406,129 0 0 344 3,712 0 1,970 29 bash
12 lenny configure.ocaml 389,900 0 0 0 3,704 0 1,956 19 dash
20 lenny configure.ocaml 641,537 0 0 0 3,704 0 1,925 20 osh-cpython
28 lenny configure.ocaml 419,788 0 0 0 3,712 0 1,947 25 osh-native
4 hoover configure.ocaml 336,875 0 0 472 4,432 0 2,024 321 bash
12 hoover configure.ocaml 318,517 0 0 0 4,432 0 2,091 329 dash
20 hoover configure.ocaml 569,484 0 0 0 4,432 0 1,959 282 osh-cpython
28 hoover configure.ocaml 349,855 1 0 0 4,440 0 2,000 391 osh-native
5 lenny configure.tcc 6,328 0 0 8 96 0 46 2 bash
13 lenny configure.tcc 5,578 0 0 0 80 0 41 2 dash
21 lenny configure.tcc 23,323 0 0 0 80 0 44 5 osh-cpython
29 lenny configure.tcc 6,894 0 0 0 88 0 41 1 osh-native
5 hoover configure.tcc 5,190 0 0 8 88 0 43 3 bash
13 hoover configure.tcc 4,336 0 0 0 88 0 44 3 dash
21 hoover configure.tcc 22,205 0 0 0 88 0 44 7 osh-cpython
29 hoover configure.tcc 6,033 0 0 0 96 0 42 9 osh-native
3 lenny configure.util-linux 2,123,171 9 0 19,088 40,984 0 17,852 257 bash
11 lenny configure.util-linux 2,275,775 0 0 8 41,176 0 16,503 202 dash
19 lenny configure.util-linux 5,573,702 0 0 0 37,768 0 15,741 176 osh-cpython
27 lenny configure.util-linux 2,327,903 0 0 0 37,752 0 16,100 186 osh-native
3 hoover configure.util-linux 1,757,499 56 0 18,480 35,920 0 19,294 1,583 bash
11 hoover configure.util-linux 1,922,164 0 0 0 35,920 0 17,132 1,782 dash
19 hoover configure.util-linux 4,999,372 3 0 0 35,928 0 17,397 2,312 osh-cpython
27 hoover configure.util-linux 1,968,086 0 0 0 35,928 0 17,348 2,082 osh-native
6 lenny configure.yash 231,618 0 0 264 3,528 0 878 14 bash
14 lenny configure.yash 221,900 0 0 0 3,128 0 860 9 dash
22 lenny configure.yash 365,796 0 0 0 3,128 0 848 20 osh-cpython
30 lenny configure.yash 235,826 0 0 0 3,136 0 852 9 osh-native
6 hoover configure.yash 177,148 5 0 592 3,616 0 884 159 bash
14 hoover configure.yash 164,590 0 0 0 3,616 0 997 179 dash
22 hoover configure.yash 311,509 0 0 0 3,616 0 864 273 osh-cpython
30 hoover configure.yash 183,899 0 0 0 3,632 0 859 184 osh-native
0 lenny hello-world 133 0 0 0 8 0 1 0 bash
8 lenny hello-world 71 0 0 0 8 0 1 0 dash
16 lenny hello-world 3,401 0 0 0 8 0 7 3 osh-cpython
24 lenny hello-world 334 0 0 0 16 0 1 0 osh-native
0 hoover hello-world 161 0 0 0 8 0 1 0 bash
8 hoover hello-world 89 0 0 0 8 0 1 0 dash
16 hoover hello-world 3,536 0 0 0 8 0 6 5 osh-cpython
24 hoover hello-world 397 0 0 0 16 0 1 0 osh-native

Shell and Host

shell label shell id
bash bash-9af8f89f
dash dash-9ff48631
mksh mksh-e3755400
zsh zsh-a785003a
osh-cpython osh-a83a5d4f
osh-ovm osh-a18d1f9f
osh-native osh-1fe436f0
python2 python2-e809d907
bash bash-0d42a1ee
dash dash-d40a0412
mksh mksh-84e6599c
zsh zsh-25b8aadb
python2 python2-e2e2dfb7
host label host id
lenny lenny-1ff4d011
hoover hoover-ecba003d