WILD/cloud/kubernetes/cluster - Parsing and Translating Shell Scripts with Oil

Attempted to parse 118 shell scripts totalling 21,116 lines.
Failed to parse 2 scripts, leaving 20,673 lines parsed in 48.79 seconds (423.7 lines/sec).
38 OSH-to-Oil translations failed.

Files	Max Lines	Total Lines	Parse Failures	Max Parse Time (secs)	Total Parse Time (secs)	Parsed Lines/sec	Translation Failures	Directory
3	190	242	0	0.32	0.51	474.5	0	addons/
1	26	26	0	0.09	0.09	288.9	0	aws/
18	387	1,631	0	0.58	3.36	485.4	10	centos/
23	2,187	9,609	0	4.96	22.73	422.7	7	gce/
2	266	303	0	0.65	0.77	393.5	0	images/
4	150	231	1	0.15	0.44	184.1	1	juju/
3	136	195	0	0.25	0.46	423.9	0	kubemark/
1	171	171	0	0.33	0.33	518.2	1	lib/
3	459	550	0	0.78	1.06	518.9	1	libvirt-coreos/
1	32	32	0	0.13	0.13	246.2	0	local/
1	373	373	0	0.75	0.75	497.3	1	log-dump/
11	293	675	1	0.16	1.14	335.1	1	openstack-heat/
10	1,110	1,820	0	2.11	3.67	495.9	5	photon-controller/
1	62	62	0	0.26	0.26	238.5	0	pre-existing/
9	112	575	0	0.27	1.37	419.7	2	saltbase/
1	99	99	0	0.16	0.16	618.8	0	skeleton/
9	391	1,222	0	0.77	2.87	425.8	5	vagrant/

Files

Max Lines

Total Lines

Parse Failures

Max Parse Time (secs)

Total Parse Time (secs)

Parsed Lines/sec

Translation Failures

Directory

190

242

0.32

0.51

474.5

addons/

0.09

288.9

aws/

387

1,631

0.58

3.36

485.4

centos/

2,187

9,609

4.96

22.73

422.7

gce/

266

303

0.65

0.77

393.5

images/

150

231

0.15

0.44

184.1

juju/

136

195

0.25

0.46

423.9

kubemark/

171

0.33

518.2

lib/

459

550

0.78

1.06

518.9

libvirt-coreos/

0.13

246.2

local/

373

0.75

497.3

log-dump/

293

675

0.16

1.14

335.1

openstack-heat/

1,110

1,820

2.11

3.67

495.9

photon-controller/

0.26

238.5

pre-existing/

112

575

0.27

1.37

419.7

saltbase/

0.16

618.8

skeleton/

391

1,222

0.77

2.87

425.8

vagrant/

Side By Side	Lines	Parsed?	Parse Process Time (secs)	Internal Parse Time (secs)	Parsed Lines/sec	Translated?	Filename
view	105	OK	0.31	0.31	338.7	OK	clientbin.sh
view	1,397	OK	4.53	4.53	308.4	FAIL	common.sh
view	226	OK	0.47	0.47	480.9	OK	get-kube-binaries.sh
view	199	OK	0.41	0.41	485.4	FAIL	get-kube-local.sh
view	247	OK	0.37	0.37	667.6	OK	get-kube.sh
view	37	OK	0.09	0.09	411.1	OK	kube-down.sh
view	96	OK	0.19	0.19	505.3	OK	kube-push.sh
view	108	OK	0.19	0.19	568.4	FAIL	kube-up.sh
view	96	OK	0.18	0.18	533.3	OK	kube-util.sh
view	43	OK	0.13	0.13	330.8	OK	kubeadm.sh
view	77	OK	0.15	0.15	513.3	OK	kubectl.sh
view	256	OK	0.6	0.6	426.7	OK	restore-from-backup.sh
view	33	OK	0.12	0.12	275.0	OK	test-e2e.sh
view	30	OK	0.1	0.1	300.0	OK	test-network.sh
view	32	OK	0.11	0.11	290.9	OK	test-smoke.sh
view	138	OK	0.24	0.24	575.0	OK	update-storage-objects.sh
view	180	OK	0.5	0.5	360.0	FAIL	validate-cluster.sh

Side By Side

Lines

Parsed?

Parse Process Time (secs)

Internal Parse Time (secs)

Parsed Lines/sec

Translated?

Filename

view

105

0.31

338.7

clientbin.sh

view

1,397

4.53

308.4

FAIL

common.sh

view

226

0.47

480.9

get-kube-binaries.sh

view

199

0.41

485.4

FAIL

get-kube-local.sh

view

247

0.37

667.6

get-kube.sh

view

0.09

411.1

kube-down.sh

view

0.19

505.3

kube-push.sh

view

108

0.19

568.4

FAIL

kube-up.sh

view

0.18

533.3

kube-util.sh

view

0.13

330.8

kubeadm.sh

view

0.15

513.3

kubectl.sh

view

256

0.6

426.7

restore-from-backup.sh

view

0.12

275.0

test-e2e.sh

view

0.1

300.0

test-network.sh

view

0.11

290.9

test-smoke.sh

view

138

0.24

575.0

update-storage-objects.sh

view

180

0.5

360.0

FAIL

validate-cluster.sh

stderr

Translating get-kube-local.sh

        Traceback (most recent call last):
  File "bin/osh", line 469, in <module>
    main(sys.argv)
  File "bin/osh", line 452, in main
    sys.exit(OilMain(argv))
  File "bin/osh", line 435, in OilMain
    status = OshMain(main_argv, login_shell)
  File "bin/osh", line 350, in OshMain
    osh2oil.PrintAsOil(arena, node, opts.debug_spans)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil
    fixer.DoCommand(node, None, at_top_level=True)  # no local symbols yet
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 632, in DoCommand
    self.DoCommand(child, new_local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 561, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 536, in DoCommand
    self.DoRedirect(r, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 270, in DoRedirect
    raise NotImplementedError(node.__class__.__name__)
NotImplementedError: HereDoc

Translating validate-cluster.sh

        Traceback (most recent call last):
  File "bin/osh", line 469, in <module>
    main(sys.argv)
  File "bin/osh", line 452, in main
    sys.exit(OilMain(argv))
  File "bin/osh", line 435, in OilMain
    status = OshMain(main_argv, login_shell)
  File "bin/osh", line 350, in OshMain
    osh2oil.PrintAsOil(arena, node, opts.debug_spans)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil
    fixer.DoCommand(node, None, at_top_level=True)  # no local symbols yet
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 726, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 549, in DoCommand
    self.DoAssignment(node, at_top_level, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 463, in DoAssignment
    self.DoWordAsExpr(pair.rhs, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 838, in DoWordAsExpr
    self.DoWordInCommand(node, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 989, in DoWordInCommand
    self.DoWordPart(part, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 1167, in DoWordPart
    self.DoArithExpr(node.anode, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 1186, in DoArithExpr
    r_id = word.LeftMostSpanForWord(node.right.w)
AttributeError: 'ArithVarRef' object has no attribute 'w'

Translating common.sh

        Traceback (most recent call last):
  File "bin/osh", line 469, in <module>
    main(sys.argv)
  File "bin/osh", line 452, in main
    sys.exit(OilMain(argv))
  File "bin/osh", line 435, in OilMain
    status = OshMain(main_argv, login_shell)
  File "bin/osh", line 350, in OshMain
    osh2oil.PrintAsOil(arena, node, opts.debug_spans)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil
    fixer.DoCommand(node, None, at_top_level=True)  # no local symbols yet
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 632, in DoCommand
    self.DoCommand(child, new_local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 549, in DoCommand
    self.DoAssignment(node, at_top_level, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 463, in DoAssignment
    self.DoWordAsExpr(pair.rhs, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 820, in DoWordAsExpr
    self.DoWordInCommand(node, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 989, in DoWordInCommand
    self.DoWordPart(part, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 999, in DoWordPart
    span_id = word.LeftMostSpanForPart(node)
  File "/home/andy/git/oil/bin/../core/word.py", line 105, in LeftMostSpanForPart
    return LeftMostSpanForWord(part.words[0])  # Hm this is a=(1 2 3)
IndexError: list index out of range

Translating kube-up.sh

        Traceback (most recent call last):
  File "bin/osh", line 469, in <module>
    main(sys.argv)
  File "bin/osh", line 452, in main
    sys.exit(OilMain(argv))
  File "bin/osh", line 435, in OilMain
    status = OshMain(main_argv, login_shell)
  File "bin/osh", line 350, in OshMain
    osh2oil.PrintAsOil(arena, node, opts.debug_spans)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil
    fixer.DoCommand(node, None, at_top_level=True)  # no local symbols yet
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 682, in DoCommand
    self.DoCommand(node.body, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 652, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 726, in DoCommand
    self.DoCommand(child, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 536, in DoCommand
    self.DoRedirect(r, local_symbols)
  File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 270, in DoRedirect
    raise NotImplementedError(node.__class__.__name__)
NotImplementedError: HereDoc

Translating get-kube-local.sh	Traceback (most recent call last): File "bin/osh", line 469, in <module> main(sys.argv) File "bin/osh", line 452, in main sys.exit(OilMain(argv)) File "bin/osh", line 435, in OilMain status = OshMain(main_argv, login_shell) File "bin/osh", line 350, in OshMain osh2oil.PrintAsOil(arena, node, opts.debug_spans) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil fixer.DoCommand(node, None, at_top_level=True) # no local symbols yet File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 632, in DoCommand self.DoCommand(child, new_local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 561, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 536, in DoCommand self.DoRedirect(r, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 270, in DoRedirect raise NotImplementedError(node.__class__.__name__) NotImplementedError: HereDoc
Translating validate-cluster.sh	Traceback (most recent call last): File "bin/osh", line 469, in <module> main(sys.argv) File "bin/osh", line 452, in main sys.exit(OilMain(argv)) File "bin/osh", line 435, in OilMain status = OshMain(main_argv, login_shell) File "bin/osh", line 350, in OshMain osh2oil.PrintAsOil(arena, node, opts.debug_spans) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil fixer.DoCommand(node, None, at_top_level=True) # no local symbols yet File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 726, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 549, in DoCommand self.DoAssignment(node, at_top_level, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 463, in DoAssignment self.DoWordAsExpr(pair.rhs, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 838, in DoWordAsExpr self.DoWordInCommand(node, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 989, in DoWordInCommand self.DoWordPart(part, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 1167, in DoWordPart self.DoArithExpr(node.anode, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 1186, in DoArithExpr r_id = word.LeftMostSpanForWord(node.right.w) AttributeError: 'ArithVarRef' object has no attribute 'w'
Translating common.sh	Traceback (most recent call last): File "bin/osh", line 469, in <module> main(sys.argv) File "bin/osh", line 452, in main sys.exit(OilMain(argv)) File "bin/osh", line 435, in OilMain status = OshMain(main_argv, login_shell) File "bin/osh", line 350, in OshMain osh2oil.PrintAsOil(arena, node, opts.debug_spans) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil fixer.DoCommand(node, None, at_top_level=True) # no local symbols yet File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 632, in DoCommand self.DoCommand(child, new_local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 549, in DoCommand self.DoAssignment(node, at_top_level, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 463, in DoAssignment self.DoWordAsExpr(pair.rhs, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 820, in DoWordAsExpr self.DoWordInCommand(node, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 989, in DoWordInCommand self.DoWordPart(part, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 999, in DoWordPart span_id = word.LeftMostSpanForPart(node) File "/home/andy/git/oil/bin/../core/word.py", line 105, in LeftMostSpanForPart return LeftMostSpanForWord(part.words[0]) # Hm this is a=(1 2 3) IndexError: list index out of range
Translating kube-up.sh	Traceback (most recent call last): File "bin/osh", line 469, in <module> main(sys.argv) File "bin/osh", line 452, in main sys.exit(OilMain(argv)) File "bin/osh", line 435, in OilMain status = OshMain(main_argv, login_shell) File "bin/osh", line 350, in OshMain osh2oil.PrintAsOil(arena, node, opts.debug_spans) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 69, in PrintAsOil fixer.DoCommand(node, None, at_top_level=True) # no local symbols yet File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 473, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 682, in DoCommand self.DoCommand(node.body, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 652, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 726, in DoCommand self.DoCommand(child, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 536, in DoCommand self.DoRedirect(r, local_symbols) File "/home/andy/git/oil/bin/../tools/osh2oil.py", line 270, in DoRedirect raise NotImplementedError(node.__class__.__name__) NotImplementedError: HereDoc