Use the provided image for all tezos binaries (#177)

* Use the provided image for all tezos binaries

To do this, we switch from making containers based on
tezos/tezos:v8-release to making containers that either:

	1. use the provided tezos image and execute
	   a script which is passed in via the
	   statefulset configuration, or

	2. are not based on the tezos image and do not
	   contain tezos binaries.  These containers
	   prepare configuration, files, etc., for the
	   later containers.

* Rename config-generator to "utils" and fold in wait-for-bootstrap

As a first step on consolidating to fewer docker images, we fold
wait-for-bootstrap into config-generator calling the result "utils".

* Make utils/entrypoint.sh into a spawner of the other scripts.

And fix a few things along the way.  I think that I'll squash this
into the previous commits before merging to the master branch.

* Forgot to git add utils/config-generator.sh.

* Rename file to correct place.

* Make tezos-node also use an external script.

This lets us put some debugging in there.

* utils/Dockerfile: pin to python:3.8-alpine and streamline.

* Various changes to get zerotier working (not the invite)

The most interesting one here is that for some reason, zerotier
races with tezos-node.  If the latter wins, then it fails to bind
to the p2p listen-address because zerotier hasn't yet setup the VPN.
We solve this by restarting tezos-node a number of times on failure.

We also sleep 3600 on tezos-node exit to give us a chance to get a
shell on the node.  We may want to consider this, though.

* Fix zerotier invites...

The issue was that the CHAIN_NAME env var wasn't percolating to
the inviter and so the host was labeled _bootstrap.  The invitee
then looked for ${CHAIN_NAME}_bootstrap, which obviously wasn't
there...

* Eliminate spurious error.

* Use Python 3.9 rather than 3.8.

* Remove confusing comment.

charts/tezos/scripts/README

@@ -0,0 +1,37 @@
+In this directory, please find a number of scripts that are used
+to start various components in the tezos docker images.
+
+These script snippets are not executed as files by /bin/sh, but rather
+as a single line passed in via -c.
+
+We also ask for the helm template system to be passed over the snippet
+before it is inserted into the chart.
+
+YAML ISSUES
+
+    Because we are actually including this snippet into YAML as
+    an indented blob, it is a requirement that the first character
+    in the file is in column zero.
+
+TEMPLATING
+
+    Because the templating system is used, you can use any of the
+    features provided.  Mostly, variable expansion, i.e.
+
+	{{ .Values.protocol.command }}
+
+    If you use braces within the file, you will have to escape them,
+    though:
+
+	{{ "{{" }}
+
+SCRIPT SIZE LIMIT
+
+    Each OS has limit on the number of bytes that can be passed to
+    a command to be executed.  In some cases, this may be relatively
+    small, but we should be able to assume that it's at least 64KB.
+    So, be explicit, take care of all of the error cases, but do not
+    write your great novel.
+
+    In practice, you are unlikely to run into this limit, but it
+    exists and so it must be mentioned.

charts/tezos/scripts/baker-endorser.sh

@@ -0,0 +1,24 @@
+set -ex
+
+TEZ_VAR=/var/tezos
+TEZ_BIN=/usr/local/bin
+CLIENT_DIR="$TEZ_VAR/client"
+NODE_DIR="$TEZ_VAR/node"
+NODE_DATA_DIR="$TEZ_VAR/node/data"
+
+proto_command="{{ .Values.protocol.command }}"
+
+if [ "${DAEMON}" == "baker" ]; then
+    extra_args="with local node $NODE_DATA_DIR"
+fi
+
+my_baker_account="$(cat /etc/tezos/baker-account )"
+
+CLIENT="$TEZ_BIN/tezos-client -d $CLIENT_DIR"
+CMD="$TEZ_BIN/tezos-$DAEMON-$proto_command -d $CLIENT_DIR"
+
+while ! $CLIENT rpc get chains/main/blocks/head; do
+    sleep 5
+done
+
+exec $CMD run ${extra_args} ${my_baker_account}

charts/tezos/scripts/chain-initiator.sh

@@ -0,0 +1,18 @@
+CLIENT="/usr/local/bin/tezos-client -A tezos-node-rpc -P 8732"
+
+until $CLIENT rpc get /version; do
+    sleep 2
+done
+
+echo /etc/tezos/parameters.json contains:
+echo ------------------------------------------------------------
+cat /etc/tezos/parameters.json
+echo ------------------------------------------------------------
+echo Activating chain:
+set -x
+$CLIENT -d /var/tezos/client -l --block					\
+	genesis activate protocol					\
+	{{ .Values.activation.protocol_hash }}				\
+	with fitness -1 and key						\
+	{{ .Values.node_config_network.activation_account_name }}	\
+	and parameters /etc/tezos/parameters.json

charts/tezos/scripts/config-init.sh

@@ -0,0 +1,12 @@
+echo "Writing custom configuration for public node"
+mkdir -p /etc/tezos/data
+
+#
+# This is my comment.
+
+/usr/local/bin/tezos-node config init		\
+    --config-file /etc/tezos/data/config.json	\
+    --data-dir /etc/tezos/data			\
+    --network $CHAIN_NAME
+
+cat /etc/tezos/data/config.json

charts/tezos/scripts/snapshot-importer.sh

@@ -0,0 +1,19 @@
+set -ex
+
+bin_dir="/usr/local/bin"
+
+data_dir="/var/tezos"
+node_dir="$data_dir/node"
+node_data_dir="$node_dir/data"
+node="$bin_dir/tezos-node"
+snapshot_file=${node_dir}/chain.snapshot
+
+if [ -d ${node_data_dir}/context ]; then
+    echo "Blockchain has already been imported, exiting"
+    exit 0
+fi
+
+${node} snapshot import ${snapshot_file} --data-dir ${node_data_dir} \
+    --network $CHAIN_NAME --config-file /etc/tezos/config.json
+find ${node_dir}
+rm -rvf ${snapshot_file}

charts/tezos/scripts/tezos-node.sh

@@ -0,0 +1,20 @@
+set -x
+
+set
+
+#
+# Not every error is fatal on start.  In particular, with zerotier,
+# the listen-addr may not yet be bound causing tezos-node to fail.
+# So, we try a few times with increasing delays:
+
+for d in 1 1 5 10 20 60 120; do
+	/usr/local/bin/tezos-node run				\
+			--bootstrap-threshold 0			\
+			--config-file /etc/tezos/config.json
+	sleep $d
+done
+
+#
+# Keep the container alive for troubleshooting on failures:
+
+sleep 3600

charts/tezos/templates/_containers.tpl

@@ -15,11 +15,36 @@
 {{- end }}
 {{- end }}
 
+{{- define "tezos.init_container.config_init" }}
+{{- if include "tezos.shouldConfigInit" . }}
+- image: "{{ .Values.images.tezos }}"
+  command:
+    - /bin/sh
+  args:
+    - "-c"
+    - |
+{{ tpl (.Files.Get "scripts/config-init.sh") . | indent 6 }}
+  imagePullPolicy: IfNotPresent
+  name: config-init
+  volumeMounts:
+    - mountPath: /etc/tezos
+      name: config-volume
+    - mountPath: /var/tezos
+      name: var-volume
+  envFrom:
+    - configMapRef:
+        name: tezos-config
+  env:
+{{- include "tezos.localvars.pod_envvars" . | indent 4 }}
+{{- end }}
+{{- end }}
+
 {{- define "tezos.init_container.config_generator" }}
-- image: {{ .Values.tezos_k8s_images.config_generator }}
+- image: {{ .Values.tezos_k8s_images.utils }}
   imagePullPolicy: IfNotPresent
   name: config-generator
   args:
+    - "config-generator"
     - "--generate-config-json"
   envFrom:
     - secretRef:
@@ -37,7 +62,9 @@
 
 {{- define "tezos.init_container.wait_for_bootstrap" }}
 {{- if include "tezos.shouldWaitForBootstrapNode" . }}
-- image: {{ .Values.tezos_k8s_images.wait_for_bootstrap }}
+- image: {{ .Values.tezos_k8s_images.utils }}
+  args:
+    - wait-for-bootstrap
   imagePullPolicy: IfNotPresent
   name: wait-for-bootstrap
   envFrom:
@@ -51,9 +78,11 @@
 
 {{- define "tezos.init_container.snapshot_downloader" }}
 {{- if include "tezos.shouldDownloadSnapshot" . }}
-- image: "{{ .Values.tezos_k8s_images.snapshot_downloader }}"
+- image: "{{ .Values.tezos_k8s_images.utils }}"
   imagePullPolicy: IfNotPresent
   name: snapshot-downloader
+  args:
+    - snapshot-downloader
   volumeMounts:
     - mountPath: /var/tezos
       name: var-volume
@@ -67,15 +96,37 @@
 {{- end }}
 {{- end }}
 
-{{- define "tezos.container.node" }}
-- args:
-    - run
-    - "--bootstrap-threshold"
-    - '0'
-    - "--config-file"
-    - /etc/tezos/config.json
+{{- define "tezos.init_container.snapshot_importer" }}
+{{- if include "tezos.shouldDownloadSnapshot" . }}
+- image: "{{ .Values.images.tezos }}"
+  imagePullPolicy: IfNotPresent
+  name: snapshot-importer
   command:
-    - /usr/local/bin/tezos-node
+    - /bin/sh
+  args:
+    - "-c"
+    - |
+{{ tpl (.Files.Get "scripts/snapshot-importer.sh") . | indent 6 }}
+  volumeMounts:
+    - mountPath: /var/tezos
+      name: var-volume
+    - mountPath: /etc/tezos
+      name: config-volume
+  envFrom:
+    - configMapRef:
+        name: tezos-config
+  env:
+{{- include "tezos.localvars.pod_envvars" . | indent 4 }}
+{{- end }}
+{{- end }}
+
+{{- define "tezos.container.node" }}
+- command:
+    - /bin/sh
+  args:
+    - "-c"
+    - |
+{{ tpl (.Files.Get "scripts/tezos-node.sh") . | indent 6 }}
   image: "{{ .Values.images.tezos }}"
   imagePullPolicy: IfNotPresent
   name: tezos-node
@@ -92,30 +143,44 @@
 {{- end }}
 
 {{- define "tezos.container.baker" }}
-- image: "{{ .Values.tezos_k8s_images.baker_endorser }}"
+- image: "{{ .Values.images.tezos }}"
+  command:
+    - /bin/sh
+  args:
+    - "-c"
+    - |
+{{ tpl (.Files.Get "scripts/baker-endorser.sh") . | indent 6 }}
   imagePullPolicy: IfNotPresent
   name: baker
   volumeMounts:
-  - mountPath: /var/tezos
-    name: var-volume
+    - mountPath: /etc/tezos
+      name: config-volume
+    - mountPath: /var/tezos
+      name: var-volume
   envFrom:
     - configMapRef:
         name: tezos-config
-    - secretRef:
-        name: tezos-secret
   env:
 {{- include "tezos.localvars.pod_envvars" . | indent 4 }}
     - name: DAEMON
       value: baker
 {{- end }}
 
 {{- define "tezos.container.endorser" }}
-- image: "{{ .Values.tezos_k8s_images.baker_endorser }}"
+- image: "{{ .Values.images.tezos }}"
+  command:
+    - /bin/sh
+  args:
+    - "-c"
+    - |
+{{ tpl (.Files.Get "scripts/baker-endorser.sh") . | indent 6 }}
   imagePullPolicy: IfNotPresent
   name: endorser
   volumeMounts:
-  - mountPath: /var/tezos
-    name: var-volume
+    - mountPath: /etc/tezos
+      name: config-volume
+    - mountPath: /var/tezos
+      name: var-volume
   envFrom:
     - configMapRef:
         name: tezos-config
@@ -134,6 +199,8 @@
 {{- define "tezos.init_container.zerotier" }}
 {{- if (include "tezos.doesZerotierConfigExist" .) }}
 - envFrom:
+    - configMapRef:
+        name: tezos-config
     - configMapRef:
         name: zerotier-config
   image: "{{ .Values.tezos_k8s_images.zerotier }}"
@@ -147,6 +214,8 @@
         - SYS_ADMIN
     privileged: true
   volumeMounts:
+    - mountPath: /etc/tezos
+      name: config-volume
     - mountPath: /var/tezos
       name: var-volume
     - mountPath: /dev/net/tun

charts/tezos/templates/_helpers.tpl

@@ -80,3 +80,18 @@
 {{- "" }}
 {{- end }}
 {{- end }}
+
+{{/*
+  Checks if we need to run tezos-node config init to help
+  config-generator obtain the appropriate parameters to run
+  a network.
+  Returns a string "true" or empty string which is falsey.
+*/}}
+*/}}
+{{- define "tezos.shouldConfigInit" }}
+{{- if not (.Values.node_config_network.genesis) }}
+{{- "true" }}
+{{- else }}
+{{- "" }}
+{{- end }}
+{{- end }}