Run hpcc platform on microk8s on Ubuntu 20.04

I've just installed Ubuntu 20.04 Desktop on my new laptop, so now I need to set up docker and kubernetes on it. I used docker desktop on OSX and Windows 10, and minikube on OSX, but this time I wanted to try microk8s. Here's what I did to run the containerized version of hpcc platform on it.

1. Install with snap:

sudo snap install microk8s --classic

2. Make microk8s more convenient to use:

Add your user to the microk8s group so that you don't have to sudo:

sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
su - $USER

Add the following to ~/.bashrc to create aliases to save some typing:

alias k="microk8s kubectl"
alias h="microk8s helm"
alias m="microk8s"

Then

source ~/.bashrc

3. Enable add-on's needed:

m enable helm dns storage ingress

Check the add-on's are really enabled:

m status

4. Init helm

h init

Wait for a couple of minutes before continuing to give some time for tiller pod to be created. To check if the pod is ready, run

k get pods -A | grep tiller

and if it's ready, something like this will be printed:

kube-system   tiller-deploy-74548b7c5f-5dg5q              1/1     Running             0          7m12s

5. Make necessary changes to your values.yaml

microk8s doesn't support LoadBalancer for a single node, so it's better to always use ClusterIP. In your copy of values.yaml, change esp's "public" and roxie's "external" attributes to "false".

This won't cause any issue unless you want to access esp or roxie from another box.

Note: You don't really have to make the change. If you don't, the external ip will show as "Pending", which doesn't look pretty but it doesn't actually hurt anything.

6. Install the helm chart:

h install --name mycluster --values myvalues.yaml hpcc

7. If you do need to access esp (eclwatch and ws_ecl) and roxie cluster from another box, just create an ingress:

Create an ingress.yaml file that looks like this:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: esp
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /eclwatch
        pathType: Prefix
        backend:
          serviceName: eclwatch
          servicePort: 8010
      - path: /wsecl
        pathType: Prefix
        backend:
          serviceName: eclqueries
          servicePort: 8002

Create the ingress with kubectl:

k apply -f ingress.yaml

8. Access eclwatch

You can now browse url http://ClusterIP:8010, or if you created the ingress, http://localhost/eclwatch

9. Note on local images

One thing worth mentioning is that microk8s is not able to see the local images you built with docker. To list the microk8s images use this command:

m ctr images ls

You can either upload your local image to a public repository like docker hub, or import the image into microk8s. The following is an example:

docker save hpccsystems/platform-core:ac7c0353b-Debug -o platform-core-ac7c0353b-Debug.tar
m ctr image import platform-core-ac7c0353b-Debug.tar

10. Optionally enable and use local repository

Another way to share the image with microk8s is to use its built-in local repository, which seems to be faster than saving and importing.

First enable the repository:

microk8s enable registry:size=40Gi

Then tag and push your images:

docker tag b1417fe2f157 localhost:32000/hpccsystems/platform-core:b779e071c-Debug
docker push localhost:32000/hpccsystems/platform-core
docker tag 0277f3ee0c22 localhost:32000/hpccsystems/platform-build:b779e071c-Debug
docker push localhost:32000/hpccsystems/platform-build

Or you can build the image with the local repository in mind:

sudo ./incr.sh -u mayx -d localhost:32000/hpccsystems b779e071c-Debug
docker push localhost:32000/hpccsystems/platform-core:2bd3dff17-Debug
docker push localhost:32000/hpccsystems/platform-build:2bd3dff17-Debug

11. Allow privileged

To allow privileged security context, add "--allow-privileged" argument to the kube-apiserver args:

sudo vim /var/snap/microk8s/current/args/kube-apiserver

Then restart the api server:

sudo systemctl restart snap.microk8s.daemon-apiserver.service

In your values.yaml file, you can enable privileged mode by adding the line to the global config:

  privileged: true

With this, you can connect to the containers and run gdb and attach to the processes to debug issues.

12. Uninstall/delete deployment

First list the releases that are running:

h list

The output is something like this:

NAME     	REVISION	UPDATED                 	STATUS  	CHART     	APP VERSION	NAMESPACE
mycluster	1       	Thu Sep  3 16:48:16 2020	DEPLOYED	hpcc-0.1.0	0.1.0      	default  

Delete the deployment by name "myclsuter":

h delete --purge mycluster