WIP: Add proof of concept kubernetes support

[geoah]

@metasoarous this is a very basic POC just to get a feel on the direction you are hoping for.

The instructions in the README should result in a hopefully working instance of polis without email support.
I need to figure out how to run tests against this.

---

Warning: This is work in progress and should not be used in production.

Under the manifests folder you can find a first version of polis running under Kubernetes. It uses an in-cluster postegres as a stateful set, with a persistent volume claim, and exposes the polis server using the cluster's ingress.

[metasoarous]

Thanks again for submitting this @geoah! Starting to take a look here.

Tagging @crkrenn, who was mentioning wanting to work on container orchestration (autoscaling, etc); This seems like a great start! And coincidentally, I feel like part of what is missing here is better orchestrated handling of credentials and configuration, which I think connects to challenges with #1341.

Thanks again.

[pmc30]

Hello @geoah, I am not very experienced with kubernetes but I was able to set up an instance by following the steps in the README. I couldn't figure out though how to update any of the docker images once I deployed it. The only way I could get changes I made to apply was to run minikube delete to remove the entire cluster and then re-add everything from scratch. It could be useful to add to the instructions something about how to apply updates while using locally built containers.

The other issue I ran into was when I tried to vote on a test conversation I created, it would always fail with the alert "Apologies, your vote failed to send. Please check your connection and try again". Here is more info on the error from the response to the POST to http://polis.local/api/v3/votes:

TypeError [ERR_INVALID_ARG_TYPE]: The "password" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
at Cipher.createCipher (internal/crypto/cipher.js:112:14)
at new Cipher (internal/crypto/cipher.js:128:16)
at Object.createCipher (crypto.js:119:10)
at Object.deprecated (internal/util.js:89:15)
at encrypt (/app/dist/src/session.js:29:35)
at addParticipantAndMetadata (/app/dist/src/server.js:2845:41)
at handle_POST_votes (/app/dist/src/server.js:6830:15)
at callbacks (/app/node_modules/express/lib/router/index.js:164:37)
at /app/dist/src/utils/parameter.js:131:17

I couldn't figure out the exact cause of this error but when I ran this with docker compose up using the same locally built containers I did not get the error so it could be related to the kubernetes cluster in some way.

[metasoarous]

Thanks so much for trying this out @pmc30!

I believe the error you're getting there is related to one of the configuration parameters being missing from the kubernetes configuration. As I mentioned in my previous post, I think most of the remaining work associated with this issue is getting the configuration story situated, so this is not terribly surprising.

I think the steps to fixing this involve going through the configuration options in the #1341 and ensuring that they're all reflected in the kubernetes configuration.

Thanks again

[geoah]

Hey @pmc30 sorry for missing this, I hadn't subscribed to the notifications for this PR; I'll fix that.

---

A couple of comments on your issue and the kubernetes stuff in general.

1. Without waiting for WIP: java/typescript node convict configuration integration #1341 you need to add your domain to the domain whitelist in two different polis.config.template.js files. I've added the polis.local domain to the template config js in ths PR.

2. As for the error you get about the password, it seems to be because of the following codepath:
   

   polis/server/src/session.ts

   Lines 8 to 10 in 9a29414

   	const password = process.env.ENCRYPTION_PASSWORD_00001;
   	//
   	// TODO replace deprecated createCipher method with current createCipheriv method

I've pushed a commit to add this env var to the server which should resolve this error.

4. In regards to the local development flow it was indeed a pain in the butt so I've introduced skaffold into the mix to help with that. The README has been updated on how to use this, but TL;DR is just run skaffold dev to have the containers be rebuild every time something that has been COPY-ed into the container changes, or just run skaffold run every time you want to rebuild/update the containers.

[tevko]

Hey @geoah , do you have any updates on this?

[sonalranjit]

I was able to get this up and running last night, i rebased @geoah's branch to the latest edge branch and updated some of the data in ConfigMap for the polis-server, but as a proof of concept i can get polis running on minikube locally, it's currently on my fork.

Screen.Recording.2024-11-22.at.7.14.08.PM.mov

[sonalranjit]

I've opened this PR for a more up to date version of this PR. Since i see it was last active in 2022.

#1847