alertmanager livenessProbe and readinessProbe port

[TerraTech]

What keywords did you search in tectonic-installer issues before filing this one?

alertmanager livenessProbe port
(and several other variations)

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

• Tectonic version (release or commit hash): 1.8.4-tectonic.3
• Terraform version (terraform version): v0.11.2
• Platform (aws|azure|openstack|metal): metal

What happened?

The pods started by alertmanager-main (statefulset) are failing the liveness probe and being restarted.

What you expected to happen?

That it would connect to port 9093 for the probes.

Anything else we need to know?

Should the probing port (liveness/readiness) defined in the pod spec be port 9093 instead of 'www' (80) ? The url is '/api/v1/status' and a curl request does respond on 9093, however trying port 80 I get connection refused. Jumping into the container via kubectl exec and running a netstat shows that it is not listening to port 80 at all. Only ports 9093 and 6783.

*container: alertmanager

livenessProbe:
  failureThreshold: 10
  httpGet:
    path: /api/v1/status
    port: www
    scheme: HTTP
  periodSeconds: 10
  successThreshold: 1
  timeoutSeconds: 3

As an aside, while I like operator driven specs, it can be frustrating to troubleshoot when you edit the statefulset spec and manually change the probe ports, only for the operator to step in and clobber/reset your (testing) changes back to what it wants it to be. If there is a way to temporarily disable an operator from doing this, please let me know. Thanks!

[sym3tri]

/cc @brancz