feat: introduce ceilometer helm chart overrides (#128)

* feat: introduce ceilometer helm chart overrides

This begins to add the overrides for the Ceilometer helm chart.
Ceilometer provides metering, monitoring, and alarming capabilities in
Openstack for billing, performance, optimization, and capacity planning
purposes.

* fix: ceilometer-api image is deprecated

Trying to pull the wallaby image for ceilometer-api results in a 404 not
found now. ceilometer-api is in the base image as of commit
cd67930 per the upstream kolla repo, so pull that instead.

* fix: remove ceilometer-collector from config

ceilometer collector was removed from ceilometer code base[1]

[1] https://review.openstack.org/504244

* fix: ceilometer-api is fully deprecated so rm it

This disables deployment of the api pod and removes related api
configuration as ceilometer no longer has a rest API. It is simply a
worker service at this point. Gnocchi API is preferred over ceilometer.

* fix: set database keys to fake values

The database section is not used, but the base chart still tries to set
some sane default values, so to avoid confusion, just override those to
a string value that makes it obvious this section is not used.

The recommended storage location for meters and events is Gnocchi, which
is automatically discovered and used by means of keystone.

* fix: set gnocchi as the publisher

This was explicitly set to notify:// without any context as to what that
is or does. The configuration does not list that as a valid value, so
let's replace the publisher with the default, `gnocchi`.

* fix: disable the ks-endpoint job

There is no endpoint for ceilometer anymore so remove the related job
that makes a service in keystone for one.

* fix: bump ceilometer images to yoga

This was the newest tagged image that I could find for Ceilometer. We
will need to investigate building our own Ceilometer images for a later
release of Genstack.

* fix: enable db-sync to init gnocchi resource types

The helm chart has a db_sync job which executes ceilometer-upgrade which
executes the storage upgrade function that initiates the resource types
in gnocchi with their attributes.

* fix: add updated event definitions from yoga

The event definitions defined in the helm chart were very dated, update
them to match those found in the yoga release.

* fix: update gnocchi resources to yoga

The gnocchi resources were outdated. This updates them to match what was
released with Yoga.

* fix: update ceilometer meters to yoga

The existing meters were outdated. This brings them up to date with the
yoga release.

* fix: simplify pipeline sinks for now

This removes some complexity that the original helm chart introduced
which defines custom meter sinks relating to instance cpu, disk, and net
metrics. We may find ourselves disabling pollsters for individual
instances, so let's not inundate the pipeline with un-necessary
complexity yet. If we find they are useful or needed, we can re-enable
them after verifying their proper operation.

The polled metrics will still be stored in Gnocchi, just not transformed
according to the defined sinks. Iff re-introduced, these pipeline sinks
may need to be further tweaked to work with the updated event defs.

* fix: enable postgresql backup jobs

* fix: add gnocchi API replicas & enable daemonsets

This should make Gnocchi more reliable and have better overall perf.

* fix: disable resource limits for ceilometer

We don't enforce pod resource limits in other helm charts so set this to
false as the default.

* fix: remove apache2 config for ceilometer

Ceilometer no longer has a rest API so let's remove this section from
the overrides.

* fix: Add default loglevels to aid troubleshooting

When troubleshooting, it helps to raise or lower default log levels
of specific modules, setting requests related loggers to DEBUG for
example can help one diagnose ceilometer CRUD operations.

* doc: add openstack ceilometer installation

* fix: set postgresql cron backup to 0015 once a day

The default was midnight but a lot of jobs run then; kick this off a
little later to help avoid the thundering herd affect.

docs/openstack-ceilometer.md

@@ -0,0 +1,80 @@
+# Deploy Ceilometer
+
+## Create Secrets
+
+```shell
+kubectl --namespace openstack create secret generic ceilometer-keystone-admin-password \
+        --type Opaque \
+        --from-literal=password="$(< /dev/urandom tr -dc _A-Za-z0-9 | head -c${1:-32};echo;)"
+kubectl --namespace openstack create secret generic ceilometer-keystone-test-password \
+        --type Opaque \
+        --from-literal=password="$(< /dev/urandom tr -dc _A-Za-z0-9 | head -c${1:-32};echo;)"
+kubectl --namespace openstack create secret generic ceilometer-rabbitmq-password \
+        --type Opaque \
+        --from-literal=password="$(< /dev/urandom tr -dc _A-Za-z0-9 | head -c${1:-32};echo;)"
+```
+
+## Run the package deployment
+
+```shell
+cd /opt/genestack/submodules/openstack-helm
+helm upgrade --install ceilometer ./ceilometer \
+    --namespace=openstack \
+    --wait \
+    --timeout 10m \
+    -f /opt/genestack/helm-configs/ceilometer/ceilometer-helm-overrides.yaml \
+    --set endpoints.identity.auth.admin.password="$(kubectl --namespace openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d)" \
+    --set endpoints.identity.auth.ceilometer.password="$(kubectl --namespace openstack get secret ceilometer-keystone-admin-password -o jsonpath='{.data.password}' | base64 -d)" \
+    --set endpoints.identity.auth.test.password="$(kubectl --namespace openstack get secret ceilometer-keystone-test-password -o jsonpath='{.data.password}' | base64 -d)" \
+    --set endpoints.oslo_messaging.auth.admin.username="$(kubectl --namespace openstack get secret rabbitmq-default-user -o jsonpath='{.data.username}' | base64 -d)" \
+    --set endpoints.oslo_messaging.auth.admin.password="$(kubectl --namespace openstack get secret rabbitmq-default-user -o jsonpath='{.data.password}' | base64 -d)" \
+    --set endpoints.oslo_messaging.auth.ceilometer.password="$(kubectl --namespace openstack get secret ceilometer-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)" \
+    --set conf.ceilometer.oslo_messaging_notifications.transport_url="\
+rabbit://ceilometer:$(kubectl --namespace openstack get secret ceilometer-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/ceilometer"\
+    --set conf.ceilometer.notification.messaging_urls.values="{\
+rabbit://ceilometer:$(kubectl --namespace openstack get secret ceilometer-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/ceilometer,\
+rabbit://cinder:$(kubectl --namespace openstack get secret cinder-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/cinder,\
+rabbit://glance:$(kubectl --namespace openstack get secret glance-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/glance,\
+rabbit://heat:$(kubectl --namespace openstack get secret heat-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/heat,\
+rabbit://keystone:$(kubectl --namespace openstack get secret keystone-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/keystone,\
+rabbit://neutron:$(kubectl --namespace openstack get secret neutron-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/neutron,\
+rabbit://nova:$(kubectl --namespace openstack get secret nova-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)@rabbitmq.openstack.svc.cluster.local:5672/nova}"
+```
+
+!!! tip
+
+    In a production like environment you may need to include production specific files like the example variable file found in `helm-configs/prod-example-openstack-overrides.yaml`.
+
+## Verify Ceilometer Workers
+
+As there is no Ceilometer API, we will do a quick validation against the
+Gnocchi API via a series of `openstack metric` commands to confirm that
+Ceilometer workers are ingesting metric and event data then persisting them
+storage.
+
+### Verify metric resource types exist
+
+The Ceilomter db-sync job will create the various resource types in Gnocchi.
+Without them, metrics can't be stored, so let's verify they exist. The
+output should include named resource types and some attributes for resources
+like `instance`, `instance_disk`, `network`, `volume`, etc.
+
+```shell
+kubectl exec -it openstack-admin-client -n openstack -- openstack metric resource-type list
+```
+
+### Verify metric resources
+
+Confirm that resources are populating in Gnocchi
+
+```shell
+kubectl exec -it openstack-admin-client -n openstack -- openstack metric resource list
+```
+
+### Verify metrics
+
+Confirm that metrics can be retrieved from Gnocchi
+
+```shell
+kubectl exec -it openstack-admin-client -n openstack -- openstack metric list
+```