Event Stream Service

[usingtechnology]

Description

Add Event Stream Service support. The underlying technology is NATS.

There is also a new configuration section specifically for the service. Note that the deployment will split the configuration across a config map and a secret as we need a password stored. See notes in the openshift/readme about creating the secret for a particular instance. If the service cannot be connected, the application will still run, log entries for failed connections as each event will attempt to connect to send the event message.

Due to external requirements for expediting this into production we removed a feature flag (enable the service per deployment) and added a flag per form. There is no administrative UX for this (currently) so each form owner will have to ask CHEFS support team to enable Event Streaming for their form. This will be extremely limited until the service is proven reliable and robust. As a fallback, we've added a flag for the current webhook subscription (again, no UX - db updates only) that will send the same event stream message metadata but over a webhook.

Event Stream will notify on Submission created, updated and deleted and on (Form) publish or unpublish.

There is a new folder named event-stream-service that contains a docker-compose file to standup a local instance of NATS that our local instance of CHEFs can use. There are examples on how to connect to the stream and consume messages pullConsumer.js is the most complete example of a consumer that listens to a given server and CHEFS stream). Important note is that locally, we can connect using the nats protocol but current deployment to openshift requires connecting with websockets protocol.

Also in event-stream-service are the helm charts used to deploy the instance. The first instance is deployed in the a191b5-dev namespace and is accessible at ess-a191b5-dev.apps.silver.devops.gov.bc.ca via websockets.

Types of changes

feat (a new feature)

Checklist

• I have read the CONTRIBUTING doc
• I have checked that unit tests pass locally with my changes
• I have run the npm script lint on the frontend and backend
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)
• I have approval from the product owner for the contribution in this pull request

Further comments

To test this out functionally, you can use the running PR here and follow the guide here which contains a sample Node script that you can use as a listener/client. A more up to date client can be found at ./event-stream-service/pullConsumer.js. Note that running the PR requires us to use WebSockets, so pullConsumer must be started with WEBSOCKETS=true.

When setting up your form to fire events, click Enable Public Stream and Enable Private Stream, also enter the encryption key used in the sample clients: ad5520469720325d1694c87511afda28a0432dd974cb77b5b4b9f946a5af6985.

You can publish and unpublish your forms, see the listener/client pull the messages, and take a look at the payloads. You may see some errors about encryption because we are wildcarding and listening for all forms, some of which will use different encryption keys.

Note about running locally... start the CHEFS API before starting the pullConsumer script. In a new/clean environment the event stream has to be initialized before a consumer can listen. CHEFS API will initialize the streams.

To run locally, start with a clean environment, use docker-compose to bring up .devcontainer/chefs_local/docker-compose.yml, ensure the db has been migrated and use docker-compose to bring up event-stream-service/docker-compose.yml. Note that there are 3 nodes for the NATS server and that when running locally we can use the NATS protocols (pullConsumer can be started without the WEBSOCKETS environment variable).

Assuming you are using the devcontainer, bring all existing containers (chefs_local) down and remove the volumes. Run the Dev Containers: Rebuild Container without Cache and start with a pristine environment. docker-compose up both chefs_local and event-stream-service and then Run > CHEFS. Once CHEFS is running, open a terminal in your devcontainer and run ENCRYPTION_KEY=ad5520469720325d1694c87511afda28a0432dd974cb77b5b4b9f946a5af6985 node pullConsumer.js

[WalterMoar]

I think this also needs a rule that the key will fit into the table.

[WalterMoar]

The reloader container doesn't have resources defined, so it's using the default: 50m->250m CPU, 256Mi->1Gi memory. I tried tuning it down in dev and it seems to function fine with:

Suggested change

	max_payload: 1MB
	max_payload: 1MB
	reloader:
	merge:
	resources:
	limits:
	cpu: 50m
	memory: 64Mi
	requests:
	cpu: 10m
	memory: 32Mi

Editing the configmap the logs show:

2024/11/21 22:37:34 Starting NATS Server Reloader v0.16.0
2024/11/21 22:37:34 Watching file: /etc/nats-config/nats.conf
2024/11/21 22:37:34 Live, ready to kick pid 7 on config changes (files=1)
2024/11/21 22:39:21 Starting ticker to re-add all tracked files.
2024/11/21 22:39:21 Changed config; file="/etc/nats-config/nats.conf" existing=true total-files=1
2024/11/21 22:39:21 Updated files: [/etc/nats-config/nats.conf]
2024/11/21 22:39:21 Sending pid 7 'hangup' signal to reload changes from: [/etc/nats-config/nats.conf]
2024/11/21 22:39:22 Tracking files [/etc/nats-config/nats.conf]
2024/11/21 22:39:22 All monitored files detected.

[WalterMoar]

Where does the chefs_pwd go?

[usingtechnology]

that needs to go into the chefs deployment. chefs-event-stream-service secret.

[github-actions]

Release a86f055 deployed at https://chefs-dev.apps.silver.devops.gov.bc.ca/pr-1444

discussed