EVEREST-107: config and secrets consistency via owner references

[michal-kralik]

CHANGE DESCRIPTION

Problem:
EVEREST-107

Short explanation of the problem.
When it comes to create/update/delete backup storage and the associated secret, we have issues with data consistency and keeping secrets around if they're not needed.

Solution:
This PR aims to resolve the issues. The benefits include:

1. Delete requires removal of the backup storage only. Secrets are deleted via owner references by k8s
2. Update makes sure the data is always in a consistent state by creating a new secret and then replacing it in backup storage
3. Create does not leave secret in k8s in case backup storage cannot be created properly

Create backup storage

1. Create backup storage in Initializing state
2. Create a secret with owner reference to backup storage
3. Update backup storage with secret name and set state to Ready

Update backup storage

1. Create new secret - if needed
2. Update backup storage
3. Delete old secret

Delete backup storage

1. Delete backup storage
2. k8s deletes all related secrets

Open topics
Listed in the code with TODO comments.

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?

Tests

• Is an Integration test/test case added for the new feature/change?
• Are unit tests added where appropriate?

[recharte]

Create backup storage

1. Create backup storage in Initializing state
2. Create a secret with owner reference to backup storage
3. Update backup storage with secret name and set state to Ready

I like the idea of using the status field to help us manage what is currently an inconsistent state and I think we can leverage this to achieve the data consistency we're looking for.
However, I don't like the fact that the backend is the entity responsible for setting this status.
For all intents and purposes, the backend is just a client of the everest CRDs and as such should only be responsible for setting their spec and never a status.

[oksana-grishchenko]

Let's have the possible statuses defined as constants in one place

[oksana-grishchenko]

Do we need to expose the status to the API? Looks like the "Initializing" will never returned to the API user anyway.