Secret deleted and recreated in the apply pipeline

[jackstockley89]

Service name

Namespace: effected legal-framework-api-uat
Effected Resource: lfa-secrets

slack thread

Problem description

User raised an issue with a secret not working anymore due to it being blank, on investigation the secret had been deleted and recreated Sunday deleted, recreated

unsure how the terraform destroy action triggered. as there weren't any changes to this pull request or state file secret terraform

We need to understand how this happened and make sure it can't happen again

Contact person

[email protected]

[jaskaransarkaria]

It's hard to replicate this issue but my intuition is telling me this is a race condition where the dir the the apply is happening is misaligned with the namespace. We've added a check to enforce that this should always be the same namespace and dir before any apply and added some debug logs to monitor if this happens again.

ministryofjustice/cloud-platform-cli#681