Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INCIDENT-001: postgres doesnt have Point of Recoverability so it might fail when there's no replicas left #72

Merged
merged 4 commits into from
Aug 16, 2024

Conversation

LuisDuarte1
Copy link
Member

Incident report (you need Notion access): https://www.notion.so/rcachaldora/INCIDENT-001-Postgres-primary-failure-due-lack-of-state-backups-5b15d04f85a34df4a91bf8f2315ff8ff

Also upgrade cnpg version while I'm at it.
We backup now to an R2 bucket that has the postgres backups.

…t fail when there's no replicas left

Also upgrade cnpg version while I'm at it. We backup now to an R2 bucket that has the postgres backups.
@LuisDuarte1 LuisDuarte1 requested a review from a team July 23, 2024 22:57
Copy link
Member

@tomaspalma tomaspalma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

A quick note: As I've now noticed the values of the cnpg-backup-secrets as <FILL-IN, is it a bad idea to write in the Secrets Management's page that there is a script (the check_secrets.sh) that checks for secrets with <FILL-IN> and warns about it and that should be the standard practice for putting secret values on the files that committed to the remote repository? I've not found that information in the wiki, I apologize if it's already there.

@LuisDuarte1
Copy link
Member Author

A quick note: As I've now noticed the values of the cnpg-backup-secrets as <FILL-IN, is it a bad idea to write in the Secrets Management's page that there is a script (the check_secrets.sh) that checks for secrets with <FILL-IN> and warns about it and that should be the standard practice for putting secret values on the files that committed to the remote repository? I've not found that information in the wiki, I apologize if it's already there.

That's a great point, I'll add it! Thank you!

@LuisDuarte1 LuisDuarte1 requested a review from tomaspalma July 28, 2024 10:16
@LuisDuarte1 LuisDuarte1 requested a review from tomaspalma August 8, 2024 21:55
@LuisDuarte1 LuisDuarte1 merged commit 74c4e2c into main Aug 16, 2024
1 check passed
@LuisDuarte1 LuisDuarte1 deleted the fix/postgres-backups branch August 16, 2024 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants