Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[database] 7 things a developer should know #5

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

octo-topi
Copy link
Owner

@octo-topi octo-topi commented Dec 30, 2024

Problem

Most of web developer use a database behind their REST API, but when a problem happens, they're helpless because some concerns have not been properly address before going into production.

Solution

Provide a handful of down-to-earth, clear advice on how to monitor the database and client-side pools.

Remarks

The title is a reference to 97 things a developer should know

@octo-topi octo-topi force-pushed the seven-things-a-developer-should-know-about-database branch from d36b374 to da68181 Compare December 30, 2024 11:18
@octo-topi octo-topi force-pushed the seven-things-a-developer-should-know-about-database branch from 3b8c29e to 5635ca6 Compare January 3, 2025 07:01

**TL;DR: use a pool; make sure when scaling you do not reach max_connections**

### Ride safe
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

est-ce qu'il ne faudrait pas aussi dire un mot sur les droits utilisateur ? les comptes utilisés en production pour du debug pourraient avoir seulement des droits en lecture, et il devraient être nominatifs

Copy link
Owner Author

@octo-topi octo-topi Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Je l'ai exprimé ainsi pour ne allonger l'article. As-tu une autre idée en quelques mots ?

If you can't get a read-only account

The only proper way to do this is using a SQL client and call one of these methods:

- `pg_cancel_backend($PID)`;
- `pg_terminate_backend($PID, $TIMEOUT)`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expliquer la différence entre les 2 méthodes, et ajouter un lien vers la doc

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Je vais supprimer pg_cancel_backend et ajouter un exemple, comme ça plus de complexité.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah mais en fait, ils ne font pas du tout la même chose - et la doc n'est pas claire .. Je vais expliciter ça

https://stackoverflow.com/questions/18866865/difference-between-terminating-and-cancelling-a-process

@octo-topi octo-topi force-pushed the seven-things-a-developer-should-know-about-database branch from 36b6b1d to 3346308 Compare January 6, 2025 14:27

You may think concurrency is a concern for programming langage designers or architects, something you shouldn't worry about, something that has been taken care of at the beginning of the project. Especially in database. Well, if you want to ride full throttle, beware of concurrency.

Let's consider the worst case: we deploy a REST API back-end on a PaaS which offers horizontal auto-scaling, plus DBaS. If we want to max out the database performance, we should consider 2 levels : inside the database, and outside the database. You want a small pool, saturated with threads waiting for connections.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Let's consider the worst case: we deploy a REST API back-end on a PaaS which offers horizontal auto-scaling, plus DBaS. If we want to max out the database performance, we should consider 2 levels : inside the database, and outside the database. You want a small pool, saturated with threads waiting for connections.
2 levels:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DBaS > c'est écrit DBaaS plus bas

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

je ne suis pas sûre de comprendre pourquoi c'est le pire des cas possibles où en tout cas le ou les éléments décrits qui en font un cas compliqué

Copy link
Owner Author

@octo-topi octo-topi Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

J'ai explicité tout ça, ça valait le coup !
Peux-tu me dire si c'est compréhensible maintenant ?


Well, to find who's not releasing the lock, [pg_locks] native view is the way to go. As it's not human friendly, and list a lock per row, use [this version](https://wiki.postgresql.org/wiki/Lock_dependency_information#Recursive_View_of_Blocking) which displays the lock tree.

Here, session 3 is blocked by session 2, itself blocked by session 1. The root blocking session, session 1, stays on the first line, and each indent shows the session it blocks, session 2. The lock held by session 1 is on `foo` table has not been released, because the session 1 is waiting for lock on `bar` table to be granted. Now it's your job to know this lock has not been granted.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

un espace en trop après la virgule avant stays session 1, stays
le Here fait référence au lien sur le wiki postresql cité juste avant ?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

J'ai corrigé l'espace. En fait, il manque une image, bien vu !

Copy link

@annemarie35 annemarie35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work ! Very usefull ^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants