Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pg: reset seq num in replication after commit msg #597

Merged
merged 1 commit into from
Mar 11, 2024

Conversation

jchappelow
Copy link
Member

@jchappelow jchappelow commented Mar 11, 2024

Untracked transaction are intended to have a sequence number of -1, indicating there was no sequence update, which is used by transactions from a pg.DB instance to correlate WAL data updates with a certain transaction.

This commit fixes the replication monitor to work correctly with concurrent untracked transactions by, such as those from the event store:

  • reset the seq var to -1 in the captureRepl loop after each commit
  • when a commit is reached with seq still -1 (not updated by an untracked transaction), do NOT send the commit ID on the channel

This also beefs up the logging and error handling in unexpected cases, which corresponds to improper use of replMon methods or concurrent (ab)use of the sentry table to break correct transaction sequencing.

@charithabandi Pls rebase spam-test on this branch to test.

Untracked transaction are intended to have a sequence number of -1,
indicating there was no sequence update, which is used by transactions
from a pg.DB instance to correlate WAL data updates with a certain
transaction.

This commit fixes the replication monitor to work correctly with
concurrent untracked transactions by:
- reset the seq var to -1 in the captureRepl loop after each commit
- when a commit is reached with seq still -1 (not updated by an
  untracked transaction), do NOT send the commit ID on the channel

This also beefs up the logging and error handling in unexpected cases,
which corresponds to improper use of replMon methods or concurrent
(ab)use of the sentry table to break correct transaction sequencing.
@jchappelow jchappelow added this to the v0.7.0 final milestone Mar 11, 2024
Copy link
Contributor

@charithabandi charithabandi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! also tested them out against the spammer listener.

@jchappelow
Copy link
Member Author

Thanks for testing @charithabandi. Any other comments or concerns? Will open a backport pr

@jchappelow jchappelow merged commit ee2dda3 into kwilteam:main Mar 11, 2024
1 check passed
@jchappelow jchappelow deleted the fix-pg-seq branch March 11, 2024 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants