Skip to content

Commit

Permalink
Document data tracking in non-public schemas and infra region
Browse files Browse the repository at this point in the history
  • Loading branch information
exAspArk committed Jun 12, 2024
1 parent f0bdd25 commit 470c2ca
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 6 deletions.
1 change: 1 addition & 0 deletions docs/docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ keywords: ['Bemi Changelog', 'Bemi New Features', 'Postgres Audit Trails', 'Chan
## 2024-06

* Platform
* Enable tracking changes from non-`public` PostgreSQL schemas
* Allow setting ignore-change column rules across all tables (e.g., `*.updatedAt`)

## 2024-05
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/postgresql/destination-database.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ You have full control over this database which comes with additional features:
* Standard cloud support
* Control plane and monitoring through Bemi Dashboard (coming soon)

The infrastructure is automatically provisioned in the `us-west-1` region. If you have specific data residency requirements, please [contact us](mailto:[email protected]).

## Data Structure

Changes performed by creating, updating, or deleting each row are stored in a table called `changes` and have the following structure:
Expand Down
17 changes: 11 additions & 6 deletions docs/docs/postgresql/source-database.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ To connect a [Neon](https://neon.tech/) database, specify your database credenti

And that's it, everything should just work!


#### Read-only credentials {#neon-read-only}

Alternatively, you can manually create read-only PostgreSQL database credentials to connect to the primary instance's WAL.
Expand Down Expand Up @@ -267,10 +268,6 @@ See the [AWS RDS user guides](https://docs.aws.amazon.com/AmazonRDS/latest/UserG

### GCP Cloud SQL

#### WAL level {#gcp-wal-level}

Logical replication is turned off by default. To turn it on, you have to update the [cloud flag](https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication#configure-your-postgresql-instance): `cloudsql.logical_decoding` = `on`. This will need a restart of your instance before `SHOW wal_level;` returns `logical`.

#### Connection {#gcp-connection}

Run the below command and then you can connect with the same credentials on the Bemi dashboard!
Expand All @@ -279,6 +276,10 @@ Run the below command and then you can connect with the same credentials on the
ALTER USER [user] WITH REPLICATION;
```

#### WAL level {#gcp-wal-level}

Logical replication is turned off by default. To turn it on, you have to update the [cloud flag](https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication#configure-your-postgresql-instance): `cloudsql.logical_decoding` = `on`. This will need a restart of your instance before `SHOW wal_level;` returns `logical`.

### Self-managed PostgreSQL

#### Connection {#self-managed-connection}
Expand Down Expand Up @@ -372,11 +373,15 @@ During the Source Database connection setup or any time after, you can configure

![](/img/tracked-tables.png)

Bemi automatically tracks changes in the default `public` schema. If you would like to enable tracking for other schemas in your Bemi organization, please [contact us](mailto:[email protected]).

### Ignoring by Columns

Bemi allows to configure ignore-change columns, such as `myTable.updatedAt`. This prevents the creation of a new audit trail entry (called "change") for a record in `myTable` if `updatedAt` was the only value that was updated.
Bemi allows to configure ignore-change columns, such as `myTable.updatedAt`, to ignore meaningless data changes.
This prevents the creation of a new audit trail entry (called "change") for a record in `myTable` if `updatedAt` was the only column value that was updated.

In other words, `myTable.updatedAt` is used to determine whether an audit trail entry should be recorded or not. But this column will always be persisted if there were other columns that were updated.
In other words, `myTable.updatedAt` is used to determine whether an audit trail entry should be recorded or not.
Note that this column will always be recorded if there were updated values in other columns.

## SSH Tunnel

Expand Down

0 comments on commit 470c2ca

Please sign in to comment.