v0.18.7

What's Changed

• kms: cache results by @serprex in #2180
• validate: 1 minute timeout by @serprex in #2183
• [clickhouse] misc type fixes by @heavycrystal in #2181
• Don't alert when drop flow is stuck dropping slot because active PID by @serprex in #2186
• feat(nexus): add flag for disabling migrations by @iamKunalGupta in #2188
• env: PEERDB_SNOWFLAKE_ON_ERROR by @serprex in #2189

Full Changelog: v0.18.6...v0.18.7

v0.18.6

What's Changed

v0.18.6 release polishes the custom sorting key behaviour for ClickHouse mirrors in the creation page. Be sure to read this when setting custom sorting keys.

• Create ClickHouse mirror: New sorting key UI and logic by @Amogh-Bharadwaj in #2142
• Alerts for drop mirror by @Amogh-Bharadwaj in #2172
• [alerting/metrics] added gauge/alert for time since last normalize by @heavycrystal in #2153
• Add logs around gRPC requests by @iskakaushik in #2159
• [clickhouse] fix checkTablesEmptyAndEngine sql query by @udev-21 in #2163
• [OPS] incident.io wire-up by @pjhampton in #2147
• [alerting/ui] minor bugfixes and optimizations by @heavycrystal in #2165
• temporal: replace deprecated SearchAttributes with TypedSearchAttributes by @serprex in #2167
• Mirror stats: ensure we dont send nulls by @Amogh-Bharadwaj in #2169
• [postgres] application name has mirror suffix by @heavycrystal in #2150
• remove excluded columns from PrimaryKeyColumns during processing by @heavycrystal in #2170
• fix(sync-flow): alert on destination connector failures by @iamKunalGupta in #2177
• KMS encrypt PEERDB_ENC_KEYS by @serprex in #2179

New Contributors

• @udev-21 made their first contribution in #2163

Full Changelog: v0.18.4...v0.18.6

v0.18.5

What's Changed

• PG Resync Query: Fix syntax by @Amogh-Bharadwaj in #2156
• docs(ui): fix peer setup links by @iamKunalGupta in #2157
• Change to debug log for unsupported message types by @Amogh-Bharadwaj in #2158
• [alerting/metrics] added gauge/alert for time since last normalize by @heavycrystal in #2153
• Add error log entry for maintainpull failed by @Amogh-Bharadwaj in #2129
• Add logs around gRPC requests by @iskakaushik in #2159
• [clickhouse] fix checkTablesEmptyAndEngine sql query by @udev-21 in #2163
• [OPS] incident.io wire-up by @pjhampton in #2147
• chore: update dependencies by @serprex in #2166
• [alerting/ui] minor bugfixes and optimizations by @heavycrystal in #2165
• temporal: replace deprecated SearchAttributes with TypedSearchAttributes by @serprex in #2167

New Contributors

• @udev-21 made their first contribution in #2163

Full Changelog: v0.18.4...v0.18.5

v0.18.4

What's Changed

• Validate mirror: fix get table schema by @Amogh-Bharadwaj in #2149
• SetupReplication: clean up replicationErr handling by @serprex in #2144
• Add alerting for bad WAL status by @iskakaushik in #2143
• feat(otel): make metrics namespace configurable by @iamKunalGupta in #2152
• Validate mirror: allow tables empty if initial load is not selected by @Amogh-Bharadwaj in #2097
• [ui/alerting] display state in UI, add appname filter by @heavycrystal in #2154
• PG to PG: Optimize resync query by @Amogh-Bharadwaj in #2155

Full Changelog: v0.18.3...v0.18.4

v0.18.3

What's Changed

• also allow for per column nullable enabled on clickhouse by @serprex in #2138
• remove clickhouse qrep metadata logic by @serprex in #2115
• clickhouse: fix normalizing updates to primary key by @serprex in #2113
• fix: slack invite url by @iamKunalGupta in #2118
• ch: retry queries for some kinds of errors by @serprex in #2119
• [clickhouse] defer dropping table during ValidateMirror by @heavycrystal in #2116
• feat: add supabase oauth state as env var by @iamKunalGupta in #2109
• Replace 32 sync limit with ContinueAsNewSuggested by @serprex in #2014
• ch retries: also retry on io.EOF by @serprex in #2122
• [clickhouse] test for replident full with unchanged TOAST by @heavycrystal in #2123
• Error message fix for CH validate by @Amogh-Bharadwaj in #2124
• SSH host key parsing: use ParseAuthorizedKey by @Amogh-Bharadwaj in #2125
• routes: remove ok/error_message by @serprex in #2128
• [cdc] avoid spamming connections on slot invalidation by @heavycrystal in #2132
• improve CreatePeerNoValidate error message by @serprex in #2133
• Refactor ClickHouse table creation SQL generation by @iskakaushik in #2136
• chore: update dependencies by @serprex in #2137
• version check: use ServerVersion method instead of select version by @serprex in #2139
• [ui] list version in peerinfo for Postgres and ClickHouse peers by @heavycrystal in #2135
• refactor(otel-metrics): use UCUM standards by @iamKunalGupta in #2145

Full Changelog: v0.18.1...v0.18.3

v0.18.2

Merge remote-tracking branch 'origin/main' into stable

v0.18.1

What's Changed

• Clickhouse: nullable fix for normalize in CDC by @Amogh-Bharadwaj in #2106
• Remove TableNameSchemaMapping from temporal state by @serprex in #2090
• RenameTableOptions: remove TableSchema by @serprex in #2107
• PG to Clickhouse: map date to date32 by @Amogh-Bharadwaj in #2108
• [postgres] update end batch time for Postgres destinations by @heavycrystal in #2103
• UI: Drop stats checkbox only drop mirror by @Amogh-Bharadwaj in #2105
• chore: update dependencies by @serprex in #2111
• _peerdb_uid: be uuid by @serprex in #2112

Full Changelog: v0.18.0...v0.18.1

v0.18.0

🚀 What's New in PeerDB

• Edit Mirror Upgrades: 🪞 Updated table removal logic and enabled per-mirror environment updates directly from Edit Mirror. #2025
• Advance LSN Handling: 🏁 The Log Sequence Number (LSN) now advances even when a PKM request is made before receiving any records. #2068
• CDC Efficiency Boosts: 🚀 We now bypass CDC storage when handling logical emit messages, resulting in faster processing. #2080
• API Enhancements: 🛠️ You now have direct access to CDC and initial load stats in the API, giving more transparency into your data processes. #2085
• Limit for CDC Stats: 🎛️ We've added a limit field to control CDC stats requests, offering finer control over your data queries. #2086
• Enhanced CDC Updates: 🔄 Now updating not-present columns from old tuples as well for better CDC data accuracy. #2092

---

🛠 Improvements & Fixes

• Numeric Type Interface: 🔢 Improved numeric type interface and fixed ClickHouse support, enhancing compatibility and usability. #2084
• Kafka Logging Optimization: 📝 Switched Kafka client logging to use activity logger, enhancing logging clarity and efficiency. #2088
• Clickhouse Standardization: 🔗 We standardized ClickHouse naming conventions for smoother operation. #2089
• Merge Tree for Raw Tables: 🌲 Implemented merge tree and adjusted order by for ClickHouse raw tables to improve query performance. #2104
• Dependency Updates & Logs Fixes: 🧹 General maintenance: updated dependencies and fixed multiple "!BADKEY" logs. #2093, #2100

---

🧑‍💻 Developer Experience

• Temporal Version Upgrade: ⏳ Upgraded Temporal version in the Docker stack for better performance and new features. #2081
• Revert Avro File Changes: 🔄 Reverted a previous method to write Avro files to S3 to avoid unnecessary complexity. #2095
• Environment Validation: ✔️ JSON validation added to envString, ensuring more robust environment configurations. #2082

---

Enjoy the improved experience and as always, happy syncing! 😊

v0.17.5

What's Changed

• per mirror alerting by @heavycrystal in #2063
• chore: update dependencies by @serprex in #2062
• drop ORDER BY _peerdb_timestamp in CH normalize statement by @heavycrystal in #2072
• Update normalize batch end time in stats by @iskakaushik in #2076
• feat: Add method to write Avro files to S3 in parts by @iskakaushik in #1770
• Supabase peer by @serprex in #2074
• nexus: don't rewrite single identifier relations by @serprex in #2079
• chore: update dependencies by @serprex in #2077

Full Changelog: v0.17.4...v0.17.5

v0.17.4

What's Changed

• Fix issues with CH qrep by @iskakaushik in #2064

Full Changelog: v0.17.3...v0.17.4