Skip to content

Latest commit

 

History

History
21 lines (11 loc) · 824 Bytes

scalable_incremental_data_aggregation_on_postgres_and_citus.md

File metadata and controls

21 lines (11 loc) · 824 Bytes

Source

Medium by Marco Slot

tl;dr

  • Rollup table exists in postgresql

  • We assume all events have an identifier which is drawn from a sequence and provide a simple SQL function that enables you to incrementally aggregate ranges of sequence values in a safe, transactional manner. => this is really fucking hard and right now postgresql can't handle it

  • Need to wait for in-progress writes to finish by using lock

  • This solution really put emphasis on Availability and perfo over consistency

  • BRIN index is performant on sequence

  • Really put lot of marketing for citus perf but what we saw is still relevant for normal postgresql database

Afterword

Would like to see this technique but with the new pub sub of postgresql