Readit News logoReadit News
dgea005 commented on Launch HN: PeerDB (YC S23) – Fast, Native ETL/ELT for Postgres    · Posted by u/saisrirampur
dgea005 · 2 years ago
Looks awesome! Maybe I missed it in the docs but I wonder how you solved the problem of a source table's initial snapshot when performing CDC (locking a large table on a production DB can be problematic).

Did you implement something like described here? https://netflixtechblog.com/dblog-a-generic-change-data-capt...

Lastly, I wonder what plugin you're using for the Postgres logical replication?

Congrats!

Edit: re-read it and saw about the DuckDB blog's inspired ctid solution for the initial snapshot. Do you find querying by ctid okay performance wise? As far as I remember it will use sequential scans

u/dgea005

KarmaCake day4July 28, 2023View Original