CDC & Replication — Debezium, Outbox, End-to-End Exactly-Once¶
Every serious data platform eventually faces the same question: the truth lives in an OLTP database — how do we get every insert, update, and delete out of it, in order, without losing anything, without hammering the database, and without paying for a second copy of reality that silently drifts from the first? The answer that has won across the industry is change data capture (CDC): instead of repeatedly asking the database what changed, you subscribe to the database's own crash-recovery log — the append-only journal it already writes to survive power loss — and replay it into downstream systems. This chapter takes you from zero to expert on that idea: why polling-based extraction is structurally lossy, how Postgres logical decoding turns the write-ahead log into a stream of typed row events, how Debezium packages those events into Kafka with a well-defined envelope, how the outbox pattern solves the dual-write problem for microservices, and how to combine at-least-once transport with idempotent application to get an end-to-end effectively-exactly-once pipeline into a data lake. …
🔒 La suite est en accès freemium — lecture complète 100 % gratuite
Tu lis ici l'aperçu libre. Le reste du chapitre (code, schémas, maths, exercices) fait partie du livre complet : crée un compte gratuit (30 secondes, aucun paiement) pour tout lire.