Featured Posts

Tags
UI WebAssembly ai analytics announcement announcement discussion survey apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc cdc tables channels charts chicory cockroachdb community community stories connectors containers core cqrs custom data data engineering data replication datalake db2 ddd debezium debezium server debezium ui debugging deduplication demo discussion dlt dlthub docker elasticsearch embedded engine event sourcing exactly once semantics example examples features fedora flamegraphs flink go graalvm hiring ibmi iceberg images informix integration internals introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse lineage machine learning mandrel mariadb metrics microservices mongo mongodb monitoring mysql native news newsletter notifications online learning openlineage operator oracle outbox performance platform postgres presentation production pydbzengine python quarkus questdb rag rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb tinygo topics tracing transactions troubleshooting tx log ui vagrant vitess wasm website

Debezium Blog

SQL Server offers change data capture (CDC) through change tables - a special set of system tables that record modifications in selected “ordinary” tables. If you want to monitor changes in real time, you query these change tables periodically. That’s exactly how Debezium works today: it polls SQL Server’s change tables at configured intervals and turns the results into a continuous stream of CDC records. This approach works fine, but could we do better?

Captured tables are filled by the SQL Server Agent, which reads the transaction log, extracts changes, and stores them in change tables. In theory, we could skip the middleman and parse the transaction log directly. That’s how tools like OpenLogReplicator handle CDC for Oracle databases. Let’s peek inside the SQL Server internals and explore little bit how it works and stores the records.

In this post, we’ll:

  • Prepare a local SQL Server instance for experimentation

  • Explore the internal structure of the SQL Server transaction log

  • Understand how the records are stored on the disk

Read More    
×