Debezium Blog

Welcome to the Debezium community newsletter in which we share all things CDC related including blog posts, group discussions, as well as StackOverflow questions that are relevant to our user community.

History is in the making as Debezium begins to sprint to its 1.0 milestone. It’s my pleasure to announce the release of Debezium 1.0.0.Beta1!

This new Debezium release includes several notable new features, enhancements, and fixes:

  • ExtractNewDocumentState and EventRouter SMTs propagate heartbeat & schema change messages (DBZ-1513)

  • Provides alternative mapping for INTERVAL columns via interval.handling.mode (DBZ-1498)

  • Ensure message keys have the right column order (DBZ-1507)

  • Warn of table locking problems in connector logs (DBZ-1280)

Let’s talk about TOAST. Toast? No, TOAST!

So what’s that? TOAST (The Oversized-Attribute Storage Technique) is a mechanism in Postgres which stores large column values in multiple physical rows, circumventing the page size limit of 8 KB.

TOAST!

Typically, TOAST storage is transparent to the user, so you don’t really have to care about it. There’s an exception, though: if a table row has changed, any unchanged values that were stored using the TOAST mechanism are not included in the message that Debezium receives from the database, unless they are part of the table’s replica identity. Consequently, such unchanged TOAST column value will not be contained in Debezium data change events sent to Apache Kafka. In this post we’re going to discuss different strategies for dealing with this situation.

On behalf of the Debezium community it’s my great pleasure to announce the release of Debezium 0.10.0.Final!

As you’d expect it, there were not many changes since last week’s CR2, one exception being a performance fix for the pgoutput plug-in of the Postgres connector, which may have suffered from slow processing when dealing with many small transactions in a short period of time (DBZ-1515).

This release finalizes the work of overall eight preview releases. We have discussed the new features and changes in depth in earlier announcements, but here are some highlights of Debezium 0.10:

It is a common requirement for business applications to maintain some form of audit log, i.e. a persistent trail of all the changes to the application’s data. If you squint a bit, a Kafka topic with Debezium data change events is quite similar to that: sourced from database transaction logs, it describes all the changes to the records of an application. What’s missing though is some metadata: why, when and by whom was the data changed? In this post we’re going to explore how that metadata can be provided and exposed via change data capture (CDC), and how stream processing can be used to enrich the actual data change events with such metadata.

back to top