Debezium Blog

Did you know December 12th is National Ding-a-Ling Day? It’s the day to call old friends you haven’t heard from in a while. So we thought we’d get in touch (not that is has been that long) with our friends, i.e. you, and share the news about the release of Debezium 1.0.0.CR1!

It’s the first, and ideally only, candidate release; so Debezium 1.0 should be out very soon. Quite a few nice features found their way into CR1:

While fall weather is in full swing, the Debezium community is not letting the unusually low, frigid temperatures get the best of us. It is my pleasure to announce the release of Debezium 1.0.0.Beta3!

This new Debezium release includes several notable new features, enhancements, and fixes:

  • Built against Kafka Connect 2.3.1 (DBZ-1612)

  • Renamed drop_on_stop configuration parameter to drop.on.stop (DBZ-1595)

  • Standardized source information for Cassandra connector (DBZ-1408)

  • Propagate MongoDB replicator exceptions so they are visible from Kafka Connect’s status endpoint (DBZ-1583)

  • Envelope methods should accept Instant rather than long values for timestamps (DBZ-1607)

  • Erroneously reporting no tables captured (DBZ-1519)

  • Avoid Oracle connector attempting to analyze tables (DBZ-1569)

  • Toasted columns should contain null in before rather than __debezium_unavailable_value (DBZ-1570)

  • Support PostgreSQL 11+ TRUNCATE operations using pgoutput decoder (DBZ-1576)

  • PostgreSQL connector times out in schema discovery for databases with many tables (DBZ-1579)

  • Value of ts_ms is not correct duing snapshot processing (DBZ-1588)

  • Heartbeats are not generated for non-whitelisted tables (DBZ-1592)

It is my pleasure to announce the release of Debezium 1.0.0.Beta2!

This new Debezium release includes several notable new features, enhancements, and fixes:

  • Support PostgreSQL LTREE columns with a logical data type (DBZ-1336)

  • Support for PostgreSQL 12 (DBZ-1542)

  • Validate configured PostgreSQL replication slot not contains no invalid characters (DBZ-1525)

  • Add MySQL DDL parser support for index creation VISIBLE and INVISIBLE keywords (DBZ-1534)

  • Add MySQL DDL parser support for granting SESSION_VARIABLES_ADMIN (DBZ-1535)

  • Fix MongoDB collection source struct field when collection name contains a dot (DBZ-1563)

  • Close idle transactions after performing a PostgreSQL snapshot (DBZ-1564)

History is in the making as Debezium begins to sprint to its 1.0 milestone. It’s my pleasure to announce the release of Debezium 1.0.0.Beta1!

This new Debezium release includes several notable new features, enhancements, and fixes:

  • ExtractNewDocumentState and EventRouter SMTs propagate heartbeat & schema change messages (DBZ-1513)

  • Provides alternative mapping for INTERVAL columns via interval.handling.mode (DBZ-1498)

  • Ensure message keys have the right column order (DBZ-1507)

  • Warn of table locking problems in connector logs (DBZ-1280)

Let’s talk about TOAST. Toast? No, TOAST!

So what’s that? TOAST (The Oversized-Attribute Storage Technique) is a mechanism in Postgres which stores large column values in multiple physical rows, circumventing the page size limit of 8 KB.

TOAST!

Typically, TOAST storage is transparent to the user, so you don’t really have to care about it. There’s an exception, though: if a table row has changed, any unchanged values that were stored using the TOAST mechanism are not included in the message that Debezium receives from the database, unless they are part of the table’s replica identity. Consequently, such unchanged TOAST column value will not be contained in Debezium data change events sent to Apache Kafka. In this post we’re going to discuss different strategies for dealing with this situation.

back to top