As I mentioned last week, we are in the homestretch for the next major release. We’re happy to announce the next candidate release, Debezium 3.0.0.CR2!. Let’s take a quick look at the changes and improvements in this new release…​

New features and improvements

Debezium 3.0.0.CR2 introduces several improvements and features, lets take a look at each individually.

PostgreSQL isolation level support

A longstanding enhancement for snapshot isolation support for PostgreSQL is now here! A new connector configuration property, snapshot.isolation.mode, allows the connector to control the consistency used while executing the initial and ad-hoc blocking snapshot steps. There are four isolation levels: serializable (the default), repeatable_read, read_committed, and read_uncommitted. You can find details about these isolation levels and how they work with PostgreSQL in the documentation (DBZ-1252).

JDBC retry flushes on specific failures

The JDBC sink uses a set of buffers to improve the throughput writes to the target database. In some use cases, the flush operation of these buffers may face specific exceptions due to locks due to other applications that may have locked a specific row or table. To improve the user experience, two new configuration properties have been added:

flush.failure.max.retries

Defines the number of retries when a flush failure occurs.

flush.failure.retries.wait.ms

Defines the number of milliseconds to wait between a retry.

The retry feature is enabled by default, attempting to retry up to a maximum 5 attempts, with a 1-second delay between retries. If you prefer retries disabled, setting flush.failure.max.retries to 0 would disable this feature (DBZ-7291).

Oracle EXTENDED max string size support

Oracle extended strings is a feature that allows the traditional 4000 byte limit on character data to be raised to 32K. This is done by applying a database upgrade to set the database parameter max_string_size to EXTENDED. The extended string feature then allows using the same SQL syntax used for 4000 byte or smaller character data to be used for character data up to 32K without forcing you to use CLOB-based operations.

With Debezium 3, you can now use the Oracle connector with databases that use extended strings and capture the changes directly from the transaction logs (DBZ-8039). As extended strings are effectively CLOB operations on the database level, mining such column types require setting lob.enabled to true.

As this new feature is experimental, we’d love to hear any feedback from the community!

Oracle CLOB/BLOB default value support

In some cases, Oracle users may define tables with a CLOB or BLOB as required, using the EMPTY_BLOB() or EMPTY_CLOB() function to define a default when the field isn’t supplied. In previous builds, these special functions were not evaluated by Debezium, and such columns would have been emitted as optional rather than not optional.

Starting with Debezium 3, when an EMPTY_BLOB() or EMPTY_CLOB() default value is specified, the field will be emitted as not optional. Additionally, the field contain the appropriate default value, an empty byte array or an empty string respectively (DBZ-8248).

Other changes

  • Debezium does not restart automatically after throwing an ORA-00600 krvrdccs30 error DBZ-8223

  • JDBC sink doesn’t include fields as per documentation DBZ-8224

  • Broken link to Streams doc about configuring logging DBZ-8231

  • Document passthrough hibernate.* properties for the JDBC connector DBZ-8232

  • Unbounded number of processing threads in async engine DBZ-8237

  • Streaming metrics are stuck after an ad-hoc blocking snapshot DBZ-8238

  • DDL statement couldn’t be parsed with IF EXISTS DBZ-8240

  • Random engine factory used by default DBZ-8241

  • JDBC sink test suite should use the debezium/connect:nightly image for e2e tests DBZ-8245

  • Bump Infinispan to 15.0.8.Final DBZ-8246

  • AbstractConnectorTest consumeRecordsUntil may prematurely exit loop DBZ-8250

  • Performance Regression in Debezium Server Kafka after DBZ-7575 fix DBZ-8251

  • Add a note to the docs about JDBC batch retry configs DBZ-8252

  • Fix conditionalization in shared MariaDB/MySQL file DBZ-8254

  • Error Prone library included in MySQL connector DBZ-8258

  • Debezium.text.ParsingException: DDL statement couldn’t be parsed DBZ-8259

  • Add Oracle FUTC license DBZ-8260

  • Remove Oracle libs from product assembly package DBZ-8261

  • debezium-connector-binlog does not need MariaDB dependency DBZ-8263

  • Provide subset package for Debezium Server DBZ-8264

Altogether, 44 issues were fixed in this release. Here are a list of some additional noteworthy changes:

A huge thank you to all contributors from the community who worked on this release: Andrei Leibovski, Anisha Mohanty, Chris Cranford, Cody Holmes, Inki Hwang, Jiri Pechanec, Mario Fiore Vitale, Robert Roldan, Vojtěch Juránek, and martin!

What’s next & Outlook?

First and foremost, assuming there are no critical issues, you can expect the final release of Debezium 3 sometime next week. As we begin to shift toward the remainder of 2024, it’s important to discuss what lies ahead for the remainder of this year and 2025.

For the remainder of the 2024 calendar year, the team will continue to deliver maintenance and bugfix releases for Debezium 3. These will continue at our normal cadence, approximately every 2-3 weeks, barring issues reported and those fixed. However, what will be changing this quarter is that there won’t be a minor release in December like years past. This decision is to allow the team to focus on a number of low-hanging fruit activities that don’t necessarily or directly tie to development tasks.

In addition, as we get closer to the end of the year, we’ll be putting out our vision of the 2025 roadmap and what that includes. Now is a perfect time to think about what you’d like to see on that roadmap and get involved. You can always reach us on our mailing list or in our Zulip chat!

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   


About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.