The MongoDB connector explicitly preferred to use the secondary under specific scenarios previously. This explicit usage created problems for users who wanted to connect to the primary node. Thanks to recent changes (DBZ-6521), this is no longer the case and the connection string setting is used instead.

The Vitess connector’s change event structure has been slightly adjusted thanks to changes (DBZ-6617). The change event’s source information block now includes a new field that identifies the shard the event originated from.

Users often express the need to manipulate connector offsets for various reasons. This can often be very difficult for those who may not be familiar with Kafka’s CLI tools or Java if you use Debezium Server. Thanks to a contribution (DBZ-6338) by Nathan Smit, you can now use an editor to manipulate the offsets from the command line or a web-based interface.

Head to our examples repository and follow the README.md to get started.

Some Debezium connectors previously used a connector property, errors.max.retries. This property controlled how often a Debezium connector failure exception would be explicitly wrapped in a RetriableException but the connector threw the raw exception up to the runtime. While this may sound similar to Kafka Connect’s errors.retry.timeout, this effectively gave users a common way to deal with retries across multiple Debezium runtimes, including Kafka Connect, Debezium Server, and Debezium Embedded.

With this release, DBZ-6573 unifies this behavior making it available to all connectors.

Debezium’s new notification subsystem provides an easy way to integrate third-party tools and applications with Debezium to gain insight into the ongoing change data capture process, above and beyond the traditional JMX approach. In 2.4, the notification subsystem now includes the ability to notify you about the status of the ongoing initial snapshot (DBZ-6416).

Initial snapshot notifications are emitted with an aggregatetType of Initial Snapshot and contain a type field that exposes the current status of the snapshot. The possible values include: STARTED, ABORTED, PAUSED, RESUMED, IN_PROGRESS, TABLE_SCAN_COMPLETED, and COMPLETED.

Thanks to a contribution provided by Harvey Yue (DBZ-6472), the MySQL connector will use parallelization to generate schema events during its snapshot phase. This should improve the overall performance when capturing the schema for many tables in your database. We plan to investigate how this can be extended to other relational connectors.

The MongoDB connector continues to see lots of active development. This release introduces several new features specifically for MongoDB, which include:

As we continue to make further improvements to the MongoDB connector, please let us know if there are still rough edges or enhancements that will help streamline its usage.

Debezium 2.4 supports several new Oracle data types, which include XML_TYPE and RAW (DBZ-3605). Two new Oracle dependencies were necessary to support XML: xdb and xmlparserv2. These dependencies are not redistributable, so they’re not included in the connector plugin archive by default, much like the connector’s driver. You must obtain these directly from Maven Central or oracle, just like the driver dependency.

In addition, XML works similarly to CLOB and BLOB data types; therefore, the connector must be configured with lob.enabled set to true to ingest XML changes. We’d love to hear your feedback on this new feature as it’s been requested for quite some time.

Thanks to a contribution from Nicholas Fwang (DBZ-6595), the JDBC sink connector can now reference values from the change event’s source information block as a part of the connector’s configuration property table.name.format. If you want to reference such fields, simply use ${source.<field-name>} in the configuration, and the field’s value will be used.

In addition, Roman Kudryashov also contributed the ability to resolve a row’s primary key from a header defined on the change event (DBZ-6602). To use this new feature, specify the connector configuration property primary.key.mode as record_header. If the header value is a primitive type, you will need to define a primary.key.fields configuration similar to how you would if the event’s record key was a primitive. If the header value is a struct type, all fields of the structure will be used by default, but specifying the primary.key.fields property allows you to choose a subset of fields from the header as the key.

It was possible due to certain conditions that a Spanner connector may not advance from the START_INITIAL_SYNC state during initialization. After investigation by Nancy Xu, a new configuration option was introduced to supply a configurable timeout. This can be done by adding the following to the connector’s configuration:

The Debezium UI project allows you to easily deploy any Debezium connector onto Kafka Connect using a web-based interface. This release has improved the interface by including several connector metrics (DBZ-5321) on the main connector listing view. We’d love your feedback on this change and welcome any suggestions on other metrics you may find useful.