It’s my pleasure to announce the second release of the Debezium 1.8 series, 1.8.0.Alpha2!
With the holiday season just around the corner, the team’s release schedule remains steadfast. While Debezium 1.8.0.Alpha2 delivers quite a lot of bugfixes and minor changes, there are a few notable changes:
MySQL support for heartbeat action queries
Configurable transaction topic name
This release does include several breaking changes. Please see the release notes for details on what changed and how to upgrade.
MySQL heartbeat action query support
A heartbeat action query can be enabled by supplying a
heartbeat.action.query configuration option in the connector’s configuration. This property is meant to supply a SQL statement that the connector will execute periodically.
The initial implementation of the heartbeat action query was specifically for PostgreSQL to handle dealing with WAL growth under specific conditions. But a heartbeat action query has many uses and is entirely connector or even user driven.
For example, you may want to notify downstream consumers that your MySQL topology has changed by supplying consumers with an event with the GTID. The following configuration shows how to capture changes from the heartbeat action query table that can then be consumed easily by your CDC pipeline.
"table.include.list": "gtid_history", "heartbeat.action.query": "INSERT INTO gtid_history( select * from mysql.gtid_executed )"
Configurable transaction topic names
The Debezium transaction metadata topic had previously used a relatively non-configurable naming convention of
<database.server.name>.transaction. While it was possible to manipulate the topic name using a single message transform (SMT) as a workaround, we felt that allowing this to be a bit more flexible in Debezium proper made sense.
A new configuration option,
transaction.topic.prefix, has been introduced that allows the connector configuration to adjust the naming of the transaction metadata topic. The configuration option value specifies what will be used as a direct replacement for the `<database.server.name>~ portion of the topic name. If this configuration option is not supplied, the prior topic naming behavior will continue to be used; requiring no changes for existing connector deployments.
There were quite a number of bugfixes and stability changes in this release, some noteworthy are:
Altogether, 45 issues were fixed for this release. A big thank you to all the contributors from the community who worked on this release: Abhishek Hodavdekar, Alexander Schwartz, Andrey Yegorov, Harvey Yue, Hussain Ansari, Jiabao Sun, Jiri Novotny, Jose Luis, Juan Fiallo, Katerina Galieva, Mike Kamornikov, and Sergei Morozov!
In addition to this release, we also released Debezium 1.7.1.Final, a bugfix update for the 1.7 series. The 1.7.1.Final release includes many of the bugfixes in the 1.8 series that have been done since 1.7.0.Final. For more information on what changed in 1.7.1.Final, please see the release notes.
The holiday season is upon us, but we intend to stick to our release cadence as closely as possible. If you haven’t already taken an opportunity, we’d love your feedback on the open discussion regarding Debezium 2.0 on the mailing list. In the meantime, you can expect the first beta release of 1.8 in a couple of weeks.
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.