We are accelerating! Three weeks after the 0.6.1 release, the Debezium team is bringing Debezium 0.6.2 to you!
This release revolves mostly around bug fixes, but there are a few new features, too. Let’s take a closer look at some of the changes.
The big news for the Postgres connector is that Debezium now runs against PostgreSQL 10 thanks to a contribution from Scofield Xu. As a part of this change we are providing a Docker Image with PostgreSQL 10, too, and we have set up a daily run of our integration tests against it.
If you are building Postgres yourself using the Debezium logical decoding plug-in, you can save quite some megabytes if you don’t need the PostGIS geometric extension: thanks to the work by Danila Kiver, it’s now possible to omit that extension.
We’ve received multiple reports related to parsing MySQL DDL statements, e.g. there were a few specific invocations of the
ALTER TABLE statement which weren’t handled correctly. Those as well as a few other parser bugs have been fixed.
If you work with the
TIMESTAMP column type and your Kafka Connect server isn’t using UTC as timezone, then the fix for DBZ-260 is applying to you. In that case, the ISO 8601 formatted String emitted by Debezium would have, incorrectly, contained the UTC date and time plus the zone offset (as per the time zone the Kafka Connect server is located in) before. Whereas now it will contain the date and time adjusted to the zone offset. This may require adjustments to to downstream consumers if they were relying on the previous, incorrect behavior.
DBZ-217 gives you more flexibility for handling corrupt events encountered in the MySQL binlog. By default, the connector will stop at the problematic event in such case. But you now also have the option to just log the event and its position and continue the processing after it.
Another nice improvement for the MySQL connector is a much reduced CPU load after the snapshot has been completed, when using the "snapshot only" mode (DBZ-396).
This connector received an important fix applying when more than one thread is used to performing the initial snapshot (DBZ-438). Before, it could happen that single messages got lost during snapshotting which is fixed now.
Examples and Docker Images
We have expanded our examples repository with an Avro example, which may be interesting to you if you’d like to not work with JSON messages but rather the compact Avro binary format and the Confluent schema registry.
As a part of our release process we are now creating micro tags for our Docker images for every released version. While tags in the format
x.y.z are fixed in time, tags in the format
x.y are rolling updates and always point to the latest micro release of that image.
Please see the full change log for more details and the complete list of fixed issues.
The Debezium 0.7 release is planned to be out in two to three weeks from now.
It will contain the move to Apache Kafka 1.0.0 and bring support for the wal2json logical decoding plug-in for Postgres. This will eventually allow to use the Debezium Postgres connector on Amazon RDS (once the correct wal2json version is available there).
In parallel, the work around handling updates to the whitelist configuration of the MySQL connector continues (it may be ready for 0.7.0), and so does the work on the Oracle connector (which will be shipping in a future release).
If you’d like to contribute, please let us know. We’re happy about any help and will work with you to get you started quickly. Check out the details below on how to get in touch.
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.