It is my pleasure to announce the immediate release of Debezium 2.3.2.Final.
This release includes several bug fixes to address regressions, stability, documentation updates. If you are currently looking to upgrade to the Debezium 2.3.x release stream, we highly recommend you consider using this release. Let’s take a quick look into the regressions and bug fixes.
SQL Server refuses to start
If you have recently tried to upgrade to Debezium 2.3.1.Final, you may have found when using the SQL Server connector that you received an unusual error when starting the connector which said, "Configuration
query.fetch.size is defined twice."
Unfortunately, this error was not intended and there is no workaround to remedy the issue. Thankfully, Debezium 2.3.2.Final is here to the recue; this release addresses this regression, allowing SQL Server connectors to start once again. If you are looking to upgrade, and you rely on the SQL Server connector, we strongly recommend that you avoid the 2.3.1.Final build and instead move directly to 2.3.2.Final.
Oracle default fetch size changed
Debezium uses JDBC in order to communicate with the Oracle database. The Debezium for Oracle connector relies on two configuration properties,
query.fetch.size to control how much data is returned for a query on each database "fetch" call.
When these properties are configured too low, this can cause Debezium to perform more network round trips to the database to read data and that network latency can add up, particularly when working with large result sets. When these properties are configured too high, this can cause Debezium to consume more memory, but reduces the network latency incurred for the fetch round trips to the database. Ultimately, it’s important to strike a good balance based both on the what your ideal data size may be but also based on the memory and hardware constraints of your environment.
While discussing performance with one community member, we concluded that adjusting these values from their default of
10000 increased the connector’s throughput quite substantially for their environment. So in this release, we felt it made logical sense to consider increasing the default to
10000 to provide a better out-of-the-box experience for Oracle connector users.
Now, these configuration properties are performance tuning knobs, and unfortunately there isn’t a guarantee that what works well for some environments is going to necessarily be universally good. Please take note of this change and if you experience any issues, you can always set the
query.fetch.size properties in your connector configuration, even setting them back to their previous default of
2000 if necessary.
Debezium 2.3.2.Final also includes quite a number of bug fixes and stability improvements, see below:
Highlight information about how to configure the schema history topic to store data only for intended tables DBZ-6219
Should use topic.prefix rather than connector.server.name in MBean namings DBZ-6690
Upstream documentation missing types for configurations DBZ-6707
Custom properties step not working correctly in validation of the properties added by user DBZ-6711
Oracle fails to process a DROP USER DBZ-6716
Oracle LogMiner mining distance calculation should be skipped when upper bounds is not within distance DBZ-6733
MariaDB: Unparseable DDL statement (ALTER TABLE IF EXISTS) DBZ-6736
Decouple Debezium Server and Extension Quarkus versions DBZ-6744
MySQL dialect does not properly recognize non-default value longblob types due to typo DBZ-6753
Please refer to the release notes to learn more about all fixed bugs, update procedures, etc.
Outlook and what’s next?
A great deal of work has already gone into the new preview release of Debezium 2.4. We plan to do the next Alpha2 build in the middle of next week, which will include a plethora of new features and improvements. There is still time to share your feedback and suggestions if there are things you’d like to see in 2.4, so take a look at our road map and reach out on the mailing list or our chat.
Finally, Debezium 2.3 will continue to receive maintenance updates. We’ll likely release 2.3.3.Final later in the month barring the community feedback on regressions and bug fixes.
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.