It’s my pleasure to announce the second release of the Debezium 1.9 series, 1.9.0.Alpha2!
This release includes support for Oracle 21c, improvements around Redis for Debezium Server, configuring the
kafka.query.timeout.ms option, and a number of bug fixes around DDL parsers, build infrastructure, etc.
Overall, the community fixed 51 issues for this release. Let’s take a closer look at some of the highlights.
Support for Oracle 21c
The Debezium Oracle connector has been tested with the latest release of Oracle 21c, 126.96.36.199, and is compatible. If you use either the LogMiner or the Xstreams adapter, you should now be able to use Oracle’s latest flagship version and stream change events without any changes. If you are on Oracle 12 or Oracle 19 and perform a database upgrade, your connector configuration should require no changes and remain compatible.
When using the Kafka Admin Client and issuing API calls, the default timeout is 3 seconds. The new
kafka.query.timeout.ms field can be used to provide a custom timeout to the Kafka Admin Client to avoid possible timeout problems in environments that may use TLS or SSL encryption or where network latency causes an unexpected timeout.
Thanks to the great work done by community member, Snigdhajyoti Ghosh.
Improvements in Redis for Debezium Server
We have three new fields in the Redis support for Debezium Server
Redis allows specifying a maximum memory limit using the
maxmemory configuration; however, if this field is not configured then Redis will continue to allocate memory. If all memory is consumed, an OutOfMemory exception occurs. The Redis sink now uses
redis.retry.max.delay.ms to set an initial and max-retry delay to be more resilient to this and connection-related issues. If you have or are experiencing such exceptions, we urge you to try these new settings to improve the sink’s resilience and experience.
Pipeline-based transactions can substantially increase Redis queries. In order to leverage pipeline-based transactions, the
batch.size configuration option can be specified which will allow Redis to write batches of change records rather than each record one by one.
Thanks to Yossi Shirizli, for these amazing improvements.
Some notable bug fixes and upgrades are:
Suspected inconsistent documentation for 'Ad-hoc read-only Incremental snapshot' DBZ-4171
Oracle Logminer: snapshot→stream switch misses DB changes in ongoing transactions DBZ-4367
DDL parsing issue: ALTER TABLE … MODIFY PARTITION … DBZ-4649
OracleSchemaMigrationIT fails with Xstream adapter DBZ-4703
Migrating UI from webpack-dev-server v3 to v4 DBZ-4642
Upgrade postgres driver to version 42.3.2 DBZ-4658
Upgrade to Quarkus 2.7.0.Final DBZ-4677
Update shared UG deployment file for use with downstream OCP Install Guide DBZ-4700
Indicate ROWID is not supported by XStream DBZ-4702
Incremental snapshots does not honor column case sensitivity DBZ-4584
Build trigger issues DBZ-4672
Cannot expand JSON payload with nested arrays of objects DBZ-4704
We will also be backporting the critical bugfixes to the 1.8 branch and will release Debezium 1.8.1.Final later this week.
A big thank you to all the contributors from the community who worked on this release: Andrei Isac, Anisha Mohanty, Bob Roldan, Chris Cranford, David Haglund, Dominique Chanet, Farid Uyar, Gunnar Morling, Harvey Yue, Ismail Simsek, Jason Schweier, Jiri Novotny, Jiri Pechanec, Mark Drilling, Nathan Smit, Paweł Malon, René Kerner, Shichao An, Snigdhajyoti Ghosh, Vojtech Juranek, and Yossi Shirizli!
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.