While we remain on track with the upcoming Debezium 2.6 release, we are continuously looking at improvements and fixes that are being made that ensures that older releases continue to provide the best possible experience. With that, the team is pleased to announce the first maintenance release of Debezium 2.5, Debezium 2.5.1.Final. Let’s dive into what this release includes and what you should be aware of while upgrading…
The team aims to avoid any potential breaking changes between minor releases; however, such changes are sometimes inevitable.
- Re-select Columns Post Processor
The re-select columns post processor used the key based on the
message.key.columnswhen building the query. This is not correct for most tables with primary keys. The default behavior has changed and the table primary key is used by default. A new configuration option was introduced to allow the user to choose between whether to use the primary key or the generated key,
Improvements and changes
MongoDB post-image changes
The MongoDB connector’s event payload can be configured to include the full document that was changed in an update. The connector previously made an opinionated choice about how the full document would be fetched as part of the change stream; however, this behavior was not consistent with our expectations in all use cases.
Debezium 2.5+ introduces a new configuration option,
capture.mode.full.update.type, allowing the connector to explicitly control how the change stream’s full document lookup should be handled (DBZ-7299). The default value for this option is
lookup, meaning that the database will make a separate look-up to fetch the full document. If you are working with MongoDB 6+, you can also elect to use
post_image to rely on MongoDB change stream’s post-image support.
Altogether, 21 issues were fixed in this release:
Connector is getting stopped while processing bulk update(50k) records in debezium server 2.0.1.Final DBZ-6955
Empty object sent to GCP Pub/Sub after DELETE event DBZ-7098
Debezium-ddl-parser crashes on parsing MySQL DDL statement (sub-query with UNION) DBZ-7259
Oracle DDL parsing error in PARTITION REFERENCE DBZ-7266
PostgreSQL ad-hoc blocking snapshots fail when snapshot mode is "never" DBZ-7311
Ad-hoc blocking snapshot dies with "invalid snapshot identifier" immediately after connector creation DBZ-7312
Specifying a table include list with spaces between elements cause LogMiner queries to miss matches DBZ-7315
Debezium heartbeat.action.query does not start before writing to WAL: part 2 DBZ-7316
Oracle connector is occasionally unable to find SCN DBZ-7345
Initial snapshot notifications should use full identifier. DBZ-7347
Debezium fails after table split operation DBZ-7360
MSSQL wrong default values in db schema for varchar, nvarchar, char columns DBZ-7374
Replace additional role binding definition in kubernetes.yml with @RBACRule DBZ-7381
Kinesis Sink Exception on PutRecord DBZ-7417
ParsingException (MariaDB Only): alterSpec drop foreign key with 'tablename.' prefix DBZ-7420
Outlook & What’s next?
The team remains dedicated toward Debezium 2.6 over the next few months. However, in that time, you can expect that we’ll continue to review regressions and bugfixes, and provide maintenance releases for Debezium 2.5.
In the meantime, if you’re interested in what the team has in store for Debezium 2.6 and beyond throughout 2024, we encourage you to review our road map, which includes details about our path to Debezium 3.0 later this year. If you have any suggestions or ideas, please feel free to get in touch with us on our mailing list or in our Zulip chat.
We have two upcoming talks later this week at Fosdem 2024 in Brussels. Ondrej Babec will be speaking about performance testing and why it’s important and Vojtech Juranek will use Debezium to feed AI models in real-time. If you plan to attend, be sure to check out these talks and meet and greet with our Debezium engineers.
Until next time…
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.