Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

I’m thrilled to announce the release of Debezium 1.5.0.Final!

With Debezium 1.5, the LogMiner-based CDC implementation for Oracle moves from Incubating to Stable state, and there’s a brand-new implementation of the MySQL connector, which brings features like transaction metadata support. Other key features include support for a new "signalling table", which for instance can be used to implement schema changes with the Oracle connector, and support for TRUNCATE events with Postgres. There’s also many improvements to the community-led connectors for Vitess and Apache Cassandra, as well as wide range of bug fixes and other smaller improvements.

Across all the 1.5 preview and the final releases, a grand total of 236 issues has been addressed.

For more details, please see the earlier announcements for the 1.5.0 Alpha1, Beta1, Beta2, and CR1 releases.

Since the CR1 release, we’ve primarily focused on documentation improvements and some bug fixes. But there are two last-minute feature additions, too:

  • Support for Redis Streams in Debezium Server (DBZ-2879), allowing to propagate Debezium data change events into Redis-based logs

  • Provide LSN coordinates as standardized sequence field in Postgres change events (DBZ-2911), allowing consumers to identify duplicated events and exclude them from processing, for instance after an un-clean connector shut-down; this field will be added to the source block of other connectors going forward, too

Please refer to the release notes of Debezium 1.5.0.Final for the complete list of resolved issues as well as procedures for upgrading to earlier versions.

As always, a big thank you to all the members from the community who helped with this release, be it via code contributions, bug report, testing, providing insight and expertise, etc. In particular for the LogMiner-based CDC implementation for Oracle, we’ve received a huge number of contributions of all kinds. We’re deeply grateful for that and look forward a lot to further grow and improve this connector implementation! Kudos to the following individuals from the community which contributed to Debezium 1.5, bringing the overall number of contributors to the Debezium core repository to 253:

Adhaam Ehab, Ahmed Eljami, Bingqin Zhou, Dave Cramer, David Seapy, Fahim Farook, Frank Koornstra, Ganesh Ramasubramanian, Harvey Yue, Hoa Le, Jeremy Vigny. Kewei Shang, Kevin Pullin, Martín Pérez, Martin Sillence, Meng Qiu, Michael Cizmar, Naveen Kumar, Nick Murray, Nitin Agarwal, Russell Ballard, Shane Paul, Thomas Prelle, Thomas Thornton, Victar Malinouski, Vladimir Osin, Yilong Chang, and Xiao Fu!

Outlook

Following our quarterly release cadence, Debezium 1.6 is planned for the end of June. A key issue we’re planning to work on for this version is explorations of how to improve the notion of initial snapshots, where we plan to touch on topics like resumeability, parallelization, changes of filter configuration, and more. This is going to be an open-ended investigation, but we hope to have at least a proof-of-concept implementation for some of these features, which constantly show up high on the wish list of Debezium users.

Another focus area will again be the Debezium connector for Oracle, where we have planned several functional and performance improvements. We are also discussing to move to Java 11 as a minimum baseline for running Debezium. This is primarily caused by external dependencies which are moving on from Java 8. In case you have specific questions or potential concerns around this change, please chime into the discussion. Also, if you got specific feature requests or other input for the roadmap and future releases, please let us know via the mailing list!

Gunnar Morling

Gunnar is a software engineer at Decodable and an open-source enthusiast by heart. He has been the project lead of Debezium over many years. Gunnar has created open-source projects like kcctl, JfrUnit, and MapStruct, and is the spec lead for Bean Validation 2.0 (JSR 380). He’s based in Hamburg, Germany.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.