Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

I’m very excited to announce the release of Debezium 1.8.0.CR1!

As were near the final release due out next week, this release focused heavily on bugfixes. Yet this release includes incremental snapshot support for MongoDB! Overall, not less than 34 issues have been fixed for this release.

Let’s take a closer look at some of them.

MongoDB incremental snapshots

Incremental snapshotting is a feature that we first introduced as a part of Debezium 1.6 nearly six months ago. The goals of incremental snapshots is to primarily address to very common user pain-points:

  • the necessity to execute consistent snapshots before streaming can begin upon connector restart

  • inability to trigger full or partial snapshots after connector has begun streaming

The implementation of this feature is based on a novel approach to snapshotting originally invented by the DBLog Framework from Netflix. Debezium’s implementation is described in the design document, and we also published an in-depth blog post discussing our implementation in greater detail.

With this release, we’re excited to finally debut this feature for MongoDB. All Debezium core connectors now support this feature; an amazing milestone!

I’d like to thank our very own Jiri Pechanec and Kate Galieva from Shopify for their amazing efforts these last few months at refining and delivering on this feature as well as the entire community for testing and offering solid feedback.

Further fixes

With 1.8 Final release scheduled for next week, a vast majority of the changes in this release focus on stability and bugfixes. Some resolved issues include:

  • [Oracle] None of log files contains offset SCN (SCN offset is no longer available in the online redo logs) (DBZ-3635)

  • [Oracle] Add support for truncate in Oracle connector (DBZ-4385)

  • [Oracle] Support binary_handling_mode for Oracle connector (DBZ-4400)

  • [Debezium Server] Event Hubs exporter slow/Event data was too large (DBZ-4277)

  • [Vitess] Enforce consistent vgtid representation in Vitess connector (DBZ-4409)

  • [Vitess] VStream gRPC connection closed after being idle for a few minutes (DBZ-4389)

  • Several fixes to DML and DDL parsing for MySQL (DBZ-4397, DBZ-4402) and Oracle (DBZ-4388, DBZ-4396)

Please refer to the release notes to learn more about these and further fixes in this release.

As always, a big thank you to everyone contributing to this release:

Anisha Mohanty, Chris Cranford, Gunnar Morling, Jakub Cechacek, Jiri Pechanec, Mark Drilling, Robert Roldan, Shichao An, Tom Billiet, and Yang Wu.

Outlook

As the year is coming to close, we’re actively preparing some holiday treats!

You can expect 1.7.2.Final to be released early next week including many bugfixes and improvements. Additionally, we intend to release 1.8.0.Final in the middle of next week barring no unforeseen bug reports with CR1.

After the holiday break, we plan to be back in full swing on Debezium 1.9. Keep at eye on our road map as we’ll be updating this to include our focus for next quarter’s release cycle.

We’re also actively working on the planning and scope of Debezium 2.0 which we intend to release sometime in 2022. We would love your feedback on any features or changes you’d like to see so join the discussion on this topic on the mailing list.

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.