Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

I’m delighted to announce the release of Debezium 0.9 Final!

This release only adds a small number of changes since last week’s CR1 release; most prominently there’s some more metrics for the SQL Server connector (lag behind master, number of transactions etc.) and two bug fixes related to the handling of partitioned tables in MySQL (DBZ-1113) and Postgres (DBZ-1118).

Having been in the works for six months after the initial Alpha release, Debezium 0.9 comes with a brand new connector for SQL Server, lots of new features and improvements for the existing connectors, updates to the latest versions of Apache Kafka and the supported databases as well as a wide range of bug fixes.

Some key features of the release besides the aforementioned CDC connector for SQL Server are:

  • Initial snapshotting for the Oracle connector (which remains to be a "tech preview" at this point)

  • Brand-new metrics for the SQL Server and Oracle connectors and extended metrics for the MySQL connector

  • Field filtering and renaming for MongoDB

  • A new handler interface for the embedded engine

  • Lots of improvements around the "event flattening" SMT for MongoDB

  • More detailed source info in CDC events and optional metadata such as a column’s source type

  • Option to delay snapshots for a given time

  • Support for HSTORE columns in Postgres

  • Incubating support for picking up changes to the whitelist/blacklist configuration of the MySQL connector

As a teaser on the connector metrics support, here’s a screenshot of Java Mission Control displaying the SQL Server connector metrics:

Monitoring the Debezium SQL Server connector

The list above is far from being exhaustive; please take a look at the preview release announcements (Alpha1, Alpha2, Beta1, Beta2 and CR 1) as well as the full list of a whopping 176 fixed issues in JIRA.

It’s hard to say which of the changes and new features I’m most excited about, but one thing surely sticking out is the tremendous amount of community work on this release. Not less than 34 different members of Debezium’s outstanding community have contributed to this release. A huge and massive "Thank You!" to all of you:

Addison Higham, Andrey Pustovetov, Anton Martynov, Amit Sela, Artiship Artiship, Cliff Wheadon, Deepak Barr, Eero Koplimets, Gagan Agrawal, Grzegorz Kołakowski, Ian Axelrod, Ilia Bogdanov, Ivan Kovbas, Joy Gao, Kevin Pullin, Lao Mei, Liu Hanlin, Maciej Bryński, Moira Tagle, Olavi Mustanoja, Olivier Lemasle, Ori Popowski, Peng Lyu, Peter Larsson, Philip Sanetra, Renato Mefi, Sagar Rao, Shivam Sharma, Syed Muhammad Sufyian, Tautvydas Januskevicius, Tapani Moilanen, Tony Rizko Wout Scheepers and Zheng Wang.

When upgrading from earlier Debezium releases, please make sure to read the information regarding update procedures and breaking changes in the release notes. One relevant change to the users of the Debezium connector for MySQL is that our new Antlr-based DDL parser is used by default now. After lots of honing we felt it’s time for using the new parser by default now. While the existing parser can still be used as a fallback as of Debezium 0.9, it will be phased out in 0.10.

Next Steps

After some drinks to celebrate this release, the plan is to do a 0.9.1 release rather quickly (probably in two weeks from now), providing improvements and potential bug fixes to the features and changes done in 0.9. We’ll also begin the work on Debezium 0.10, stay tuned for the details on that!

For further plans beyond that, take a look at our road map. Any suggestions and ideas are very welcomed on mailing list or in the comments below.

If you’re just about to begin using Debezium for streaming changes out of your database, you might be interested in join us for the upcoming webinar on February 7th. After a quick overview, you’ll see Debezium in action, as it streams changes to a browser-based dashboard and more. You can also find lots of resources around Debezium and change data capture such as blog posts and presentations in our curated list of online resources.

Gunnar Morling

Gunnar is a software engineer at Decodable and an open-source enthusiast by heart. He has been the project lead of Debezium over many years. Gunnar has created open-source projects like kcctl, JfrUnit, and MapStruct, and is the spec lead for Bean Validation 2.0 (JSR 380). He’s based in Hamburg, Germany.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.