Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

The Debezium team is happy to announce the release of Debezium 0.9.3.Final!

This is mostly a bug-fix release and a drop-in replacement for earlier Debezium 0.9.x versions, but there are few significant new features too. Overall, 17 issues were resolved.

Container images will be released with a small delay due to some Docker Hub configuration issues.

New Features

The 0.9.3 release comes with two larger new features:

  • A feature request was made to execute a partial recovery of the replication process after losing the replication slot with the PostgreSQL database, e.g. after failing over to a secondary database host (DBZ-1082). Instead of adding yet another snapshotting mode, we took a step back and decided to make the Postgres snapshotting process more customizable by introducing a service provider interface (SPI). This lets you implement and register your own Java class for controlling the snaphotting process. See the issue description of DBZ-1082 for one possible custom implementation of this SPI, which is based on Postgres' catalog_xmin property and selects all records altered after the last known xmin position. To learn more about the SPI, see the the Snapshotter contract. Note that the feature is still in incubating phase and the SPI should be considered unstable for the time being.

  • Not long ago we published blogpost about implementing the outbox pattern with Debezium for propagating data changes between microservices. Community member Renato Mefi expanded the idea and created a ready-made implementation of the single message transform (SMT) described in the post for routing events from the outbox table to specific topics. This SMT is part of the Debezium core library now (DBZ-1169). Its usage will be described in the documentation soon; for the time being please refer to the EventRouter type and the accompanying configuration class.

Bug fixes

We did a couple of fixes related to the Debezium Postgres connector:

  • A regression that introduced a deadlock in snapshotting process has been fixed (DBZ-1161)

  • The hstore datatype works correctly in snapshot phase (DBZ-1162)

  • The wal2json plug-in processes also empty events (DBZ-1181) as e.g. originating from materialize view updates; this should help to resolve some of the issues where log files in Postgres couldn’t be discarded due to Debezium’s replication slot not advancing.

  • The commit time is propely converted to microseconds (DBZ-1174)

Also the Debezium MySQL connector saw a number of fixes especially in SQL parser:

  • The SERIAL datatype and default value is now supported (DBZ-1185)

  • A specific detail in the MySQL grammar that allows to enumerate table options in ALTER TABLE without comma works (DBZ-1186)

  • A false alarm for empty MySQL password is no longer reported (DBZ-1188)

  • It is no longer necessary to create history topic manually for broker without default topic replication value (DBZ-1179)

It is now possible to process multiple schemas with a single Oracle connector (DBZ-1166).

Check out the release notes for the complete list of issues fixed in Debezium 0.9.3.

Many thanks to Debezium community members Renato Mefi, Shubham Rawat, Addison Higham, Jon Casstevens, Ashar Hassan and Josh Stanfield for their contributions to this release!

Jiri Pechanec

Jiri is a software developer (and a former quality engineer) at Red Hat. He spent most of his career with Java and system integration projects and tasks. He lives near Brno, Czech Republic.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.