Featured Posts

Tags
analytics announcement announcement discussion survey apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg images informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb metrics microservices mongo mongodb monitoring mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions ui vagrant vitess website

Just shy of a month after the 0.6.0 release, I’m happy to announce the release of Debezium 0.6.1!

This release contains several bugfixes, dependency upgrades and a new option for controlling how BIGINT UNSIGNED columns are conveyed. We also expanded the set of Docker images and Docker Compose files accompanying our tutorial, so you can run it now with all the databases we support.

Let’s take a closer look at some of the changes.

New connector option for controlling BIGINT UNSIGNED representation

BIGINT UNSIGNED columns from MySQL databases have been represented using Kafka Connect’s Decimal type until now. This type allows to represent all possible values of such columns, but its based on a byte array, so it can be a bit cumbersome to handle for consumers. Therefore we added a new option named bigint.unsigned.handling.mode to the MySQL connector that allows to represent such columns using long.

For the very most cases that’s the preferable option, only if your column contains values larger than 2^63 (which MySQL doesn’t recommend due to potential value losses when performing calculations), you should stick to the Decimal representation.

Using long will be the default as of Debezium 0.7, for the 0.6.x timeline we decided to go with the previous behavior (i.e. using Decimal) for the sake of backwards compatibility.

Thanks a lot to Ben Williams who contributed this feature!

New example Docker images and Docker Compose files

In the Debezium examples repository we now provide Docker Compose files which let you run the tutorial with all the three databases we currently support, MySQL, Postgres and MongoDB.

Just choose the Compose file for your preferred database and get a all the required components (ZooKeeper, Apache Kafka, Kafka Connect and the database) running within a few seconds.

We’ve also deployed Docker images for Postgres and MongoDB to the Debezium organization on Docker Hub, so you got some data to play with.

Version upgrades

We’ve upgraded our images from Kafka 0.11.0.0 to 0.11.0.1. Also the binlog client library used by the MySQL connector was upgraded from 0.9.0 to 0.13.0.

Bugfixes

Finally, several bugs were fixed in 0.6.1. E.g. you can now name a column column in MySQL (DBZ-408), generated DROP TEMP TABLE statements won’t flood the DB history topic (DBZ-295) and we’ve fixed a case where the Postgres connector would stop working due to an internal error but fail to report though via the task/connector status (DBZ-380).

Please see the full change log for more details and the complete list of fixed issues.

What’s next?

The work on Debezium 0.7 has already begun and we’ve merged the first set of changes. You can expect to see support for using the wal2json logical decoding plug-in with the Postgres connector, which will finally allow it to use Debezium with Postgres on Amazon RDS! We’ve also started our explorations of providing a connector for Oracle (DBZ-20) and hope to report some progress here soon.

While the work on Debezium 0.7 continues, you will likely continue to see one or more 0.6.x bugfix releases. We’ve automated the release process as much as possible, making it a breeze to ship a new release and getting fixes into your hands quickly.

If you’d like to contribute, please let us know. We’re happy about any help and will work with you to get you started quickly. Check out the details below on how to get in touch.

Gunnar Morling

Gunnar is a software engineer at Decodable and an open-source enthusiast by heart. He has been the project lead of Debezium over many years. Gunnar has created open-source projects like kcctl, JfrUnit, and MapStruct, and is the spec lead for Bean Validation 2.0 (JSR 380). He’s based in Hamburg, Germany.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.