Featured Posts

Tags
analytics announcement announcement discussion survey apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg images informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb metrics microservices mongo mongodb monitoring mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions ui vagrant vitess website

I am pleased to announce the first maintenance release for the Debezium 3 release stream, 3.0.1.Final. This maintenance release introduces several new features including support for Cassandra 5, PostgreSQL 17, and MySQL 9.1. In addition, there are some configuration improvements with Debezium Server supporting YAML.

Breaking changes

We announced the discontinuation of publishing Debezium container images to Docker Hub here and that they are now solely available on quay.io (DBZ-8327). Please be sure to update your container pipelines to continue to use the most up-to-date images.

New features and improvements

Debezium 3.0.1.Final introduces a couple of improvements and features, lets take a look at each individually.

Cassandra 5 support

The Apache Cassandra Community released Cassandra 5 in September 2024, and we are happy to report that Debezium is now compatible and supports the latest major milestone of Cassandra (DBZ-8347).

A new Debezium connector artifact is now available for Cassandra 5:

<dependency>
    <groupId>io.debezium</groupId>
    <artifactId>debezium-connector-cassandra-5</artifactId>
    <version>3.0.1.Final</version>
</dependency>

In addition, the new Cassandra 5 connector is included with Debezium Server.

MySQL 9.1 support

Oracle released MySQL 9.1 just a short weeks ago, and we’re happy to also report that Debezium now supports MySQL 9.1 (DBZ-8424). This update also includes an update to the MySQL JDBC driver version 9.1.0.

For existing users, this should require no additional changes.

PostgreSQL 17 support

The PostgreSQL team released the latest PostgreSQL 17 database at the end of September 2024, and we’re happy to report that Debezium officially supports PostgreSQL 17 (DBZ-8275). For any existing users, this update should require no additional changes.

Using YAML with Debezium Server

Debezium Server is a standalone environment for running Debezium connectors outside of a Kafka Connect environment on bare metal, VMs, or Kubernetes. Debezium Server users are familiar with the configuration property file called application.properties, where you supply the source and sink connector configurations to define your Debezium Server pipeline.

We are excited to share that we’ve added support for YAML when using Debezium Server, allowing you to now define your configuration in application.yml (DBZ-8313). Please review the Quarkus documentation on using YAML configuration should you have any questions.

Other fixes

In total there were 35 issues resolved in Debezium 3.0.1.Final. The list of changes can also be found in our release notes.

Here are some noteworthy changes:

  • Oracle DDL parsing will fail if the DDL ends with a new line character DBZ-7040

  • Missing documentation for MongoDb SSL configuration DBZ-7927

  • Conditionalization implemented for single-sourcing MySQL/MariaDB content isn’t working as expected DBZ-8094

  • Support batch write to AWS Kinesis DBZ-8193

  • Debezium is replaying all events from an older offset DBZ-8194

  • Embedded MySqlConnector "Unable to find minimal snapshot lock mode" since 2.5.4.Final DBZ-8271

  • Reselect Post Processor not working when pkey of type uuid etc. DBZ-8277

  • BinlogStreamingChangeEventSource totalRecordCounter is never updated DBZ-8290

  • Restart Oracle connector when ORA-01001 invalid cursor exception is thrown DBZ-8292

  • Connector uses incorrect partition names when creating offsets DBZ-8298

  • ReselectPostProcessor fails when reselecting columns from Oracle DBZ-8304

  • Debezium MySQL DDL parser: SECONDARY_ENGINE=RAPID does not support DBZ-8305

  • SQL Server Documentation for CDC on Server table DBZ-8314

  • Oracle DDL failure - subpartition list clause does not support in-memory clause DBZ-8315

  • DDL statement couldn’t be parsed DBZ-8316

  • Binary Log Client doesn’t process the TRANSACTION_ PAYLOAD header DBZ-8340

  • Oracle connector: archive.log.only.mode stop working after reach SYSDATE SCN DBZ-8345

A big thank you to all the contributors from the community who worked diligently on this release: Anisha Mohanty, Kunal Bhatnagar, Chris Cranford, Dongwook Chan, Franz Emberger, Gaurav Miglani, Jiri Pechanec, Jonas Thelemann, Katerina Galieva, Mario Fiore Vitale, Nathan Smit, Ondrej Babec, Philippe Labat, Robert Roldan, Stefan Miklosovic, and Vojtech Juranek!

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.