Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions ui vagrant vitess website

As the summer temperatures continue to rise, the Debezium community is pleased to announce Debezium 3.0.0.Alpha2 is now available for testing.

This release includes a host of new features and improvements, including being built on top of Kafka 3.8, the relocation of the JDBC sink connector, custom converters support in Debezium Server, and several improvements to our community-led connectors.

Breaking changes

With any new major release of software, there is often several breaking changes. The Debezium 3.0.0.Alpha2 release is no exception, so let’s discuss the major changes you should be aware about.

Built on Kafka 3.8

This release moves to Kafka 3.8 as our baseline for testing and building Debezium. Kafka 3.8 changed a number of internal APIs that needed to be adapted for Debezium’s use (DBZ-8105).

For most users, this change has no impact; however, if you are extending Debezium, it’s important to be aware of these changes.

New features and improvements

The upgrade to Debezium 3.0.0.Alpha2 introduces a number of new improvements to several components:

JDBC Sink

Relocation of JDBC sink repository

The JDBC sink repository has been relocated from debezium-connector-jdbc to debezium main repository (DBZ-8008). With the introduction of the MongoDB sink connector in Debezium 3.0.0.Alpha1, this allows the team to easily share common contracts across our sink connectors.

Moving forward, to raise pull requests for the JDBC sink, please use the main Debezium repository, as the old repository will be archived and placed into read-only mode after Debezium 2.7.1.Final this week.

Debezium Server

Custom converter types support

In prior releases of Debezium Server, there were a finite number of converters that could be used for headers, keys, and values. These included Json, JsonByteArray, CloudEvents, Avro, Protobuf, Binary, and SimpleString. While these often satisfied a vast majority of use cases, it’s not uncommon that someone may have a unique requirement specific to their environment that is outside these options.

In this release, a new ClientProvided converter option has been added, which allows for extending the header, key, and value converters with a custom, user-supplied implementation (DBZ-8040).

Google Spanner

Support for 32-bit floats

The Google Spanner database introduced support for a 32-bit float data type. The Debezium Google Spanner connector has been adjusted to support this new data type (DBZ-8043).

Vitess

Empty shard support

In Vitess, it is possible for a keyspace to have shards that have no tablets. Debezium for Vitess has improved working with this use case, and now gracefully handles such a keyspace without fault (DBZ-8053).

Other changes

  • Embedded Infinispan tests fail to start with Java 23 DBZ-7840

  • Clarify that Oracle connector does not read from physical standby DBZ-7895

  • Bump Infinispan to 14.0.29.Final DBZ-8010

  • StackOverflow exception on incremental snapshot DBZ-8011

  • Write a blog post about async engine DBZ-8013

  • Test offset/history store configurations DBZ-8015

  • JDBC primary.key.fields cannot be empty when i set insert.mode to upsert and primary.key.mode record_value DBZ-8018

  • Unable to acquire buffer lock, buffer queue is likely full DBZ-8022

  • Add Status ObservedGeneration to Operator DBZ-8025

  • Release process sets incorrect images for k8s for the next development version DBZ-8041

  • Use recreate as (default) rollout strategy for deployments DBZ-8047

  • "Unexpected input: ." when snapshot incremental empty Database DBZ-8050

  • Debezium Operator Using RollingUpdate Strategy DBZ-8051

  • Debezium Operator Using RollingUpdate Strategy DBZ-8052

  • Refactor LogMining implementation to allow alternative cache implementations DBZ-8054

  • Oracle connector inconsistency in redo log switches DBZ-8055

  • Blocking snapshot can fail due to CommunicationsException DBZ-8058

  • FakeDNS not working with JDK version > 18 DBZ-8059

  • Debezium Operator with a provided Service Account doesn’t spin up deployment DBZ-8061

  • Upgrade postgres server version to 15 DBZ-8062

  • Standard Webhooks signatures for HTTP sink DBZ-8063

  • ParsingException (MySQL/MariaDB): rename table syntax DBZ-8066

  • Disable DebeziumResourceNoTopicCreationIT - no longer compatible with Java 21 DBZ-8067

  • Oracle histogram metrics are no longer printed in logs correctly DBZ-8068

  • Vitess-connector should provide a topic naming strategy that supports separate connectors per-table DBZ-8069

  • In hybrid log.mining.strategy reconstruction logs should be set to DEBUG DBZ-8070

  • Speed-up PostgresShutdownIT DBZ-8075

  • Support capturing BLOB column types during snapshot for MySQL/MariaDB DBZ-8076

  • Standard Webhooks auth secret config value is not marked as PASSWORD_PATTERN DBZ-8078

  • Vitess transaction Epoch should not reset to zero when tx ID is missing DBZ-8087

  • Add MariaDB to debezium/connect image DBZ-8088

  • After changing the column datatype from int to float the Debezium fails to round it and i get a null value for this field in the stream DBZ-8089

  • MySQL and MariaDB keyword YES cannot be parsed as a column name DBZ-8092

  • Update third-party LICENSE with LGPL forMariaDB Connector/J DBZ-8099

  • NotificationIT tests seemingly seem to fail due to stepping on one another DBZ-8100

  • ORA-26928 - Unable to communicate with XStream apply coordinator process should be retriable DBZ-8102

  • Transformations are not closed in emebdded engine DBZ-8106

  • Rabbitmq native stream Failed DBZ-8108

  • Don’t close connection after loading timescale metadata in TimescaleDb SMT DBZ-8109

In total, 43 issues were resolved in Debezium 3.0.0.Alpha2. The list of changes can also be found in our release notes.

A big thank you to all the contributors from the community who worked diligently on this release: Andrey Pustovetov, Anisha Mohanty, Aravind, Bertrand Paquet, Chris Cranford, Christian Jacob Mencias, Duc Le Tu, Ganesh Bankar, Gaurav Miglani, Gunnar Morling, Harvey Yue, Henk van Dyk, Hossein Torabi, Inki Hwang, Jakub Cechacek, Jeremy Ford, Jiri Pechanec, M. Gökhan Akgül, Mario Fiore Vitale, Michal Pioun, Mickael Maison, Minh Son Nguyen, Ondrej Babec, Robert Roldan, Roman Kudryashov, Ryan van Huuksloot, Seongjoon Jeong, Thomas Thornton, Timo Wilhelm, Hossein Torabi, Vadzim Ramanenka, Vojtech Juranek, Yanjie Wang, and 张展业!

What’s next & Outlook

We are about half-way through the Debezium 3.0 release cycle, with many more new features still in the works. As we continue to work through these features, we will continue to update Debezium 2.7 with bug fixes and various improvements. You can expect the first maintenance release 2.7.1.Final for Debezium later this week.

Please check out our road map for more details, and get involved in the conversation. You can contact us on the mailing list or Zulip chat if you have any questions or suggestions about anything Debezuim.

Until next time, stay cool and safe…​

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.