Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

It is my pleasure to announce the next Debezium 2.3 release, 2.3.0.CR1!

The main focus of this release is to stabilize the Debezium 2.3 release in preparation for a final release in the coming weeks, which typically means we’re focusing on bugfixes; however, this release includes two new features. Lets take a moment and dive into these new features and any bug fixes that are noteworthy!

Breaking Changes

This release includes no breaking changes, so upgrading from Debezium 2.3.0.Beta1 to 2.3.0.CR1 should be a simple drop-in replacement.

If you are upgrading from a prior version, please review the release notes for any migration steps that may be necessary.

Debezium Server Kubernetes Operator

This release introduces a preview version of the new Debezium Operator, providing the ability to deploy and manage Debezium Server instances within Kubernetes. Debezium Server allows you to stream change events from your data sources to a wide variety of messaging infrastructures. Our goal is to provide a Kafka-less alternative for the Debezium community who wish to utilize Kubernetes for scalability and high availability deployments.

Presently, the documentation is sparse as the operator is in early incubation stages; however, we do intend to improve upon this throughout the remainder of the 2.3 as well into the Debezium 2.4 release cycles. You can find a deployment example and basic description of the custom resource specification in the github repository that you can use as a reference for the short-term.

We do not recommend a production deployment of this component at this time; however, we encourage users to provide community feedback. The feedback will be valuable in evaluating if the component is feature ready or if there are still areas of improvement to meet everyone’s needs.

JMX signals and notifications

Debezium 2.3 previously introduced both a new signal channel and notification feature. This feature allows external applications to easily integrate with Debezium, sending signals to perform various tasks such as ad-hoc incremental snapshots, and to receive notifications about the progress of such tasks. This release builds on top of that functionality to allow the ability to send signals and receive notifications via JMX.

Sending signals

In this release, the signal channel subsystem has been improved to support sending signals via JMX. From the jconsole window, you can now see there are two new subsections for a connector, a notifications and signal section shown below:

JConsole JMX metrics

The new signals section allows you to invoke an operation on the JMX bean in order to transmit a signal to Debezium. This signal resembles the logical signal table structure where it accepts 3 parameters, a unique identifier, the signal type, and finally the signal payload. The following illustrates what this looks like from jconsole:

JConsole Sending Signals

Receiving notifications

The new notifications section allows you to receive and react to notifications captured by the JMX bean from Debezium. The Debezium JMX bean will buffer all notifications to ensure that no notification is missed. The following illustrates what this looks like from jconsole:

JConsole Receive Notifications

The JMX bean does not automatically clear the notification queue. In order to avoid memory concerns, be sure to invoke the reset method on the notifications bean once you’ve processed all notifications.

We look forward to your feedback on this new way to integrate signals and notifications with Debezium over JMX.

Other fixes

  • Code Improvements for skip.messages.without.change DBZ-6366

  • FileSignalChannel is not loaded DBZ-6509

  • Utilize event.processing.failure.handling.mode in Vitess replication connection DBZ-6510

  • MySqlReadOnlyIncrementalSnapshotChangeEventSource enforces Kafka dependency during initialization DBZ-6511

  • Debezium incremental snapshot chunk size documentation unclear or incorrect DBZ-6512

  • Debezium incremental snapshot chunk size documentation unclear or incorrect DBZ-6515

  • Only use error processing mode on certain errors DBZ-6523

  • [PostgreSQL] LTree data is not being captured by streaming DBZ-6524

  • Use better hashing function for PartitionRouting DBZ-6529

  • Start publishing nightly images for Debezium Operator DBZ-6541

  • Start releasing images for Debezium Operator DBZ-6542

Altogether, 24 issues were fixed for this release. A big thank you to all the contributors from the community who worked on this release: Anisha Mohanty, Jakub Cechacek, Jesse Ehrenzweig, Jiri Pechanec, Mario Fiore Vitale, Ronak Jain, Thomas Thornton, Tommy Karlsson, and Vojtech Juranek!

What’s next?

With Debezium 2.3 quickly approaching a Final release, Debezium 2.4 is just around the corner.

Debezium 2.4 planning is underway, and we would love to have your feedback. Please reach out to us on the mailing list or our chat if you have any ideas or suggestions. The roadmap for Debezium 2.4 and beyond will be updated in conjunction with Debezium 2.3 final in the coming week.

And until next time, stay cool as summer approaches!

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.