Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

I’m excited to announce the release of Debezium 1.3.0.Alpha1!

This initial pass in the 1.3 release line provides a number of useful new features:

  • A new Debezium Server sink adapter for Azure Event Hubs

  • A new SQL Server connector snapshot mode, initial_only

  • Additional connection timeout options for the MongoDB Connector

Overall, the community fixed not less than 31 issues for this release. Let’s take a closer look at some of them in the remainder of this post.

Azure Event Hubs sink adapter

Debezium’s standalone server is one of the newest features in the Debezium ecosystem. The standalone server provides a ready-to-use application that can stream change events from a source database to a variety of messaging infrastructures.

Driven by the community, the Debezium Server now supports Azure Event Hubs (DBZ-2282). This now enables the Debezium Server to stream change events to Amazon Kinesis, Apache Pulsar, Google Cloud Pub/Sub, and Azure Event Hubs.

Additional MongoDB connection options

The Debezium MongoDB connector has traditionally used the driver default connection options and timeouts. There are use cases where customization of these defaults are necessary to support latency or performance concerns in your deployment.

There additional configuration options are now available for MongoDB:

mongodb.connect.timeout.ms
The number of milliseconds the driver waits for a new conneciton before the attempt is aborted.
Defaults to 10000.

mongodb.server.selection.timeout.ms
The number of milliseconds the driver will wait to select a server before it times out, throwing an error.
Defaults to 30000.

mongodb.socket.timeout.ms
The number of milliseconds the driver waits before a send/receive on the socket may timeout.
A value of 0 disables this behavior. Defaults to 0.

Other Features

Besides these key features, there’s a number of other new features coming with the 1.3.0.Alpha1 release:

  • New SQL Server snapshot mode initial_only (DBZ-2379)

  • Postgres and possibly other DB connections are not properly shutdown when the task encounters thread interrupt (DBZ-2133)

Bugfixes

Also a number of bugs were fixed, e.g.:

  • Ignore non-existing table reported on Aurora via SHOW TABLES (DBZ-1939)

  • Cassandra connector not getting events (DBZ-2086)

  • PubSub Sink sends empty records (DBZ-2277)

  • Skipping LSN is inefficient and does not forward slot position (DBZ-2310)

  • message size is at least 68x larger for changes with bit varying columns (DBZ-2315)

  • Change events lost when connnector is restarted while processing transaction with PK update (DBZ-2329)

  • Error when processing commitLogs related to list-type columns (DBZ-2345)

  • Fix dependency groupId on Outbox Quarkus Extension documentation (DBZ-2367)

  • Cannot detect Azure Sql Version (DBZ-2373)

  • ParallelSnapshotReader sometimes throws NPE (DBZ-2387)

Please refer to the release notes for the complete list of resolved issues as well as procedures for upgrading from earlier Debezium versions.

A big thank you to all the contributors from the community who worked on this release: Abhishek Gupta, Cory Harper, Arik Cohen, Moira Tagle, Victor Xiang, Grzegorz Kołakowski, Björn Häuser, Lukasz Korzeniowski, and Jonas Lins!

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.