Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

The Debezium release cadence is in full swing as I’m excited to announce Debezium 2.1.2.Final!

This release focuses primarily on bug fixes and stability; and it is the recommended update for all users from earlier versions. This release contains 28 resolved issues, so let’s take a moment and discuss a critical breaking change.

Breaking Change

An edge case was reported in DBZ-5996 where if a temporal column used ZonedTimestamp and if the column’s value had 0 micro or nanoseconds, rather than emitting the value as 2023-01-19T12:30:00.123000Z, the value would be emitted in a truncated way as 2023-01-19T12:30:00.123Z. This could lead to other issues with converters used in the event pipeline when the output from that column could be formatted inconsistently.

In order to remedy the edge case, the ZonedTimestamp implementation will now pad the fraction-based seconds value of the column’s value to the length/scale of the source database column. Using the example above of a TIMESTAMP(6) MySQL column type, the emitted value will now properly reflect a value of 2023-01-19T12:30:00.123000Z.

While this change in behavior is likely to have minimal impact to most users, we wanted to bring attention to it in the event that you’ve perhaps used other means to handle this edge case in your pipelines. If you have, you should be able to rely on Debezium to emit the value consistently, even when the fraction-based seconds is 0.

Other changes

A few noteworthy bug fixes and stability improvements include:

  • Data type conversion failed for mysql bigint DBZ-5798

  • Oracle cannot undo change DBZ-5907

  • Truncate records incompatible with ExtractNewRecordState DBZ-5966

  • Computed partition must not be negative DBZ-5967

  • NPE in execute snapshot signal with exclude.tables config on giving wrong table name DBZ-5988

  • There is a problem with postgresql connector parsing the boundary value of money type DBZ-5991

  • Nullable columns marked with "optional: false" in DDL events DBZ-6003

  • Vitess: Handle the shard list difference between current db shards and persisted shards DBZ-6011

  • Postgres LSN check should honor event.processing.failure.handling.mode DBZ-6012

  • Enhance the Spanner connector by adding features and/or solving bugs DBZ-6014

  • DDL statement with TokuDB engine specific "CLUSTERING KEY" couldn’t be parsed DBZ-6016

  • DDL parse fail for role revoke with "user-like" role name DBZ-6019

  • DDL parse fail for ALTER USER x DEFAULT ROLE y; DBZ-6020

  • Offsets are not flushed on connect offsets topic when encountering an error on Postgres connector DBZ-6026

  • Unexpected format for TIME column: 8:00 DBZ-6029

  • Oracle does not support compression/logging clauses after an LOB storage clause DBZ-6031

  • Debezium is logging the full message along with the error DBZ-6037

  • Improve resilience during internal schema history recovery from Kafka DBZ-6039

Please refer to the release notes to learn more about all fixed bugs, update procedures, etc.

Many thanks to the following individuals from the community who contributed to Debezium 2.1.2.Final: Akshansh Jain, Animesh Kumar, Anisha Mohanty, Bob Roldan, Chris Cranford, Harvey Yue, Henry Cai, Indra Shukla, Jiri Pechanec, Luca Scannapieco, Mario Fiore Vitale, Mark Lambert, Sergei Morozov, Vojtech Juranek, Yohei Yoshimuta, and yohei yoshimuta!

Outlook, What’s next?

Debezium 2.1 will continue to receive bug fix and maintenance changes throughout this quarter. I expect there will be at least one additional release likely toward the middle or late February or March timeframe as we begin to wrap up up the work on Debezium 2.2.

Regarding Debezium 2.2, we intend to deliver another Alpha build in the coming weeks. We have lots of features still in the works, including the JDBC Sink Connector, configurable signal channels, new message transformations, and much more.

Stay tuned, and until then …​

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.