Featured Posts

Tags
apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels community community stories connectors containers cqrs custom datalake db2 ddd debezium debezium server debezium ui deduplication discussion docker elasticsearch event sourcing exactly once semantics example examples features fedora flink hiring ibmi iceberg informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb microservices mongo mongodb mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb topics tracing transactions vagrant vitess website

I’m pleased to announce the release of Debezium 1.4.0.Beta1!

This release includes support for distributed tracing, lowercase table and schema naming for Db2, specifying MySQL snapshot records as create or read operations, and enhancements to Vitess for nullable and primary key columns.

Overall, the community fixed 39 issues for this release. Let’s take a closer look at some of the highlights.

Distributed Tracing

In a nutshell, distributed tracing is a pattern used to profile and monitor applications to allow quick identification of failures or performance concerns. Tracing works by having each component in a distributed process contribute a block of metadata called a "span". Each span contains unique details about that component’s unit of work. Typically a full distributed trace consists of a sequence of multiple spans.

Distributed tracing in Debezium is enabled by using the ActivateTracingSpan SMT:

"transforms": "tracing"
"transforms.tracing.type": "io.debezium.transforms.tracing.ActivateTracingSpan"

The above configuration will lead to the emitted message header containing the tracing key/value pairs.

A blog post discussing the distributed tracing support in depth, including end-to-end tracing for microservices data exchange via the outbox pattern, will follow up shortly.

Bugfixes

Also a number of bugs were fixed, e.g.:

  • DDL parser: Allow stored procedure variables in LIMIT clause DBZ-2692

  • Wrong mysql command in openshift dpeloyment docs DBZ-2746

  • long running transaction will be abandoned and ignored DBZ-2759

  • MS SQL Decimal with default value not matching the scale of the column definition cause exception DBZ-2767

  • Cassandra Connector doesn’t shut down completely DBZ-2768

  • MySQL Parser fails for BINARY collation shortcut DBZ-2771

  • PostgresConnectorIT.shouldResumeStreamingFromSlotPositionForCustomSnapshot is failing for wal2json on CI DBZ-2772

  • Connector configuration property "database.out.server.name" is not relevant for Logminer implementation but cannot be omitted DBZ-2801

  • CHARACTER VARYING mysql identifier for varchar is not supported in debezium DBZ-2821

  • try-with-resources should not be used when OkHttp Response object is returned DBZ-2827

  • EmbeddedEngine does not shutdown when commitOffsets is interrupted DBZ-2830

  • Rename user command parsing fails DBZ-2743

A big thank you to all the contributors from the community who worked on this release: Jeremy Ford, Matt Beary, Vadzim Ramanenka, John Martin, Kewei Shang, Hoa Le, Ramesh Reddy, and Denis Andrejew.

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.