Debezium Blog

It’s my pleasure to announce the first release of the Debezium 2.1 series, 2.1.0.Alpha1!

The Debezium 2.1.0.Alpha1 release includes quite a number of bug fixes but also some noteworthy improvements and new features including but not limited to:

  • Support for PostgreSQL 15

  • Single Message Transformation (SMT) predicate support in Debezium engine

  • Capturing TRUNCATE as change event in MySQL table topics

  • Oracle LogMiner performance improvements

  • New Redis-based storage module

I’m excited to announce the release of Debezium 1.9.7.Final!

This release focuses on bug fixes and stability; and is the recommended update for all users from earlier versions. This release contains 22 resolved issues overall.

Today it’s my great pleasure to announce the availability of Debezium 2.0.0.Final!

Since our 1.0 release in December 2019, the community has worked vigorously to build a comprehensive open-source low-latency platform for change data capture (CDC). Over the past three years, we have extended Debezium’s portfolio to include a stable connector for Oracle, a community led connector for Vitess, the introduction of incremental snapshots, multi-partition support, and so much more. With the help of our active community of contributors and committers, Debezium is the de facto leader in the CDC space, deployed to production within lots of organizations from across multiple industries, using hundreds of connectors to stream data changes out of thousands of database platforms.

The 2.0 release marks a new milestone for Debezium, one that we are proud to share with each of you.

I am excited to announce the release of Debezium 2.0.0.CR1!

This release contains breaking changes, stability fixes, and bug fixes, all to inch us closer to 2.0.0.Final. Overall, this release contains a total of 53 issues that were fixed.

I’m excited to announce the release of Debezium 1.9.6.Final!

This release focuses on bug fixes and stability; and is the recommended update for all users from earlier versions. This release contains 78 resolved issues overall.

I am excited to announce the release of Debezium 2.0.0.Beta2!

This release contains several breaking changes, stability fixes, and bug fixes, all to inch us closer to 2.0.0.Final. Overall, this release contains a total of 107 issues that were fixed.

I am thrilled to share that Debezium 2.0.0.Beta1 has been released!

This release contains several new features including a pluggable topic selector, the inclusion of database user who committed changes for Oracle change events, and improved handling of table unique indices as primary keys. In addition, there are several breaking changes such as the move to multi-partition mode as default and the introduction of the debezium-storage module and its implementations. So lets take a look at all these in closer detail.

With the summer in full swing, the team is pleased to announce the release of Debezium 1.9.5.Final!

This release primarily focuses on bugfixes and stability; and is the recommended update for all users from earlier versions. This release contains 24 resolved issues overall.

I am thrilled to share that Debezium 2.0.0.Alpha3 has been released!

While this release contains a plethora of bugfixes, there are a few noteworthy improvements, which include providing a timestamp in transaction metadata events, the addition of several new fields in Oracle’s change event source block, and a non-backward compatible change to the Oracle connector’s offsets.

Lets take a look at these in closer detail.

I’m pleased to announce the release of Debezium 1.9.4.Final!

This release primarily focuses on bugfixes and stability; and is the recommended update for all users from earlier versions. This release contains 32 resolved issues overall.

I am thrilled to share that Debezium 2.0.0.Alpha2 has been released!

This release is packed with tons of bugfixes and improvements, 110 issues resolved in total. Just, WOW!

A few noteworthy changes include incremental snapshots gaining support for regular expressions and a new stop signal. We also did some housekeeping and removed a number of deprecated configuration options and as well as the legacy MongoDB oplog implementation.

Lets take a look at these in closer detail.

As the summer nears, I’m excited to announce the release of Debezium 1.9.3.Final!

This release primarily focuses on bugfixes and stability; however, there are some notable feature enhancements. Lets take a moment to cool off and "dive" into these new features in a bit of detail :).

I am excited to share that Debezium 2.0.0.Alpha1 has been released!

This release is the first of several planned pre-releases of Debezium 2.0 over the next five months. Each pre-release plans to focus on strategic changes in the hope that as we move forward, changes can be easily tested and regressions addressed quickly.

In this release, some of the most notable changes include requiring Java 11 to use Debezium or any of its components, the removal of wal2json support for PostgreSQL and the legacy MySQL connector implementation, as well as some notable features such as improved Debezium Server Google Pub/Sub sink support, and a multitude of bugfixes. Let’s take a look at a few of these.

I’m excited to announce the release of Debezium 1.9.1.Final!

This release primarily focuses on bugfixes and stability concerns after the 1.9.0.Final release.

I am very happy to share the news that Debezium 1.9.0.Final has been released!

Besides the usual set of bug fixes and improvements, key features of this release are support for Apache Cassandra 4, multi-database support for the Debezium connector for SQL Server, the ability to use Debezium Server as a Knative event source, as well as many improvements to the integration of Debezium Server with Redis Streams.

Exactly 276 issues have been fixed by the community for the 1.9 release; a big thank you to each and everyone who helped to make this happen!

I am happy to announce the release of Debezium 1.9.0.CR1!

Besides a range of bugfixes, this release brings the long-awaited support for Apache Cassandra 4! Overall, 52 issues have been fixed for this release.

Let’s take a closer look at both the Cassandra 3 changes & Cassandra 4 support.

I am happy to announce the release of Debezium 1.9.0.Beta1!

This release includes many new features for Debezium Server, including Knative Eventing support and offset storage management with the Redis sink, multi-partitioned scaling for the SQL Server connector, and various of bugfixes and improvements. Overall, 56 issues have been fixed for this release.

Let’s take a closer look at a couple of them.

It’s my pleasure to announce the second release of the Debezium 1.9 series, 1.9.0.Alpha2!

This release includes support for Oracle 21c, improvements around Redis for Debezium Server, configuring the kafka.query.timeout.ms option, and a number of bug fixes around DDL parsers, build infrastructure, etc.

Overall, the community fixed 51 issues for this release. Let’s take a closer look at some of the highlights.

It’s my pleasure to announce the first release of the Debezium 1.9 series, 1.9.0.Alpha1!

With the new year comes a new release! The Debezium 1.9.0.Alpha1 release comes with quite a number of fixes and improvements, most notably improved metrics and Oracle ROWID data type support.

It’s my great pleasure to announce the release of Debezium 1.8.0.Final!

Besides a strong focus on the Debezium connector for MongoDB (more on that below), the 1.8 release brings support for Postgres' logical decoding messages, support for configuring SMTs and topic creation settings in the Debezium UI, and much more.

Overall, the community has fixed 242 issues for this release. A big thank you to everyone who helped to make this release happen on time, sticking to our quarterly release cadence!

I’m very excited to announce the release of Debezium 1.8.0.CR1!

As were near the final release due out next week, this release focused heavily on bugfixes. Yet this release includes incremental snapshot support for MongoDB! Overall, not less than 34 issues have been fixed for this release.

Let’s take a closer look at some of them.

I’m very happy to announce the release of Debezium 1.8.0.Beta1!

This release is packed with exciting new features like support for MongoDB 5.0, an outbox event router for the MongoDB connector and support for Postgres logical decoding messages, as well as tons of bugfixes and other improvements. Overall, not less than 63 issues have been fixed for this release.

Let’s take a closer look at some of them.

It’s my pleasure to announce the second release of the Debezium 1.8 series, 1.8.0.Alpha2!

With the holiday season just around the corner, the team’s release schedule remains steadfast. While Debezium 1.8.0.Alpha2 delivers quite a lot of bugfixes and minor changes, there are a few notable changes:

  • MySQL support for heartbeat action queries

  • Configurable transaction topic name

In addition, the latest 1.2 tag of the debezium/tooling image is available. The newest version includes all the latest tools, including kcctl, a super simple, cuddly CLI for Apache Kafka Connect.

It’s my pleasure to announce the first release of the Debezium 1.8 series, 1.8.0.Alpha1!

With the colors of Autumn upon us, the team has been hard at work painting lines of code for this release. With Debezium 1.8.0.Alpha1 comes quite a number of improvements but most notably is the new native MongoDB 4.0 change streams support!

It’s with great pleasure that I am announcing the release of Debezium 1.7.0.Final!

Key features of this release include substantial improvements to the notion of incremental snapshotting (as introduced in Debezium 1.6), a web-based user Debezium user interface, NATS support in Debezium Server, and support for running Apache Kafka without ZooKeeper via the Debezium Kafka container image.

Also in the wider Debezium community some exciting things happened over the last few months; For instance, we saw a CDC connector for ScyllaDB based on the Debezium connector framework, and there’s work happening towards a Debezium Server connector for Apache Iceberg (details about this coming soon in a guest post on this blog).

We are very happy to announce the release of Debezium 1.7.0.CR2!

As we are moving ahead towards the final release we include mostly bugfixes. Yet this release contains important performance improvements and a new feature for read-only MySQL incremental snapshots.

I am very happy to announce the release of Debezium 1.7.0.CR1!

For this release, we’ve reworked how column filters are handled during snapshotting, the Debezium container images have been updated to use Fedora 34 as their base, there’s support for MySQL INVISIBLE columns, and much more.

It’s my pleasure to announce the second release of the Debezium 1.7 series, 1.7.0.Beta1!

This release brings NATS Streaming support for Debezium Server along with many other fixes and enhancements. Also this release is the first one tested with Apache Kafka 2.8.

It’s my pleasure to announce the first release of the Debezium 1.7 series, 1.7.0.Alpha1!

With the summer in a full-swing, this release brings additional improvements to the Debezium Oracle connector but also to the others as well.

I’m pleased to announce the release of Debezium 1.6.0.Final!

This release is packed full with tons of new features, including support for incremental snapshotting that can be toggled using the new the Signal API. Based on the excellent paper DBLog: A Watermark Based Change-Data-Capture Framework by Netflix engineers Andreas Andreakis and Ioannis Papapanagiotou, the notion of incremental snapshotting addresses several requirements around snapshotting that came up repeatedly in the Debezium community:

It’s my pleasure to announce the release of Debezium 1.6.0.CR1!

This release adds skipped operations optimizations for SQL Server, introduces Heartbeat support to the Oracle connector, Oracle BLOB/CLOB support is now opt-in only, and provides a range of bug fixes and other improvements across different Debezium connectors.

It’s my pleasure to announce the release of Debezium 1.6.0.Beta2!

This release adds support for Pravega to Debezium Server, expands the snapshotting options of the Debezium Oracle connector, and provides a range of bug fixes and other improvements across different Debezium connectors.

Let me announce the bugfix release of Debezium 1.5, 1.5.2.Final!

This release is a rebuild of 1.5.1.Final using Java 8.

Let me announce the bugfix release of Debezium 1.5, 1.5.1.Final!

This release fixes a small set of issues discovered since the original release and few improvements into the documentation.

I’m pleased to announce the release of Debezium 1.6.0.Beta1!

This release introduces incremental snapshot support for SQL Server and Db2, performance improvements for SQL Server, support for BLOB/CLOB for Oracle, and much more. Lets take a few moments and explore some of these new features in the following.

It’s my pleasure to announce the first release of the Debezium 1.6 series, 1.6.0.Alpha1!

This release brings the brand new feature called incremental snapshots for MySQL and PostgreSQL connectors, a Kafka sink for Debezium Server, as well as a wide range of bug fixes and other small feature additions.

I’m thrilled to announce the release of Debezium 1.5.0.Final!

With Debezium 1.5, the LogMiner-based CDC implementation for Oracle moves from Incubating to Stable state, and there’s a brand-new implementation of the MySQL connector, which brings features like transaction metadata support. Other key features include support for a new "signalling table", which for instance can be used to implement schema changes with the Oracle connector, and support for TRUNCATE events with Postgres. There’s also many improvements to the community-led connectors for Vitess and Apache Cassandra, as well as wide range of bug fixes and other smaller improvements.

It’s my pleasure to announce the release of Debezium 1.5.0.CR1!

As we begin moving toward finalizing the Debezium 1.5 release stream, the Oracle connector has been promoted to stable and there were some TLS improvements for the Cassandra connector, as well as numerous bugfixes. Overall, 50 issues have been addressed for this release.

We are very happy to announce the release of Debezium 1.5.0.Beta2!

The main features of this release is the new Debezium Signaling Table support, Vitess SET type support, and a continued focus to minor improvements, bugfixes, and polish as we sprint to the finish line for the 1.5 release.

Overall, the community fixed 54 issues since the Beta1 release, some of which we’ll explore more in-depth below.

I’m very happy to announce the release of Debezium 1.5.0.Beta1!

This release adds a brand-new component — the web-based Debezium UI --, transaction metadata support for the MySQL connector, a large number of improvements to the LogMiner-based capture implementation for the Debezium Oracle connector, support for Vitess 9.0, and much more. Let’s explore some of the new features in the following.

It’s my pleasure to announce the first release of the Debezium 1.5 series, 1.5.0.Alpha1!

This release brings many improvements to the LogMiner-based capture implementation for the Debezium Oracle connector, a large overhaul of the MySQL connector, as well as a wide range of bug fixes and other small feature additions.

I’m pleased to announce the release of Debezium 1.4.1.Final!

We highly recommend upgrading from 1.4.0.Final and earlier versions as this release includes bug fixes and enhancements to several Debezium connectors which includes some of the following:

I am pleased to announce the release of Debezium 1.4.0.Final!

This release concludes the major work put into Debezium over the last three months. Overall, the community fixed 117 issues during that time, including the following key features and changes:

  • New Vitess connector, featured in an in-depth blog post by Kewei Shang

  • Fine-grained selection of snapshotted tables

  • PostgreSQL Snapshotter completion hook

  • Distributed Tracing

  • MySQL support for create or read records emitted during snapshot

  • Many Oracle Logminer adapter improvements

  • Full support for Oracle JDBC connection strings

  • Improved reporting of DDL errors

I’m excited to announce the release of Debezium 1.3.1.Final!

This release primarily focuses on bugs that were reported after the 1.3 release. Most importantly, the following bugs were fixed related to the Debezium connector for Oracle LogMiner adapter thanks to the continued feedback by the Debezium community.

  • SQLExceptions thrown when using Oracle LogMiner (DBZ-2624)

  • LogMiner mining session stopped due to WorkerTask killed (DBZ-2629)

Welcome to the first edition of "Debezium Community Stories With…​", a new series of interviews with members of the Debezium and change data capture community, such as users, contributors or integrators. We’re planning to publish more parts of this series in a loose rhythm, so if you’d like to be part of it, please let us know. In today’s edition it’s my pleasure to talk to Renato Mefi, a long-time Debezium user and contributor.

It’s with great please that I’m announcing the release of Debezium 1.3.0.Final!

As per Debezium’s quarterly release cadence, this wraps up the work of the last three months. Overall, the community has fixed 138 issues during that time, including the following key features and changes:

  • A new incubating LogMiner-based implementation for ingesting change events from Oracle

  • Support for Azure Event Hubs in Debezium Server

  • Upgrade to Apache Kafka 2.6

  • Revised filter option names

  • A new SQL Server connector snapshot mode, initial_only

  • Support for database-filtered columns for SQL Server

  • Additional connection options for the MongoDB connector

  • Improvements to ByteBufferConverter for implementing the outbox pattern with Avro as the payload format

I’m very happy to announce the release of Debezium 1.3.0.CR1!

As we approach the final stretch of Debezium 1.3 Final, we took this opportunity to add delegate converter support for the ByteBufferConverter and introduce a debezium-scripting module. In addition, there’s also a range of bug fixes and quite a bit of documentation polish; overall, not less than 15 issues have been resolved for this release.

I’m very happy to announce the release of Debezium 1.3.0.Beta2!

In this release we’ve improved support for column filtering for the MySQL and SQL Server connectors, and there’s a brand-new implementation for ingesting change events from Oracle, using the LogMiner package. As we’re on the home stretch towards Debezium 1.3 Final, there’s also a wide range of smaller improvements, bug fixes and documentation clarifications; overall, not less than 44 issues have been resolved for this release.

It’s my pleasure to announce the release of Debezium 1.3.0.Beta1!

This release upgrades to the recently released Apache Kafka version 2.6.0, fixes several critical bugs and comes with a renaming of the connector configuration options for selecting the tables to be captured. We’ve also released Debezium 1.2.2.Final, which is a drop-in replacement for all users of earlier 1.2.x releases.

Release early, release often! After the 1.1 Beta1 and 1.0.1 Final releases earlier this week, I’m today happy to share the news about the release of Debezium 1.1.0.Beta2!

The main addition in Beta2 is support for integration tests of your change data capture (CDC) set-up using Testcontainers. In addition, the Quarkus extension for implementing the outbox pattern as well as the SMT for extracting the after state of change events have been re-worked and offer more configuration flexibility now.

This article is a dive into the realms of Event Sourcing, Command Query Responsibility Segregation (CQRS), Change Data Capture (CDC), and the Outbox Pattern. Much needed clarity on the value of these solutions will be presented. Additionally, two differing designs will be explained in detail with the pros/cons of each.

So why do all these solutions even matter? They matter because many teams are building microservices and distributing data across multiple data stores. One system of microservices might involve relational databases, object stores, in-memory caches, and even searchable indexes of data. Data can quickly become lost, out of sync, or even corrupted therefore resulting in disastrous consequences for mission critical systems.

Solutions that help avoid these serious problems are of paramount importance for many organizations. Unfortunately, many vital solutions are somewhat difficult to understand; Event Sourcing, CQRS, CDC, and Outbox are no exception. Please look at these solutions as an opportunity to learn and understand how they could apply to your specific use cases.

As you will find out at the end of this article, I will propose that three of these four solutions have high value, while the other should be discouraged except for the rarest of circumstances. The advice given in this article should be evaluated against your specific needs, because, in some cases, none of these four solutions would be a good fit.

Outbox as in that folder in my email client? No, not exactly but there are some similarities!

The term outbox describes a pattern that allows independent components or services to perform read your own write semantics while concurrently providing a reliable, eventually consistent view to those writes across component or service boundaries.

You can read more about the Outbox pattern and how it applies to microservices in our blog post, Reliable Microservices Data Exchange With the Outbox Patttern.

So what exactly is an Outbox Event Router?

In Debezium version 0.9.3.Final, we introduced a ready-to-use Single Message Transform (SMT) that builds on the Outbox pattern to propagate data change events using Debezium and Kafka. Please see the documentation for details on how to use this transformation.