Debezium Blog

I’m pleased to announce the immediate availability of Debezium 2.6.1.Final. This release is the first maintenance release that focuses on addressing several critical stability issues with the 2.6.0.Final release, including classpath loading problems with Debezium Server, MongoDB silently stops gathering changes, and a race condition with the Oracle Infinispan buffer implementation.

Let’s take a few moments and dive into these and more…​

As the team has sprung forward into action, with spring upon us, and we’re in the summer spirit, we are pleased to announce the immediate release of Debezium 2.6.0.Final. This release includes dozens of new features, bug fixes, and improvements from the valiant efforts of the team and community contributors. Overall, there were 249 issues resolved with contributions from over 56 contributors. Lets a moment and review all the changes.

As we are just a week away from Debezium 2.6.0.Final, I am pleased to announce Debezium 2.6.0.CR1, the first release candidate for the 2.6 release stream. This release includes a number of improvements, including XML support for the Oracle OpenLogReplicator adapter, TRACE level logging support for Debezium Server, configurable partition modes for Cassandra, the new Snapshot API for MongoDB and Db2, and more.

Additionally, this release includes a variety of bug fixes and several breaking changes.

Let’s take a closer look at all these changes and improvements included in Debezium 2.6.0.CR1; as always, you can find the complete list of changes for this release in the release notes. Please remember to take special note to any breaking changes that could affect your upgrade path.

We are happy to announce the third maintenance release of the Debezium 2.5 release stream, Debezium 2.5.3.Final. This release includes some improvements and numerous bug fixes, so let’s dive right in…​

We are pleased to announce the release of Debezium 2.6.0.Beta1. We enter the home stretch with this release, packed with many improvements, enhancements, bug fixes, and yes a brand new Db2 connector for iSeries. There is a lot to cover in this release, so lets dive right in! Breaking changes The team aims to avoid any potential breaking changes between minor releases; however, such changes are sometimes inevitable. Oracle In older versions of Debezium, users...

We are happy to announce the second maintenance release of the Debezium 2.5 release stream, Debezium 2.5.2.Final. This release includes some improvements and numerous bug fixes, so let’s dive right in…​

As we’ve hit the mid-mark of the quarter, the team is pleased to announce the second installment of the Debezium 2.6 release stream, Debezium 2.6.0.Alpha2. This release is filled to the brim with new features, improvements, and bug fixes, so let’s dive into these…​

While we remain on track with the upcoming Debezium 2.6 release, we are continuously looking at improvements and fixes that are being made that ensures that older releases continue to provide the best possible experience. With that, the team is pleased to announce the first maintenance release of Debezium 2.5, Debezium 2.5.1.Final. Let’s dive into what this release includes and what you should be aware of while upgrading…​

A new year, a new preview release, in true Debezium fashion. The team is pleased to announce the first installment of the Debezium 2.6 release stream, Debezium 2.6.0.Alpha1. Let’s take a moment and dive into these new features, understand how to use these to improve your change data capture experience…​

As the winter chill settles in, and we transition to the festive holiday season, our dedicated team has been busy preparing a special gift for the Debezium community. I am excited to share the immediate release of Debezium 2.5.0.Final, just in time for the holiday celebrations. Get ready to unwrap this latest minor version, filled with holiday cheer and exciting new features!

Let’s take a moment and review all the new features, changes, and improvements that are included in Debezium 2.5, which includes 181 issues resolved by 37 unique contributors.

As we are just one step away from the Debezium 2.5 final release, I am pleased to announce that Debezium 2.5.0.CR1 is now available. This release includes a number of improvements like AWS SQS sink for Debezium Server, INSERT/DELETE semantics for incremental snapshot watermarking, ReselectColumnsPostProcessor, uniform Oracle LOB behavior.

Additionally, this release includes a variety of bug fixes and several breaking changes.

Let’s take a closer look at all these changes and improvements included in Debezium 2.5.0.CR1; as always, you can find the complete list of changes for this release in the release notes. Please remember to take special note to any breaking changes that could affect your upgrade path.

As we begin to approach the final stretch for Debezium 2.5, I am pleased to announce that Debezium 2.5.0.Beta1 is now available. This release includes a number of improvements like support for MariaDB GTID, partitioning for Debezium Server EventHub’s sink, native RabbitMQ streams Debezium Server sink, streaming from PostgreSQL 16 stand-by databases, MySQL high-precision source timestamps, field inclusion/exclusion with JDBC sink, some additional notifications for initial snapshots, and service account support for Debezium Operator CRDs. Additionally, this release includes a variety of bug fixes and several breaking changes.

Let’s take a closer look at all these changes and improvements included in Debezium 2.5.0.Beta1; as always, you can find the complete list of changes for this release in the release notes. Please remember to take special note to any breaking changes that could affect your upgrade path.

As the year starts to come to a close, I am happy to announce the first maintenance release for the Debezium 2.4 release series, Debezium 2.4.1.Final.

While our maintenance releases focus primarily on bug fixes and stability improvements, there are a few new features we back ported from our most recent Debezium 2.5 development series. All these new features focus on the Debezium Operator for Kubernetes, so lets take a quick deep dive into those details. As always, you can find the complete list of changes for this release in the release notes.

While it has only been two short weeks since our first preview release for the Debezium 2.5 release stream, I am happy to announce the immediate availability of the next preview release, Debezium 2.5.0.Alpha2.

This release includes a variety of improvements, batch support for the JDBC Sink connector, seamless support for MongoDB documents that exceed the 16MB barrier, MySQL 8.2 compatibility, and signal improvements for SQL Server. Additionally, this release includes a variety of bug fixes and several breaking changes.

Let’s take a closer look at these changes and improvements that are included in Debezium 2.5.0.Alpha2; as always, you can find the complete list of changes for this release in the release notes. Please remember to take special note to any breaking changes that could affect your upgrade path.

It’s been about three weeks since we released Debezium 2.4, and in that time the team has been diligently working on what comes next in the evolution of Debezium. I am pleased to announce that today we have released Debezium 2.5.0.Alpha1, the first preview release of Debezium’s 2.5 release stream. This release includes many new exciting features as well as bug fixes, e.g. a brand-new IBM Informix connector, a preview support for MariaDB with the...

As the summer months wind down and we enter autumn with cooler temperatures, the team has diligently prepared the next major milestone of Debezium. It’s my pleasure to announce the immediate release of the next minor version, Debezium 2.4.0.Final.

As the team begins the journey toward the next development iteration, let’s take a moment and review all the new features, changes, and improvements that are included in Debezium 2.4, which includes 231 issues resolved by 68 unique contributors.

It has been nearly two weeks since our last preview release of the Debezium 2.4 series, and I am thrilled to announcement the next installation of that series, Debezium 2.4.0.Beta2.

While typically beta releases focus on stability and bugs, this release includes quite a number of noteworthy improves and new features including a new ingestion method for Oracle using OpenLogReplicator, a new single message transform to handle timezone conversions, custom authentication support for MongoDB, configurable order for the MongoDB aggregation pipeline, and lastly support for MongoDB 7.

Let’s take a few moments and dive into all these new features, improvements, and changes in more detail.

It is my pleasure to announce the immediate release of Debezium 2.3.3.Final.

This release includes several bug fixes to address regressions, stability, documentation updates. If you are currently looking to upgrade to the Debezium 2.3.x release stream, we highly recommend you consider using this release. Let’s take a quick look into the regressions and bug fixes.

While development remains steadfast as we continue forward on Debezium 2.4, I am thrilled to announce the immediate availability of Debezium 2.4.0.Beta1.

While this release focuses on stability and bug fixes, there are several new noteworthy features including TimescaleDB support, JMX notifications using JSON payloads, multiple improvements to the Oracle connector’s metrics and embedded Infinispan buffer implementation, SQL Server heartbeats, Vitess shardless strategy, JDBC sink with SQL Server identity-based inserts, and much more. Let’s dive into each of thees new features and others in more detail.

Despite summer being well underway, Debezium contributors remain hard at work, and it’s my pleasure to announce the next preview release of Debezium 2.4 series, 2.4.0.Alpha2. This preview release includes a mix of improvements, bug fixes, and new features that are available for the Debezium community to test and offer feedback. Some highlights from this release include ad-hoc blocking snapshots, source-to-sink column name propagation, support for alternative MySQL drivers, and all Cassandra connectors with Debezium...

It is my pleasure to announce the immediate release of Debezium 2.3.2.Final.

This release includes several bug fixes to address regressions, stability, documentation updates. If you are currently looking to upgrade to the Debezium 2.3.x release stream, we highly recommend you consider using this release. Let’s take a quick look into the regressions and bug fixes.

It has been several weeks since we released the first installment of Debezium 2.3, and I’m excited to announce the next iteration of Debezium 2.3 with 2.3.1.Final. As with any micro-release, the focus is on stability and bug fixes, as well as adjustments to our documentation; however there are some changes that are noteworthy that I would like to take a few moments to highlight.

It’s been a busy month in Debezium-land, and it’s my pleasure to announce the first release of Debezium 2.4 series, 2.4.0.Alpha1. This release includes a plethora of changes, 59 changes to be exact, that cover a wide range of resolved issues, improvement to stability, new features, and several breaking changes. Let’s dive into each of these and discuss them in more depth. Breaking changes New features Other changes Breaking changes MongoDB The MongoDB connector explicitly...

It’s my pleasure to announce the next release of the Debezium 2.3 series, 2.3.0.Beta1!

While this release focuses primarily on bug fixes and stability improvements, there are some new improvements with the PostgreSQL connector and the new notification and channels subsystem. In addition, there are also some compatibility breaking changes.

This release contains changes for 22 issues, so lets take a moment and dive into the new features and any potential bug fixes or breaking changes that are noteworthy!

It’s my pleasure to announce the first release of the Debezium 2.3 series, 2.3.0.Alpha1!

This release brings many new and exciting features as well as bug fixes, including Debezium status notifications, storage of Debezium state into a JDBC data store, configurable signaling channels, the ability to edit connector configurations via Debezium UI, the parallelization of Vitess shards processing, and much more.

This release contains changes for 59 issues, so lets take a moment and dive into several of these new features and any potential bug fixes or breaking changes that are noteworthy!

Today, it’s with great joy that we can announce the availability of Debezium 2.2.0.Final!

Many of you may have noticed, this release cadence took a bit longer than our traditional three-months. While we normally prefer to keep to our usual cadence, this shift gives us a unique opportunity to ship Debezium 2.2 with tons of new features and bug fixes, but also major upgrades to several core components.

The Debezium team is excited to announce the first release candidate of Deebzium 2.2, Debezium 2.2.0.CR1.

This release primarily focuses on stability improvements and bug fixes; however, there are a number of new features and breaking changes. In this release, Debezium migrated to Quarkus 3.0.0.Final, there are performance improvements to Debezium Server Pulsar sink, Jolokia can be enabled inside Debezium’s Kafka Connect container image, incubating support for incremental snapshots on MongoDB multi-replica and sharded clusters, and the deprecation usage of Docker Hub for images.

Let’s take a moment and dive into several of these and what it means moving forward!

The team is excited to announce the first beta release of the Debezium 2.2 release stream, Debezium 2.2.0.Beta1.

This release includes a plethora of bug fixes, improvements, and a number of new features including, but not limited to, a new JDBC sink connector implementation, MongoDB sharded cluster improvements, Google Spanner PostgreSQL dialect support, and a RabbitMQ sink implementation for Debezium Server to just name a few.

Let’s take moment and dive into what’s new!

Today, I am pleased to announce the third alpha release in the 2.2 release stream, Debezium 2.2.0.Alpha3.

This release includes a plethora of bug fixes, improvements, breaking changes, and a number of new features including, but not limited to, optional parallel snapshots, server-side MongoDB change stream filtering, surrogate keys for incremental snapshots, a new Cassandra connector for Cassandra Enterprise, much more.

Let’s take moment and dive into some of these new features, improvements, and breaking changes.

Today, I am pleased to announce the second alpha release in the 2.2 release stream, Debezium 2.2.0.Alpha2. This release includes a plethora of bug fixes, improvements, breaking changes, and a number of new features including, but not limited to, a new ExtractRecordChanges single message transformation, a Reactive-based implementation of the Debezium Outbox extension for Quarkus, a Debezium Storage module for Apache RocketMQ, and much more. Let’s take moment and dive into these new features, improvements, and breaking changes.

The Debezium release cadence is in full swing as I’m excited to announce Debezium 2.1.2.Final!

This release focuses primarily on bug fixes and stability; and it is the recommended update for all users from earlier versions. This release contains 28 resolved issues, so let’s take a moment and discuss a critical breaking change.

It’s my pleasure to announce not only the first release of the Debezium 2.2 series, but also the first release of Debezium in 2023, 2.2.0.Alpha!

The Debezium 2.2.0.Alpha1 release includes some breaking changes, a number of bug fixes, and some noteworthy improvements and features, including but not limited to:

  • [Breaking Change] - ZonedTimestamp values will no longer truncate fractional seconds.

  • [New] - Support ingesting changes from an Oracle logical stand-by database

  • [New] - Support Amazon S3 buckets using the Debezium Storage API

  • [New] - Support retrying database connections during connector start-up

  • [New] - Debezium Server sink connector support for Apache RocketMQ and Infinispan

Today it’s my great pleasure to announce the availability of Debezium 2.1.0.Final!

You might recently noticed that Debezium went a bit silent for the last few weeks. No, we are not going away. In fact the elves in Google worked furiously to bring you a present under a Christmas tree - Debezium Spanner connector.

It’s my pleasure to announce the first release of the Debezium 2.1 series, 2.1.0.Alpha1!

The Debezium 2.1.0.Alpha1 release includes quite a number of bug fixes but also some noteworthy improvements and new features including but not limited to:

  • Support for PostgreSQL 15

  • Single Message Transformation (SMT) predicate support in Debezium engine

  • Capturing TRUNCATE as change event in MySQL table topics

  • Oracle LogMiner performance improvements

  • New Redis-based storage module

I’m excited to announce the release of Debezium 1.9.7.Final!

This release focuses on bug fixes and stability; and is the recommended update for all users from earlier versions. This release contains 22 resolved issues overall.

Today it’s my great pleasure to announce the availability of Debezium 2.0.0.Final!

Since our 1.0 release in December 2019, the community has worked vigorously to build a comprehensive open-source low-latency platform for change data capture (CDC). Over the past three years, we have extended Debezium’s portfolio to include a stable connector for Oracle, a community led connector for Vitess, the introduction of incremental snapshots, multi-partition support, and so much more. With the help of our active community of contributors and committers, Debezium is the de facto leader in the CDC space, deployed to production within lots of organizations from across multiple industries, using hundreds of connectors to stream data changes out of thousands of database platforms.

The 2.0 release marks a new milestone for Debezium, one that we are proud to share with each of you.

I am excited to announce the release of Debezium 2.0.0.CR1!

This release contains breaking changes, stability fixes, and bug fixes, all to inch us closer to 2.0.0.Final. Overall, this release contains a total of 53 issues that were fixed.

I’m excited to announce the release of Debezium 1.9.6.Final!

This release focuses on bug fixes and stability; and is the recommended update for all users from earlier versions. This release contains 78 resolved issues overall.

I am excited to announce the release of Debezium 2.0.0.Beta2!

This release contains several breaking changes, stability fixes, and bug fixes, all to inch us closer to 2.0.0.Final. Overall, this release contains a total of 107 issues that were fixed.

I am thrilled to share that Debezium 2.0.0.Beta1 has been released!

This release contains several new features including a pluggable topic selector, the inclusion of database user who committed changes for Oracle change events, and improved handling of table unique indices as primary keys. In addition, there are several breaking changes such as the move to multi-partition mode as default and the introduction of the debezium-storage module and its implementations. So lets take a look at all these in closer detail.

With the summer in full swing, the team is pleased to announce the release of Debezium 1.9.5.Final!

This release primarily focuses on bugfixes and stability; and is the recommended update for all users from earlier versions. This release contains 24 resolved issues overall.

I am thrilled to share that Debezium 2.0.0.Alpha3 has been released!

While this release contains a plethora of bugfixes, there are a few noteworthy improvements, which include providing a timestamp in transaction metadata events, the addition of several new fields in Oracle’s change event source block, and a non-backward compatible change to the Oracle connector’s offsets.

Lets take a look at these in closer detail.

I’m pleased to announce the release of Debezium 1.9.4.Final!

This release primarily focuses on bugfixes and stability; and is the recommended update for all users from earlier versions. This release contains 32 resolved issues overall.

I am thrilled to share that Debezium 2.0.0.Alpha2 has been released!

This release is packed with tons of bugfixes and improvements, 110 issues resolved in total. Just, WOW!

A few noteworthy changes include incremental snapshots gaining support for regular expressions and a new stop signal. We also did some housekeeping and removed a number of deprecated configuration options and as well as the legacy MongoDB oplog implementation.

Lets take a look at these in closer detail.

As the summer nears, I’m excited to announce the release of Debezium 1.9.3.Final!

This release primarily focuses on bugfixes and stability; however, there are some notable feature enhancements. Lets take a moment to cool off and "dive" into these new features in a bit of detail :).

I am excited to share that Debezium 2.0.0.Alpha1 has been released!

This release is the first of several planned pre-releases of Debezium 2.0 over the next five months. Each pre-release plans to focus on strategic changes in the hope that as we move forward, changes can be easily tested and regressions addressed quickly.

In this release, some of the most notable changes include requiring Java 11 to use Debezium or any of its components, the removal of wal2json support for PostgreSQL and the legacy MySQL connector implementation, as well as some notable features such as improved Debezium Server Google Pub/Sub sink support, and a multitude of bugfixes. Let’s take a look at a few of these.

I’m excited to announce the release of Debezium 1.9.1.Final!

This release primarily focuses on bugfixes and stability concerns after the 1.9.0.Final release.

I am very happy to share the news that Debezium 1.9.0.Final has been released!

Besides the usual set of bug fixes and improvements, key features of this release are support for Apache Cassandra 4, multi-database support for the Debezium connector for SQL Server, the ability to use Debezium Server as a Knative event source, as well as many improvements to the integration of Debezium Server with Redis Streams.

Exactly 276 issues have been fixed by the community for the 1.9 release; a big thank you to each and everyone who helped to make this happen!

I am happy to announce the release of Debezium 1.9.0.CR1!

Besides a range of bugfixes, this release brings the long-awaited support for Apache Cassandra 4! Overall, 52 issues have been fixed for this release.

Let’s take a closer look at both the Cassandra 3 changes & Cassandra 4 support.

I am happy to announce the release of Debezium 1.9.0.Beta1!

This release includes many new features for Debezium Server, including Knative Eventing support and offset storage management with the Redis sink, multi-partitioned scaling for the SQL Server connector, and various of bugfixes and improvements. Overall, 56 issues have been fixed for this release.

Let’s take a closer look at a couple of them.

It’s my pleasure to announce the second release of the Debezium 1.9 series, 1.9.0.Alpha2!

This release includes support for Oracle 21c, improvements around Redis for Debezium Server, configuring the kafka.query.timeout.ms option, and a number of bug fixes around DDL parsers, build infrastructure, etc.

Overall, the community fixed 51 issues for this release. Let’s take a closer look at some of the highlights.

It’s my pleasure to announce the first release of the Debezium 1.9 series, 1.9.0.Alpha1!

With the new year comes a new release! The Debezium 1.9.0.Alpha1 release comes with quite a number of fixes and improvements, most notably improved metrics and Oracle ROWID data type support.

It’s my great pleasure to announce the release of Debezium 1.8.0.Final!

Besides a strong focus on the Debezium connector for MongoDB (more on that below), the 1.8 release brings support for Postgres' logical decoding messages, support for configuring SMTs and topic creation settings in the Debezium UI, and much more.

Overall, the community has fixed 242 issues for this release. A big thank you to everyone who helped to make this release happen on time, sticking to our quarterly release cadence!

I’m very excited to announce the release of Debezium 1.8.0.CR1!

As were near the final release due out next week, this release focused heavily on bugfixes. Yet this release includes incremental snapshot support for MongoDB! Overall, not less than 34 issues have been fixed for this release.

Let’s take a closer look at some of them.

I’m very happy to announce the release of Debezium 1.8.0.Beta1!

This release is packed with exciting new features like support for MongoDB 5.0, an outbox event router for the MongoDB connector and support for Postgres logical decoding messages, as well as tons of bugfixes and other improvements. Overall, not less than 63 issues have been fixed for this release.

Let’s take a closer look at some of them.

It’s my pleasure to announce the second release of the Debezium 1.8 series, 1.8.0.Alpha2!

With the holiday season just around the corner, the team’s release schedule remains steadfast. While Debezium 1.8.0.Alpha2 delivers quite a lot of bugfixes and minor changes, there are a few notable changes:

  • MySQL support for heartbeat action queries

  • Configurable transaction topic name

In addition, the latest 1.2 tag of the debezium/tooling image is available. The newest version includes all the latest tools, including kcctl, a super simple, cuddly CLI for Apache Kafka Connect.

It’s my pleasure to announce the first release of the Debezium 1.8 series, 1.8.0.Alpha1!

With the colors of Autumn upon us, the team has been hard at work painting lines of code for this release. With Debezium 1.8.0.Alpha1 comes quite a number of improvements but most notably is the new native MongoDB 4.0 change streams support!

It’s with great pleasure that I am announcing the release of Debezium 1.7.0.Final!

Key features of this release include substantial improvements to the notion of incremental snapshotting (as introduced in Debezium 1.6), a web-based user Debezium user interface, NATS support in Debezium Server, and support for running Apache Kafka without ZooKeeper via the Debezium Kafka container image.

Also in the wider Debezium community some exciting things happened over the last few months; For instance, we saw a CDC connector for ScyllaDB based on the Debezium connector framework, and there’s work happening towards a Debezium Server connector for Apache Iceberg (details about this coming soon in a guest post on this blog).

It’s my pleasure to announce the second release of the Debezium 1.7 series, 1.7.0.Beta1!

This release brings NATS Streaming support for Debezium Server along with many other fixes and enhancements. Also this release is the first one tested with Apache Kafka 2.8.

It’s my pleasure to announce the first release of the Debezium 1.7 series, 1.7.0.Alpha1!

With the summer in a full-swing, this release brings additional improvements to the Debezium Oracle connector but also to the others as well.

I’m pleased to announce the release of Debezium 1.6.0.Final!

This release is packed full with tons of new features, including support for incremental snapshotting that can be toggled using the new the Signal API. Based on the excellent paper DBLog: A Watermark Based Change-Data-Capture Framework by Netflix engineers Andreas Andreakis and Ioannis Papapanagiotou, the notion of incremental snapshotting addresses several requirements around snapshotting that came up repeatedly in the Debezium community:

It’s my pleasure to announce the release of Debezium 1.6.0.CR1!

This release adds skipped operations optimizations for SQL Server, introduces Heartbeat support to the Oracle connector, Oracle BLOB/CLOB support is now opt-in only, and provides a range of bug fixes and other improvements across different Debezium connectors.

It’s my pleasure to announce the release of Debezium 1.6.0.Beta2!

This release adds support for Pravega to Debezium Server, expands the snapshotting options of the Debezium Oracle connector, and provides a range of bug fixes and other improvements across different Debezium connectors.

Let me announce the bugfix release of Debezium 1.5, 1.5.2.Final!

This release is a rebuild of 1.5.1.Final using Java 8.

Let me announce the bugfix release of Debezium 1.5, 1.5.1.Final!

This release fixes a small set of issues discovered since the original release and few improvements into the documentation.

I’m pleased to announce the release of Debezium 1.6.0.Beta1!

This release introduces incremental snapshot support for SQL Server and Db2, performance improvements for SQL Server, support for BLOB/CLOB for Oracle, and much more. Lets take a few moments and explore some of these new features in the following.

It’s my pleasure to announce the first release of the Debezium 1.6 series, 1.6.0.Alpha1!

This release brings the brand new feature called incremental snapshots for MySQL and PostgreSQL connectors, a Kafka sink for Debezium Server, as well as a wide range of bug fixes and other small feature additions.

I’m thrilled to announce the release of Debezium 1.5.0.Final!

With Debezium 1.5, the LogMiner-based CDC implementation for Oracle moves from Incubating to Stable state, and there’s a brand-new implementation of the MySQL connector, which brings features like transaction metadata support. Other key features include support for a new "signalling table", which for instance can be used to implement schema changes with the Oracle connector, and support for TRUNCATE events with Postgres. There’s also many improvements to the community-led connectors for Vitess and Apache Cassandra, as well as wide range of bug fixes and other smaller improvements.

It’s my pleasure to announce the release of Debezium 1.5.0.CR1!

As we begin moving toward finalizing the Debezium 1.5 release stream, the Oracle connector has been promoted to stable and there were some TLS improvements for the Cassandra connector, as well as numerous bugfixes. Overall, 50 issues have been addressed for this release.

We are very happy to announce the release of Debezium 1.5.0.Beta2!

The main features of this release is the new Debezium Signaling Table support, Vitess SET type support, and a continued focus to minor improvements, bugfixes, and polish as we sprint to the finish line for the 1.5 release.

Overall, the community fixed 54 issues since the Beta1 release, some of which we’ll explore more in-depth below.

I’m very happy to announce the release of Debezium 1.5.0.Beta1!

This release adds a brand-new component — the web-based Debezium UI --, transaction metadata support for the MySQL connector, a large number of improvements to the LogMiner-based capture implementation for the Debezium Oracle connector, support for Vitess 9.0, and much more. Let’s explore some of the new features in the following.

It’s my pleasure to announce the first release of the Debezium 1.5 series, 1.5.0.Alpha1!

This release brings many improvements to the LogMiner-based capture implementation for the Debezium Oracle connector, a large overhaul of the MySQL connector, as well as a wide range of bug fixes and other small feature additions.

I’m pleased to announce the release of Debezium 1.4.1.Final!

We highly recommend upgrading from 1.4.0.Final and earlier versions as this release includes bug fixes and enhancements to several Debezium connectors which includes some of the following:

I am pleased to announce the release of Debezium 1.4.0.Final!

This release concludes the major work put into Debezium over the last three months. Overall, the community fixed 117 issues during that time, including the following key features and changes:

  • New Vitess connector, featured in an in-depth blog post by Kewei Shang

  • Fine-grained selection of snapshotted tables

  • PostgreSQL Snapshotter completion hook

  • Distributed Tracing

  • MySQL support for create or read records emitted during snapshot

  • Many Oracle Logminer adapter improvements

  • Full support for Oracle JDBC connection strings

  • Improved reporting of DDL errors

I’m pleased to announce the release of Debezium 1.4.0.CR1!

This release focuses primarily on polishing the 1.4 release.

I’m pleased to announce the release of Debezium 1.4.0.Beta1!

This release includes support for distributed tracing, lowercase table and schema naming for Db2, specifying MySQL snapshot records as create or read operations, and enhancements to Vitess for nullable and primary key columns.

I’m excited to announce the release of Debezium 1.4.0.Alpha2!

This second pass of the 1.4 release line provides a few useful new features:

  • New API hook for the PostgreSQL Snapshotter interface

  • Field renaming using ExtractNewRecordState SMT’s add.fields and add.headers configurations

I’m excited to announce the release of Debezium 1.3.1.Final!

This release primarily focuses on bugs that were reported after the 1.3 release. Most importantly, the following bugs were fixed related to the Debezium connector for Oracle LogMiner adapter thanks to the continued feedback by the Debezium community.

  • SQLExceptions thrown when using Oracle LogMiner (DBZ-2624)

  • LogMiner mining session stopped due to WorkerTask killed (DBZ-2629)

I am excited to announce the release of Debezium 1.4.0.Alpha1!

This first pass of the 1.4 release line provides a few useful new features:

  • New Vitess connector

  • Allow fine-grained selection of snapshotted tables

Overall, the community fixed 41 issues for this release. Let’s take a closer look at some of the highlights.

It’s with great please that I’m announcing the release of Debezium 1.3.0.Final!

As per Debezium’s quarterly release cadence, this wraps up the work of the last three months. Overall, the community has fixed 138 issues during that time, including the following key features and changes:

  • A new incubating LogMiner-based implementation for ingesting change events from Oracle

  • Support for Azure Event Hubs in Debezium Server

  • Upgrade to Apache Kafka 2.6

  • Revised filter option names

  • A new SQL Server connector snapshot mode, initial_only

  • Support for database-filtered columns for SQL Server

  • Additional connection options for the MongoDB connector

  • Improvements to ByteBufferConverter for implementing the outbox pattern with Avro as the payload format

I’m very happy to announce the release of Debezium 1.3.0.CR1!

As we approach the final stretch of Debezium 1.3 Final, we took this opportunity to add delegate converter support for the ByteBufferConverter and introduce a debezium-scripting module. In addition, there’s also a range of bug fixes and quite a bit of documentation polish; overall, not less than 15 issues have been resolved for this release.

I’m very happy to announce the release of Debezium 1.3.0.Beta2!

In this release we’ve improved support for column filtering for the MySQL and SQL Server connectors, and there’s a brand-new implementation for ingesting change events from Oracle, using the LogMiner package. As we’re on the home stretch towards Debezium 1.3 Final, there’s also a wide range of smaller improvements, bug fixes and documentation clarifications; overall, not less than 44 issues have been resolved for this release.

It’s my pleasure to announce the release of Debezium 1.3.0.Beta1!

This release upgrades to the recently released Apache Kafka version 2.6.0, fixes several critical bugs and comes with a renaming of the connector configuration options for selecting the tables to be captured. We’ve also released Debezium 1.2.2.Final, which is a drop-in replacement for all users of earlier 1.2.x releases.

I’m excited to announce the release of Debezium 1.3.0.Alpha1!

This initial pass in the 1.3 release line provides a number of useful new features:

  • A new Debezium Server sink adapter for Azure Event Hubs

  • A new SQL Server connector snapshot mode, initial_only

  • Additional connection timeout options for the MongoDB Connector

Overall, the community fixed not less than 31 issues for this release. Let’s take a closer look at some of them in the remainder of this post.

I’m very happy to announce the release of Debezium 1.2.0.Final!

Over the last three months, the community has resolved nearly 200 issues. Key features of this release include:

  • New Kafka Connect single message transforms (SMTs) for content-based event routing and filtering; Upgrade to Apache Kafka 2.5

  • Schema change topics for the Debezium connectors for SQL Server, Db2 and Oracle

  • Support for SMTs and message converters in the Debezium embedded engine

  • Debezium Server, a brand-new runtime which allows to propagate data change events to a range of messaging infrastructures like Amazon Kinesis, Google Cloud Pub/Sub, and Apache Pulsar

  • A new column masking mode "consistent hashing", allowing to anonymize column values while still keeping them correlatable

  • New metrics for the MongoDB connector

  • Improved re-connect capability for the SQL Server connector

It’s my pleasure to announce the release of Debezium 1.2.0.CR1!

This release includes several notable features, enhancements, and fixes:

  • PostgreSQL can restrict the set of tables with a publication while using pgoutput (DBZ-1813).

  • Metrics MBean registration is skipped if a platform MBean server does not exist (DBZ-2089).

  • SQL Server reconnection improved during shutdown and connection resets (DBZ-2106).

  • EventRouter SMT can now pass non-String based keys (DBZ-2152).

  • PostgreSQL include.unknown.datatypes can now return strings rather than hashes (DBZ-1266).

  • Debezium Server now supports Google Cloud PubSub (DBZ-2092).

  • Debezium Server now supports Apache Pulsar sink (DBZ-2112).

You can find the complete list of addressed issues, upgrade procedures, and notes on any backward compatibility changes in the release notes.

I’m very happy to share the news that Debezium 1.2.0.Beta2 has been released!

Core feature of this release is Debezium Server, a dedicated stand-alone runtime for Debezium, opening up its open-source change data capture capabilities towards messaging infrastructure like Amazon Kinesis.

Overall, the community has fixed 25 issues since the Beta1 release, some of which we’re going to explore in more depth in the remainder of this post.

With great happiness I’m announcing the release of Debezium 1.2.0.Beta1!

This release brings user-facing schema change topics for the SQL Server, Db2 and Oracle connectors, a new message transformation for content-based change event routing, support for a range of array column types in Postgres and much more. We also upgraded the Debezium container images for Apache Kafka and Kafka Connect to version 2.5.0.

As it’s the answer to all questions in life, the number of issues fixed for this release is exactly 42!

I’m very happy to announce the release of Debezium 1.2.0.Alpha1!

This first drop of the 1.2 release line provides a number of useful new features:

  • Support for message transformations (SMTs) and converters in the Debezium embedded engine API

  • A new SMT for filtering out change events using scripting languages

  • Automatic reconnects for the SQL Server connector

  • A new column masking mode using consistent hash values

Overall, the community fixed not less than 41 issues for this release. Let’s take a closer look at some of them in the remainder of this post.

It’s with great excitement that I’m announcing the release of Debezium 1.1.0.Final!

About three months after the 1.0 release, this new version comes with many exciting new features such as:

It’s my pleasure to announce the release of Debezium 1.1.0.Beta1!

This release adds support for transaction marker events, an incubating connector for the IBM Db2 database as well as a wide range of bug fixes. As the 1.1 release still is under active development, we’ve backported an asorted set of bug fixes to the 1.0 branch and released Debezium 1.0.1.Final, too.

At the time of writing this, not all connector archives have been synched to Maven Central yet; this should be the case within the next few others.

Today it’s my great pleasure to announce the availability of Debezium 1.0.0.Final!

Since the initial commit in November 2015, the Debezium community has worked tirelessly to realize the vision of building a comprehensive open-source low-latency platform for change data capture (CDC) for a variety of databases.

Within those four years, Debezium’s feature set has grown tremendously: stable, highly configurable CDC connectors for MySQL, Postgres, MongoDB and SQL Server, incubating connectors for Apache Cassandra and Oracle, facilities for transforming and routing change data events, support for design patterns such as the outbox pattern and much more. A very active and welcoming community of users, contributors and committers has formed around the project. Debezium is deployed to production at lots of organizations from all kinds of industries, some with huge installations, using hundreds of connectors to stream data changes out of thousands of databases.

The 1.0 release marks an important milestone for the project: based on all the production feedback we got from the users of the 0.x versions, we figured it’s about time to express the maturity of the four stable connectors in the version number, too.

While fall weather is in full swing, the Debezium community is not letting the unusually low, frigid temperatures get the best of us. It is my pleasure to announce the release of Debezium 1.0.0.Beta3!

This new Debezium release includes several notable new features, enhancements, and fixes:

  • Built against Kafka Connect 2.3.1 (DBZ-1612)

  • Renamed drop_on_stop configuration parameter to drop.on.stop (DBZ-1595)

  • Standardized source information for Cassandra connector (DBZ-1408)

  • Propagate MongoDB replicator exceptions so they are visible from Kafka Connect’s status endpoint (DBZ-1583)

  • Envelope methods should accept Instant rather than long values for timestamps (DBZ-1607)

  • Erroneously reporting no tables captured (DBZ-1519)

  • Avoid Oracle connector attempting to analyze tables (DBZ-1569)

  • Toasted columns should contain null in before rather than __debezium_unavailable_value (DBZ-1570)

  • Support PostgreSQL 11+ TRUNCATE operations using pgoutput decoder (DBZ-1576)

  • PostgreSQL connector times out in schema discovery for databases with many tables (DBZ-1579)

  • Value of ts_ms is not correct duing snapshot processing (DBZ-1588)

  • Heartbeats are not generated for non-whitelisted tables (DBZ-1592)

It is my pleasure to announce the release of Debezium 1.0.0.Beta2!

This new Debezium release includes several notable new features, enhancements, and fixes:

  • Support PostgreSQL LTREE columns with a logical data type (DBZ-1336)

  • Support for PostgreSQL 12 (DBZ-1542)

  • Validate configured PostgreSQL replication slot not contains no invalid characters (DBZ-1525)

  • Add MySQL DDL parser support for index creation VISIBLE and INVISIBLE keywords (DBZ-1534)

  • Add MySQL DDL parser support for granting SESSION_VARIABLES_ADMIN (DBZ-1535)

  • Fix MongoDB collection source struct field when collection name contains a dot (DBZ-1563)

  • Close idle transactions after performing a PostgreSQL snapshot (DBZ-1564)

History is in the making as Debezium begins to sprint to its 1.0 milestone. It’s my pleasure to announce the release of Debezium 1.0.0.Beta1!

This new Debezium release includes several notable new features, enhancements, and fixes:

  • ExtractNewDocumentState and EventRouter SMTs propagate heartbeat & schema change messages (DBZ-1513)

  • Provides alternative mapping for INTERVAL columns via interval.handling.mode (DBZ-1498)

  • Ensure message keys have the right column order (DBZ-1507)

  • Warn of table locking problems in connector logs (DBZ-1280)

On behalf of the Debezium community it’s my great pleasure to announce the release of Debezium 0.10.0.Final!

As you’d expect it, there were not many changes since last week’s CR2, one exception being a performance fix for the pgoutput plug-in of the Postgres connector, which may have suffered from slow processing when dealing with many small transactions in a short period of time (DBZ-1515).

This release finalizes the work of overall eight preview releases. We have discussed the new features and changes in depth in earlier announcements, but here are some highlights of Debezium 0.10:

I’m very happy to announce the release of Debezium 0.10.0.CR2!

After the CR1 release we decided to do another candidate release, as there was not only a good number of bug fixes coming in, but also a few very useful feature implementations were provided by the community, which we didn’t want to delay. So we adjusted the original plan a bit and now aim for Debezium 0.10 Final in the course of next week, barring any unforeseen regressions.

As usual, let’s take a closer look at some of the new features and resolved bugs.

The Debezium community is on the homestretch towards the 0.10 release and we’re happy to announce the availability of Debezium 0.10.0.CR1!

Besides a number of bugfixes to the different connectors, this release also brings a substantial improvement to the way initial snapshots can be done with Postgres. Unless any major regressions show up, the final 0.10 release should follow very soon.

This post originally appeared on the WePay Engineering blog.

In the first half of this blog post series, we explained our decision-making process of designing a streaming data pipeline for Cassandra at WePay. In this post, we will break down the pipeline into three sections and discuss each of them in more detail:

  1. Cassandra to Kafka with CDC agent

  2. Kafka with BigQuery with KCBQ

  3. Transformation with BigQuery view

This post originally appeared on the WePay Engineering blog.

Historically, MySQL had been the de-facto database of choice for microservices at WePay. As WePay scales, the sheer volume of data written into some of our microservice databases demanded us to make a scaling decision between sharded MySQL (i.e. Vitess) and switching to a natively sharded NoSQL database. After a series of evaluations, we picked Cassandra, a NoSQL database, primarily because of its high availability, horizontal scalability, and ability to handle high write throughput.