I’m pleased to announce the release of Debezium 1.6.0.Final!
This release is packed full with tons of new features, including support for incremental snapshotting that can be toggled using the new the Signal API. Based on the excellent paper DBLog: A Watermark Based Change-Data-Capture Framework by Netflix engineers Andreas Andreakis and Ioannis Papapanagiotou, the notion of incremental snapshotting addresses several requirements around snapshotting that came up repeatedly in the Debezium community:
It’s my pleasure to announce the release of Debezium 1.6.0.CR1!
This release adds skipped operations optimizations for SQL Server, introduces Heartbeat support to the Oracle connector, Oracle BLOB/CLOB support is now opt-in only, and provides a range of bug fixes and other improvements across different Debezium connectors.
It’s my pleasure to announce the release of Debezium 1.6.0.Beta2!
This release adds support for Pravega to Debezium Server, expands the snapshotting options of the Debezium Oracle connector, and provides a range of bug fixes and other improvements across different Debezium connectors.
Let me announce the bugfix release of Debezium 1.5, 1.5.2.Final!
This release is a rebuild of 1.5.1.Final using Java 8.
Let me announce the bugfix release of Debezium 1.5, 1.5.1.Final!
This release fixes a small set of issues discovered since the original release and few improvements into the documentation.
I’m pleased to announce the release of Debezium 1.6.0.Beta1!
This release introduces incremental snapshot support for SQL Server and Db2, performance improvements for SQL Server, support for BLOB/CLOB for Oracle, and much more. Lets take a few moments and explore some of these new features in the following.
It’s my pleasure to announce the first release of the Debezium 1.6 series, 1.6.0.Alpha1!
This release brings the brand new feature called incremental snapshots for MySQL and PostgreSQL connectors, a Kafka sink for Debezium Server, as well as a wide range of bug fixes and other small feature additions.
I’m thrilled to announce the release of Debezium 1.5.0.Final!
With Debezium 1.5, the LogMiner-based CDC implementation for Oracle moves from Incubating to Stable state, and there’s a brand-new implementation of the MySQL connector, which brings features like transaction metadata support. Other key features include support for a new "signalling table", which for instance can be used to implement schema changes with the Oracle connector, and support for
TRUNCATE events with Postgres. There’s also many improvements to the community-led connectors for Vitess and Apache Cassandra, as well as wide range of bug fixes and other smaller improvements.
It’s my pleasure to announce the release of Debezium 1.5.0.CR1!
As we begin moving toward finalizing the Debezium 1.5 release stream, the Oracle connector has been promoted to stable and there were some TLS improvements for the Cassandra connector, as well as numerous bugfixes. Overall, 50 issues have been addressed for this release.
We are very happy to announce the release of Debezium 1.5.0.Beta2!
The main features of this release is the new Debezium Signaling Table support, Vitess SET type support, and a continued focus to minor improvements, bugfixes, and polish as we sprint to the finish line for the 1.5 release.
Overall, the community fixed 54 issues since the Beta1 release, some of which we’ll explore more in-depth below.
I’m very happy to announce the release of Debezium 1.5.0.Beta1!
This release adds a brand-new component — the web-based Debezium UI --, transaction metadata support for the MySQL connector, a large number of improvements to the LogMiner-based capture implementation for the Debezium Oracle connector, support for Vitess 9.0, and much more. Let’s explore some of the new features in the following.
It’s my pleasure to announce the first release of the Debezium 1.5 series, 1.5.0.Alpha1!
This release brings many improvements to the LogMiner-based capture implementation for the Debezium Oracle connector, a large overhaul of the MySQL connector, as well as a wide range of bug fixes and other small feature additions.
I’m pleased to announce the release of Debezium 1.4.1.Final!
We highly recommend upgrading from 1.4.0.Final and earlier versions as this release includes bug fixes and enhancements to several Debezium connectors which includes some of the following:
I am pleased to announce the release of Debezium 1.4.0.Final!
This release concludes the major work put into Debezium over the last three months. Overall, the community fixed 117 issues during that time, including the following key features and changes:
Fine-grained selection of snapshotted tables
MySQL support for create or read records emitted during snapshot
Many Oracle Logminer adapter improvements
Full support for Oracle JDBC connection strings
Improved reporting of DDL errors
I’m pleased to announce the release of Debezium 1.4.0.CR1!
This release focuses primarily on polishing the 1.4 release.
I’m pleased to announce the release of Debezium 1.4.0.Beta1!
This release includes support for distributed tracing, lowercase table and schema naming for Db2, specifying MySQL snapshot records as create or read operations, and enhancements to Vitess for nullable and primary key columns.
I’m excited to announce the release of Debezium 1.4.0.Alpha2!
This second pass of the 1.4 release line provides a few useful new features:
New API hook for the PostgreSQL
Field renaming using
I’m excited to announce the release of Debezium 1.3.1.Final!
This release primarily focuses on bugs that were reported after the 1.3 release. Most importantly, the following bugs were fixed related to the Debezium connector for Oracle LogMiner adapter thanks to the continued feedback by the Debezium community.
I am excited to announce the release of Debezium 1.4.0.Alpha1!
This first pass of the 1.4 release line provides a few useful new features:
New Vitess connector
Allow fine-grained selection of snapshotted tables
Overall, the community fixed 41 issues for this release. Let’s take a closer look at some of the highlights.
It’s with great please that I’m announcing the release of Debezium 1.3.0.Final!
As per Debezium’s quarterly release cadence, this wraps up the work of the last three months. Overall, the community has fixed 138 issues during that time, including the following key features and changes:
A new incubating LogMiner-based implementation for ingesting change events from Oracle
Support for Azure Event Hubs in Debezium Server
Upgrade to Apache Kafka 2.6
Revised filter option names
A new SQL Server connector snapshot mode,
Support for database-filtered columns for SQL Server
Additional connection options for the MongoDB connector
ByteBufferConverterfor implementing the outbox pattern with Avro as the payload format
I’m very happy to announce the release of Debezium 1.3.0.CR1!
As we approach the final stretch of Debezium 1.3 Final, we took this opportunity to add delegate converter support for the
ByteBufferConverter and introduce a
debezium-scripting module. In addition, there’s also a range of bug fixes and quite a bit of documentation polish; overall, not less than 15 issues have been resolved for this release.
I’m very happy to announce the release of Debezium 1.3.0.Beta2!
In this release we’ve improved support for column filtering for the MySQL and SQL Server connectors, and there’s a brand-new implementation for ingesting change events from Oracle, using the LogMiner package. As we’re on the home stretch towards Debezium 1.3 Final, there’s also a wide range of smaller improvements, bug fixes and documentation clarifications; overall, not less than 44 issues have been resolved for this release.
It’s my pleasure to announce the release of Debezium 1.3.0.Beta1!
This release upgrades to the recently released Apache Kafka version 2.6.0, fixes several critical bugs and comes with a renaming of the connector configuration options for selecting the tables to be captured. We’ve also released Debezium 1.2.2.Final, which is a drop-in replacement for all users of earlier 1.2.x releases.
I’m excited to announce the release of Debezium 1.3.0.Alpha1!
This initial pass in the 1.3 release line provides a number of useful new features:
A new Debezium Server sink adapter for Azure Event Hubs
A new SQL Server connector snapshot mode,
Additional connection timeout options for the MongoDB Connector
Overall, the community fixed not less than 31 issues for this release. Let’s take a closer look at some of them in the remainder of this post.
I am happy to announce the release of Debezium 1.2.1.Final!
This release includes several bug fixes to different Debezium connectors, and we highly recommend the upgrade from 1.2.0.Final and earlier versions:
The Debezium Postgres connector may have missed events from concurrent transactions when transitioning from snapshotting to streaming events from the WAL (DBZ-2288); this is fixed now when using the exported snapshotting mode; this mode should preferably be used, and for Debezium 1.3 we’re planning for this to be the basis for all the existing snapshotting modes
The Postgres JDBC driver got upgraded to 42.2.14 (DBZ-2317), which fixes a CVE in the driver related to processing XML column values sourced from untrusted XML input
The MySQL connector automatically filters out specific DML binlog entries from internal tables when using it with Amazon RDS (DBZ-2275)
The Debezium MongoDB connector got more resilient against connection losses (DBZ-2141)
I’m very happy to announce the release of Debezium 1.2.0.Final!
Over the last three months, the community has resolved nearly 200 issues. Key features of this release include:
Support for SMTs and message converters in the Debezium embedded engine
Debezium Server, a brand-new runtime which allows to propagate data change events to a range of messaging infrastructures like Amazon Kinesis, Google Cloud Pub/Sub, and Apache Pulsar
A new column masking mode "consistent hashing", allowing to anonymize column values while still keeping them correlatable
New metrics for the MongoDB connector
Improved re-connect capability for the SQL Server connector
It’s my pleasure to announce the release of Debezium 1.2.0.CR1!
This release includes several notable features, enhancements, and fixes:
PostgreSQL can restrict the set of tables with a publication while using pgoutput (DBZ-1813).
Metrics MBean registration is skipped if a platform MBean server does not exist (DBZ-2089).
SQL Server reconnection improved during shutdown and connection resets (DBZ-2106).
EventRouter SMT can now pass non-String based keys (DBZ-2152).
include.unknown.datatypescan now return strings rather than hashes (DBZ-1266).
Debezium Server now supports Google Cloud PubSub (DBZ-2092).
Debezium Server now supports Apache Pulsar sink (DBZ-2112).
You can find the complete list of addressed issues, upgrade procedures, and notes on any backward compatibility changes in the release notes.
Many thanks to all the community members contributing to this release: Andy Teijelo Pérez, Balázs Németh, Bingqin Zhou, Brandon Brown, cobolbaby, Dave Cumberland, Ed Laur, Emmanuel Brard, Fabian Aussems, Ivan Trusov, Justin Hiza, Jeremy Finzel, Kewei Shang, Lukas Krejci, and Robert B. Hanviriyapunt.
I’m very happy to share the news that Debezium 1.2.0.Beta2 has been released!
Core feature of this release is Debezium Server, a dedicated stand-alone runtime for Debezium, opening up its open-source change data capture capabilities towards messaging infrastructure like Amazon Kinesis.
Overall, the community has fixed 25 issues since the Beta1 release, some of which we’re going to explore in more depth in the remainder of this post.
With great happiness I’m announcing the release of Debezium 1.2.0.Beta1!
This release brings user-facing schema change topics for the SQL Server, Db2 and Oracle connectors, a new message transformation for content-based change event routing, support for a range of array column types in Postgres and much more. We also upgraded the Debezium container images for Apache Kafka and Kafka Connect to version 2.5.0.
As it’s the answer to all questions in life, the number of issues fixed for this release is exactly 42!
I’m very happy to announce the release of Debezium 1.2.0.Alpha1!
This first drop of the 1.2 release line provides a number of useful new features:
Support for message transformations (SMTs) and converters in the Debezium embedded engine API
A new SMT for filtering out change events using scripting languages
Automatic reconnects for the SQL Server connector
A new column masking mode using consistent hash values
Overall, the community fixed not less than 41 issues for this release. Let’s take a closer look at some of them in the remainder of this post.
It’s with great excitement that I’m announcing the release of Debezium 1.1.0.Final!
About three months after the 1.0 release, this new version comes with many exciting new features such as:
a Quarkus extension facilitating the outbox pattern
support for the CloudEvents specification
an incubating connector for the IBM Db2 database
transaction marker events
support for CDC integration testing via Testcontainers
It’s my pleasure to announce the release of Debezium 1.1.0.CR1!
This release brings a brand-new API module, including a facility for overriding the schema and value conversion of specific columns. The Postgres connector gained the ability to reconnect to the database after a connection loss, and the MongoDB connector supports the metrics known from other connectors now.
Release early, release often! After the 1.1 Beta1 and 1.0.1 Final releases earlier this week, I’m today happy to share the news about the release of Debezium 1.1.0.Beta2!
The main addition in Beta2 is support for integration tests of your change data capture (CDC) set-up using Testcontainers. In addition, the Quarkus extension for implementing the outbox pattern as well as the SMT for extracting the
after state of change events have been re-worked and offer more configuration flexibility now.
Did you know January 16th is National Nothing Day? It’s the one day in the year without celebrating, observing or honoring anything.
Well, normally, that is. Because we couldn’t stop ourselves from sharing the news of the Debezium 1.1.0.Alpha1 release with you! It’s the first release after Debezium 1.0, and there are some really useful features coming with it. Let’s take a closer look.
Today it’s my great pleasure to announce the availability of Debezium 1.0.0.Final!
Since the initial commit in November 2015, the Debezium community has worked tirelessly to realize the vision of building a comprehensive open-source low-latency platform for change data capture (CDC) for a variety of databases.
Within those four years, Debezium’s feature set has grown tremendously: stable, highly configurable CDC connectors for MySQL, Postgres, MongoDB and SQL Server, incubating connectors for Apache Cassandra and Oracle, facilities for transforming and routing change data events, support for design patterns such as the outbox pattern and much more. A very active and welcoming community of users, contributors and committers has formed around the project. Debezium is deployed to production at lots of organizations from all kinds of industries, some with huge installations, using hundreds of connectors to stream data changes out of thousands of databases.
The 1.0 release marks an important milestone for the project: based on all the production feedback we got from the users of the 0.x versions, we figured it’s about time to express the maturity of the four stable connectors in the version number, too.
Did you know December 12th is National Ding-a-Ling Day? It’s the day to call old friends you haven’t heard from in a while. So we thought we’d get in touch (not that is has been that long) with our friends, i.e. you, and share the news about the release of Debezium 1.0.0.CR1!
It’s the first, and ideally only, candidate release; so Debezium 1.0 should be out very soon. Quite a few nice features found their way into CR1:
Graceful handling of MongoDB 4.0 transaction events (DBZ-1215)
While fall weather is in full swing, the Debezium community is not letting the unusually low, frigid temperatures get the best of us. It is my pleasure to announce the release of Debezium 1.0.0.Beta3!
This new Debezium release includes several notable new features, enhancements, and fixes:
Built against Kafka Connect 2.3.1 (DBZ-1612)
drop_on_stopconfiguration parameter to
Standardized source information for Cassandra connector (DBZ-1408)
Propagate MongoDB replicator exceptions so they are visible from Kafka Connect’s status endpoint (DBZ-1583)
Envelope methods should accept
longvalues for timestamps (DBZ-1607)
Erroneously reporting no tables captured (DBZ-1519)
Avoid Oracle connector attempting to analyze tables (DBZ-1569)
Toasted columns should contain
nullin before rather than
Support PostgreSQL 11+
PostgreSQL connector times out in schema discovery for databases with many tables (DBZ-1579)
ts_msis not correct duing snapshot processing (DBZ-1588)
Heartbeats are not generated for non-whitelisted tables (DBZ-1592)
It is my pleasure to announce the release of Debezium 1.0.0.Beta2!
This new Debezium release includes several notable new features, enhancements, and fixes:
LTREEcolumns with a logical data type (DBZ-1336)
Support for PostgreSQL 12 (DBZ-1542)
Validate configured PostgreSQL replication slot not contains no invalid characters (DBZ-1525)
Add MySQL DDL parser support for index creation
Add MySQL DDL parser support for granting
collectionsource struct field when collection name contains a dot (DBZ-1563)
Close idle transactions after performing a PostgreSQL snapshot (DBZ-1564)
History is in the making as Debezium begins to sprint to its 1.0 milestone. It’s my pleasure to announce the release of Debezium 1.0.0.Beta1!
This new Debezium release includes several notable new features, enhancements, and fixes:
ExtractNewDocumentState and EventRouter SMTs propagate heartbeat & schema change messages (DBZ-1513)
Provides alternative mapping for
Ensure message keys have the right column order (DBZ-1507)
Warn of table locking problems in connector logs (DBZ-1280)
On behalf of the Debezium community it’s my great pleasure to announce the release of Debezium 0.10.0.Final!
As you’d expect it, there were not many changes since last week’s CR2, one exception being a performance fix for the
pgoutput plug-in of the Postgres connector, which may have suffered from slow processing when dealing with many small transactions in a short period of time (DBZ-1515).
This release finalizes the work of overall eight preview releases. We have discussed the new features and changes in depth in earlier announcements, but here are some highlights of Debezium 0.10:
I’m very happy to announce the release of Debezium 0.10.0.CR2!
After the CR1 release we decided to do another candidate release, as there was not only a good number of bug fixes coming in, but also a few very useful feature implementations were provided by the community, which we didn’t want to delay. So we adjusted the original plan a bit and now aim for Debezium 0.10 Final in the course of next week, barring any unforeseen regressions.
As usual, let’s take a closer look at some of the new features and resolved bugs.
The Debezium community is on the homestretch towards the 0.10 release and we’re happy to announce the availability of Debezium 0.10.0.CR1!
Besides a number of bugfixes to the different connectors, this release also brings a substantial improvement to the way initial snapshots can be done with Postgres. Unless any major regressions show up, the final 0.10 release should follow very soon.
The temperatures are slowly cooling off after the biggest summer heat, an the Debezium community is happy to announce the release of Debezium 0.10.0.Beta4. In this release we’re happy to share some news we don’t get to share too often: with Apache Cassandra, another database gets added to the list of databases supported by Debezium!
In addition, we finished our efforts for rebasing the existing Postgres connector to Debezium framework structure established for the SQL Server and Oracle connectors. This means more shared coded between these connectors, and in turn reduced maintenance efforts for the development team going forward; but there’s one immediately tangible advantage for you coming with this, too: the Postgres connector now exposes the same metrics you already know from the other connectors.
Finally, the new release contains a range of bugfixes and other useful improvements. Let’s explore some details below.
The summer is at its peak but Debezium community is not relenting in its effort so the Debezium 0.10.0.Beta3 is released.
This version not only continues in incremental improvements of Debezium but also brings new shiny features.
All of you who are using PostgreSQL 10 and higher as a service offered by different cloud providers definitely felt the complications when you needed to deploy logical decoding plugin necessary to enable streaming. This is no longer necessary. Debezium now supports (DBZ-766) pgoutput replication protocol that is available out-of-the-box since PostgreSQL 10.
It’s my pleasure to announce the release of Debezium 0.10.0.Beta2!
This further stabilizes the 0.10 release line, with lots of bug fixes to the different connectors. 23 issues were fixed for this release; a couple of those relate to the DDL parser of the MySQL connector, e.g. around
RENAME INDEX (DBZ-1329),
SET NEW in triggers (DBZ-1331) and function definitions with the
COLLATE keyword (DBZ-1332).
For the Postgres connector we fixed a potential inconsistency when flushing processed LSNs to the database (DBZ-1347). Also the "include.unknown.datatypes" option works as expected now during snapshotting (DBZ-1335) and the connector won’t stumple upon materialized views during snapshotting any longer (DBZ-1345).
Another week, another Debezium release — I’m happy to announce the release of Debezium 0.10.0.Beta1!
A very welcomed usability improvement is that the connectors will log a warning now if not at least one table is actually captured as per the whitelist/blacklist configuration (DBZ-1242). This helps to prevent the accidental exclusion all tables by means of an incorrect filter expression, in which case the connectors "work as intended", but no events are propagated to the message broker.
Please see the release notes for the complete list of issues fixed in this release. Also make sure to examine the upgrade guidelines for 0.10.0.Alpha1 and Alpha2 when upgrading from earlier versions.
Release early, release often — Less than a week since the Alpha1 we are announcing the release of Debezium 0.10.0.Alpha2!
This is an incremental release that completes some of the tasks started in the Alpha1 release and provides a few bugfixes and also quality improvements in our Docker images.
I’m very happy to announce the release of Debezium 0.10.0.Alpha1!
The major theme for Debezium 0.10 will be to do some clean-up (that’s what you do at this time of the year, right?); we’ve planned to remove a few deprecated features and to streamline some details in the structure the CDC events produced by the different Debezium connectors.
This means that upgrading to Debezium 0.10 from earlier versions might take a bit more planning and consideration compared to earlier upgrades, depending on your usage of features and options already marked as deprecated in 0.9 and before. But no worries, we’re describing all changes in great detail in this blog post and the release notes.
It’s my pleasure to announce the release of Debezium 0.9.5.Final!
This is a recommended update for all users of earlier versions; besides bug fixes also a few new features are provide. The release contains 18 resolved issues overall.
The Debezium team is happy to announce the release of Debezium 0.9.3.Final!
This is mostly a bug-fix release and a drop-in replacement for earlier Debezium 0.9.x versions, but there are few significant new features too. Overall, 17 issues were resolved.
|Container images will be released with a small delay due to some Docker Hub configuration issues.|
The Debezium team is happy to announce the release of Debezium 0.9.2.Final!
This is mostly a bug-fix release and a drop-in replacement for earlier Debezium 0.9.x versions. Overall, 18 issues were resolved.
A couple of fixes relate to the Debezium Postgres connector:
Quickly following up to last week’s release of Debezium 0.9, it’s my pleasure today to announce the release of Debezium 0.9.1.Final!
This release fixes a couple of bugs which were reported after the 0.9 release. Most importantly, there are two fixes to the new Debezium connector for SQL Server, which deal with correct handling of LSNs after connector restarts (DBZ-1128, DBZ-1131). The connector also uses more reasonable defaults for the
fetchSize options of the SQL Server JDBC driver (DBZ-1065), which can help to significantly increase through-put and reduce memory consumption of the connector.
I’m delighted to announce the release of Debezium 0.9 Final!
This release only adds a small number of changes since last week’s CR1 release; most prominently there’s some more metrics for the SQL Server connector (lag behind master, number of transactions etc.) and two bug fixes related to the handling of partitioned tables in MySQL (DBZ-1113) and Postgres (DBZ-1118).
Having been in the works for six months after the initial Alpha release, Debezium 0.9 comes with a brand new connector for SQL Server, lots of new features and improvements for the existing connectors, updates to the latest versions of Apache Kafka and the supported databases as well as a wide range of bug fixes.
Reaching the home stretch towards Debezium 0.9, it’s with great pleasure that I’m announcing the first release of Debezium in 2019, 0.9.0.CR1!
For this release we’ve mainly focused on sorting out remaining issues in the Debezium connector for SQL Server; the connector comes with greatly improved performance and has received a fair number of bug fixes.
Other changes include a new interface for event handlers of Debezium’s embedded engine, which allows for bulk handling of change events, an option to export the scale of numeric columns as schema parameter, as well as a wide range of bug fixes for the Debezium connectors for MySQL, Postgres and Oracle.
With only a few days left for the year, it’s about time for another Debezium release; so it’s with great pleasure that I’m announcing Debezium 0.9.0.Beta2!
This release comes with support for MySQL 8 and Oracle 11g; it includes a first cut of metrics for monitoring the SQL Server and Oracle connectors, several improvements to the MongoDB event flattening SMT as well as a wide range of bug fixes. Overall, not less than 42 issues were addressed; very clearly, there has to be some deeper sense in that ;)
A big shout out goes to the following members Debezium’s amazing community, who contributed to this release: Eero Koplimets, Grzegorz Kołakowski, Hanlin Liu, Lao Mei, Renato Mefi, Tautvydas Januskevicius, Wout Scheepers and Zheng Wang!
In the following, let’s take a closer look at some of the changes coming with the 0.9 Beta2 release.
It’s my pleasure to announce the release of Debezium 0.9.0.Beta1! Oh, and to those of you who are celebrating it — Happy Thanksgiving!
This new Debezium release comes with several great improvements to our work-in-progress SQL Server connector:
Initial snapshots can be done using the
snapshotisolation level if enabled in the DB (DBZ-941)
Changes to the structures of captured tables after the connector has been set up are supported now (DBZ-812)
It’s my pleasure to announce the release of Debezium 0.9.0.Alpha2!
While the work on the connectors for SQL Server and Oracle continues, we decided to do another Alpha release, as lots of fixes and new features - many of them contributed by community members - have piled up, which we wanted to get into your hands as quickly as possible.
This release supports Apache Kafka 2.0, comes with support for Postgres' HSTORE column type, allows to rename and filter fields from change data messages for MongoDB and contains multiple bug fixes and performance improvements. Overall, this release contains 55 fixes (note that a few of these have been merged back to 0.8.x and are contained in earlier 0.8 releases, too).
Just two weeks after the Debezium 0.8 release, I’m very happy to announce the release of Debezium 0.9.0.Alpha1!
The main feature of the new version is a first work-in-progress version of the long-awaited Debezium connector for MS SQL Server. Based on the CDC functionality available in the Enterprise and Standard editions, the new connector lets you stream data changes out of Microsoft’s popular RDBMS.
Besides that we’ve continued the work on the Debezium Oracle connector. Most notably, it supports initial snapshots of captured tables now. We’ve also upgraded Apache Kafka in our Docker images to 1.1.1 (DBZ-829).
Please take a look at the change log for the complete list of changes in 0.9.0.Alpha1 and general upgrade notes.