1. Releases
  2. 2.5
  3. Release Notes for Debezium 2.5

Release Notes for Debezium 2.5

All notable changes for Debezium releases are documented in this file. Release numbers follow Semantic Versioning.

Release 2.5.0.Beta1 (December 4th 2023)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 3.6.0 and has been tested with version 3.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.5.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.5.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 2.5.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

MongoDB no longer allows doing collection snapshot on a specific shard in sharded deployment. This was never intended and supported feature and was removed as a step in complete replica set streaming mode removal (DBZ-7139).

ComputePartition SMT was deprecated and replaced with significantly improved PartitionRouting SMT. The original SMT is now completely removed (DBZ-7141).

JDBC Sink connector was storing default values in database columns when null value was streamed and such value existed. This was incorrect behaviour and is now fixed (DBZ-7191).

New features

  • Support for mariadb GTID DBZ-1482

  • Include only certain columns in JDBC sink connector DBZ-6636

  • Support native RabbitMQ Streams DBZ-6703

  • Add support for partitioning with Azure EventHubs DBZ-6723

  • Enhance Notification information and more notifications for Initial Snapshots DBZ-6878

  • Add handling for CDB and non-CDB / PDB in Oracle REST Extension tests DBZ-7091

  • Check schema length when create value to find missed DDL by SQL_BIN_LOG=OFF DBZ-7093

  • Add service account parameter to DebeziumServer CRD DBZ-7111

  • Inactivity pause in MongoDB connector should be configurable DBZ-7146

  • Oracle Infinispan event processor speed-up using in memory cache DBZ-7153

  • Add last event process time, number of events, number of heartbeat events metrics to MongoDb connector DBZ-7162

  • LogMiner ISPN event buffer recent transaction optimization DBZ-7169

  • Support logical decoding from Postgres 16 stand-bys DBZ-7181

  • Support MySQL 8 high resolution replication timestamps from GTID events DBZ-7183

  • Use buffer queue when reading MongoDB change stream events DBZ-7184

  • Cleanup event processing loop in streaming event source of MongoDB connector DBZ-7186

  • Oracle Infinispan - implement support for abandoned transactions DBZ-7192

  • Add ability to avoid throwing an exception for missing additional fields DBZ-7197

  • XStream attach should be retriable DBZ-7207

Fixes

  • Test Avro adjustment for MongoDb connector and ExtractNewDocumentState SMT DBZ-6809

  • The DefaultDeleteHandlingStrategy couldn’t add the rewrite "__deleted" field to a non-struct value DBZ-7066

  • Debezium server has no default for offset.flush.interval.ms DBZ-7099

  • Failed to authenticate to the MySQL database after snapshot DBZ-7132

  • Failure reading CURRENT_TIMESTAMP on Informix 12.10 DBZ-7137

  • Debezium-ddl-parser crashes on parsing MySQL DDL statement (specific UNION) DBZ-7140

  • outbox.EventRouter SMT throws NullPointerException when there is a whitespace in fields.additional.placement value DBZ-7142

  • Debezium-ddl-parser crashes on parsing MySQL DDL statement (specific UPDATE) DBZ-7152

  • JsonSerialisation is unable to process changes from sharded collections with composite sharding key DBZ-7157

  • Log sequence check should treat each redo thread independently DBZ-7158

  • Fix DebeziumMySqlConnectorResource not using the new MySQL adatper structure to support different MySQL flavors DBZ-7179

  • Parsing MySQL indexes for JSON field fails, when casting is used with types double and float DBZ-7189

  • Unchanged toasted array columns are substituted with unavailable.value.placeholder, even when REPLICA IDENTITY FULL is configured. DBZ-7193

  • MongoDB streaming pauses for Blocking Snapshot only when there is no event DBZ-7206

  • NPE on AbstractInfinispanLogMinerEventProcessor.logCacheStats DBZ-7211

Other changes

  • Generate sundrio fluent builders for operator model DBZ-6550

  • Convert operator source into multi module project DBZ-6551

  • Implement "validate filters" endpoint in connector-specific Connect REST extensions DBZ-6762

  • Implement IT tests against Cloud Spanner emulator in main repo. DBZ-6906

  • Implement strategy pattern for MariaDB and MySQL differences DBZ-7083

  • Run MySQL CI builds in parallel DBZ-7135

  • Add matrix strategy to workflows DBZ-7154

  • Add Unit Tests for ServiceAccountDependent Class in Debezium Operator Repository DBZ-7155

  • Fail fast during deserialization if a value is not a CloudEvent DBZ-7159

  • Correctly calculate Max LSN DBZ-7175

  • Upgrade to Infinispan 14.0.20 DBZ-7187

  • Upgrade Outbox Extension to Quarkus 3.5.3 DBZ-7188

  • Enable ability to stream changes against Oracle 23c for LogMiner DBZ-7194

  • Add modify range_partitions to modify_table_partition rule in parsing PL/SQL DBZ-7196

  • Handle Drop Tablespace in PL/SQL DBZ-7208

  • Upgrade logback to 1.2.12 DBZ-7209

Release 2.5.0.Alpha2 (November 10th 2023)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 3.6.0 and has been tested with version 3.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.5.0.Alpha2 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.5.0.Alpha2 plugin files, and restart the connector using the same configuration. Upon restart, the 2.5.0.Alpha2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

MySQL 5.7 is end of life. Debezium is no longer officially supporting it or testing with it. Further support is on best-effort basis only (DBZ-6874).

Configuration option metadata.location was renamed to metadata.source for CloudEvents converter (DBZ-7060).

MongoDB default connection mode is switched from replica_set to sharded. This is the first step in removing the replica_set mode completely. Please note that the offsets will be invalidated and snapshot triggered for existing deployments unless the configuration option is explicitly set (DBZ-7108).

Using deprecated embbeded engine API is no longer possible. Only Debezium Engine API is available (DBZ-7110).

New features

  • JDBC Sink Connector - Support batch operations DBZ-6317

  • Utilize $changeStreamSplitLargeEvent to handle large change events with post and pre images DBZ-6726

  • Add support for MySQL 8.2 DBZ-6873

  • Kinesis Sink Reliability DBZ-7032

  • Upgrade MSSQL JDBC driver to support sensitivity classification DBZ-7109

  • Add maximum retry limit to Redis Schema History DBZ-7120

  • Emit a notification when completed reading from a capture instance DBZ-7043

Fixes

  • Oracle RAC throws ORA-00310: archive log sequence required DBZ-5350

  • oracle missing CDC data DBZ-5656

  • Missing oracle cdc records DBZ-5750

  • Connector frequently misses commit operations DBZ-6942

  • Missing events from Oracle 19c DBZ-6963

  • Debezium Embedded Infinispan Performs Slowly DBZ-7047

  • Field exclusion does not work with events of removed fields DBZ-7058

  • JDBC sink connector not working with CloudEvent DBZ-7065

  • JDBC connection leak when error occurs during processing DBZ-7069

  • Some server tests fail due to @com.google.inject.Inject annotation DBZ-7077

  • HttpIT fails with "Unrecognized field subEvents" DBZ-7092

  • MySQL parser does not conform to arithmetical operation priorities DBZ-7095

  • When RelationalBaseSourceConnector#validateConnection is called with invalid config [inside Connector#validate()] can lead to exceptions DBZ-7105

  • Debezium crashes on parsing MySQL DDL statement (specific INSERT) DBZ-7119

Other changes

  • Add (integration) tests for Oracle connector-specific Debezium Connect REST extension DBZ-6763

  • Intermittent failure of MongoDbReplicaSetAuthTest DBZ-6875

  • Mongodb tests in RHEL system testsuite are failing with DBZ 2.3.4 DBZ-6996

  • Use DebeziumEngine instead of EmbeddedEngine in the testsuite DBZ-7007

  • Update transformation property "delete.tombstone.handling.mode" to debezium doc DBZ-7062

  • Add MariaDB driver for testing and distribution DBZ-7085

  • Allow DS JMX to use username-password authentication on k8 DBZ-7087

  • VitessConnectorIT.shouldTaskFailIfColumnNameInvalid fails DBZ-7104

Release 2.5.0.Alpha1 (October 26th 2023)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 3.6.0 and has been tested with version 3.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.5.0.Alpha1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.5.0.Alpha1 plugin files, and restart the connector using the same configuration. Upon restart, the 2.5.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

ExtractNewRecordState and ExtractNewDocumentState SMT configuration was reworked. Two disting options the separately controlled handling of delete and tombstone records were merged into a single one (DBZ-6907).

Support for MongoDB 4.4 is deprecated (DBZ-6881).

New features

  • Provide first class support for MariaDB DBZ-2913

  • Support for IBM Informix DBZ-4999

  • Add support for honouring MongoDB read preference in change stream after promotion DBZ-5953

  • Enable Spanner Connector against Cloud Spanner Emulator DBZ-6845

  • Refactor Oracle streaming metrics beans DBZ-6899

  • Provide capability to set image pull secrets in DS k8s CRD DBZ-6962

  • Upgrade to Vitess 17 for integration tests DBZ-6981

  • Add the ability to sanitize field name when inferencing json schema DBZ-6983

  • Allow OLM Bundle scripts to download from maven central by default DBZ-6995

  • Enhance README.md with Instructions for Creating a Kubernetes Namespace DBZ-7004

  • Support OKD/Openshift catalog in OH release script DBZ-7010

  • Add displayName and description metadata to DebeziumServer CRD in OLM Bundle DBZ-7011

  • Upgrade Kafka to 3.6.0 DBZ-7033

  • DebeziumConnector always attempts to contact Quay.io to determine latest stable version DBZ-7044

  • Support snapshot with automatic retry DBZ-7050

  • Provide resources to set pod requests and limits in DS k8s CRD DBZ-7052

  • Provide svc to better collects dbz-server metrics in DS k8s DBZ-7053

  • Improve logging at DEBUG level for Commit events DBZ-7067

  • Replace schema tracking restriction for SYS/SYSTEM users with configurable option DBZ-7071

Fixes

  • Multiple debezium:offsets Redis clients DBZ-6952

  • Wrong case-behavior for non-avro column name in sink connector DBZ-6958

  • Handle properly bytea field for jdbc sink to postgresql DBZ-6967

  • Debezium jdbc sink process truncate event failure DBZ-6970

  • Single quote replication includes escaped quotes for N(CHAR/VARCHAR) columns DBZ-6975

  • Debezium jdbc sink should throw not supporting schema change topic exception DBZ-6990

  • Debezium doesn’t compile with JDK 21 DBZ-6992

  • OLM bundle version for GA releases is invalid DBZ-6994

  • ALTER TABLE fails when adding multiple columns to JDBC sink target DBZ-6999

  • Invalid Link to zulip chat in CSV metadata DBZ-7000

  • Make sure to terminate the task once connectivity is lost to either the rebalance or sync topic DBZ-7001

  • Missing .metadata.annotations.repository field in CSV metadata DBZ-7003

  • Single quote replication and loss of data DBZ-7006

  • Oracle connector: Payload size over 76020 bytes are getting truncated DBZ-7018

  • DDL statement couldn’t be parsed DBZ-7030

  • Blocking ad-hoc snapshot is not really blocking for MySQL DBZ-7035

  • Fake ROTATE event on connection restart cleans metadata DBZ-7037

Other changes

  • Adding Debezium Server example using MySQL and GCP PubSub DBZ-4471

  • Test Debezium against MSSQL 2016 DBZ-6693

  • Test Debezium against DB2 1.5.8.0 DBZ-6694

  • Add MSSQL 2022 to test matrix DBZ-6695

  • Edit test matrix after team evaluation DBZ-6696

  • Edit test automation to run both DB2 1.5.8.0 and 1.5.0.0a DBZ-6697

  • Refactor ElapsedTimeStrategy DBZ-6778

  • Provide configuration option to exclude extension attributes from a CloudEvent DBZ-6982

  • Further refactoring to correct downstream rendering of incremental snapshots topics DBZ-6997

  • Remove deprecated embedded engine code DBZ-7013

  • Enable replication slot advance check DBZ-7015

  • Add configuration option to CloudEventsConverter to retrieve id and type from headers DBZ-7016

  • Use optional schema for Timezone Converter tests DBZ-7020

  • Debezium Operator blogpost DBZ-7025

  • Apply 2.3.4 updates to main branch DBZ-7039

  • Update documentation with Postgres’s pgoutput limitation DBZ-7041

  • Use oracle container registry for MySQL images DBZ-7042

  • Updates to fix build of downstream doc DBZ-7046

  • Update operator dependencies and add qosdk platform bom DBZ-7048

  • Upgrade maven-surefire-plugin to 3.1.2 DBZ-7055

  • Consolidate resource labels and annotations DBZ-7064

  • Disable time sync in Testing farm test runs DBZ-7074