Release Notes for Debezium 2.1

All notable changes for Debezium releases are documented in this file. Release numbers follow Semantic Versioning.

Release 2.1.4.Final (April 4th 2023)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.4.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.4.Final plugin files, and restart the connector using the same configuration. Upon restart, the 2.1.4.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

There are no breaking changes in this release.

New features

There are no new features in this release.

Fixes

  • java.lang.NullPointerException in MySQL connector with max.queue.size.in.bytes DBZ-6104

  • debezium-connector-mysql failed to parse serveral DDLs of 'CREATE TABLE' DBZ-6124

  • Zerofill property failed for different int types DBZ-6185

  • GRANT DELETE HISTORY couldn’t be parsed in mariadb DBZ-6186

  • ddl parse failed for key partition table DBZ-6188

  • Use CHARSET for alterByConvertCharset clause DBZ-6194

  • Data loss upon connector restart DBZ-6204

  • DDL statement couldn’t be parsed: no viable alternative at input 'GRANT APPLICATION_PASSWORD_ADMIN' DBZ-6211

  • ParsingException: DDL statement couldn’t be parsed DBZ-6217

  • The CHARACTER/CHARACTER(p)/CHARACTER VARYING(p) data types not recognized as JDBC type CHAR DBZ-6221

  • MySQL singleDeleteStatement parser does not support table alias DBZ-6243

  • Missing GEOMETRY keyword which can be used as column name DBZ-6250

  • MariaDB’s UUID column type cannot be parsed when scheme is loaded DBZ-6255

  • Multiplatform build of example-postres fails DBZ-6258

  • Add protoc version property to postgres connector pom.xml DBZ-6261

  • Table names with spaces are not correctly deserialized when using an Infinispan cache as the transaction buffer DBZ-6273

  • Transaction buffer state can become corrupted when using Infinispan cache with LOBs DBZ-6275

Other changes

  • Update connector configuration examples in deployment instructions DBZ-6153

  • Insert missing Nebel annotations for Oracle connector FAQ topic DBZ-6215

  • Add metadata for MongoDB change streams topic DBZ-6223

  • Deprecate MongoDb 4.0 DBZ-6246

  • Fix broken link to Streams documentation in shared deployment files DBZ-6263

  • Update config example in Installing Debezium on OpenShift DBZ-6267

  • Address review feedback in downstream RHEL and OCP installation guides DBZ-6272

  • Infinispan cache configuration used by Oracle tests are not compatible with Infinispan 14.0.2 DBZ-6274

  • Upgrade MySQL JDBC driver to 8.0.32 DBZ-6304

Release 2.1.3.Final (March 9th 2023)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.3.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.3.Final plugin files, and restart the connector using the same configuration. Upon restart, the 2.1.3.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

SSN field from Oracle connector was propagated as INT32 in the source info block. This could lead to overflows on certain installations so the field is now propagated as INT64 (DBZ-6091).

Debezium was truncating on timezoned types milli/microsecond zeroes regardless of the schema width setting. This is no longer the case and Debezium provides the correct number of trailing zeroes (DBZ-6163).

New features

  • Reduce verbosity of skipped transactions if transaction has no events relevant to captured tables DBZ-6094

  • Support String type for key in Mongo incremental snapshot DBZ-6116

  • Add hostname validator to connector configuration DBZ-6156

Fixes

  • ActivateTracingSpan wrong timestamps reported DBZ-5827

  • Postgresql Data Loss on restarts DBZ-5915

  • debezium-connector-cassandra 2.1.0.Alpha2 plugin can no longer run "out of the box" DBZ-5925

  • MongoDB Incremental Snapshot not Working DBZ-5973

  • Mask password in log statement DBZ-6064

  • Loading Custom offset storage fails with Class not found error DBZ-6075

  • SQL Server tasks fail if the number of databases is smaller than maxTasks DBZ-6084

  • GCP Spanner connector start failing when there are multiple indexes on a single column DBZ-6101

  • When using LOB support, an UPDATE against multiple rows can lead to inconsistent event data DBZ-6107

  • Negative remaining attempts on MongoDB reconnect case DBZ-6113

  • Tables with spaces or non-ASCII characters in their name are not captured by Oracle because they must be quoted. DBZ-6120

  • Offsets are not advanced in a CDB deployment with low frequency of changes to PDB DBZ-6125

  • Oracle TIMESTAMP WITH TIME ZONE is emitted as GMT during snapshot rather than the specified TZ DBZ-6143

  • Config options internal.schema.history.internal.ddl.filter not working DBZ-6190

Other changes

  • Prepare MongoDB ExtractNewDocumentState SMT doc for downstream GA DBZ-6006

  • Refactor OCP deployment job DBZ-6044

  • Refactor ARO deployment job DBZ-6045

  • Invalid links breaking downstream documentation build DBZ-6069

  • Remove references to adding configuration settings to a .properties file DBZ-6130

  • Upgrade Quarkus dependencies to 2.16.3.Final DBZ-6150

  • Disable advance slot checking DBZ-6191

Release 2.1.1.Final (December 22nd 2022)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.1.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.1.Final plugin files, and restart the connector using the same configuration. Upon restart, the 2.1.1.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

There are no breaking changes in this release.

New features

There are no new features in this release.

Fixes

  • Spanner connector is missing JSR-310 dependency DBZ-5959

Other changes

There are no other changes in this release.

Release 2.1.2.Final (January 26th 2023)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.2.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.2.Final plugin files, and restart the connector using the same configuration. Upon restart, the 2.1.2.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

ZonedTimestamp strings were sent with fractional second trailing zeroes removed. Current behaviour is to provide the trailing zeroes padded to the length/scale of the source column (DBZ-5996).

New features

  • Remove redundant modifiers of members for interface fields DBZ-2439

  • Update the DBZ-UI documentation page to incorporate the recently added "Custom properties" step details DBZ-5878

  • Postgres LSN check should honor event.processing.failure.handling.mode DBZ-6012

  • Enhance the Spanner connector by adding features and/or solving bugs DBZ-6014

Fixes

  • Data type conversion failed for mysql bigint DBZ-5798

  • Oracle cannot undo change DBZ-5907

  • Truncate records incompatible with ExtractNewRecordState DBZ-5966

  • Computed partition must not be negative DBZ-5967

  • NPE in execute snapshot signal with exclude.tables config on giving wrong table name DBZ-5988

  • There is a problem with postgresql connector parsing the boundary value of money type DBZ-5991

  • Run PostgresConnectorIT.shouldReceiveChangesForChangeColumnDefault() failed DBZ-6002

  • Nullable columns marked with "optional: false" in DDL events DBZ-6003

  • Vitess: Handle the shard list difference between current db shards and persisted shards DBZ-6011

  • DDL statement with TokuDB engine specific "CLUSTERING KEY" couldn’t be parsed DBZ-6016

  • DDL parse fail for role revoke with "user-like" role name DBZ-6019

  • DDL parse fail for ALTER USER x DEFAULT ROLE y; DBZ-6020

  • Offsets are not flushed on connect offsets topic when encountering an error on Postgres connector DBZ-6026

  • Unexpected format for TIME column: 8:00 DBZ-6029

  • Oracle does not support compression/logging clauses after an LOB storage clause DBZ-6031

  • Debezium is logging the full message along with the error DBZ-6037

  • Improve resilience during internal schema history recovery from Kafka DBZ-6039

Other changes

  • Plug-in version information duplicated DBZ-4669

  • Remove incubating documentation text for MongoDB ExtractNewDocumentState SMT DBZ-5975

  • Upgrade Apicurio to 2.4.1.Final DBZ-5977

  • Upgrade JDBC driver to 42.5.1 DBZ-5980

  • Migrate connector triggers to gitlab DBZ-5992

  • SQL Server IncrementalSnapshotWithRecompileIT fails randomly DBZ-6035

Release 2.1.0.Final (December 22nd 2022)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.0.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.0.Final plugin files, and restart the connector using the same configuration. Upon restart, the 2.1.0.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

MongoDB Debezium connector required streaming from primary node in the cluster. This is no longer necessary and the connector prefers reading from non-primary node (DBZ-4339).

Vitess Debezium connector now supports snapshotting. This means that upon new connector start the existing content will be snapshotted by default (DBZ-5930).

New features

  • Implement support for JSON_TABLE in MySQL parser DBZ-3575

  • Provide Debezium Spanner connector DBZ-5937

  • Print the readable data class name in JdbcValueConverters.handleUnknownData DBZ-5946

Fixes

  • Cannot expand JSON payload with nested arrays of objects DBZ-5344

  • field.exclude.list in MongoDB Connector v2.0 doesn’t accept * as a wildcard for collectionName DBZ-5818

  • Debezium UI documentation link is not accessible to the user via documentation side navigation menu. DBZ-5900

  • Toasted json/int/bigint arrays are not properly processed DBZ-5936

  • No table filters found for filtered publication DBZ-5949

Other changes

There are no other changes in this release.

Release 2.1.0.Beta1 (December 16th 2022)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 2.1.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

The Cassandra range tombstone information was a plain string. Now logical object with parsed data and types is provided (DBZ-5912).

The Cassandra TimeUUID datatype was propadagated as binary object but documented as string. This was incorrect and it is now propagated as string (DBZ-5923).

New features

  • Postgres: Disable LSN confirmation to database DBZ-5811

  • Realize data distribution according to specified fields DBZ-5847

  • Support predicate parameters in Debezium Server DBZ-5940

  • Use the Patternfly database icon as a placeholder for Oracle Database DBZ-5941

Fixes

  • Handle toasted String array DBZ-4941

  • Cassandra deletes log files on exit when real time processing is enabled DBZ-5776

  • ReplicationConnectionIT test fails DBZ-5800

  • MongoDB docs for incremental snapshots is SQL specific DBZ-5804

  • Conflicting documentation for snapshot.mode property in MongoDB connector v2.0 DBZ-5812

  • IllegalStateException is thrown if task is recovering while other tasks are running DBZ-5855

  • Negative decimal number scale is not supported by Avro DBZ-5880

  • Connector deployment instructions provide incorrect Maven path for Debezium scripting component DBZ-5882

  • Incorrect Streams Kafka version in connector deployment instructions for creating a custom image DBZ-5883

  • Run postgres connector RecordsStreamProducerIT failed DBZ-5895

  • Suppport INSERT INTO statements with dots in column names DBZ-5904

  • Incorrect default value for additional-condition docs DBZ-5906

  • ConnectorLifecycle is not logging anymore the exception stacktrace when startup fails DBZ-5908

  • Debezium Server stops with NPE when Redis does not report the "maxmemory" field in "info memory" command DBZ-5911

  • PostgresConnectorIT#shouldAckLsnOnSourceByDefault and #shouldNotAckLsnOnSource fails DBZ-5914

  • SQL Server connector database.instance config option is ignored DBZ-5924

  • Wrong java version in Installing Debezium documentation DBZ-5928

  • Toasted varchar array is not correctly processed DBZ-5944

Other changes

  • Use static import for Assertions in all tests DBZ-2432

  • Test window function in MySQL parser DBZ-3576

  • Run test against Apicurio registry DBZ-5838

  • Add tests against multinode RS and (ideally) sharded cluster DBZ-5857

  • Update documentation for Debezium Server with Cassandra Connector DBZ-5885

  • Allow CI deploy clusters to PSI DBZ-5887

  • Mariadb and Mysql have different syntax DBZ-5888

  • Execute IT tests in alphabetical order DBZ-5889

  • Migrate debezium-server-nats-jetstream to AssertJ DBZ-5901

  • Reduce jenkins jobs footprint DBZ-5905

  • Move Debezium Cassandra connector out from incubation DBZ-5922

  • Clean up "doSnapshot" config code DBZ-5931

  • Version badge on README in Cassandra connector is stuck DBZ-5932

  • Make startup of Cassandra container faster DBZ-5933

  • Fix logging for tests for Cassandra connector DBZ-5934

Release 2.1.0.Alpha2 (November 30th 2022)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.0.Alpha2 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.0.Alpha2 plugin files, and restart the connector using the same configuration. Upon restart, the 2.1.0.Alpha2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

Debezium REST extension was not deployed in Debezium container and has to be added by the user. This is no longer necessary as the REST expension is included (DBZ-4303).

Debezium images were upgraded to use Fedora 37 (DBZ-5461).

PostgreSQL connector could resume streaming from re-created replication slot even if it no longer contained data that connector has not seen. The result could be a silent data loss. Now the connector checks if the resume point is present and fails to start if it is not (DBZ-5739).

New features

  • Expose Cassandra Connector via Debezium Server DBZ-2098

  • Validate Debezium Server configuration properties DBZ-4720

  • Enable pass-thru of additional config options in Debezium UI DBZ-5324

  • Sink adapter for Nats JetStream DBZ-5772

  • Replace obsolete DebeziumDownload attribute DBZ-5835

  • Reduce container image sizes by consolidating operations per layer DBZ-5864

  • Typo error in Oracle connector documentation 2.0 DBZ-5877

Fixes

  • Embedded Engine or Server retrying indefinitely on all types of retriable errors DBZ-5661

  • PostgreSQL missing metadata info DBZ-5789

  • For outbox transformation, when 'table.expand.json.payload' is set to true null values are not correctly deserialized DBZ-5796

  • Cassandra decimal values are not deserialized using Debezium Cassandra Connector DBZ-5807

  • Cassandra varint type is currently not supported DBZ-5808

  • 'topic.prefix' default value in MongoDB connector v2.0 DBZ-5817

  • Quarkus outbox extention never finishes the open tracing span DBZ-5821

  • fix names of range fields in schema to comply with Avro standard DBZ-5826

  • ExtractNewDocumentState does not support updateDescription.updatedFields field DBZ-5834

  • CREATE/ALTER user does not support COMMENT token DBZ-5836

  • Invalid Java object for schema with type FLOAT64: class java.lang.Float DBZ-5843

  • Message contents might not get logged in case of error DBZ-5874

  • CREATE/ALTER user does not support ATTRIBUTE token DBZ-5876

Other changes

  • SQL table rename affect on Kafka connector and topic DBZ-5423

  • Create RHAF version of Debezium docs DBZ-5729

  • Add Debezium doc section to RHAF DBZ-5730

  • Create new Debezium section in the docs. DBZ-5731

  • Add Debezium docs to DDF DBZ-5732

  • Create ARO provisioning job DBZ-5742

  • Amend Confluent Avro converter installation documentation DBZ-5762

  • Modify ocp system tests to archive test results and logs DBZ-5785

  • GitHub Actions: Deprecating save-state and set-output commands DBZ-5824

  • Change logging levels of several schema change handler log entries DBZ-5833

  • Revert running tests against Apicurio registry DBZ-5839

  • Add Kubernetes plugin to Jenkins DBZ-5844

  • OracleConnectorIT shouldIgnoreAllTablesInExcludedSchemas test may randomly fail DBZ-5850

  • Upgrade wildfly-elytron to 1.15.5 / 1.16.1 due to CVE-2021-3642 DBZ-5854

  • Upgrade PostgreSQL example images to Postgres 15 DBZ-5860

  • GitHub Actions deprecation of Node 12 - actions/checkout DBZ-5870

Release 2.1.0.Alpha1 (November 10th 2022)

Kafka compatibility

This release has been built against Kafka Connect 3.3.1 and has been tested with version 3.3.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 2.1.0.Alpha1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 2.1.0.Alpha1 plugin files, and restart the connector using the same (when upgrading from the same major version) or updated (when upgrading from an older major version) configuration. Upon restart, the 2.1.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from DockerHub.

Breaking changes

There are no breaking changes in this release.

New features

  • Support for Postgres 15 DBZ-5370

  • Add support for SMT predicates in Debezium Engine DBZ-5530

  • MySQL Connector capture TRUNCATE command as message in table topic DBZ-5610

  • Improve LogMiner query performance by reducing REGEXP_LIKE disjunctions DBZ-5648

  • Expose heartbeatFrequency setting for mongodb connector DBZ-5736

  • Provide Redis storage as store module DBZ-5749

  • Redis Sink wait for Redis Replica writes DBZ-5752

  • Redis sink back-pressure mechanism when Redis memory is almost full DBZ-5782

  • Enhance the ability to sanitize topic name DBZ-5790

Fixes

  • Using snapshot boundary mode "all" causes DebeziumException on Oracle RAC DBZ-5302

  • ORA-01003: no statement parsed DBZ-5352

  • Missing snapshot pending transactions DBZ-5482

  • Db2 documentation refers to invalid SMALLMONEY and MONEY data types DBZ-5504

  • Using snapshot.mode ALWAYS uses SCN from offsets DBZ-5626

  • MongoDB multiple tasks monitor misalignment DBZ-5629

  • UNIQUE INDEX with NULL value throws exception when lob.enabled is true DBZ-5682

  • Oracle SQL parsing error when collation used DBZ-5726

  • Columns are not excluded when doing incremental snapshots DBZ-5727

  • Unparseable DDL statement DBZ-5734

  • NullPointerException thrown during snapshot of tables in Oracle source connector DBZ-5738

  • Remove note from snapshot metrics docs file that flags incremental snapshots as TP feature DBZ-5748

  • Hostname not available for load balanced ocp services in ARO DBZ-5753

  • Exclude Oracle Compression Advisor tables from capture to avoid infinite loop DBZ-5756

  • More Oracle logging DBZ-5759

  • Oracle should only log row contents at TRACE level DBZ-5760

  • Update system test artifact preparation to reflect naming changes in downstream DBZ-5767

  • Outbox Router documentation outdated regarding value converter DBZ-5770

  • Using DBMS_LOB.ERASE by itself can lead to an unexpected UPDATE with null BLOB value DBZ-5773

  • Suppress logging of undetermined optionality for explicitly excluded columns DBZ-5783

  • Oracle connector does not attempt restart when ORA-01089 exception is nested DBZ-5791

  • Message with LSN 'LSN{XYZ}' not present among LSNs seen in the location phase DBZ-5792

  • The merge method of configuration is not work DBZ-5801

  • Mysql connector alter table with database name parse failed DBZ-5802

Other changes

  • Execute tests with Apicurio converters DBZ-2131

  • Revision info missing on website DBZ-5083

  • Debezium on ARO sanity testing DBZ-5647

  • SQL Server connector docs should mention multi-task support DBZ-5714

  • Remove downstream TP designation for RAC content in Oracle connector docs DBZ-5735

  • Update Pulsar client to 2.10.1 DBZ-5737

  • Parametrize Strimzi operator name to enable multiple testsuites running on same cluster DBZ-5744

  • Enable CI to report results to ReportPortal instance DBZ-5745

  • Debezium connectors ship with an old version of google-protobuf vulnerable to CVE-2022-3171 DBZ-5747

  • Testsuite unable to connect to SQLServer due to encryption DBZ-5763

  • Testsuite uses incorrect jdbc driver class for SQLServer with docker DBZ-5764

  • Upgrade com.jayway.jsonpath:json-path DBZ-5766

  • Product profile is not used when running Oracle matrix against downstream DBZ-5768

  • Upgrade to Quarkus 2.14.CR1 DBZ-5774

  • Switch from Fest to AssertJ DBZ-5779

  • Upgrade postgres driver to version 42.5.0 DBZ-5780

  • Upgrade to Quarkus 2.14.0.Final DBZ-5786

  • Doc Typo in cloudevents DBZ-5788

  • Fix DB2 reporting script path DBZ-5799

  • Add ORA-01555 to Oracle documentation DBZ-5816

  • Change visibility of BaseSourceTask#logStatistics method to protected DBZ-5822

  • Upgrade Postgres images to Debian 11 DBZ-5823