Release Notes for Debezium 3.0

All notable changes for Debezium releases are documented in this file. Release numbers follow Semantic Versioning.

Release 3.0.6.Final (December 19th 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.9.0 and has been tested with version 3.9.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.6.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.6.Final plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.6.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

There are no breaking changes in this release.

New features

There are no new features in this release.

Fixes

Other changes

There are no other changes in this release.

Release 3.0.5.Final (December 18th 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.9.0 and has been tested with version 3.9.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.5.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.5.Final plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.5.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

Debezium was reprocessing signals upon connector restarts. This could lead to an unpredictable behaviour and can have unintended side-effects. This feature was thus removed and if a connector stops then the signal must be re-sent (DBZ-7856).

The snapshot enumeration in the source info block schema was extended to cover all possible cases (DBZ-8496).

New features

  • List all examples in root README.md of Debezium’s Example Repo DBZ-2535

  • Test the MS SQL Server Plugin with Transparent data encryption (TDE) DBZ-4590

  • Allow adhoc snapshot on tables whose schemas have not been captured DBZ-4903

  • Support Postgres 17 failover slots DBZ-8412

  • Improve error handling in dispatchSnapshotEvent of EventDispatcher DBZ-8433

  • Connector configuration logging improvement DBZ-8472

  • Handle un-parseable DDLs gracefully DBZ-8479

  • Track LogMiner partial rollback events in metrics DBZ-8491

  • Support JDBC offset/history configuration in CRD DBZ-8501

Fixes

  • Error with debezium.sink.pulsar.client.serviceUrl and debezium-server DBZ-3720

  • MySQL regression - Defaults store.only.captured.tables.ddl to true DBZ-6709

  • ExtractNewRecordState value of optional null field which has default value DBZ-7094

  • DebeziumException: No column '' where ' found in table DBZ-8034

  • MySQL Connector Does Not Act On CREATE DATABASE Records In The Binlog DBZ-8291

  • Vgtid doesn’t contain multiple shard GTIDs when multiple tasks are used DBZ-8432

  • Object ID cache may fail with concurent modification expcetion DBZ-8465

  • Oracle gathers and logs object attributes for views unnecessarily DBZ-8492

  • ReselectColumnPostProcessor can throw ORA-01003 "no statement parsed" when using fallback non-flashback area query DBZ-8493

  • Oracle DDL ALTER TABLE ADD CONSTRAINT fails to be parsed DBZ-8494

  • Edit Source/Destination on adding new configuration properties its removing old once DBZ-8495

  • Invalid property name in JDBC Schema History DBZ-8500

  • Fix the URL in Pipeline log page DBZ-8502

  • Failed to start LogMiner mining session due to "Required Start SCN" error message DBZ-8503

  • Oracle data pump TEMPLATE_TABLE clause not supported DBZ-8504

  • Postgres alpine images require lang/llvm 19 for build DBZ-8505

  • TimezoneConverter include.list should be respected if set DBZ-8514

  • Missing log classes debezium-platform-conductor DBZ-8515

  • Debezium Server fails to start when using the sink Kinesis DBZ-8517

  • Skip GoldenGate REPLICATION MARKER events DBZ-8533

Other changes

  • Add example for SSL-enabled Kafka DBZ-1937

  • Create smoke test to make sure Debezium Server container image works DBZ-3226

  • Align MySQL and MariaDB grammars with upstream versions DBZ-8270

  • Support MongoDB 8.0 DBZ-8451

  • Update description of message.key.columns and format admonitions in PG doc DBZ-8455

  • Add Basic validation in UI to check for form completion before submitting. DBZ-8474

  • Use schema evolution tool to manage the conductor database DBZ-8486

  • Update Quarkus Outbox Extension to Quarkus 3.17.3 DBZ-8506

  • Merge conductor and stage into single platform repository DBZ-8508

  • Container Tests are executed with -DskipITs DBZ-8509

  • Add github workflow for UI unit testing on PRs DBZ-8526

Release 3.0.4.Final (November 28th 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.9.0 and has been tested with version 3.9.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.4.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.4.Final plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.4.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

There are no breaking changes in this release.

New features

  • Update the UI to pass on the backend URL at runtime from ENV Var while running the container image DBZ-8424

  • Add support for mysql_clear_password in mysql-binlog-connector DBZ-8445

Fixes

  • Debezium db2i CDC source connector does not seem to pickup JOURNAL_ENTRY_TYPES ⇒ 'DR' records DBZ-8453

  • Randomly failing tests after migration to async engine DBZ-8461

  • Invalid label used for API service discriminator DBZ-8464

Other changes

  • Migrate rest of the testsuite to async engine DBZ-7977

  • Update QOSDK to version 6.9.1 DBZ-8452

  • Add JDBC storage module in Debezium Server DBZ-8460

Release 3.0.3.Final (November 25th 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.9.0 and has been tested with version 3.9.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.3.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.3.Final plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.3.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

There are no breaking changes in this release.

New features

  • Add support for bpchar datatype DBZ-8416

  • Allow parts of DS resource to reference values from primary in configuration DBZ-8431

Fixes

  • Spanner tests fail randomly DBZ-8410

  • Engine shutdown may get stuck when error is thrown during connector stop DBZ-8414

  • JdbcOffsetBackingStore does not release lock of debezium_offset_storage gracefully DBZ-8423

  • Installation documentation typo on download link DBZ-8429

  • Asycn engine fails with NPE when transformation returns null DBZ-8434

  • Snapshot completed flag not correctly saved on offsets DBZ-8449

  • Formatting characters render in descriptions of Oracle log.mining properties DBZ-8450

  • Prevent data corruption from netty version 4.1.111.Final DBZ-8438

Other changes

  • Support config map offset store in the DS Operator DBZ-8352

  • Migrate Vitess testsuite to async engine DBZ-8377

  • Migrate Spanner testsuite to async engine DBZ-8381

  • Do not build images for unsupported database versions DBZ-8413

  • Update PatternFly version in UI from 6.beta to final 6.0 DBZ-8415

  • Fix the UI build issue DBZ-8435

  • Make AbstractConnectorTest#createEngine method abstract DBZ-8441

Release 3.0.2.Final (November 15th 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.9.0 and has been tested with version 3.9.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.2.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.2.Final plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.2.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

There are no breaking changes in this release.

New features

  • Add file signal channel documentation to the signal channel chapter DBZ-7245

  • Improve blocking snapshot reliability in case of restart DBZ-7903

  • Allow skipping exceptions related to DML parser errors DBZ-8208

  • Ability to enable DS REST API in Operator CR DBZ-8234

  • Add feature to download and stream the Pipeline logs from UI DBZ-8239

  • Add support for vitess-connector to send DDL events DBZ-8325

  • Vstream table filter to match full table names DBZ-8354

  • RowsScanned JMX metric for MongoDB differs from relational connectors DBZ-8359

  • Refactor CassandraTypeProvider to not contain getClusterName method DBZ-8373

  • Possibility for Debezium Oracle Connector to accept NLS Time Format (For Date and Timestamp Columns) DBZ-8379

  • Provide config to allow for sending schema change events without historized schemas DBZ-8392

  • Implement new config map offset store in DS DBZ-8351

Fixes

  • Race condition in stop-snapshot signal DBZ-8303

  • Debezium shifts binlog offset despite RabbitMQ Timeout and unconfirmed messages DBZ-8307

  • Debezium server with eventhubs sink type and eventhubs emulator connection string fails DBZ-8357

  • Filter for snapshot using signal doesn’t seem to work DBZ-8358

  • JDBC storage module does not use quay.io images DBZ-8362

  • Failure on offset store call to configure/start is logged at DEBUG level DBZ-8364

  • Object name is not in the list of S3 schema history fields DBZ-8366

  • Faulty "Failed to load mandatory config" error message DBZ-8367

  • Upgrade protobuf dependencies to avoid potential vulnerability DBZ-8371

  • Tests in IncrementalSnapshotIT may fail randomly DBZ-8386

  • ExtractNewRecordState transform: NPE when processing non-envelope records DBZ-8393

  • Oracle LogMiner metric OldestScnAgeInMilliseconds can be negative DBZ-8395

  • SqlServerConnectorIT.restartInTheMiddleOfTxAfterCompletedTx fails randomly DBZ-8396

  • ExtractNewDocumentStateTestIT fails randomly DBZ-8397

  • BlockingSnapshotIT fails on Oracle DBZ-8398

  • Oracle OBJECT_ID lookup and cause high CPU and latency in Hybrid mining mode DBZ-8399

  • Protobuf plugin does not compile for PostgreSQL 17 on Debian DBZ-8403

Other changes

  • Clarify signal data collection should be unique per connector DBZ-6837

  • Use DebeziumSinkRecord instead of Kafka Connect’s SinkRecord inside Debezium sink connectors DBZ-8346

  • Migrate SQL server testsuite to async engine DBZ-8353

  • Remove unnecessary converter code from parsers DBZ-8360

  • Deduplicate Cassandra Debezium tests DBZ-8363

  • Migrate MongoDB testsuite to async engine DBZ-8369

  • Migrate Oracle testsuite to async engine DBZ-8370

  • Add transform page to provide a single place to list the already configured transform plus UI to add a new transform DBZ-8374

  • Migrate rest of Debezium testsuite to async engine DBZ-8375

  • Migrate DB2 testsuite to async engine DBZ-8380

  • Migrate IBM i testsuite to async engine DBZ-8382

  • Upgrade Kafka to 3.8.1 DBZ-8385

  • Add Transform Edit and delete support. DBZ-8388

  • Log SCN existence check may throw ORA-01291 if a recent checkpoint occurred DBZ-8389

  • Upgrade Kafka to 3.9.0 DBZ-8400

  • Update Quarkus Outbox Extension to Quarkus 3.16.3 DBZ-8409

Release 3.0.1.Final (October 25th 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.8.0 and has been tested with version 3.8.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.1.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.1.Final plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.1.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

Debezium no longer publishes container images to the Docker Hub (DBZ-8327).

New features

  • Support batch write to AWS Kinesis DBZ-8193

  • Support for PostgreSQL 17 DBZ-8275

  • Extend Debezium Server to include support for application.yaml DBZ-8313

  • SQL Server Documentation for CDC on Server table DBZ-8314

  • Add support for MySQL 9.1 DBZ-8324

  • Support Cassandra 5.0 DBZ-8347

Fixes

  • Oracle DDL parsing will fail if the DDL ends with a new line character DBZ-7040

  • Missing documentation for MongoDb SSL configuration DBZ-7927

  • Conditionalization implemented for single-sourcing MySQL/MariaDB content isn’t working as expected DBZ-8094

  • Debezium is replaying all events from an older offset DBZ-8194

  • Embedded MySqlConnector "Unable to find minimal snapshot lock mode" since 2.5.4.Final DBZ-8271

  • Reselect Post Processor not working when pkey of type uuid etc. DBZ-8277

  • BinlogStreamingChangeEventSource totalRecordCounter is never updated DBZ-8290

  • Restart Oracle connector when ORA-01001 invalid cursor exception is thrown DBZ-8292

  • Connector uses incorrect partition names when creating offsets DBZ-8298

  • ReselectPostProcessor fails when reselecting columns from Oracle DBZ-8304

  • Debezium MySQL DDL parser: SECONDARY_ENGINE=RAPID does not support DBZ-8305

  • Oracle DDL failure - subpartition list clause does not support in-memory clause DBZ-8315

  • DDL statement couldn’t be parsed DBZ-8316

  • Binary Log Client doesn’t process the TRANSACTION_ PAYLOAD header DBZ-8340

  • Oracle connector: archive.log.only.mode stop working after reach SYSDATE SCN DBZ-8345

Other changes

  • Provide example for activity monitoring metrics DBZ-8174

  • Write blog post on how detect data mutation patterns with Debezium DBZ-8256

  • Formatting characters render literally in docs DBZ-8293

  • REST tests fail due to unable to execute cp DBZ-8294

  • Create MariaDB systemtests DBZ-8306

  • Refactor MySqlTests and MariaDBTests to share the tests via parent base class DBZ-8309

  • Document how to work with ServiceLoader and bundled jars DBZ-8318

  • Broken system tests for upstream DBZ-8326

  • Upstream system tests are stuck in Retrieving connector metrics DBZ-8330

  • Fix upstream JDBC system tests DBZ-8331

  • Add version for Cassandra 5 to debezium-build-parent DBZ-8348

Release 3.0.0.Final (October 2nd 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.8.0 and has been tested with version 3.8.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.0.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.0.Final plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.0.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

The deprecated additional-condition field of execute-snapshot signal was removed. The field is fully replaced with previously introduced additional-conditions field (DBZ-8278).

New features

  • Add documentation for custom converters in PG DBZ-7820

  • Create REST bridge for DBZ signal channels DBZ-8101

  • Support int/bigint arrays in reselect colums postprocessors DBZ-8212

  • Log the record key when debezium fails to send the record to Kafka DBZ-8282

Fixes

  • Custom convert (all to strings) and SQLServer default '0' type issue DBZ-7045

  • UnsupportedClassVersionError while running debezium-connector docker Image DBZ-7751

  • Error writing data to target database. (Caused by: java.lang.RuntimeException: org.postgresql.util.PSQLException: The column index is out of range: 140, number of columns: 139.) DBZ-8221

  • Debezium Server messages not being sent to Pub/Sub after restart DBZ-8236

  • An aborted ad-hoc blocking snapshot leaves the connector in a broken state DBZ-8244

  • JDBC Sink truncate event also add event to updateBufferByTable DBZ-8247

  • mysql-binlog-connector-java doesn’t compile with java 21 DBZ-8253

  • DDL statement couldn’t be parsed. 'mismatched input 'NOCACHE' expecting {'AS', 'USAGE', ';'} DBZ-8262

  • journal processing loops after journal offset reset DBZ-8265

Other changes

  • Add async engine config options to server documentation DBZ-8133

  • Bump apicurio schema registry to 2.6.2.Final DBZ-8145

  • Correct description of the all_tables option for the PG publication.autocreate.mode property DBZ-8268

  • Test docs for productization and fix broken links and rendering errors DBZ-8284

Release 3.0.0.CR2 (September 25th 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.8.0 and has been tested with version 3.8.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.0.CR2 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.0.CR2 plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.0.CR2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

There are no breaking changes in this release.

New features

  • Snapshot isolation level options for postgres DBZ-1252

  • Retry flush records if LockAcquisitionException occured in mysql DBZ-7291

  • Add support for MAX_STRING_SIZE set to EXTENDED DBZ-8039

  • Add invalid value logger for dates to Debezium Vitess Connector DBZ-8235

  • Support BLOB with EMPTY_BLOB() as default DBZ-8248

Fixes

  • Debezium does not restart automatically after throwing an ORA-00600 krvrdccs30 error DBZ-8223

  • JDBC sink doesn’t include fields as per documentation DBZ-8224

  • Unbounded number of processing threads in async engine DBZ-8237

  • Streaming metrics are stuck after an ad-hoc blocking snapshot DBZ-8238

  • DDL statement couldn’t be parsed with IF EXISTS DBZ-8240

  • Random engine factory used by default DBZ-8241

  • JDBC sink test suite should use the debezium/connect:nightly image for e2e tests DBZ-8245

  • Performance Regression in Debezium Server Kafka after DBZ-7575 fix DBZ-8251

  • Error Prone library included in MySQL connector DBZ-8258

  • Debezium.text.ParsingException: DDL statement couldn’t be parsed DBZ-8259

Other changes

  • Test and check compatibility with ojdbc11 DBZ-3658

  • Broken link to Streams doc about configuring logging DBZ-8231

  • Document passthrough hibernate.* properties for the JDBC connector DBZ-8232

  • Bump Infinispan to 15.0.8.Final DBZ-8246

  • AbstractConnectorTest consumeRecordsUntil may prematurely exit loop DBZ-8250

  • Add a note to the docs about JDBC batch retry configs DBZ-8252

  • Fix conditionalization in shared MariaDB/MySQL file DBZ-8254

  • Add Oracle FUTC license DBZ-8260

  • Remove Oracle libs from product assembly package DBZ-8261

  • debezium-connector-binlog does not need MariaDB dependency DBZ-8263

  • Provide subset package for Debezium Server DBZ-8264

  • Bump container images to Fedora 40 DBZ-8266

Release 3.0.0.CR1 (September 13rd 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.8.0 and has been tested with version 3.8.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.0.CR1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.0.CR1 plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.0.CR1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

JMX signalling and notification did not work correctly for deployments with SQL Server configured for multiple tasks. To fix the issue it was necessary to change the naming of signalling and notification MBeans to make them unique per each task (DBZ-8137).

Deprecated Oracle connector configuration options were removed from the project (DBZ-8181).

Vector datatype names introduced for PostgreSQL were too related to PostgreSQL datatype naming. The type names were changed to more generic ones and are shared between PostgreSQL and MySQL (DBZ-8183).

New features

  • Add support for MySQL 9 DBZ-8030

  • Add support for MySQL vector datatype DBZ-8157

  • Refactor engine signal support DBZ-8160

  • Add feature to inherit shard epoch DBZ-8163

  • Avoid 3 second delay in Oracle when one of the RAC nodes is offline DBZ-8177

  • Truncate byte buffer should return a new array DBZ-8189

  • Support for older MongoDb versions DBZ-8202

  • Add VECTOR functions to MySQL grammar DBZ-8210

  • Support MariaDB 11.4.3 DBZ-8226

  • Add information about helm chart installation to operator readme DBZ-8233

Fixes

  • Make ORA-00600 - krvrdccs10 automatically retriable DBZ-5009

  • Incremental snapshot fails with NPE if surrogate key doesn’t exist DBZ-7797

  • MySQL 8.4 incompatibility due to removed SQL commands DBZ-7838

  • Postgres connector - null value processing for "money" type column. DBZ-8027

  • Using snapshot.include.collection.list with Oracle raises NullPointerException DBZ-8032

  • Performance degradation when reconstructing (log.mining.stragtegy hybrid mode) DBZ-8071

  • The source data type exceeds the debezium data type and cannot deserialize the object DBZ-8142

  • Incorrect use of generic types in tests DBZ-8166

  • Postgres JSONB Fields are not supported with Reselect Post Processor DBZ-8168

  • NullPointerException (schemaUpdateCache is null) when restarting Oracle engine DBZ-8187

  • XStream may fail to attach on retry if previous attempt failed DBZ-8188

  • Exclude Oracle 23 VECSYS tablespace from capture DBZ-8198

  • AbstractProcessorTest uses an incorrect database name when run against Oracle 23 Free edition DBZ-8199

  • DDL statement couldn’t be parsed: REVOKE IF EXISTS DBZ-8209

  • System testsuite fails with route name being too long DBZ-8213

  • Oracle TableSchemaBuilder provides wrong column name in error message DBZ-8217

  • Using ehcache in Kafka connect throws an XMLConfiguration parse exception DBZ-8219

  • OcpJdbcSinkConnectorIT fails DBZ-8228

  • Container image does not install correct apicurio deps DBZ-8230

Other changes

  • Documentation for signals provides incorrect data-collection format for some connectors DBZ-8090

  • Latest Informix JDBC Driver DBZ-8167

  • upgrade Adobe s3mock to version 3.10.0 DBZ-8169

  • Include Jackson libraries to JDBC connector Docker image distribution DBZ-8175

  • Ehcache fails to start, throwing "Invaild XML Configuration" DBZ-8178

  • Enable snapshot.database.errors.max.retriesEnable during Oracle tests DBZ-8184

  • Change event for a logical decoding message doesn’t contain transaction field DBZ-8185

  • Add MariaDB connector server distribution DBZ-8186

  • Update Vitess example to Debezium 2.7/Vitess 19 DBZ-8196

  • OracleConnectorIT test shouldGracefullySkipObjectBasedTables can timeout prematurely DBZ-8197

  • Reduce log verbosity of OpenLogReplicator SCN confirmation DBZ-8201

  • Implement separate source and sink connector sections in documentation navigation DBZ-8220

Release 3.0.0.Beta1 (August 22nd 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.8.0 and has been tested with version 3.8.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

Debezim Kafka sink could wait indefinitely in case of Kafka broker unavailablity. A support for configurable timeout was added and the default behaviour is timeout after 30 seconds (DBZ-7575).

RabbitMQ native stream sink was sending changes all messages into a static single stream. With the new default behaviour the changes are sent to a distinct stream for each table (DBZ-8118).

New features

  • Implement Ehcache event buffer DBZ-7758

  • Expose a metric for number of create, update, delete events per table DBZ-8035

  • Log additional details about abandoned transactions DBZ-8044

  • Introduce timeout for replication slot creation DBZ-8073

  • ConverterBuilder doesn’t pass Headers to be manipulated DBZ-8082

  • Add SMT to decode binary content of a logical decoding message DBZ-8103

  • Support DECIMAL(p) Floating Point DBZ-8114

  • Support for PgVector datatypes DBZ-8121

  • Implement in process signal channel DBZ-8135

  • Validate log position method missing gtid info from SourceInfo DBZ-8140

  • Vitess Connector Epoch should support parallelism & shard changes DBZ-8154

  • Add an option for publication.autocreate.mode to create a publication with no tables DBZ-8156

Fixes

  • Incremental snapshots don’t work with CloudEvent converter DBZ-7601

  • Snapshot retrying logic falls into infinite retry loop DBZ-7860

  • Primary Key Update/ Snapshot Race Condition DBZ-8113

  • Docs: connect-log4j.properties instead log4j.properties DBZ-8117

  • Recalculating mining range upper bounds causes getScnFromTimestamp to fail DBZ-8119

  • ORA-00600: internal error code, arguments: [krvrdGetUID:2], [18446744073709551614], [], [], [], [], [], [], [], [], [], [] DBZ-8125

  • ConvertingFailureIT#shouldFailConversionTimeTypeWithConnectModeWhenFailMode fails randomly DBZ-8128

  • ibmi Connector does not take custom properties into account anymore DBZ-8129

  • Unpredicatable ordering of table rows during insertion causing foreign key error DBZ-8130

  • schema_only crashes ibmi Connector DBZ-8131

  • Support larger database.server.id values DBZ-8134

  • Open redo thread consistency check can lead to ORA-01291 - missing logfile DBZ-8144

  • SchemaOnlyRecoverySnapshotter not registered as an SPI service implementation DBZ-8147

  • When stopping the Oracle rac node the Debezium server throws an expections - ORA-12514: Cannot connect to database and retries DBZ-8149

  • Issue with Debezium Snapshot: DateTimeParseException with plugin pgoutput DBZ-8150

  • JDBC connector validation fails when using record_value with no primary.key.fields DBZ-8151

  • Taking RAC node offline and back online can lead to thread inconsistency DBZ-8162

Other changes

  • MySQL has deprecated mysql_native_password usage DBZ-7049

  • Upgrade to Apicurio 2.5.8 or higher DBZ-7357

  • Write and publish Debezium Orchestra blog post DBZ-7972

  • Move Debezium Conductor repository under Debezium Organisation DBZ-7973

  • Decide on name, jira components, etc…​ for Debezium Orchestra platform DBZ-7975

  • Migrate Postgres testsuite to async engine DBZ-8077

  • Conditionalize reference to the MySQL default value in description of schema.history.internal.store.only.captured.databases.ddl DBZ-8081

  • Bump Debezium Server to Quarkus 3.8.5 DBZ-8095

  • Converters documentation uses incorrect examples DBZ-8104

  • Remove reference to`additional condition` signal parameter from ad hoc snapshots doc DBZ-8107

  • TimescaleDbDatabaseTest.shouldTransformCompressedChunks is failing DBZ-8123

  • Update Oracle connector doc to describe options for restricting access permissions for the Debezium LogMiner user DBZ-8124

  • Use SQLSTATE to handle exceptions for replication slot creation command timeout DBZ-8127

  • Re-add check to test for if assembly profile is active DBZ-8138

  • Add LogMiner start mining session retry attempt counter to logs DBZ-8143

  • Reduce logging verbosity of XStream DML event data DBZ-8148

  • Upgrade Outbox Extension to Quarkus 3.14.0 DBZ-8164

Release 3.0.0.Alpha2 (August 2nd 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.8.0 and has been tested with version 3.8.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.0.Alpha2 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.0.Alpha2 plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.0.Alpha2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

Debezium is now build with Kafka 3.8.0. There were few changes in Kafka internal APIs Debezium is using. The codebase was modified to run with both pre-3.8.0 and 3.8.0 versions (DBZ-8105).

New features

  • Add Status ObservedGeneration to Operator DBZ-8025

  • Support Custom Converters in Debezium Server DBZ-8040

  • Support FLOAT32 type in debezium-connector-spanner DBZ-8043

  • Debezium should auto exclude empty shards (no tablets) and not crash on keyspaces with empty shards DBZ-8053

  • Refactor LogMining implementation to allow alternative cache implementations DBZ-8054

  • Standard Webhooks signatures for HTTP sink DBZ-8063

  • Vitess-connector should provide a topic naming strategy that supports separate connectors per-table DBZ-8069

  • Update third-party LICENSE with LGPL forMariaDB Connector/J DBZ-8099

  • Rabbitmq native stream Failed DBZ-8108

Fixes

  • Embedded Infinispan tests fail to start with Java 23 DBZ-7840

  • Clarify that Oracle connector does not read from physical standby DBZ-7895

  • StackOverflow exception on incremental snapshot DBZ-8011

  • JDBC primary.key.fields cannot be empty when i set insert.mode to upsert and primary.key.mode record_value DBZ-8018

  • Unable to acquire buffer lock, buffer queue is likely full DBZ-8022

  • Release process sets incorrect images for k8s for the next development version DBZ-8041

  • Use recrate as (default) rollout strategy for deployments DBZ-8047

  • "Unexpected input: ." when snapshot incremental empty Database DBZ-8050

  • Debezium Operator Using RollingUpdate Strategy DBZ-8051

  • Debezium Operator Using RollingUpdate Strategy DBZ-8052

  • Oracle connector inconsistency in redo log switches DBZ-8055

  • Blocking snapshot can fail due to CommunicationsException DBZ-8058

  • FakeDNS not working with JDK version > 18 DBZ-8059

  • Debezium Operator with a provided Service Account doesn’t spin up deployment DBZ-8061

  • ParsingException (MySQL/MariaDB): rename table syntax DBZ-8066

  • Oracle histogram metrics are no longer printed in logs correctly DBZ-8068

  • In hybrid log.mining.strategy reconstruction logs should be set to DEBUG DBZ-8070

  • Support capturing BLOB column types during snapshot for MySQL/MariaDB DBZ-8076

  • Standard Webhooks auth secret config value is not marked as PASSWORD_PATTERN DBZ-8078

  • Vitess transaction Epoch should not reset to zero when tx ID is missing DBZ-8087

  • After changing the column datatype from int to float the Debezium fails to round it and i get a null value for this field in the stream DBZ-8089

  • MySQL and MariaDB keyword YES cannot be parsed as a column name DBZ-8092

  • NotificationIT tests seemingly seem to fail due to stepping on one another DBZ-8100

  • ORA-26928 - Unable to communicate with XStream apply coordinator process should be retriable DBZ-8102

  • Transformations are not closed in emebdded engine DBZ-8106

  • Don’t close connection after loading timescale metadata in TimescaleDb SMT DBZ-8109

Other changes

  • Bump Infinispan to 14.0.29.Final DBZ-8010

  • Write a blog post about async engine DBZ-8013

  • Test offset/history store configurations DBZ-8015

  • Upgrade postgres server version to 15 DBZ-8062

  • Disable DebeziumResourceNoTopicCreationIT - no longer compatible with Java 21 DBZ-8067

  • Speed-up PostgresShutdownIT DBZ-8075

  • Add MariaDB to debezium/connect image DBZ-8088

Release 3.0.0.Alpha1 (July 11st 2024)

Kafka compatibility

This release has been built against Kafka Connect 3.7.0 and has been tested with version 3.7.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 3.0.0.Alpha1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 3.0.0.Alpha1 plugin files, and restart the connector using the same configuration. Upon restart, the 3.0.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our container images, then please do not forget to pull them fresh from Quay.io.

Breaking changes

Debezium connectors now require Java 17 for runtime and Java 21 for building. Debezium Server, Debezium Operator, and Debezium Outbox extension require Java 21 both for build and runtime (DBZ-6795).

New features

  • Provide MongoDB sink connector DBZ-7223

  • Extends process of finding Bundle path DBZ-7992

  • Support FLOAT32 type in debezium-connector-spanner DBZ-8043

Fixes

  • Debezium postgres jdbc sink not handling infinity values DBZ-7920

  • JdbcSinkTask doesn’t clear offsets on stop DBZ-7946

  • ibmi as400 connector config isn’t prefixed with "database." DBZ-7955

  • Duplicate downstream annotation comments incorrectly refer to Db2 connector DBZ-7968

  • Issue with Hybrid mode and DDL change DBZ-7991

  • Incorrect offset/history property mapping generatated DBZ-8007

  • Debezium Server Operator on minikube with java.lang.NullPointerException': java.lang.NullPointerException DBZ-8019

  • ORA-65090: operation only allowed in a container database when connecting to a non-CDB database DBZ-8023

  • Added type to Prometheus JMX exporter DBZ-8036

  • Add kafka.producer metrics to debezium-server jmx exporter config DBZ-8037

Other changes

  • Use Java 17 as baseline DBZ-7224

  • Document new MariaDB connector DBZ-7786

  • Move to Maven 3.9.8 as build requirement DBZ-7965

  • Add disclaimer that PostProcessors and CustomConverters are Debezium source connectors only DBZ-8031

  • Typos in Bug report template DBZ-8038

  • Find an alternative way to manually deploy the connector with local changes that is compatible with Debezium 3 DBZ-8046