1. Releases
  2. 1.4
  3. Release Notes for Debezium 1.4

Release Notes for Debezium 1.4

All notable changes for Debezium releases are documented in this file. Release numbers follow Semantic Versioning.

Release 1.4.2.Final (March 1st, 2021)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.1 and has been tested with version 2.6.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.4.2.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.2.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.2.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

In earlier versions of Debezium, the MySQL connector incorrectly emitted snapshot events using the c (create) operation type instead of the correct type r (read). If you have consumers which rely on that earlier behavior, you can use the io.debezium.connector.mysql.transforms.ReadToInsertEvent single message transform to emulate that earlier behavior (DBZ-2788). A connector option which accidentally was introduced in 1.4.0 for this same purpose, snapshot.events.as.inserts, got removed again, and the SMT should be used instead in this situation. This SMT is meant for migration purposes only and will be removed in a future Debezium version.

The (incubating) Debezium connector for Oracle emits transaction ids in lower-case now, differing from the previous behavior of returning them as upper-case (DBZ-3165).

The previously deprecated snapshot mode INITIAL_SCHEMA_ONLY of the Oracle connector has been removed. Please use SCHEMA_ONLY instead (DBZ-3034).

New Features

  • Make field descriptions consistent for time values (milliseconds, ms, sec, seconds, etc) DBZ-2858

  • Detect and skip non-parent index-organized tables DBZ-3036

  • Capture additional JMX metrics for LogMiner DBZ-3038

  • SqlServerConnector does not implement validate DBZ-3056

  • Improve DML parser performance DBZ-3078

  • Add ability to skip tests based on available database options DBZ-3110

  • Capture LogMiner session parameters when session fails to start DBZ-3153

Fixes

  • Wrong reference to KafkaConnector in setting up Debezium DBZ-2745

  • Oracle Connector(Using Logminer) with Oracle RDS (v12) does not capture changes DBZ-2754

  • Oracle connector causes ORA-65090 when connecting to an Oracle instance running in non-CDB mode DBZ-2795

  • Warnings and notifications from PostgreSQL are ignored by the connector until the connection is closed DBZ-2865

  • ExtractNewRecord SMT incorrectly extracts ts_ms from source info DBZ-2984

  • Replication terminates with ORA-01291: missing log file DBZ-3001

  • Support multiple schemas with Oracle LogMiner DBZ-3009

  • Documentation LOCK_TABLES should be LOCK TABLES DBZ-3013

  • Complete support for properties that contain hyphens DBZ-3019

  • SQLException for Global temp tables from OracleDatabaseMetaData.getIndexInfo() makes Debezium snapshotting fail DBZ-3057

  • no viable alternative at input 'create or replace index' DBZ-3067

  • Strange transaction metadata for Oracle logminer connector DBZ-3090

  • Environment Variables with spaces are truncated when written to properties file DBZ-3103

  • Error: Supplemental logging not configured for table. Use command: ALTER TABLE DBZ-3109

  • Final stage of snapshot analyzes tables not present in table.include.list thus stumbles upon unsupported XMLTYPE table DBZ-3151

  • Forever stuck with new binlog parser (1.3 and later) when processing big JSON column data DBZ-3168

  • XStream does not process NUMER(1) data DBZ-3172

  • Replace MySQL connector option with SMT for mitigating wrong op flag DBZ-2788

  • DML parser IndexOutOfRangeException with where-clause using "IS NULL" DBZ-3193

  • ORA-01284 file cannot be opened error when file locked by another process DBZ-3194

  • CommitThroughput metrics can raise division by zero error DBZ-3200

Other changes

  • OSD certification DBZ-2813

  • Integration with Service Registry promoted to GA DBZ-2815

  • Use new deployment endpoint for releases to Maven Central DBZ-3069

  • Remove zero-width whitespace from option names DBZ-3087

  • Remove duplicate anchor links in Connector properties DBZ-3111

  • Config validation for Oracle DBZ-3119

  • Clarify required privileges for using pgoutput DBZ-3138

  • Update Oracle documentation DBZ-3156

  • Put IIDR license requirement into NOTE box DBZ-3163

  • Remove COLUMN_BLACK_LIST option in Oracle connector DBZ-3167

  • Minor editorial update to PostgreSQL connector documentation DBZ-3192

  • Incorrect link/anchor pair for truncate.handling.mode property in PG properties documentation DBZ-3195

Release 1.4.1.Final (January 28th, 2021)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.1 and has been tested with version 2.6.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.4.1.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.1.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.1.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Clarify information in Debezium connector for SQL Server doc DBZ-2675

  • Add support for binary.handling.mode to the SQL Server connector DBZ-2912

  • Use collation to get charset when charset is not set DBZ-2922

  • Additional logging for number and type of sql operations DBZ-2980

  • Retry on "The server failed to resume the transaction" DBZ-2959

Fixes

  • Debezium Connectors are failing while reading binlog: Unknown event type 100 DBZ-2499

  • Some column default values are not extracted correctly while reading table structure DBZ-2698

  • Supplemental logging is required for entire database rather than per monitored table DBZ-2711

  • Missing log file error when current SCN differs from snapshotted in Oracle connector and Logminer DBZ-2855

  • GitHub action for "Build Testing Workflow" is using old artifacts and not building missing dependencies DBZ-2861

  • Deadlock in the XStream handler and offset commiter call concurrently DBZ-2891

  • Sanitise DECIMAL string from VStream DBZ-2906

  • Vitess Connector download link missing on website DBZ-2907

  • DML statements longer than 4000 characters are incorrectly combined from V$LOGMNR_CONTENTS DBZ-2920

  • Default database charset is not recorded DBZ-2921

  • Instable test: PostgresConnectorIT#testCustomSnapshotterSnapshotCompleteLifecycleHook() DBZ-2938

  • Snapshot causes ORA-08181 exception DBZ-2949

  • Postgres connector config validation fails because current connector is occupying replication slot DBZ-2952

  • Labeled create procedure’s body is not parsed DBZ-2972

  • Debezium swallows DML exception in certain cases DBZ-2981

Other changes

  • Migrate website build to Hugo DBZ-575

  • Test binary/varbinary datatypes DBZ-2174

  • Implement Scn as a domain type DBZ-2518

  • Fix docs for message.key.columns and skipped.operations DBZ-2572

  • Upgrade to Apache Kafka Connect 2.6.1 DBZ-2630

  • Centralize postgres image name for test container tests DBZ-2764

  • Add missing connector options for Postgres connector DBZ-2807

  • Importing TestDatabase as QuarkusTestResource for IT tests DBZ-2868

  • Set up Pulsar via Testcontainers in PulsarIT DBZ-2915

  • Remove blacklist and whitelist from anchor link text in documentation DBZ-2918

  • Instable test: PostgresShutdownIT#shouldStopOnPostgresFastShutdown() DBZ-2923

  • Rename whitelist/blacklist configs in examples to include/exclude DBZ-2925

  • Misspelling in readme for db2 connector DBZ-2940

  • Fetch correct Apicurio version for ApicurioRegistryTest DBZ-2945

  • Incorrect link IDs in SQL Server connector snapshot metrics table DBZ-2958

Release 1.4.0.Final (January 7th, 2021)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.4.0.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Improve error reporting from DDL parser DBZ-2366

  • Support TNS Names and full RAC connection strings DBZ-2859

  • Add more comprehensible logs to FIELD event DBZ-2873

Fixes

  • AWS RDS has different role names which make connector validation fail DBZ-2800

  • Archive Log mining does not work with Logminer DBZ-2825

  • MySQL parser error for comments starting with tab DBZ-2840

  • Connector fails when using '$' sign in column name. DBZ-2849

  • Connection adapter not passed to Surefire tests DBZ-2856

  • Unsupported MariaDB syntax for generated columns DBZ-2882

  • SLF4J API should not be included in Oracle distirbution DBZ-2890

  • Vitess distro contains unaligned deps DBZ-2892

  • Changing base packages does not always trigger full builds DBZ-2896

  • LogMiner causes DataException when DATE field is specified as NOT NULL DBZ-2784

Other changes

  • Remove LegacyDdlParser and related code DBZ-2167

  • Add MongoDB connector interface DBZ-2808

  • sanitize.field.names support for Vitess Connector DBZ-2851

  • Explicitly declare to Quarkus that ORM XML mapping is required for the outbox extension DBZ-2860

  • Upgrade MySQL JDBC driver to 8.0.21 DBZ-2887

  • Upgrade Guava library to 30.0 DBZ-2888

  • Avoid exception when payload id field not present DBZ-2889

Release 1.4.0.CR1 (December 16th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.4.0.CR1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.CR1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.CR1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Documentation of the Logminer implementation needs improvement DBZ-2799

  • Update Vitess Connector documentation DBZ-2854

  • Add Cassandra to tutorial Compose set-up DBZ-1463

  • Add support for Vitess gRPC static authentication DBZ-2852

Fixes

  • Document "database.oracle.version" option DBZ-2603

  • Remove link in MySQL docs section that points to the same section DBZ-2710

  • Oracle schema history events fail on partitioned table DBZ-2841

  • outbox extension emits UPDATE events when delete is disabled DBZ-2847

Other changes

  • Move Cassandra connector to separate repository DBZ-2636

  • Invalid column name should fail connector with meaningful message DBZ-2836

  • Fix typos in downstream ModuleID declarations in monitoring.adoc DBZ-2838

  • Duplicate anchor ID in partials/ref-connector-monitoring-snapshot-metrics.adoc DBZ-2839

  • Fix additional typo in ModuleID declaration in monitoring.adoc DBZ-2843

  • Edit modularization annotations in logging.adoc DBZ-2846

  • Update Groovy version to 3.0.7 DBZ-2850

Release 1.4.0.Beta1 (December 9th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.4.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Add support for distributed tracing DBZ-559

  • Outbox Quarkus extension: Support OpenTracing DBZ-1818

  • Upgrade MongoDB driver to 4.x to run in native mode in GraalVM (for Quarkus extension) DBZ-2138

  • Allow snapshot records be generated either as create or read for MySQL connector DBZ-2775

  • Support in Db2 connector for lowercase table and schema names DBZ-2796

  • option to kill process when engine run crashes DBZ-2785

  • Add support for using Vitess primary key as Kafka message key DBZ-2578

  • Add support for Nullable columns DBZ-2579

  • Tablespace name LOGMINER_TBS should not be hardcoded in the Java code DBZ-2797

Fixes

  • DDL parser: Allow stored procedure variables in LIMIT clause DBZ-2692

  • Wrong mysql command in openshift dpeloyment docs DBZ-2746

  • long running transaction will be abandoned and ignored DBZ-2759

  • MS SQL Decimal with default value not matching the scale of the column definition cause exception DBZ-2767

  • Cassandra Connector doesn’t shut down completely DBZ-2768

  • MySQL Parser fails for BINARY collation shortcut DBZ-2771

  • PostgresConnectorIT.shouldResumeStreamingFromSlotPositionForCustomSnapshot is failing for wal2json on CI DBZ-2772

  • Connector configuration property "database.out.server.name" is not relevant for Logminer implementation but cannot be omitted DBZ-2801

  • CHARACTER VARYING mysql identifier for varchar is not supported in debezium DBZ-2821

  • try-with-resources should not be used when OkHttp Response object is returned DBZ-2827

  • EmbeddedEngine does not shutdown when commitOffsets is interrupted DBZ-2830

  • Rename user command parsing fails DBZ-2743

Other changes

  • Fix splitter annotations that control how content is modularized downstream DBZ-2824

  • VerifyRecord#isValid() compares JSON schema twice instead of Avro DBZ-735

  • Don’t rely on deprecated JSON serialization functionality of MongoDB driver DBZ-1322

  • Move website build to GitHub Actions DBZ-1984

  • Move Db2 connector to separate repository DBZ-2001

  • Modularize doc for SQL Server component DBZ-2335

  • Upgrade apicurio to 1.3.2.Final DBZ-2561

  • Remove obsolete logging files from /partials directory DBZ-2740

  • Remove obsolete monitoring files from /partials directory DBZ-2741

  • Increase Oracle CI frequency DBZ-2744

  • Make Debezium example work with Podman instead of Docker DBZ-2753

  • Disable log mining history by default DBZ-2763

  • Upgrade -setup-java action to the latest 1.4.3 DBZ-2770

  • Trigger non-core connector tests when core or DDL parser module are changed DBZ-2773

  • Add support for unsigned integer types DBZ-2776

  • Update JDK action workflow matrix with JDK 16.0.0-ea.24 DBZ-2777

  • Auto resolve latest JDK EA release number DBZ-2781

  • Update content in modularized SQL Server connector doc DBZ-2782

Release 1.4.0.Alpha2 (November 16th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.4.0.Alpha2 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Alpha2 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Alpha2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Move testcontainers changes on DebeziumContainer from UI PoC backend to Debezium main repo DBZ-2602

  • Add ability to map new name for the fields and headers DBZ-2606

  • Add close call to the Snapshotter interface DBZ-2608

  • Overriding Character Set Mapping DBZ-2673

  • Support PostgreSQL connector retry when database is restarted DBZ-2685

  • Cassandra connector documentation typos DBZ-2701

  • Fix typo in converters doc DBZ-2717

  • Add tests for DBZ-2617: PG connector does not enter FAILED state on failing heartbeats DBZ-2724

  • DBZ-2662 Control ChangeEventQueue by the size in bytes DBZ-2662

Fixes

  • Oracle throw "no snapshot found based on specified time" when running flashback query DBZ-1446

  • Exception when PK definition precedes column definition DBZ-2580

  • Patroni can’t stop PostgreSQL when Debezium is streaming DBZ-2617

  • ChangeRecord informations don’t connect with the TableSchema DBZ-2679

  • MySQL connector fails on a zero date DBZ-2682

  • Oracle LogMiner doesn’t support partition tables DBZ-2683

  • DB2 doesn’t start reliably in OCP DBZ-2693

  • Dropped columns cause NPE in SqlServerConnector DBZ-2716

  • Timestamp default value in 'yyyy-mm-dd' format fails MySQL connector DBZ-2726

  • Connection timeout on write should retry DBZ-2727

  • No viable alternative at input error on "min" column DBZ-2738

  • SQLServer CI error in SqlServerConnectorIT.whenCaptureInstanceExcludesColumnsAndColumnsRenamedExpectNoErrors:1473 DBZ-2747

  • debezium-connector-db2: DB2 SQL Error: SQLCODE=-206 on DB2 for z/OS DBZ-2755

  • no viable alternative at input 'alter table order drop CONSTRAINT' DBZ-2760

  • Tests are failing on macos DBZ-2762

Other changes

  • Move CI to Github Actions for all repositories DBZ-1720

  • Privileges missing from setup in documentation - Oracle LogMiner connector DBZ-2628

  • Add validation that replication slot doesn’t exist DBZ-2637

  • Update OpenJDK Quality Outreach jobs DBZ-2638

  • Re-unify monitoring content in the operations/monitoring.adoc file DBZ-2659

  • Pull oracle specific changes for reading table column metadata into debezium-core DBZ-2690

  • Intermittent test failure on CI - PostgresConnectorIT#shouldRegularlyFlushLsnWithTxMonitoring DBZ-2704

  • Topic routing doc formatting fix DBZ-2708

  • Re-unify logging content in the operations/logging.adoc file DBZ-2721

  • Incorporate Oracle LogMiner implementation updates DBZ-2729

  • Upgrade Vitess docker image to Vitess 8.0.0 DBZ-2749

  • Intermittent SQL Server test failure on CI - SqlServerConnectorIT DBZ-2625

  • Change initial.sync.max.threads to snapshot.max.threads DBZ-2742

Release 1.4.0.Alpha1 (October 22nd, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.4.0.Alpha1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Alpha1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Allow to specify subset of captured tables to be snapshotted DBZ-2456

  • Implement snapshot select override behavior for MongoDB DBZ-2496

  • Asciidoc block titles are rendered the same as regular text DBZ-2631

  • Allow closing of hung JDBC connection DBZ-2632

  • Hide stacktrace when default value for SQL Server cannot be parsed DBZ-2642

  • Implement a CDC connector for Vitess DBZ-2463

  • SqlServer - Skip processing of LSNs not associated with change table entries. DBZ-2582

Fixes

  • Cant override environment variables DBZ-2559

  • Inconsistencies in PostgreSQL Connector Docs DBZ-2584

  • ConcurrentModificationException during exporting data for a mongodb collection in a sharded cluster DBZ-2597

  • Mysql connector didn’t pass the default db charset to the column definition DBZ-2604

  • [Doc] "registry.redhat.io/amq7/amq-streams-kafka-25: unknown: Not Found" error occurs DBZ-2609

  • [Doc] "Error: no context directory and no Containerfile specified" error occurs DBZ-2610

  • SqlExceptions using dbz with Oracle on RDS online logs and logminer DBZ-2624

  • Mining session stopped - task killed/SQL operation cancelled - Oracle LogMiner DBZ-2629

  • Unparseable DDL: Using 'trigger' as table alias in view creation DBZ-2639

  • Antlr DDL parser fails to interpret BLOB([size]) DBZ-2641

  • MySQL Connector keeps stale offset metadata after snapshot.new.tables is changed DBZ-2643

  • WAL logs are not flushed in Postgres Connector DBZ-2653

  • Debezium server Event Hubs plugin support in v1.3 DBZ-2660

  • Cassandra Connector doesn’t use log4j for logging correctly DBZ-2661

  • Should Allow NonAsciiCharacter in SQL DBZ-2670

  • MariaDB nextval function is not supported in grammar DBZ-2671

  • Sanitize field name do not santize sub struct field DBZ-2680

  • Debezium fails if a non-existing view with the same name as existing table is dropped DBZ-2688

Other changes

  • Merge MySQL doc source files into one again DBZ-2127

  • Metrics links duplicate anchor IDs DBZ-2497

  • Slim down Vitess container image DBZ-2551

  • Modify release peipeline to support per-connector repos e.g. Vitess DBZ-2611

  • Add Vitess connector to Kafka Connect container image DBZ-2618

  • User Guide Documentation corrections for PostgreSQL DBZ-2621

  • Checkstyle should be built as a part of GH check formatting action DBZ-2623

  • Upgrade MySQL JDBC driver to version 8.0.19 DBZ-2626

  • Add support for multiple shard GTIDs in VGTID DBZ-2635

  • Add documentation for Vitess connector DBZ-2645

  • Restrict matrix job configurations to run only on Slaves DBZ-2648

  • Upgrade JUnit to 4.13.1 DBZ-2658

  • Avoid parsing generated files in Checkstyle DBZ-2669

  • Update debezium/awestruct image to use Antora 2.3.4 DBZ-2674

  • Fix doc typos and minor format glitches for downstream rendering DBZ-2681

  • Intermittent test failure on CI - RecordsStreamProducerIT#shouldReceiveHeartbeatAlsoWhenChangingNonWhitelistedTable() DBZ-2344