Release Notes for Debezium 1.2

All notable changes for Debezium releases are documented in this file. Release numbers follow Semantic Versioning.

Release 1.2.0.Beta2 (May 19th, 2020)

Kafka compatibility

This release has been built against Kafka Connect 2.5.0 and has been tested with version 2.5.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.2.0.Beta2 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.2.0.Beta2 plugin files, and restart the connector using the same configuration. Upon restart, the 1.2.0.Beta2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

The snapshot mode initial_schema_only was renamed schema_only for Db2 connector (DBZ-2051).

The previously deprecated options operation.header and add.source.fields of the ExtractNewRecordState have been removed; please use add.headers and add.fields instead (DBZ-1828).

When instantiating the Debezium container in integration tests with Testcontainers, the full image name must be given now, e.g. 1debezium/connect:1.2.0.Beta2`. This is to allow for using custom container images in tests, e.g. containing additional SMTs, converters or sink connectors (DBZ-2070).

New Features

  • Add JDBC driver versions to docs DBZ-2031

  • Add a few more loggings for Cassandra Connector DBZ-2066

  • Provide ready-to-use standalone application based on the embedded engine DBZ-651

  • Add option to skip LSN timestamp queries DBZ-1988

  • Add option to logical topic router for controlling placement of table information DBZ-2034

  • Add headers and topic name into scripting transforms DBZ-2074

  • Filter and content-based router SMTs should be restrictable to certain topics DBZ-2024

Fixes

This release includes the following fixes:

  • Avro schema doesn’t change if a column default value changes from 'foo' to 'bar' DBZ-2061

  • DDL statement throws error if compression keyword contains backticks (``) DBZ-2062

  • Error and connector stops when DDL contains algorithm=instant DBZ-2067

  • Debezium Engine advanced record consuming example broken DBZ-2073

  • Unable to parse MySQL ALTER statement with named primary key DBZ-2080

  • Missing schema-serializer dependency for Avro DBZ-2082

  • TinyIntOneToBooleanConverter doesn’t seem to work with columns having a default value DBZ-2085

Other changes

This release includes also other changes:

  • Add ability to insert fields from op field in ExtractNewDocumentState DBZ-1791

  • Test with MySQL 8.0.20 DBZ-2041

  • Update debezium-examples/tutorial README docker-compose file is missing DBZ-2059

  • Skip tests that are no longer compatible with Kafka 1.x DBZ-2068

  • Remove additional Jackson dependencies as of AK 2.5 DBZ-2076

  • Make EventProcessingFailureHandlingIT resilient against timing issues DBZ-2078

  • Tar packages must use posix format DBZ-2088

  • Remove unused sourceInfo variable DBZ-2090

Release 1.2.0.Beta1 (May 7th, 2020)

Kafka compatibility

This release has been built against Kafka Connect 2.5.0 and has been tested with version 2.5.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.2.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.2.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.2.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

Field eventType was removed from Outbox router SMT (DBZ-2014).

JDBC driver has been upgrade to the version to 42.2.12 (DBZ-2027). Due to changes in the driver behaviour it is necessary to keep Debezium and driver versions aligned.

Debezium API now allows conversion to JSON and Avro types distinctly for key and value (DBZ-1970). To enable this feature it was necessary to modify the incubating Debezium API.

New Features

  • Don’t try to database history topic if it exists already DBZ-1886

  • Deleted database history should be detected for all connectors DBZ-1923

  • Provide anchors to connector parameters DBZ-1933

  • move static methods TRUNCATE_COLUMN and MASK_COLUMN as attributes to RelationalDatabaseConnectorConfig DBZ-1972

  • Implement SKIPPED_OPERATIONS for mysql DBZ-1895

  • User facing schema history topic for SQL Server DBZ-1904

  • Multiline stack traces can be collapsed into a single log event DBZ-1913

  • Introduce column.whitelist for Postgres Connector DBZ-1962

  • Add support for Postgres time, timestamp array columns DBZ-1969

  • Add support for Postgres Json and Jsonb array columns DBZ-1990

  • Content-based topic routing based on scripting languages DBZ-2000

  • Support different converters for key/value in embedded engine DBZ-1970

Fixes

This release includes the following fixes:

  • bit varying column has value that is too large to be cast to a long DBZ-1949

  • PostgreSQL Sink connector with outbox event router and Avro uses wrong default io.confluent schema namespace DBZ-1963

  • Stop processing new commitlogs in cdc folder DBZ-1985

  • [Doc] Debezium User Guide should provide example of DB connector yaml and deployment instructions DBZ-2011

  • ExtractNewRecordState SMT spamming logs for heartbeat messages DBZ-2036

  • MySQL alias FLUSH TABLE not handled DBZ-2047

  • Embedded engine not compatible with Kafka 1.x DBZ-2054

Other changes

This release includes also other changes:

  • Blog post and demo about Debezium + Camel DBZ-1656

  • Refactor connector config code to share the configuration definition DBZ-1750

  • DB2 connector follow-up refactorings DBZ-1753

  • Oracle JDBC driver available in Maven Central DBZ-1878

  • Align snapshot/streaming semantics in MongoDB documentation DBZ-1901

  • Add MySQL 5.5 and 5.6 to test matrix. DBZ-1953

  • Upgrade to Quarkus to 1.4.1 release DBZ-1975

  • Version selector on releases page should show all versions DBZ-1979

  • Upgrade to Apache Kafka 2.5.0 and Confluent Platform 5.5.0 DBZ-1981

  • Fix broken link DBZ-1983

  • Update Outbox Quarkus extension yaml DBZ-1991

  • Allow for simplified property references in filter SMT with graal.js DBZ-1993

  • Avoid broken cross-book references in downstream docs DBZ-1999

  • Fix wrong attribute name in MongoDB connector DBZ-2006

  • Upgrade formatter and Impsort plugins DBZ-2007

  • Clarify support for non-primary key tables in PostgreSQL documentation DBZ-2010

  • Intermittent test failure on CI DBZ-2030

  • Cleanup Postgres TypeRegistry DBZ-2038

  • Upgrade to latest parent pom and checkstyle DBZ-2039

  • Reduce build output to avoid maximum log length problems on CI DBZ-2043

  • Postgres TypeRegistry makes one query per enum type at startup DBZ-2044

  • Remove obsolete metrics from downstream docs DBZ-1947

Release 1.2.0.Alpha1 (April 16th, 2020)

Kafka compatibility

This release has been built against Kafka Connect 2.4.1 and has been tested with version 2.4.1 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.2.0.Alpha1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.2.0.Alpha1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.2.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

For the SQL Server connector, the previously deprecated snapshot mode initial_schema_only has been removed. The mode schema_only should be used instead, providing the same behavior and semantics (DBZ-1945).

The previously deprecated message transformations UnwrapFromEnvelope and UnwrapMongoDbEnvelope have been removed. Instead, please use ExtractNewRecordState and ExtractNewDocumentState, respectively (DBZ-1968).

New Features

  • Expose original value for PK updates DBZ-1531

  • New column masking mode: consistent hashing DBZ-1692

  • Provide a filtering SMT DBZ-1782

  • Support converters for embedded engine DBZ-1807

  • Enhance MongoDB connector metrics DBZ-1859

  • SQL Server connector: support reconnect after the database connection is broken DBZ-1882

  • Support SMTs in embedded engine DBZ-1930

  • Snapshot metrics shows TotalNumberOfEventsSeen as zero DBZ-1932

Fixes

This release includes the following fixes:

  • java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff] DBZ-1744

  • Snapshot lock timeout setting is not documented DBZ-1914

  • AvroRuntimeException when publishing transaction metadata DBZ-1915

  • Connector restart logic throttles for the first 2 seconds DBZ-1918

  • Wal2json empty change event could cause NPE above version 1.0.3.final DBZ-1922

  • Misleading error message on lost database connection DBZ-1926

  • Cassandra CDC should not move and delete processed commitLog file under testing mode DBZ-1927

  • Broken internal links and anchors in documentation DBZ-1935

  • Dokumentation files in modules create separate pages, should be partials instead DBZ-1944

  • Validation of binlog_row_image is not compatible with MySQL 5.5 DBZ-1950

  • High CPU usage when idle DBZ-1960

  • Outbox Quarkus Extension throws NPE in quarkus:dev mode DBZ-1966

  • Cassandra Connector: unable to deserialize column mutation with reversed type DBZ-1967

Other changes

This release includes also other changes:

  • Replace Custom CassandraTopicSelector with DBZ’s TopicSelector class in Cassandra Connector DBZ-1407

  • Improve documentation on WAL disk space usage for Postgres connector DBZ-1732

  • Outbox Quarkus Extension: Update version of extension used by demo DBZ-1786

  • Community newsletter 1/2020 DBZ-1806

  • Remove obsolete SnapshotChangeRecordEmitter DBZ-1898

  • Fix typo in Quarkus Outbox extension documentation DBZ-1902

  • Update schema change topic section of SQL Server connector doc DBZ-1903

  • Documentation should link to Apache Kafka upstream docs DBZ-1906

  • Log warning about insufficient retention time for DB history topic DBZ-1905

  • The error messaging around binlog configuration is missleading DBZ-1911

  • Restore documentation of MySQL event structures DBZ-1919

  • Link from monitoring page to connector-specific metrics DBZ-1920

  • Update snapshot.mode options in SQL Server documentation DBZ-1924

  • Update build and container images to Apache Kafka 2.4.1 DBZ-1925

  • Avoid Thread#sleep() calls in Oracle connector tests DBZ-1942

  • Different versions of Jackson components pulled in as dependencies DBZ-1943

  • Remove deprecated connector option value "initial_schema_only" DBZ-1945

  • Add docs for mask column and truncate column features DBZ-1954

  • Upgrade MongoDB driver to 3.12.3 DBZ-1958

  • Remove deprecated unwrap SMTs DBZ-1968

back to top