Release Notes for Debezium 1.1

All notable changes for Debezium releases are documented in this file. Release numbers follow Semantic Versioning.

Release 1.1.0.Beta2 (February 13th, 2020)

Kafka compatibility

This release has been built against Kafka Connect 2.4.0 and has been tested with version 2.4.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.1.0.Beta2 from any of the earlier 1.1.x, 1.0.x, 0.10.x, 0.9.x, 0.8.x, 0.7.x, 0.6.x, 0.5.x, 0.4.x, 0.3.x, 0.2.x, or 0.1.x versions, first check the upgrading notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.1.0.Beta2 plugin files, and restart the connector using the same configuration. Upon restart, the 1.1.0.Beta2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Add ability to insert fields from op field in ExtractNewRecordState SMT DBZ-1452

  • Integrates with TestContainers project DBZ-1722

Fixes

This release includes the following fixes:

  • Postgres Connector losing data on restart due to commit() being called before events produced to Kafka DBZ-1766

  • DBREF fields causes SchemaParseException using New Record State Extraction SMT and Avro converter DBZ-1767

Other changes

This release includes also other changes:

  • Superfluous whitespace in intra-level ToC sidebar DBZ-1668

  • Outbox Quarkus Extension follow-up tasks DBZ-1711

  • DB2 connector follow-up tasks DBZ-1752

  • Unwrap SMT demo not compatible with ES 6.1+ DBZ-1756

  • Instable SQL Server test DBZ-1764

  • Remove Db2 JDBC driver from assembly package DBZ-1776

  • Fix PostgresConnectorIT.shouldOutputRecordsInCloudEventsFormat test DBZ-1783

  • Use "application/avro" as data content type in CloudEvents DBZ-1784

  • Update Standard Tutorials/Examples with DB2 DBZ-1558

Release 1.1.0.Beta1 (February 5th, 2020)

Kafka compatibility

This release has been built against Kafka Connect 2.4.0 and has been tested with version 2.4.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.1.0.Beta1 from any of the earlier 1.1.x, 1.0.x, 0.10.x, 0.9.x, 0.8.x, 0.7.x, 0.6.x, 0.5.x, 0.4.x, 0.3.x, 0.2.x, or 0.1.x versions, first check the upgrading notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.1.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.1.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

Before updating the DecoderBufs logical decoding plug-in in your Postgres database to this new version (or when pulling the debezium/postgres container image for that new version), it is neccessary to upgrade the Debezium Postgres connector to 1.0.1.Final or 1.1.0.Alpha2 or later (DBZ-1052).

The ExtractNewDocumentState SMT to be used with the Debezium MongoDB connector will convert Date and Timestamp fields now into the org.apache.kafka.connect.data.Timestam`p logical type, clarifying its semantics. The schema type itself remains unchanged as `int64. Please note that the resolution of Timestamp is seconds as per the semantics of that type in MongoDB. (DBZ-1717).

New Features

  • Create a plug-in for DB2 streaming DBZ-695

  • Add topic routing by field option for New Record State Extraction DBZ-1715

  • Generate date(time) field types in the Kafka Connect data structure DBZ-1717

  • Publish TX boundary markers on a TX metadata topic DBZ-1052

  • Replace connectorName with kafkaTopicPrefix in kafka key/value schema DBZ-1763

Fixes

This release includes the following fixes:

  • Connector error after adding a new not null column to table in Postgres DBZ-1698

  • MySQL connector doesn’t use default value of connector.port DBZ-1712

  • Fix broken images in Antora and brush up AsciiDoc DBZ-1725

  • ANTLR parser cannot parse MariaDB Table DDL with TRANSACTIONAL attribute DBZ-1733

  • Postgres connector does not support proxied connections DBZ-1738

  • GET DIAGNOSTICS statement not parseable DBZ-1740

  • Examples use http access to Maven repos which is no longer available DBZ-1741

  • MySql password logged out in debug log level DBZ-1748

  • Cannot shutdown PostgreSQL if there is an active Debezium connector DBZ-1727

Other changes

This release includes also other changes:

  • Add tests for using fallback values with default REPLICA IDENTITY DBZ-1158

  • Migrate all attribute name/value pairs to Antora component descriptors DBZ-1687

  • Upgrade to Awestruct 0.6.0 DBZ-1719

  • Run CI tests for delivered non-connector modules (like Quarkus) DBZ-1724

  • Remove overlap of different documentation config files DBZ-1729

  • Don’t fail upon receiving unkown operation events DBZ-1747

  • Provide a method to identify an envelope schema DBZ-1751

  • Upgrade to Mongo Java Driver version 3.12.1 DBZ-1761

  • Create initial Proposal for DB2 Source Connector DBZ-1509

  • Review Pull Request for DB2 Connector DBZ-1527

  • Test Set up of the DB2 Test Instance DBZ-1556

  • Create Documentation for the DB2 Connector DBZ-1557

  • Verify support of all DB2 types DBZ-1708

Release 1.1.0.Alpha1 (January 16th, 2020)

Kafka compatibility

This release has been built against Kafka Connect 2.4.0 and has been tested with version 2.4.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.1.0.Alpha1 from any of the earlier 1.1.x, 1.0.x, 0.10.x, 0.9.x, 0.8.x, 0.7.x, 0.6.x, 0.5.x, 0.4.x, 0.3.x, 0.2.x, or 0.1.x versions, first check the upgrading notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.1.0.Alpha1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.1.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

When using the outbox event routing SMT and configuring a column from which to obtain the Kafka record timestamp from (table.field.event.timestamp option), then that value could have been exported as milliseconds, microseconds or nanoseconds, based on the source column’s definition. As of this release, the timestamp always be exported as milliseconds (DBZ-1707).

The deprecated Postgres connector option slot.drop_on_stop has been removed; use slot.drop.on.stop instead (DBZ-1600).

New Features

  • MongoDB authentication against non-admin authsource DBZ-1168

  • Oracle: Add support for different representations of "NUMBER" Data Type DBZ-1552

  • Update Mongo Java driver to version 3.12.0 DBZ-1690

  • Support exporting change events in "CloudEvents" format DBZ-1292

  • Build Quarkus extension facilitating implementations of the outbox pattern DBZ-1478

  • Support column masking option for Postgres DBZ-1685

Fixes

This release includes the following fixes:

  • Make slot creation in PostgreSQL more resilient DBZ-1684

  • SQLserver type time(4)…​time(7) lost nanoseconds DBZ-1688

  • Support boolean as default for INT(1) column in MySQL DBZ-1689

  • SIGNAL statement is not recognized by DDL parser DBZ-1691

  • When using in embedded mode MYSQL connector fails DBZ-1693

  • MySQL connector fails to parse trigger DDL DBZ-1699

Other changes

This release includes also other changes:

  • Update outbox routing example DBZ-1673

  • Add option to JSON change event SerDe for ignoring unknown properties DBZ-1703

  • Update debezium/awestruct image to use Antora 2.3 alpha 2 DBZ-1713

back to top