Debezium Blog

I’m very happy to announce the release of Debezium 1.3.0.CR1!

As we approach the final stretch of Debezium 1.3 Final, we took this opportunity to add delegate converter support for the ByteBufferConverter and introduce a debezium-scripting module. In addition, there’s also a range of bug fixes and quite a bit of documentation polish; overall, not less than 15 issues have been resolved for this release.

I’m very happy to announce the release of Debezium 1.3.0.Beta2!

In this release we’ve improved support for column filtering for the MySQL and SQL Server connectors, and there’s a brand-new implementation for ingesting change events from Oracle, using the LogMiner package. As we’re on the home stretch towards Debezium 1.3 Final, there’s also a wide range of smaller improvements, bug fixes and documentation clarifications; overall, not less than 44 issues have been resolved for this release.

Create new topics / pipes

When you are working with Kafka Connect Distributed then you might have realized that once you start Kafka Connect there are already some internal Kafka Connect related topics created for you:

$ kafka-topics.sh --bootstrap-server $HOSTNAME:9092 --list

connect_configs
connect_offsets
connect_statuses

This is done automatically for you by Kafka Connect with a sane, customized default topic configuration that fits the needs of these internal topics.

When you start a Debezium connector the topics for the captured events are created by the Kafka broker based on a default, maybe customized, configuration in the broker if auto.create.topics.enable = true is enabled in the broker config:

auto.create.topics.enable = true
default.replication.factor = 1
num.partitions = 1
compression.type = producer
log.cleanup.policy = delete
log.retention.ms = 604800000  ## 7 days

But often, when you use Debezium and Kafka in a production environment you might choose to disable Kafka’s topic auto creation capability with auto.create.topics.enable = false, or you want the connector topics to be configured differently from the default. In this case you have to create topics for Debezium’s captured data sources upfront.
But there’s good news! Beginning with Kafka Connect version 2.6.0, this can be automated since KIP-158 is implemented to enable customizable topic creation with Kafka Connect.

It’s my pleasure to announce the release of Debezium 1.3.0.Beta1!

This release upgrades to the recently released Apache Kafka version 2.6.0, fixes several critical bugs and comes with a renaming of the connector configuration options for selecting the tables to be captured. We’ve also released Debezium 1.2.2.Final, which is a drop-in replacement for all users of earlier 1.2.x releases.

I’m excited to announce the release of Debezium 1.3.0.Alpha1!

This initial pass in the 1.3 release line provides a number of useful new features:

  • A new Debezium Server sink adapter for Azure Event Hubs

  • A new SQL Server connector snapshot mode, initial_only

  • Additional connection timeout options for the MongoDB Connector

Overall, the community fixed not less than 31 issues for this release. Let’s take a closer look at some of them in the remainder of this post.

back to top