Debezium 1.3.0.Beta1 Released

Revised Filter Options and Documentation Wording

Since the beginning of the Debezium project, there was support for specifying the tables and columns to capture. This is done via a range of configuration options like schema.whitelist, column.blacklist, etc.

While nothing was wrong with those options from a technical perspective, we’ve come to realize that the terms "whitelist" and "blacklist" are problematic and that they may even be hurtful to some members of our community. This is why we’ve decided to deprecate the existing option names and replace them with counterparts which are not only more inclusive, but also more expressive when it comes to describing their purpose.

The following changes have been made:

Old Name New Name

Old Name	New Name
`database.whitelist`	`database.include.list`
`database.blacklist`	`database.exclude.list`
`schema.whitelist`	`schema.include.list`
`schema.blacklist`	`schema.exclude.list`
`table.whitelist`	`table.include.list`
`table.blacklist`	`table.exclude.list`
`column.whitelist`	`column.include.list`
`column.blacklist`	`column.exclude.list`

database.whitelist

database.include.list

database.blacklist

database.exclude.list

schema.whitelist

schema.include.list

schema.blacklist

schema.exclude.list

table.whitelist

table.include.list

table.blacklist

table.exclude.list

column.whitelist

column.include.list

column.blacklist

column.exclude.list

The renaming has been done for all the stable Debezium connectors as of this release; the options of the incubator connectors (Oracle, Db2, Cassandra) will be renamed in the next Debezium 1.3.x preview release. Note that for the sake of backwards-compatibility, the old option names still can be used during a transition period. In this case, e.g. when upgrading an existing connector instance to the new version, a warning will be logged upon connector start-up, and you should update your configuration accordingly.

Besides renaming these filter options, we’ve also updated our documentation; in particular the description of supported database topologies has been updated from the previously used terms "master" and "slave" to "primary" (node) and "replica" (node).

This change is part of a larger effort across Red Hat and the industry as a whole, and we’re very happy that we can contribute our share for making the world of open-source projects and its communities more welcoming and inclusive.

Bug Fixes

This release fixes a number of critical bugs:

Potentially lost change events with the Postgres connector, in case of connector restarts (DBZ-2338, DBZ-2397)
NullPointerException in the logical table router (DBZ-2412)
Snapshot fails if table or schema contain hyphens (DBZ-2452)
Misc. MySQL DDL parser fixes (DBZ-2413, DBZ-2415, DBZ-2425)

Altogether, 20 issues were fixed for this release.

Please refer to the release notes for the complete list of resolved issues as well as procedures for upgrading from earlier Debezium versions.

Many thanks to community members Bingqin Zhou and Ruslan Gibaiev for their contributions to this release!

Outlook

As you’d expect it, things went a bit slower during the summer with several folks taking some well-deserved time off. Now that everyone is back, Debezium development moves forward with full steam again, and you can expect some exciting new features coming soon: the ongoing work by the community towards a LogMiner-based implementation for Oracle should soon reach a state where it can be merged into the upstream Debezium repository.

And a brand-new connector contributed by the community is showing up on the horizon, too; Bolt engineers Kewei Shang and Ruslan Gibaiev have been working on a CDC connector for the Vitess database and announced that they’d like to open-source and continue to evolve it under the Debezium umbrella.

Exciting times for open-source change data capture and Debezium 🎉!

Gunnar Morling

Gunnar is a software engineer at Decodable and an open-source enthusiast by heart. He has been the project lead of Debezium over many years. Gunnar has created open-source projects like kcctl, JfrUnit, and MapStruct, and is the spec lead for Bean Validation 2.0 (JSR 380). He’s based in Hamburg, Germany.

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.