The Debezium UI team continues to add support for more features, allowing users to configure connectors more easily. In this article, we’ll describe and demonstrate how to provide the additional properties for configuration that the UI does not expose by default. Read further for more information!
Need for custom properties step
Debezium is very flexible - each connector can be configured and fine-tuned in various ways. Debezium UI already exposes an extensive list of properties that are segregated under different steps for configuration. However, despite the UI exposing a wide range of properties, there are situations where additional custom properties may be necessary beyond those exposed out of the box. For example,
consumer., and even
driver.* properties are pass-through configuration options for configuring parts of the Kafka Connect pipeline or the JDBC driver, respectively.
Debezium must configure the Kafka producer settings to write schema change events to the schema history topic for connectors that utilize a schema history topic, like MySQL. The connector configuration includes a subset of options that act as pass-through properties that begin with the
Debezium strips the prefix from the property names before passing the properties to the Kafka client.
Custom properties step
The Debezium UI allows you to configure additional properties for connectors not exposed in the UI. For this, we have created a new extra step called Custom Properties in the configuration wizard.
The custom properties step allows you to provide the additional properties by entering a key-value pair. Once you enter the additional configuration properties
value, click the Apply button.
When clicking the Apply button, the UI validates the user entries and provides feedback if there are any problems. If there are no problems, the UI will show a green check icon on the right side of all the successfully added additional properties. You can also remove any added property by clicking on the Remove button on the right side of the property.
On the left-hand side, you can see the list of already configured properties from the previous wizard step, and the newly added custom properties in this step are highlighted in blue color.
In the screenshot above, the user added
schema.history.internal.producer.* related additional properties.
You can try out configuring the connector with some custom properties (and more) with our self-contained example UI demo - which is included under debezium-examples on GitHub. The UI demo includes a Docker Compose file, which brings up several sources with data and the UI. Please refer to the README file for more details on running the Debezium UI demo.
To learn more about the Debezium UI, please refer to the reference documentation.
More coming soon!
Stay tuned for further improvements and new feature announcements in the UI in the coming releases.
A big thank you to the team who have contributed in many ways: Anisha Mohanty, René Kerner and Chrish Cranford!
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.