Over the last five years, Debezium has become a leading open-source solution for change data capture for a variety of databases. Users from all kinds of industries work with Debezium for use cases like replication of data from operational databases into data warehouses, updating caches and search indexes, driving streaming queries via Kafka Streams or Apache Flink, synchronizing data between microservices, and many more.

When talking to Debezium users, we generally receive very good feedback on the range of applications enabled by Debezium and its flexibility: e.g. each connector can be configured and fine-tuned in many ways, depending on your specific requirements. A large number of metrics provide deep insight into the state of running Debezium connectors, allowing to safely operate CDC pipelines also in huge installations with thousands of connectors.

All this comes at the cost of a learning curve, though: users new to Debezium need to understand the different options and settings as well as learn about best practices for running Debezium in production. We’re therefore constantly exploring how the user experience of Debezium can be further improved, allowing people to set up and operate its connectors more easily.

Today it’s my great pleasure to introduce you to a proof-of-concept for a potential future Debezium graphical user interface. The goal for this PoC is to explore how a graphical UI could facilitate the getting started and operational experience of Debezium users.

The scope of the PoC is the set-up flow for configuring and instantiating a Debezium Postgres connector. The user is guided through the required configuration steps in a wizard interface, starting from mandatory information (e.g. database credentials), over selecting the tables to be captured, up to optional settings like different data mapping options. After reviewing the final configuration, the UI will instantiate the connector in Kafka Connect.

You can see a short demo of how this looks like in this video:

We focused on some core interaction patterns, e.g. the preview functionality for the selecting the captured tables. Instead of solely taking key/value pairs of configuration parameters, the UI should guide the user through the process, provide context and help, e.g. by showing the allowed options for settings in drop-downs, validating the provided settings after each step, and more.

Now this is just the beginning, there’s many more things that could be done in such Debezium UI, e.g. in the connection configuration step, we could validate whether the given user has all the required database permissions, whether the right WAL level is set in the database, etc. There could be views for monitoring and trouble-shooting connectors. When running on Kubernetes, the UI could produce resource definitions processed by a Kafka (Connector) operator like Strimzi (instead of calling the Kafka Connect REST API), and much more.

But before further progressing with this, we’d like to gather your feedback and opinions: Do you consider a graphical UI for Debezium useful in general, and is it something you would use in your projects? What is your feedback on the functionality currently implemented in the PoC? Which other functionality besides connector configuration would you like to see in a Debezium UI? We’ve provided a short survey with these and a few other questions:

Before taking the questionnaire, please watch the video or run the PoC yourself (see below). Answering these questions should take just a few minutes; your participation would be very helpful for us in order to decide whether and how we should move forward with this effort.