When I first learned about the Debezium project last year, I was very excited about it right away.
I could see how this project would be very useful for many people out there and I was very impressed by the professional way it was set up: a solid architecture for change data capture based on Apache Kafka, a strong focus on robustness and correctness also in the case of failures, the overall idea of creating a diverse eco-system of CDC connectors. All that based on the principles of open source, combined with extensive documentation from day one, a friendly and welcoming web site and a great getting-started experience.
So you can imagine that I was more than enthusiastic about the opportunity to take over the role of Debezium’s project lead. Debezium and CDC have close links to some data-centric projects I’ve been previously working on and also tie in with ideas I’ve been pursuing around CQRS, even sourcing and denormalization. As core member of the Hibernate team at Red Hat, I’ve implemented the initial Elasticsearch support for Hibernate Search (which deals with full-text index updates via JPA/Hibernate). I’ve also contributed to Hibernate OGM - a project which connects JPA and the world of NoSQL. One of the plans for OGM is to create a declarative denormalization engine for creating read models optimized for specific use cases. It will be very interesting to see how this plays together with the capabilities provided by Debezium.
Currently I am serving as the lead of the Bean Validation 2.0 specification (JSR 380) as well as its reference implementation Hibernate Validator. Two other projects close to my heart are MapStruct - a code generator for bean-to-bean mappings - and ModiTect, which is tooling for Java 9 modules and their descriptors. In general, I’m a strong believer into the idea of open source and I just love it to work with folks from all over the world to create useful tools and libraries.
Joining the Debezium community and working on change data capture is a great next step. There are so many things to do: connectors for Oracle, SQL Server and Cassandra, but also things like an entity join processor which would allow to step from row-level events to more aggregated business-level events (e.g. for updating a combined search index for an order and its order lines) or tooling for managing and visualizing histories of event schema changes.
One thing I’d like to emphasize is that the project’s direction generally isn’t going to change very much. Red Hat is fully committed to maintaining and evolving the project together with you, the Debezium community. The ride really has just begun!
Finally, let me say a huge thank you to Randall for his excellent work! You’ve been a true role model for going from an idea over pitching it - within Red Hat as well as within the wider community - to building a steadily growing and evolving project. It’s stating the obvious, but it wouldn’t be for Debezium without you. Thanks for everything and looking forward very much to working with you and the community on this great project!
Gunnar is a software engineer at Decodable and an open-source enthusiast by heart. He has been the project lead of Debezium over many years. Gunnar has created open-source projects like kcctl, JfrUnit, and MapStruct, and is the spec lead for Bean Validation 2.0 (JSR 380). He’s based in Hamburg, Germany.
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.