Debezium Blog
Most engineers working in data streaming are not SQL specialists. So you might be asking yourself: What is a CTE? More importantly, what are CTE queries, why are they useful, and how do they help you in the context of Debezium?
In this post, we’ll answer those questions, explore how the Debezium Oracle connector leverages CTE queries, and discuss the benefits and trade-offs involved.
Debezium 3.2.1.Final is here — delivering faster performance, smarter resource use, and rock-solid stability for your CDC pipelines. This release brings improved PostgreSQL TOAST handling, native MariaDB vector data type support, and major Oracle LogMiner resilience improvements, all designed to keep your data flowing smoothly and efficiently.
When I started working on Debezium, two questions came to mind: Is it possible to build a native version of Debezium? Can I receive change data capture (CDC) events directly inside my microservice without relying on additional infrastructure?
This led us to work on a new Debezium stream: I’m excited to announce the first release of Debezium Extensions for Quarkus!
Debezium 3.3.0.Alpha1 introduces an exciting wave of innovation, including exactly-once semantics for core connectors and a brand new CockroachDB connector led by the community. Featuring support for emerging data types, deeper Quarkus integration, and enhanced tooling, this release raises the bar for modern change data capture solutions. Ready to see what’s new? Let’s dive in.
Remember when debugging streaming data pipelines felt like playing detective in a crime scene where the evidence kept moving? Well, grab your magnifying glass because we’re about to turn you into Sherlock Holmes of the streaming world. After our introduction to OpenLineage integration with Debezium, it’s time to roll up our sleeves and get our hands dirty with some real detective work. We’ll build a complete order processing pipeline that captures database changes with Debezium, processes them through Apache Flink, and tracks every breadcrumb of data lineage using OpenLineage and Marquez – because losing track of your data is like losing your keys, except infinitely more embarrassing in production.
Case definition
In this showcase, we demonstrate how to leverage lineage metadata to troubleshoot issues in data pipelines. Our e-commerce order processing pipeline, despite its simplicity, effectively illustrates the benefits of lineage metadata for operational monitoring and debugging. We will simulate a configuration change in the Debezium connectors that causes the order processing job to skip records. Using the lineage graph, we’ll navigate through the pipeline components to identify the root cause of the problem and understand how metadata tracking enables faster issue resolution.
