Featured Posts

Tags
WebAssembly analytics announcement announcement discussion survey apache kafka apache kafka apicurio avro aws batch caassandra camel cassandra cdc channels chicory community community stories connectors containers cqrs custom data engineering data replication datalake db2 ddd debezium debezium server debezium ui deduplication discussion dlt dlthub docker elasticsearch event sourcing exactly once semantics example examples features fedora flink go hiring ibmi iceberg images informix integration introduction jaeger jdbc json kafka kafka streams kafka streams kogito ksql kubernetes lakehouse machine learning mariadb metrics microservices mongo mongodb monitoring mysql news newsletter notifications online learning operator oracle outbox performance postgres presentation production pydbzengine python quarkus questdb rds releases schema scylla secrets sentry serialization signaling smt snapshots spanner spark sql sqlserver tensorflow testcontainers tests time series timescaledb tinygo topics tracing transactions ui vagrant vitess wasm website

Debezium Blog

Just shy of a month after the 0.6.0 release, I’m happy to announce the release of Debezium 0.6.1!

This release contains several bugfixes, dependency upgrades and a new option for controlling how BIGINT UNSIGNED columns are conveyed. We also expanded the set of Docker images and Docker Compose files accompanying our tutorial, so you can run it now with all the databases we support.

Let’s take a closer look at some of the changes.

Read More    

In this blog post we will create a simple streaming data pipeline to continuously capture the changes in a MySQL database and replicate them in near real-time into a PostgreSQL database. We’ll show how to do this without writing any code, but instead by using and configuring Kafka Connect, the Debezium MySQL source connector, the Confluent JDBC sink connector, and a few single message transforms (SMTs).

This approach of replicating data through Kafka is really useful on its own, but it becomes even more advantageous when we can combine our near real-time streams of data changes with other streams, connectors, and stream processing applications. A recent Confluent blog post series shows a similar streaming data pipeline but using different connectors and SMTs. What’s great about Kafka Connect is that you can mix and match connectors to move data between multiple systems.

We will also demonstrate a new functionality that was released with Debezium 0.6.0: a single message transform for CDC Event Flattening.

Read More    

What’s better than getting Java 9? Getting Java 9 and a new version of Debezium at the same time! So it’s with great happiness that I’m announcing the release of Debezium 0.6 today.

Read More    

I’m very happy to announce the release of Debezium 0.5.2!

As the previous release, the 0.5.2 release fixes several bugs in the MySQL, Postgres and MongoDB connectors. But there are also several new features and options:

  • The decimal.handling.mode option already known from the MySQL connector is now also supported for PostgreSQL (DBZ-337). It lets you control how NUMERIC and DECIMAL columns are represented in change events (either using Kafka’s Decimal type or as double).

  • The MongoDB connector supports the options database.whitelist and database.blacklist now (DBZ-302)

  • The PostgreSQL connector can deal with array-typed columns as well as with quoted identifiers for tables, schemas etc. (DBZ-297, DBZ-298)

  • The Debezium Docker images run on Red Hat’s OpenShift cloud environment (DBZ-267)

Read More    

It’s my pleasure to announce the release of Debezium 0.5.1!

This release fixes several bugs in the MySQL, Postgres and MongoDB connectors. There’s also support for some new datatypes: POINT on MySQL (DBZ-222) and TSTZRANGE on Postgres (DBZ-280). This release is a drop-in replacement for 0.5.0, upgrading is recommended to all users.

Note that in the — rather unlikely — case that you happened to enable Debezium for all the system tables of MySQL, any configured table filters will be applied to these system tables now, too (DBZ-242). This may require an adjustment of your filters if you indeed wanted to capture all system tables but only selected non-system tables.

Read More