Subscribe


Debezium 0.3.6 Released


We’re happy to announce that Debezium 0.3.6 is now available for use with Kafka Connect 0.10.0.1. This release contains over a dozen fixes combined for the MySQL connector and MongoDB connectors. See the release notes for specifics on these changes.

We’ve also updated the Debezium Docker images labelled 0.3 and latest, which we use in our tutorial.

Thanks to Farid, RenZhu, Dongjun, Anton, Chris, Dennis, Sharaf, Rodrigo, Tim, and others for their help with this release, issues, discussions, contributions, and questions!

What’s next

We’ll continue to improve the MongoDB and MySQL connectors, and we also have a great PostgreSQL connector that is nearly ready to be released. With the new connector we’ll switch release numbers to 0.4.x and plan to stop issuing 0.3.x releases. Stay tuned for this next 0.4.0 release!

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Gitter, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve the MySQL connector and add more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.

releases   mysql   docker  




Debezium 0.3.5 Released


We’re happy to announce that Debezium 0.3.5 is now available for use with Kafka Connect 0.10.0.1. This release contains several fixes for the MySQL connector and adds the ability to use with multi-master MySQL servers as sources. See the release notes for specifics on these changes. We’ve also updated the Debezium Docker images labelled 0.3 and latest, which we use in our tutorial.

One of the fixes is signficant, and so we strongly urge all users to upgrade to this release from all earlier versions. In prior versions, the MySQL connector may stop without completing all updates in a transaction, and when the connector restarts it starts with the next transaction and therefore might fail to capture some of the change events in the earlier transaction. This release fixes this issue so that when restarting it will always pick up where it left off, even if that point is in the middle of a transaction. Note that this fix only takes affect once a connector is upgraded and restarted. See the issue for more details.

Thanks to Akshath, Anton, Chris, and others for their help with the release, issues, discussions, contributions, and questions!

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Gitter, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve the MySQL connector and add more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.

releases   mysql   docker  




Debezium 0.3.4 Released


We’re happy to announce that Debezium 0.3.4 is now available for use with Kafka Connect 0.10.0.1. This release contains several new features for the MySQL connector: support for MySQL’s JSON datatype, a new snapshot mode called schema_only, and JMX metrics. Also, the Debezium Docker images for Zookeeper, Kafka, and Kafka Connect have all been updated to allow optionally expose JMX metrics in these services. And, one backward-incompatible fix was made to the change event’s ts_sec field. See the release notes for specifics.

We’ve also updated the Debezium Docker images labelled 0.3 and latest, which we use in our tutorial.

Thanks to Akshath, Chris, Vitalii, Dennis, Prannoy, and others for their help with the release, issues, discussions, contributions, and questions!

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Gitter, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve the MySQL connector and add more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.

releases   mysql   docker  




Support for MySQL’s JSON type coming soon


MySQL 5.7 introduced a new data type for storing and working with JSON data. Clients can define tables with columns using the new JSON datatype, and they can store and read JSON data using SQL statements and new built-in JSON functions to construct JSON data from other relational columns, introspect the structure of JSON values, and search within and manipulate JSON data. It possible to define generated columns on tables whose values are computed from the JSON value in another column of the same table, and to then define indexes with those generated columns. Overall, this is really a very powerful feature in MySQL.

Debezium’s MySQL connector will support the JSON datatype starting with the upcoming 0.3.4 release. JSON document, array, and scalar values will appear in change events as strings with io.debezium.data.json for the schema name. This will make it natural for consumers to work with JSON data. BTW, this is the same semantic schema type used by the MongoDB connector to represent JSON data.

This sounds straightforward, and we hope it is. But implementing this required a fair amount of work. That’s because although MySQL exposes JSON data as strings to client applications, internally it stores all JSON data in a special binary form that allows the MySQL engine to efficiently access the JSON data in queries, JSON functions and generated columns. All JSON data appears in the binlog in this binary form as well, which meant that we had to parse the binary form ourselves if we wanted to extract the more useful string representation. Writing and testing this parser took a bit of time and effort, and ultimately we donated it to the excellent MySQL binlog client library that the connector uses internally to read the binlog events.

We’d like to thank Stanley Shyiko for guiding us and helping us debug the final problems with parsing JSON in the binlog, for accepting our proposed changes into his library, for releasing his library quickly when needed, and for being so responsive on this and other issues!

mysql   json  




Debezium 0.3.3 Released


We’re happy to announce that Debezium 0.3.3 is now available for use with Kafka Connect 0.10.0.1. This release contains a handful of bug fixes and minor improvements for the MySQL connector, including better handling of BIT(n) values, ENUM and SET values, and GTID sets, This release also improves the log messages output by the MySQL connectors to better represent the ongoing activity when consuming the changes from the source database. See the release notes for specifics.

We’ve also updated the Debezium Docker images labelled 0.3 and latest, which we use in our tutorial. We’ve also updated the tutorial to use the latest Docker installations on Linux, Windows, and OS X.

Thanks to Akshath, Chris, Randy, Prannoy, Umang, Horia, and others for their help with the release, issues, discussions, contributions, and questions!

About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Gitter, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve the MySQL connector and add more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.

releases   mysql   mongodb   docker  




back to top