Transformations

Connectors can be configured with transformations to make lightweight per message modifications. Debezium provides several single message transformations (SMTs) that you can use to either modify records before they are sent to Apache Kafka (by applying them to the Debezium connectors), or when they are read from Kafka by a sink connector. Also Debezium Server supports the usage of SMTs.

The following SMTs are provided by Debezium:

Transform	Description
Topic Routing	Re-routes records to different topics based on a regular expression applied to the original topic name.
Content-Based Routing	Reroute selected events to other topics, based on the event content.
New Record State Extraction	Extracts the flat structure of field names and values from Debezium change events, facilitating sink connectors which cannot process Debezium’s complex event structure.
MongoDB New Document State Extraction	The MongoDB-specific counter-part to the New Record State Extraction SMT.
Outbox Event Router	Provides a way to safely and reliably exchange data between multiple (micro) services.
MongoDB Outbox Event Router	The MongoDB-specific counter-part to the Outbox Event Router SMT.
Message Filtering	Applies a filter to the change events emitted by the connectors, based on their content. This lets you propagate only those records that are relevant to you.
HeaderToValue	Moves or copies headers into the record value.
Partition Routing	Re-routes records to specific partitions based on configured payload fields.
Timezone Converter	Converts Debezium and Kafka Connect timestamp fields in event records to a specified timezone.
TimescaleDB Integration	Routes and enriches messages that the Debezium PostgreSQL connector captures from a TimescaleDB.
Convert CloudEvents to Saveable Form	Converts values of records deserialized by Debezium CloudEvents converter to a structure suitable for Debezium JDBC sink connector.
Vitess Use Local VGTID	A transformation that reduces the size of VGTIDs that the Vitess connector emits. To reduce the amount of data in the event message, the SMT writes only the VGTID for the shard in which a change occurs. The VGTIDs for other shards are removed. This transformation is designed for use only with the Debezium connector for Vitess.
Vitess Remove Field	The Vitess-specific transformation to remove fields to reduce the size of the messages written (e.g., redundant VGTID fields).
Vitess Filter Transaction Topic Records	A transformation that filters out all messages that the connector emits to the transaction topic (when transaction metadata is enabled). This transformation is designed for use only with the Debezium connector for Vitess.

Transform

Description

Topic Routing

Re-routes records to different topics based on a regular expression applied to the original topic name.

Content-Based Routing

Reroute selected events to other topics, based on the event content.

New Record State Extraction

Extracts the flat structure of field names and values from Debezium change events, facilitating sink connectors which cannot process Debezium’s complex event structure.

MongoDB New Document State Extraction

The MongoDB-specific counter-part to the New Record State Extraction SMT.

Outbox Event Router

Provides a way to safely and reliably exchange data between multiple (micro) services.

MongoDB Outbox Event Router

The MongoDB-specific counter-part to the Outbox Event Router SMT.

Message Filtering

Applies a filter to the change events emitted by the connectors, based on their content. This lets you propagate only those records that are relevant to you.

HeaderToValue

Moves or copies headers into the record value.

Partition Routing

Re-routes records to specific partitions based on configured payload fields.

Timezone Converter

Converts Debezium and Kafka Connect timestamp fields in event records to a specified timezone.

TimescaleDB Integration

Routes and enriches messages that the Debezium PostgreSQL connector captures from a TimescaleDB.

Convert CloudEvents to Saveable Form

Converts values of records deserialized by Debezium CloudEvents converter to a structure suitable for Debezium JDBC sink connector.

Vitess Use Local VGTID

A transformation that reduces the size of VGTIDs that the Vitess connector emits. To reduce the amount of data in the event message, the SMT writes only the VGTID for the shard in which a change occurs. The VGTIDs for other shards are removed. This transformation is designed for use only with the Debezium connector for Vitess.

Vitess Remove Field

The Vitess-specific transformation to remove fields to reduce the size of the messages written (e.g., redundant VGTID fields).

Vitess Filter Transaction Topic Records

A transformation that filters out all messages that the connector emits to the transaction topic (when transaction metadata is enabled). This transformation is designed for use only with the Debezium connector for Vitess.

By means of SMT Predicates you can apply any of the transformations selectively, so that it modifies only that subset of change event messages that share a common characteristic.

Most of the above SMTs are available by default with the Debezium container image, but you need to opt-in for the scripting-based ones (Message Filtering or Content-based Routing). See the README file of the container image for more details.