HeaderToValue

Table of Contents

Example
Configuration options

The HeaderToValue SMT extracts specified header fields from event records, and then copies or moves the header fields to values in the event record. The move options removes the fields from the header entirely before adding them as values in the payload. You can configure the SMT to manipulate multiple headers in the original message. You can use dot notation to specify a node within the payload in which you want to nest the header field. For more information about configuring the SMT, see the following example.

Example

To extract message headers in an event record into the record value, configure the HeaderToValue SMT in the Kafka Connect configuration for a connector. You can configure the transformation to either remove the original headers or to copy them. To remove header fields from the record, configure the SMT to use the move operations. To retain the header fields in the original record, configure the SMT to use the copy operation. For example, to remove the headers event_timestamp and key from an event message, add the following lines to your connector configuration:

transforms=moveHeadersToValue
transforms.moveHeadersToValue.type=io.debezium.transforms.HeaderToValue
transforms.moveHeadersToValue.headers=event_timestamp,key
transforms.moveHeadersToValue.fields=timestamp,source.id
transforms.moveHeadersToValue.operation=move

The following example shows the headers and values of an event record before and after the transformation is applied.

Example 1. Effect of applying the HeaderToValue SMT

Event record before it is processed by the HeaderToValue transformation

Header before the SMT processes the event record

{
    "header_x": 0,
    "event_timestamp": 1626102708861,
    "key": 100,
}

Value before the SMT processes the event record

{
        "before": null,
        "after": {
            "id": 1,
            "first_name": "Anne",
            "last_name": "Kretchmar",
            "email": "annek@noanswer.org"
        },
        "source": {
            "version": "2.1.3.Final",
            "connector": "postgresql",
            "name": "PostgreSQL_server",
            "ts_ms": 1559033904863,
            "ts_us": 1559033904863091,
            "ts_ns": 1559033904863091000,
            "snapshot": true,
            "db": "postgres",
            "sequence": "[\"24023119\",\"24023128\"]"
            "schema": "public",
            "table": "customers",
            "txId": 555,
            "lsn": 24023128,
            "xmin": null
        },
        "op": "c",
        "ts_ms": 1559033904863,
        "ts_us": 1559033904863741,
        "ts_ns": 1559033904863741697
    }

Event record after it is processed by the HeaderToValue transformation

Header after the SMT removes the specified field

{
    "header_x": 0
}

Value after the SMT moves header fields into the value

{
        "before": null,
        "after": {
            "id": 1,
            "first_name": "Anne",
            "last_name": "Kretchmar",
            "email": "annek@noanswer.org"
        },
        "source": {
            "version": "2.1.3.Final",
            "connector": "postgresql",
            "name": "PostgreSQL_server",
            "ts_ms": 1559033904863,
            "ts_us": 1559033904863697,
            "ts_ns": 1559033904863697000,
            "snapshot": true,
            "db": "postgres",
            "sequence": "[\"24023119\",\"24023128\"]"
            "schema": "public",
            "table": "customers",
            "txId": 555,
            "lsn": 24023128,
            "xmin": null,
            "id": 100
        },
        "op": "c",
        "ts_ms": 1559033904863,
        "ts_us": 1559033904863631,
        "ts_ns": 1559033904863631584,
        "event_timestamp": 1626102708861
    }

Configuration options

The following table lists the configuration options that you can use with the HeaderToValue SMT.

Table 1. HeaderToValue SMT configuration options
Property	Description	Type	Default	Valid Values	Importance
`headers`	A comma-separated list of header names in the record whose values are to be copied or moved to the record value.	list	No default value	non-empty list	high
`fields`	A comma-separated list of field names, in the same order as the header names listed in the `headers` configuration property. Use dot notation to instruct the SMT to nest fields within specific nodes of the message payload. For information about how to configure the SMT to use dot notation, see the example that appears earlier in this topic.	list	No default value	non-empty list	high
`operation`	Specifies one of the following options: `move`:: The SMT moves header fields to values in the event record, and removes the fields from the header. `copy`:: The SMT copies header field to values in the event record, and retains the original header fields.	string	No default value	move or copy	high