EnrichCdcStream 2025.3.28.13-SNAPSHOT¶
BUNDLE¶
com.snowflake.openflow.runtime | runtime-database-cdc-processors-nar
DESCRIPTION¶
Enriches incoming FlowFiles that come from CaptureChangePostgreSQL, etc. with information pertaining to whichJournal Table to write to and relevant schema information. This Processor manages the schema versions for each table being processedin order to ensure that the correct Journal Table is used for each FlowFile.
INPUT REQUIREMENT¶
REQUIRED
Supports Sensitive Dynamic Properties¶
false
PROPERTIES¶
Property |
Description |
|---|---|
CDC Schema Registry |
Specifies the CDC Schema Registry to use for managing the schemas of the CDC data |
Record Reader |
Specifies the Record Reader to use for reading the incoming data |
Record Writer |
Specifies the Record Writer to use for writing the outgoing data |
STATE MANAGEMENT¶
SCOPES |
DESCRIPTION |
|---|---|
CLUSTER |
Tracks the current journal table version for each table being processed. |
RELATIONSHIPS¶
NAME |
DESCRIPTION |
|---|---|
schema update |
If any schema update is required in order to handle incoming Records, a FlowFile is routed to this relationship. The FlowFile will include the schema information to indicate what changes are required. |
failure |
If any FlowFile is unable to be read, it will be routed to this Relationship. |
success |
Rows to be inserted into the Snowflake table will be routed to this Relationship. |
skipped ddl event |
This Relationship will be used for any DDL / Schema Change events that do not result in a change to the destination table’s schema. |
WRITES ATTRIBUTES¶
NAME |
DESCRIPTION |
|---|---|
table.schema.generation |
The index of the journal table for incremental processing. |
destination.table.schema |
The updated schema for the destination table. This attribute is only written for DDL events. |