Snowflake Connector for Kafka release notes for 2024¶

With this release, the Snowflake Connector for Kafka can ingest data into a Snowflake-managed Apache Iceberg™ table. For more information, see Using the Snowflake Connector for Kafka with Apache Iceberg™ tables.

Fixed dependency vulnerabilities.

Upgraded the Snowflake Ingest Java SDK to version 2.3.

Closing channels in parallel is now enabled by default. This improves the speed of restarting the connector.

Fixed logging issues.

Upgraded the Snowflake Ingest Java SDK to version 2.2.2.

Fixed issues with schematization.

Upgraded the Snowflake Ingest Java SDK to version 2.2.0, which contains a critical fix for potential issues when change_tracking is enabled for streams and dynamic tables.

Upgraded the Snowflake JDBC driver from version 3.14.5 to version 3.18.0.

Improved the logging experience in various components for improved troubleshooting experience.

Improved the channel reopening logic.

Updated dependencies with known vulnerabilities.

Added support to close Snowpipe Streaming channels in parallel, which significantly reduces time for rebalancing.

Added a new SnowflakeConnectorPushTime property in the metadata that represents the time when the message was pushed by the connector.

Updated dependencies with known vulnerabilities.

Fixed an issue where the staged files are not cleaned up properly.

Added offset verification logic to make sure there is no missing or duplicate data.

Added client provider overridden map for Snowpipe Streaming. The map uses comma-separated key value pairs as input.

Upgraded to the following versions:

• JDBC version to 3.14.5.

• kafka connect-api version to 3.7.0.

• jackson-core and jackson-databind to 2.16.1

• commons-compress to 1.26.0

Cleaned up streaming ingest threads when SinkTask stop () is called.

Preserved the old data type that goes into an ARRAY column for schematization.

Added support for AVRO logical types.

Implemented changes to prevent potential data duplication because of a new channel name format.

Implemented changes to preserve the old data type that goes into an ARRAY column for schematization.

Implemented changes to make schema evolution add columns idempotent.

Enabled the Ingest SDK MAX_CLIENT_LAG configuration in Kafka connector.

Fixed schema evolution cases that could cause non-exactly once delivery.

Fixed issues with generating and building Java library.

Fixed an issue that the Kafka offset is not reset correctly.