Snowpark Connect for Spark release notes for 2026¶

Fix accessing struct field from array via getItem

Fix names for accessing array elements

Added missing compression for TEXT format

Reduce query size in DataFrame.replace, UDTF creation, and read_parquet

Emulate types on create [temp] view

Fixed casting structured types to

Fix text write type validation

Support XML read dir in parallel

Optimize conv function usage

Support both Snowflake and net.snowflake.spark.snowflake format read and write

Emulate types on create table

Fix accessing nested structs with arrays

Fix Parquet error message

Optimize to_number reducing query size

Fix UDF cache to consider query database change

Optimize mask function

Pass PATTERN to NVS fallback reader during Parquet schema inference

Null and structured type coercion

Introduce DIRECTED join hint

Integrate XML inferSchema

Added missing JDBC Type mapping

Support user provided schema in parquet

Handle invalid UTF-8 characters in JSON gracefully

Resolve LCA columns only if actually used

Optimize get_json_object query generation

Strip semicolon from SQL query

Make processInBulk=True the default for JSON reads and fix NullType schema inference

Fix bug regarding incorrect stage read

Add non check in udf registration

Tighten limit for error message

Allow missing fields in user provided schema

JSON and CSV compression inference

Fix for coalesce(1) creating a single file

Add execute_jar method to launch Java/Scala workloads

Fix error swallowing with --wait-for-completion flag

JSON and CSV compression inference.

Fix for coalesce creating a single file.

Refactor JSON read to use COPY INTO for single-file reads and add VariantType schema inference.

Allow JSON loading without explicit schema.

Fix multi_line in JSON.

Fix JSON infer schema to avoid scanning whole files.

Correctly handle casting to timestamp ltz.

Clamp hash returned value.

Fix for repartition with partitionBy.

Fix to use [connections.spark-connect] section header in config.toml.

Convert Java date/timestamp format tokens to Snowflake equivalents for CSV reads.

Calculate schema for pivot functions.

Fix UDTFs in aliased lateral join.

Align result for SQL SET command.

Fix return type for CEIL and FLOOR functions.

Improve query generation in unbase64 v2.

Fix some of option to Snowflake mapping for CSV.

Fix serialization for POJO.

Improve CSV header error messages.

Improve mapType detection logic with try_cast for Parquet reads.

Support for reduceGroups API.

Support specifying connection name inside init_spark_session.

Add config param to use UDF for unbase64.

Optimize SQL generation in function unbase64.

Fix from_json regression

Fix for records that span multiple BZ2 compression block boundaries

Fix nullability mapping in unresolved attribute

Initialize spark-connect session with any connection, not just one named spark-connect

Add XML options validation

Drop CSV ESCAPE option when it matches the quote character to prevent compilation error

Fix incorrect conversion of named tuples in productEncoder

Verify mergeSchema for CSV and JSON is not supported

Fix Parquet complex type round-trip (write + read)

Fix schema for pivot/unpivot

Fix return type for MOD and PMOD functions

Fix CSV header extraction for files with leading blank lines

Test timezones correctly and replace string-based date/time serialization with epoch-based

Update Java version check for Windows

Flatten nested withColumn calls

Change logic for Literal _IntegralType in add/sub operations

Return LongType for COUNT functions

Read JSON: test compression = bz2/bzip2/none

Improve performance of to_varchar/to_char

Make better comparison in I/O testing

Set multi_line to False by default for copy JSON

Throw error on unspecified compute pool.

Remove result scan when calling df.count()

Make sure infer schema runs on limited rows for reading JSON

Fix createDataFrame for interval types

Change logic for Literal _IntegralType in multiplication and division operations

Widen and coerce type for Set operations

Fix neo4j multi label support

Modify JAR metadata so that Grype does not detect Netty vulnerability

Return correct type for ANY_VALUE function

Return widened type for sequence

Add support for config spark.sql.parquet.inferTimestampNTZ.enabled

Batch column rename/cast in _validate_schema_and_get_writer

JDBC hang when partitioned queries given with fetch size

Return trimmed exception message when it exceeds the HTTP header limits

Fix map_type_to_snowflake_type for BigDecimal

Fix literal decimal precision and scale

Improve random string generation

Make BZ2 compressed JSON loading ignore corrupt records

Use staged files from config in Scala UDFs

Use permissive TRY_CAST in JSON reading

Make the number of server threads configurable

Adding back init_spark_session() to testing

Update snowpark-submit command line output to clarify snowflake-connection-name is required.

Cache table type when running saveAsTable

Optimize literal input for substring and type casting for coalesce

Handle decimal overflow in avg/mean and fix decimal type coercion

Iceberg - Preserve grants on overwrite

Standardize SQL passthrough mode

Optimize from_utc_timestamp/to_utc_timestamp for literal timezone

Handle JSON null values in structured types to match Spark semantics

Emulate integral types on creating tables from SQL

Fix edge case with mapping nested rows in Scala UDFs

Fix how Parquet handles read and write of complex structured datatypes

Support save ignore argument for parquet files

Add support for artifact repository

Fix array nullability in Scala UDxF

Fix log1p for args from (-1, 0) range

Fix first_value and last_value in aggregate context

Fix reading DayTimeIntervalType for Scala client

Handle timezones correctly in Scala UDFs

Support Java 11 and 17 without any configuration

Support snowpark-submit for python 3.9

Enhance init_spark_session to be usable in snowpark-submit workflow

Fixed split function issue

Downgraded snowflake-snowpark-python dependency to version 1.44

Fixed Neo4j dialect matching to improve SQL translation

Fixed operation ID returned in execute responses to be consistent

Fixed gRPC metadata handling for TCP channel connections

Added support for partition_hint in mapPartitions operations

Added XML reader support for scenarios with user-defined schemas

Preserve hidden columns after various DataFrame operators

Fix issues for scala udf input types (byte, binary, scala.math.BigDecimal)

Add snowpark-submit User Defined Args to comment

Fix config unset error for session configuration.

Use copy into to load CSV files in parallel.

Fix writes for DataFrames using outer joins.

Handle nulls in Scala UDFs.

Optimize CTE query generation with parameter protection.

Avoid casting arguments of DATEDIFF.

Fix appending partitioned files and reading of null partitions.

Make a 10X performance improvement for conversion between base 10 and 16 using SQL.

Overwrite only modified partitions for parquet files.

Updated logic to detect if Snowpark Connect for Spark is running on XP.

Support writing to a table with variant data type in Snowflake.

Remove unnecessary info logs.

Move Java tests out of Scala tests job to a separate job.

Update the dependency version for gcsfs.

Fix serializing Scala tuples.

Fix loading huge JSON files.

Implement small fixes for customer issues.

Implement fixes for struct comparisons.

Add handling for 0-column DataFrames.

Correct upload file path.

Fix Upload_files_if_needed not running in parallel.

Improve input type inference when UDF input types are not defined in the proto.

Fix NA edge cases.

Support reading single JSON BZ2 file.

Support Scala UDFs in server-side Snowpark Connect for Spark.

Implement cast between string and daytime.

Add support for Scala UDFs in group_map.

Reduce generated workload names.

Fixed JAVA_HOME handling for Windows.

Support neo4j data source via JDBC.