You are viewing documentation about an older version (1.3.0). View latest version

snowflake.snowpark.functions.udtf

snowflake.snowpark.functions.udtf(handler: Callable | None = None, *, output_schema: StructType | List[str], input_types: List[DataType] | None = None, name: str | Iterable[str] | None = None, is_permanent: bool = False, stage_location: str | None = None, imports: List[str | Tuple[str, str]] | None = None, packages: List[str | module] | None = None, replace: bool = False, if_not_exists: bool = False, session: Session | None = None, parallel: int = 4, statement_params: Dict[str, str] | None = None, strict: bool = False, secure: bool = False) → UserDefinedTableFunction | partial[source]

Registers a Python class as a Snowflake Python UDTF and returns the UDTF.

It can be used as either a function call or a decorator. In most cases you work with a single session. This function uses that session to register the UDTF. If you have multiple sessions, you need to explicitly specify the session parameter of this function. If you have a function and would like to register it to multiple databases, use session.udtf.register instead. See examples in UDTFRegistration.

Parameters:

  • handler – A Python class used for creating the UDTF.

  • output_schema – A list of column names, or a StructType instance that represents the table function’s columns. If a list of column names is provided, the process method of the handler class must have return type hints to indicate the output schema data types.

  • input_types – A list of DataType representing the input data types of the UDTF. Optional if type hints are provided.

  • name – A string or list of strings that specify the name or fully-qualified object identifier (database name, schema name, and function name) for the UDTF in Snowflake, which allows you to call this UDTF in a SQL command or via call_udtf(). If it is not provided, a name will be automatically generated for the UDTF. A name must be specified when is_permanent is True.

  • is_permanent – Whether to create a permanent UDTF. The default is False. If it is True, a valid stage_location must be provided.

  • stage_location – The stage location where the Python file for the UDTF and its dependencies should be uploaded. The stage location must be specified when is_permanent is True, and it will be ignored when is_permanent is False. It can be any stage other than temporary stages and external stages.

  • imports – A list of imports that only apply to this UDTF. You can use a string to represent a file path (similar to the path argument in add_import()) in this list, or a tuple of two strings to represent a file path and an import path (similar to the import_path argument in add_import()). These UDTF-level imports will override the session-level imports added by add_import().

  • packages – A list of packages that only apply to this UDTF. These UDTF-level packages will override the session-level packages added by add_packages() and add_requirements().

  • replace – Whether to replace a UDTF that already was registered. The default is False. If it is False, attempting to register a UDTF with a name that already exists results in a SnowparkSQLException exception being thrown. If it is True, an existing UDTF with the same name is overwritten.

  • if_not_exists – Whether to skip creation of a UDTF when one with the same signature already exists. The default is False. if_not_exists and replace are mutually exclusive and a ValueError is raised when both are set. If it is True and a UDTF with the same signature exists, the UDTF creation is skipped.

  • session – Use this session to register the UDTF. If it’s not specified, the session that you created before calling this function will be used. You need to specify this parameter if you have created multiple sessions before calling this method.

  • parallel – The number of threads to use for uploading UDTF files with the PUT command. The default value is 4 and supported values are from 1 to 99. Increasing the number of threads can improve performance when uploading large UDTF files.

  • statement_params – Dictionary of statement level parameters to be set while executing this action.

  • strict – Whether the created UDTF is strict. A strict UDTF will not invoke the UDTF if any input is null. Instead, a null value will always be returned for that row. Note that the UDTF might still return null for non-null inputs.

  • secure – Whether the created UDTF is secure. For more information about secure functions, see Secure UDFs.

Returns:

A UDTF function that can be called with Column expressions.

Note

1. When type hints are provided and are complete for a function, return_type and input_types are optional and will be ignored. See details of supported data types for UDTFs in UDTFRegistration.

  • You can use use Variant to annotate a variant, and use Geography to annotate a geography when defining a UDTF.

  • typing.Union is not a valid type annotation for UDTFs, but typing.Optional can be used to indicate the optional type.

  • Type hints are not supported on functions decorated with decorators.

2. A temporary UDTF (when is_permanent is False) is scoped to this session and all UDTF related files will be uploaded to a temporary session stage (session.get_session_stage()). For a permanent UDTF, these files will be uploaded to the stage that you specify.

3. By default, UDTF registration fails if a function with the same name is already registered. Invoking udtf() with replace set to True will overwrite the previously registered function.

See also

UDTFRegistration

Example:

>>> from snowflake.snowpark.types import IntegerType, StructField, StructType
>>> class PrimeSieve:
...     def process(self, n):
...         is_prime = [True] * (n + 1)
...         is_prime[0] = False
...         is_prime[1] = False
...         p = 2
...         while p * p <= n:
...             if is_prime[p]:
...                 # set all multiples of p to False
...                 for i in range(p * p, n + 1, p):
...                     is_prime[i] = False
...             p += 1
...         # yield all prime numbers
...         for p in range(2, n + 1):
...             if is_prime[p]:
...                 yield (p,)
>>> prime_udtf = udtf(PrimeSieve, output_schema=StructType([StructField("number", IntegerType())]), input_types=[IntegerType()])
>>> session.table_function(prime_udtf(lit(20))).collect()
[Row(NUMBER=2), Row(NUMBER=3), Row(NUMBER=5), Row(NUMBER=7), Row(NUMBER=11), Row(NUMBER=13), Row(NUMBER=17), Row(NUMBER=19)]

Instead of calling `udtf` it is also possible to use udtf as a decorator.
Copy

Example:

>>> @udtf(name="alt_int",replace=True, output_schema=StructType([StructField("number", IntegerType())]), input_types=[IntegerType()])
... class Alternator:
...     def __init__(self):
...         self._positive = True
...
...     def process(self, n):
...         for i in range(n):
...             if self._positive:
...                 yield (1,)
...             else:
...                 yield (-1,)
...             self._positive = not self._positive
>>> session.table_function("alt_int", lit(3)).collect()
[Row(NUMBER=1), Row(NUMBER=-1), Row(NUMBER=1)]
>>> session.table_function("alt_int", lit(2)).collect()
[Row(NUMBER=1), Row(NUMBER=-1)]
>>> session.table_function("alt_int", lit(1)).collect()
[Row(NUMBER=1)]
Copy
Sprache: Deutsch