object tableFunctions

Provides utility functions that generate table function expressions that can be passed to DataFrame join method and Session tableFunction method.

This object also provides functions that correspond to Snowflake system-defined table functions .

The following examples demonstrate the use of some of these functions: 

import com.snowflake.snowpark.functions.parse_json

// Creates DataFrame from Session.tableFunction
session.tableFunction(tableFunctions.flatten, Map("input" -> parse_json(lit("[1,2]"))))
session.tableFunction(tableFunctions.split_to_table, "split by space", " ")

// DataFrame joins table function
df.join(tableFunctions.flatten, Map("input" -> parse_json(df("a"))))
df.join(tableFunctions.split_to_table, df("a"), ",")

// Invokes any table function including user-defined table function
 df.join(tableFunctions.tableFunction("flatten"), Map("input" -> parse_json(df("a"))))
 session.tableFunction(tableFunctions.tableFunction("split_to_table"), "split by space", " ")
Since

0.4.0

Linear Supertypes
AnyRef , Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. tableFunctions
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def != ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  2. final def ## () : Int
    Definition Classes
    AnyRef → Any
  3. final def == ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf [ T0 ] : T0
    Definition Classes
    Any
  5. def clone () : AnyRef
    Attributes
    protected[ lang ]
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... ) @native () @HotSpotIntrinsicCandidate ()
  6. final def eq ( arg0: AnyRef ) : Boolean
    Definition Classes
    AnyRef
  7. def equals ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  8. def explode ( input: Column ) : Column

    Flattens a given array or map type column into individual rows.

    Flattens a given array or map type column into individual rows. The output column(s) in case of array input column is VALUE , and are KEY and VALUE in case of amp input column.

    Example 

    import com.snowflake.snowpark.functions._

val df = Seq("""{"a":1, "b": 2}""").toDF("a")
val df1 = df.select(
  parse_json(df("a"))
  .cast(types.MapType(types.StringType, types.IntegerType))
  .as("a"))
df1.select(lit(1), tableFunctions.explode(df1("a")), df1("a")("a")).show()
    input

    The expression that will be unseated into rows. The expression must be either MapType or ArrayType data.

    returns

    The result Column reference

    Since

    1.10.0

  9. def flatten ( input: Column , path: String , outer: Boolean , recursive: Boolean , mode: String ) : Column

    Flattens (explodes) compound values into multiple rows.

    Flattens (explodes) compound values into multiple rows.

    Example 

    import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

df.join(
  tableFunctions.flatten(parse_json(df("a")), "path", true, true, "both")
)
    input

    The expression that will be unseated into rows. The expression must be of data type VariantType, MapType or ArrayType.

    path

    The path to the element within a VariantType data structure which needs to be flattened. Can be a zero-length string (i.e. empty path) if the outermost element is to be flattened.

    outer

    Optional boolean value. If FALSE, any input rows that cannot be expanded, either because they cannot be accessed in the path or because they have zero fields or entries, are completely omitted from the output. If TRUE, exactly one row is generated for zero-row expansions (with NULL in the KEY, INDEX, and VALUE columns).

    recursive

    If FALSE, only the element referenced by PATH is expanded. If TRUE, the expansion is performed for all sub-elements recursively.

    mode

    ("object", "array", or "both") Specifies whether only objects, arrays, or both should be flattened.

    returns

    The result Column reference

    Since

    1.10.0

  10. def flatten ( input: Column ) : Column

    Flattens (explodes) compound values into multiple rows.

    Flattens (explodes) compound values into multiple rows.

    Example 

    import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

df.join(
  tableFunctions.flatten(parse_json(df("a")))
)
    input

    The expression that will be unseated into rows. The expression must be of data type VariantType, MapType or ArrayType.

    returns

    The result Column reference

    Since

    1.10.0

  11. lazy val flatten : TableFunction

    Flattens (explodes) compound values into multiple rows.

    Flattens (explodes) compound values into multiple rows.

    Argument List:

    input: Required. The expression that will be unseated into rows. The expression must be of data type VariantType, MapType or ArrayType.

    path: Optional. The path to the element within a VariantType data structure which needs to be flattened. Can be a zero-length string (i.e. empty path) if the outermost element is to be flattened. Default: Zero-length string (i.e. empty path)

    outer: Optional boolean value. If FALSE, any input rows that cannot be expanded, either because they cannot be accessed in the path or because they have zero fields or entries, are completely omitted from the output. If TRUE, exactly one row is generated for zero-row expansions (with NULL in the KEY, INDEX, and VALUE columns). Default: FALSE

    recursive: Optional boolean value If FALSE, only the element referenced by PATH is expanded. If TRUE, the expansion is performed for all sub-elements recursively. Default: FALSE

    mode: Optional String ("object", "array", or "both") Specifies whether only objects, arrays, or both should be flattened. Default: both

    Example 

    import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

df.join(
  tableFunctions.flatten,
  Map("input" -> parse_json(df("a"), "outer" -> lit(true)))
)

session.tableFunction(
  tableFunctions.flatten,
  Map("input" -> parse_json(lit("[1,2]"), "mode" -> lit("array")))
)
    Since

    0.4.0

  12. final def getClass () : Class [_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  13. def hashCode () : Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  14. final def isInstanceOf [ T0 ] : Boolean
    Definition Classes
    Any
  15. final def ne ( arg0: AnyRef ) : Boolean
    Definition Classes
    AnyRef
  16. final def notify () : Unit
    Definition Classes
    AnyRef
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  17. final def notifyAll () : Unit
    Definition Classes
    AnyRef
    Annotations
    @native () @HotSpotIntrinsicCandidate ()
  18. def split_to_table ( str: Column , delimiter: String ) : Column

    This table function splits a string (based on a specified delimiter) and flattens the results into rows.

    This table function splits a string (based on a specified delimiter) and flattens the results into rows.

    Example 

    import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

df.join(tableFunctions.split_to_table(df("a"), lit(",")))
    str

    Text to be split.

    delimiter

    Text to split string by.

    returns

    The result Column reference

    Since

    1.10.0

  19. lazy val split_to_table : TableFunction

    This table function splits a string (based on a specified delimiter) and flattens the results into rows.

    This table function splits a string (based on a specified delimiter) and flattens the results into rows.

    Argument List:

    First argument (no name): Required. Text to be split.

    Second argument (no name): Required. Text to split string by.

    Example 

    import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

df.join(tableFunctions.split_to_table, df("a"), lit(","))
session.tableFunction(
  tableFunctions.split_to_table,
  lit("split by space"),
  lit(" ")
)
    Since

    0.4.0

  20. final def synchronized [ T0 ] ( arg0: ⇒ T0 ) : T0
    Definition Classes
    AnyRef
  21. def toString () : String
    Definition Classes
    AnyRef → Any
  22. final def wait ( arg0: Long , arg1: Int ) : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... )
  23. final def wait ( arg0: Long ) : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... ) @native ()
  24. final def wait () : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... )

Deprecated Value Members

  1. def finalize () : Unit
    Attributes
    protected[ lang ]
    Definition Classes
    AnyRef
    Annotations
    @throws ( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from AnyRef

Inherited from Any

Ungrouped