Registers a file in stage or a local file as a dependency of a user-defined function (UDF).

The local file can be a JAR file, a directory, or any other file resource. If you pass the path to a local file to addDependency , the Snowpark library uploads the file to a temporary stage and imports the file when executing a UDF.

If you pass the path to a file in a stage to addDependency , the file is included in the imports when executing a UDF.

Note that in most cases, you don't need to add the Snowpark JAR file and the JAR file (or directory) of the currently running application as dependencies. The Snowpark library automatically attempts to detect and upload these JAR files. However, if this automatic detection fails, the Snowpark library reports this in an error message, and you must add these JAR files explicitly by calling addDependency .

The following example demonstrates how to add dependencies on local files and files in a stage:

session.addDependency("@my_stage/http-commons.jar")
session.addDependency("/home/username/lib/language-detector.jar")
session.addDependency("./resource-dir/")
session.addDependency("./resource.xml")

Cancel all action methods that are running currently. This does not affect on any action methods called in the future.

Close this session.

Returns an AsyncJob object that you can use to track the status and get the results of the asynchronous query specified by the query ID.

For example, create an AsyncJob by specifying a valid <query_id> , check whether the query is running or not, and get the result rows.

val asyncJob = session.createAsyncJob(<query_id>)
println(s"Is query ${asyncJob.getQueryId} running? ${asyncJob.isRunning()}")
val rows = asyncJob.getRows()

Creates a new DataFrame that uses the specified schema and contains the specified Row objects.

For example, the following code creates a DataFrame containing two columns of the types int and string with two rows of data:

For example

import com.snowflake.snowpark.types._
...
// Create an array of Row objects containing data.
val data = Array(Row(1, "a"), Row(2, "b"))
// Define the schema for the columns in the DataFrame.
val schema = StructType(Seq(StructField("num", IntegerType),
  StructField("str", StringType)))
// Create the DataFrame.
val df = session.createDataFrame(data, schema)

Creates a new DataFrame that uses the specified schema and contains the specified Row objects.

For example, the following code creates a DataFrame containing three columns of the types int , string , and variant with a single row of data:

import com.snowflake.snowpark.types._
...
// Create a sequence of a single Row object containing data.
val data = Seq(Row(1, "a", new Variant(1)))
// Define the schema for the columns in the DataFrame.
val schema = StructType(Seq(StructField("int", IntegerType),
  StructField("string", StringType),
  StructField("variant", VariantType)))
// Create the DataFrame.
val df = session.createDataFrame(data, schema)

Creates a new DataFrame containing the specified values. Currently, you can use values of the following types:

• Base types (Int, Short, String etc.). The resulting DataFrame has the column name "VALUE".
• Tuples consisting of base types. The resulting DataFrame has the column names "_1", "_2", etc.
• Case classes consisting of base types. The resulting DataFrame has column names that correspond to the case class constituents.

If you want to create a DataFrame by calling the toDF method of a Seq object, import session.implicits._ , where session is an object of the Session class that you created to connect to the Snowflake database. For example:

val session = Session.builder.configFile(..).create
// Importing this allows you to call the toDF method on a Seq object.
import session.implicits._
// Create a DataFrame from a Seq object.
val df = Seq((1, "x"), (2, "y"), (3, "z")).toDF("numCol", "varcharCol")
df.show()

Returns a FileOperation object that you can use to perform file operations on stages. For example:

session.file.put("file:///tmp/file1.csv", "@myStage/prefix1")
session.file.get("@myStage/prefix1", "file:///tmp")

Creates a new DataFrame by flattening compound values into multiple rows.

for example:

import com.snowflake.snowpark.functions._
val df = session.flatten(parse_json(lit("""{"a":[1,2]}""")), "a", false, false, "BOTH")

Creates a new DataFrame by flattening compound values into multiple rows.

For example:

import com.snowflake.snowpark.functions._
val df = session.flatten(parse_json(lit("""{"a":[1,2]}""")))

Creates a new DataFrame via Generator function.

For example:

import com.snowflake.snowpark.functions._
session.generator(10, seq4(), uniform(lit(1), lit(5), random())).show()

Creates a new DataFrame via Generator function.

For example:

import com.snowflake.snowpark.functions._
session.generator(10, Seq(seq4(), uniform(lit(1), lit(5), random()))).show()

Returns the name of the current database for the JDBC session attached to this session.

For example, if you change the current database by executing the following code:

session.sql("use database newDB").collect()

the method returns newDB .

Returns the name of the current schema for the JDBC session attached to this session.

For example, if you change the current schema by executing the following code:

session.sql("use schema newSchema").collect()

the method returns newSchema .

Returns the name of the default database configured for this session in Session.builder .

Returns the name of the default schema configured for this session in Session.builder .

Returns the list of URLs for all the dependencies that were added for user-defined functions (UDFs). This list includes any JAR files that were added automatically by the library.

Returns a Java Set of URLs for all the dependencies that were added for user-defined functions (UDFs). This list includes any JAR files that were added automatically by the library.

Returns the fully qualified name of the current schema for the session.

Returns the query tag that you set by calling setQueryTag .

Get the session information.

Returns the name of the temporary stage created by the Snowpark library for uploading and store temporary artifacts for this session. These artifacts include classes for UDFs that you define in this session and dependencies that you add when calling addDependency .

Returns the JDBC Connection object used for the connection to the Snowflake database.

Creates a new DataFrame from a range of numbers. The resulting DataFrame has the column name "ID" and a row for each number in the sequence.

Creates a new DataFrame from a range of numbers starting from 0. The resulting DataFrame has the column name "ID" and a row for each number in the sequence.

Creates a new DataFrame from a range of numbers. The resulting DataFrame has the column name "ID" and a row for each number in the sequence.

Returns a DataFrameReader that you can use to read data from various supported sources (e.g. a file in a stage) as a DataFrame.

Removes a path from the set of dependencies.

Sets a query tag for this session. You can use the query tag to find all queries run for this session.

If not set, the default value of query tag is the Snowpark library call and the class and method in your code that invoked the query (e.g. com.snowflake.snowpark.DataFrame.collect Main$.main(Main.scala:18) ).

Returns a SProcRegistration object that you can use to register Stored Procedures. For example:

val sp = session.sproc.registerTemporary((session: Session, num: Int) => num * 2)
session.storedProcedure(sp, 100).show()

Returns a new DataFrame representing the results of a SQL query.

You can use this method to execute an arbitrary SQL statement.

Returns a new DataFrame representing the results of a SQL query.

You can use this method to execute an arbitrary SQL statement.

Creates a new DataFrame from the given Stored Procedure and arguments.

val sp = session.sproc.register(...)
session.storedProcedure(
  sp, "arg1", "arg2"
).show()

Creates a new DataFrame from the given Stored Procedure and arguments.

session.storedProcedure(
  "sp_name", "arg1", "arg2"
).show()

Returns an Updatable that points to the specified table.

Returns an Updatable that points to the specified table.

Returns an Updatable that points to the specified table.

Returns an Updatable that points to the specified table.

name can be a fully qualified identifier and must conform to the rules for a Snowflake identifier.

Creates a new DataFrame from the given table function.

Example

import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

session.tableFunction(
  flatten(parse_json(lit("[1,2]")))
)

Creates a new DataFrame from the given table function and arguments.

Example

import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

session.tableFunction(
  flatten,
  Map("input" -> parse_json(lit("[1,2]")))
)
// Since 1.8.0, DataFrame columns are accepted as table function arguments:
df = Seq("[1,2]").toDF("a")
session.tableFunction((
  flatten,
  Map("input" -> parse_json(df("a")))
)

Creates a new DataFrame from the given table function and arguments.

Example

import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

session.tableFunction(
  split_to_table,
  Seq(lit("split by space"), lit(" "))
)
// Since 1.8.0, DataFrame columns are accepted as table function arguments:
df = Seq(Seq("split by space", " ")).toDF(Seq("a", "b"))
session.tableFunction((
  split_to_table,
  Seq(df("a"), df("b"))
)

Creates a new DataFrame from the given table function and arguments.

Example

import com.snowflake.snowpark.functions._
import com.snowflake.snowpark.tableFunctions._

session.tableFunction(
  split_to_table,
  lit("split by space"),
  lit(" ")
)

Returns a UDFRegistration object that you can use to register UDFs. For example:

session.udf.registerTemporary("mydoubleudf", (x: Int) => 2 * x)
session.sql(s"SELECT mydoubleudf(c) FROM table")

Returns a UDTFRegistration object that you can use to register UDTFs. For example:

class MyWordSplitter extends UDTF1[String] {
  override def process(input: String): Iterable[Row] = input.split(" ").map(Row(_))
  override def endPartition(): Iterable[Row] = Array.empty[Row]
  override def outputSchema(): StructType = StructType(StructField("word", StringType))
}
val tableFunction = session.udtf.registerTemporary(new MyWordSplitter)
session.tableFunction(tableFunction, lit("My name is Snow Park")).show()

Unset query_tag parameter for this session.

If not set, the default value of query tag is the Snowpark library call and the class and method in your code that invoked the query (e.g. com.snowflake.snowpark.DataFrame.collect Main$.main(Main.scala:18) ).

Updates the query tag that is a JSON encoded string for the current session.

Keep in mind that assigning a value via setQueryTag will remove any current query tag state.

Example 1:

session.setQueryTag("""{"key1":"value1"}""")
session.updateQueryTag("""{"key2":"value2"}""")
print(session.getQueryTag().get)
{"key1":"value1","key2":"value2"}

Example 2:

session.sql("""ALTER SESSION SET QUERY_TAG = '{"key1":"value1"}'""").collect()
session.updateQueryTag("""{"key2":"value2"}""")
print(session.getQueryTag().get)
{"key1":"value1","key2":"value2"}

Example 3:

session.setQueryTag("")
session.updateQueryTag("""{"key1":"value1"}""")
print(session.getQueryTag().get)
{"key1":"value1"}

Provides implicit methods for convert Scala objects to Snowpark DataFrame and Column objects.

To use this, import session.implicits._ :

val session = Session.builder.configFile(..).create
import session.implicits._

After you import this, you can call the toDF method of a Seq to convert a sequence to a DataFrame:

// Create a DataFrame from a local sequence of integers.
val df = (1 to 10).toDF("a")
val df = Seq((1, "one"), (2, "two")).toDF("a", "b")

You can also refer to columns in DataFrames by using $"colName" and 'colName :

// Refer to a column in a DataFrame by using $"colName".
val df = session.table("T").filter($"a" > 1)
// Refer to columns by using 'colName.
val df = session.table("T").filter('a > 1)