class Session extends Logging

Establishes a connection with a Snowflake database and provides methods for creating DataFrames and accessing objects for working with files in stages.

When you create a Session object, you provide configuration settings to establish a connection with a Snowflake database (e.g. the URL for the account, a user name, etc.). You can specify these settings in a configuration file or in a Map that associates configuration setting names with values.

To create a Session from a file:

val session = Session.builder.configFile("/path/to/file.properties").create

To create a Session from a map of configuration properties:

val configMap = Map(
"URL" -> "demo.snowflakecomputing.com",
"USER" -> "testUser",
"PASSWORD" -> "******",
"ROLE" -> "myrole",
"WAREHOUSE" -> "warehouse1",
"DB" -> "db1",
"SCHEMA" -> "schema1"
)
Session.builder.configs(configMap).create

Session contains functions to construct DataFrame s like Session.table , Session.sql , and Session.read

Since

0.1.0

Linear Supertypes
Logging , AnyRef , Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Session
  2. Logging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def != ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  2. final def ## () : Int
    Definition Classes
    AnyRef → Any
  3. final def == ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  4. def addDependency ( path: String ) : Unit

    Registers a file in stage or a local file as a dependency of a user-defined function (UDF).

    Registers a file in stage or a local file as a dependency of a user-defined function (UDF).

    The local file can be a JAR file, a directory, or any other file resource. If you pass the path to a local file to addDependency , the Snowpark library uploads the file to a temporary stage and imports the file when executing a UDF.

    If you pass the path to a file in a stage to addDependency , the file is included in the imports when executing a UDF.

    Note that in most cases, you don't need to add the Snowpark JAR file and the JAR file (or directory) of the currently running application as dependencies. The Snowpark library automatically attempts to detect and upload these JAR files. However, if this automatic detection fails, the Snowpark library reports this in an error message, and you must add these JAR files explicitly by calling addDependency .

    The following example demonstrates how to add dependencies on local files and files in a stage:

    session.addDependency("@my_stage/http-commons.jar")
    session.addDependency("/home/username/lib/language-detector.jar")
    session.addDependency("./resource-dir/")
    session.addDependency("./resource.xml")
    path

    Path to a local directory, local file, or file in a stage.

    Since

    0.1.0

  5. final def asInstanceOf [ T0 ] : T0
    Definition Classes
    Any
  6. def cancelAll () : Unit

    Cancel all action methods that are running currently.

    Cancel all action methods that are running currently. This does not affect on any action methods called in the future.

    Since

    0.5.0

  7. def clone () : AnyRef
    Attributes
    protected[ lang ]
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... ) @native ()
  8. def close () : Unit

    Close this session.

    Close this session.

    Since

    0.7.0

  9. def createAsyncJob ( queryID: String ) : AsyncJob

    Returns an AsyncJob object that you can use to track the status and get the results of the asynchronous query specified by the query ID.

    Returns an AsyncJob object that you can use to track the status and get the results of the asynchronous query specified by the query ID.

    For example, create an AsyncJob by specifying a valid <query_id> , check whether the query is running or not, and get the result rows.

    val asyncJob = session.createAsyncJob(<query_id>)
    println(s"Is query ${asyncJob.getQueryId} running? ${asyncJob.isRunning()}")
    val rows = asyncJob.getRows()
    queryID

    A valid query ID

    returns

    An AsyncJob object

    Since

    0.11.0

  10. def createDataFrame ( data: Array [ Row ] , schema: StructType ) : DataFrame

    Creates a new DataFrame that uses the specified schema and contains the specified Row objects.

    Creates a new DataFrame that uses the specified schema and contains the specified Row objects.

    For example, the following code creates a DataFrame containing two columns of the types int and string with two rows of data:

    For example

    import com.snowflake.snowpark.types._
    ...
    // Create an array of Row objects containing data.
    val data = Array(Row(1, "a"), Row(2, "b"))
    // Define the schema for the columns in the DataFrame.
    val schema = StructType(Seq(StructField("num", IntegerType),
      StructField("str", StringType)))
    // Create the DataFrame.
    val df = session.createDataFrame(data, schema)
    data

    An array of Row objects representing rows of data.

    schema

    StructType representing the schema for the DataFrame.

    returns

    A DataFrame

    Since

    0.7.0

  11. def createDataFrame ( data: Seq [ Row ] , schema: StructType ) : DataFrame

    Creates a new DataFrame that uses the specified schema and contains the specified Row objects.

    Creates a new DataFrame that uses the specified schema and contains the specified Row objects.

    For example, the following code creates a DataFrame containing three columns of the types int , string , and variant with a single row of data:

    import com.snowflake.snowpark.types._
    ...
    // Create a sequence of a single Row object containing data.
    val data = Seq(Row(1, "a", new Variant(1)))
    // Define the schema for the columns in the DataFrame.
    val schema = StructType(Seq(StructField("int", IntegerType),
      StructField("string", StringType),
      StructField("variant", VariantType)))
    // Create the DataFrame.
    val df = session.createDataFrame(data, schema)
    data

    A sequence of Row objects representing rows of data.

    schema

    StructType representing the schema for the DataFrame.

    returns

    A DataFrame

    Since

    0.2.0

  12. def createDataFrame [ T ] ( data: Seq [ T ] ) ( implicit arg0: scala.reflect.api.JavaUniverse.TypeTag [ T ] ) : DataFrame

    Creates a new DataFrame containing the specified values.

    Creates a new DataFrame containing the specified values. Currently, you can use values of the following types:

    • Base types (Int, Short, String etc.). The resulting DataFrame has the column name "VALUE".
    • Tuples consisting of base types. The resulting DataFrame has the column names "_1", "_2", etc.
    • Case classes consisting of base types. The resulting DataFrame has column names that correspond to the case class constituents.

    If you want to create a DataFrame by calling the toDF method of a Seq object, import session.implicits._ , where session is an object of the Session class that you created to connect to the Snowflake database. For example:

    val session = Session.builder.configFile(..).create
    // Importing this allows you to call the toDF method on a Seq object.
    import session.implicits._
    // Create a DataFrame from a Seq object.
    val df = Seq((1, "x"), (2, "y"), (3, "z")).toDF("numCol", "varcharCol")
    df.show()
    T

    DataType

    data

    A sequence in which each element represents a row of values in the DataFrame.

    returns

    A DataFrame

    Since

    0.1.0

  13. final def eq ( arg0: AnyRef ) : Boolean
    Definition Classes
    AnyRef
  14. def equals ( arg0: Any ) : Boolean
    Definition Classes
    AnyRef → Any
  15. lazy val file : FileOperation

    Returns a FileOperation object that you can use to perform file operations on stages.

    Returns a FileOperation object that you can use to perform file operations on stages. For example:

    session.file.put("file:///tmp/file1.csv", "@myStage/prefix1")
    session.file.get("@myStage/prefix1", "file:///tmp")
    Since

    0.4.0

  16. def finalize () : Unit
    Attributes
    protected[ lang ]
    Definition Classes
    AnyRef
    Annotations
    @throws ( classOf[java.lang.Throwable] )
  17. def flatten ( input: Column , path: String , outer: Boolean , recursive: Boolean , mode: String ) : DataFrame

    Creates a new DataFrame by flattening compound values into multiple rows.

    Creates a new DataFrame by flattening compound values into multiple rows.

    for example:

    import com.snowflake.snowpark.functions._
    val df = session.flatten(parse_json(lit("""{"a":[1,2]}""")), "a", false, false, "BOTH")
    input

    The expression that will be unseated into rows. The expression must be of data type VARIANT, OBJECT, or ARRAY.

    path

    The path to the element within a VARIANT data structure which needs to be flattened. Can be a zero-length string (i.e. empty path) if the outermost element is to be flattened.

    outer

    If false , any input rows that cannot be expanded, either because they cannot be accessed in the path or because they have zero fields or entries, are completely omitted from the output. Otherwise, exactly one row is generated for zero-row expansions (with NULL in the KEY, INDEX, and VALUE columns).

    recursive

    If false , only the element referenced by PATH is expanded. Otherwise, the expansion is performed for all sub-elements recursively.

    mode

    Specifies which types should be flattened ( "OBJECT" , "ARRAY" , or "BOTH" ).

    Since

    0.2.0

  18. def flatten ( input: Column ) : DataFrame

    Creates a new DataFrame by flattening compound values into multiple rows.

    Creates a new DataFrame by flattening compound values into multiple rows.

    For example:

    import com.snowflake.snowpark.functions._
    val df = session.flatten(parse_json(lit("""{"a":[1,2]}""")))
    input

    The expression that will be unseated into rows. The expression must be of data type VARIANT, OBJECT, or ARRAY.

    returns

    A DataFrame .

    Since

    0.2.0

  19. def generator ( rowCount: Long , col: Column , cols: Column * ) : DataFrame

    Creates a new DataFrame via Generator function.

    Creates a new DataFrame via Generator function.

    For example:

    import com.snowflake.snowpark.functions._
    session.generator(10, seq4(), uniform(lit(1), lit(5), random())).show()
    rowCount

    The row count of the result DataFrame.

    col

    the column of the result DataFrame

    cols

    A list of columns excepts the first column

    returns

    A DataFrame

    Since

    0.11.0

  20. def generator ( rowCount: Long , columns: Seq [ Column ] ) : DataFrame

    Creates a new DataFrame via Generator function.

    Creates a new DataFrame via Generator function.

    For example:

    import com.snowflake.snowpark.functions._
    session.generator(10, Seq(seq4(), uniform(lit(1), lit(5), random()))).show()
    rowCount

    The row count of the result DataFrame.

    columns

    the column list of the result DataFrame

    returns

    A DataFrame

    Since

    0.11.0

  21. final def getClass () : Class [_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native ()
  22. def getCurrentDatabase : Option [ String ]

    Returns the name of the current database for the JDBC session attached to this session.

    Returns the name of the current database for the JDBC session attached to this session.

    For example, if you change the current database by executing the following code:

    session.sql("use database newDB").collect()

    the method returns newDB .

    returns

    The name of the current database for this session.

    Since

    0.1.0

  23. def getCurrentSchema : Option [ String ]

    Returns the name of the current schema for the JDBC session attached to this session.

    Returns the name of the current schema for the JDBC session attached to this session.

    For example, if you change the current schema by executing the following code:

    session.sql("use schema newSchema").collect()

    the method returns newSchema .

    returns

    Current schema in session.

    Since

    0.1.0

  24. def getDefaultDatabase : Option [ String ]

    Returns the name of the default database configured for this session in Session.builder .

    Returns the name of the default database configured for this session in Session.builder .

    returns

    The name of the default database

    Since

    0.1.0

  25. def getDefaultSchema : Option [ String ]

    Returns the name of the default schema configured for this session in Session.builder .

    Returns the name of the default schema configured for this session in Session.builder .

    returns

    The name of the default schema

    Since

    0.1.0

  26. def getDependencies : Set [ URI ]

    Returns the list of URLs for all the dependencies that were added for user-defined functions (UDFs).

    Returns the list of URLs for all the dependencies that were added for user-defined functions (UDFs). This list includes any JAR files that were added automatically by the library.

    returns

    Set[URI]

    Since

    0.1.0

  27. def getDependenciesAsJavaSet : Set [ URI ]

    Returns a Java Set of URLs for all the dependencies that were added for user-defined functions (UDFs).

    Returns a Java Set of URLs for all the dependencies that were added for user-defined functions (UDFs). This list includes any JAR files that were added automatically by the library.

    Since

    0.2.0

  28. def getFullyQualifiedCurrentSchema : String

    Returns the fully qualified name of the current schema for the session.

    Returns the fully qualified name of the current schema for the session.

    returns

    The fully qualified name of the schema

    Since

    0.2.0

  29. def getQueryTag () : Option [ String ]

    Returns the query tag that you set by calling setQueryTag .

    Returns the query tag that you set by calling setQueryTag .

    Since

    0.1.0

  30. def getSessionInfo () : String

    Get the session information.

    Get the session information.

    Since

    0.11.0

  31. def getSessionStage : String

    Returns the name of the temporary stage created by the Snowpark library for uploading and store temporary artifacts for this session.

    Returns the name of the temporary stage created by the Snowpark library for uploading and store temporary artifacts for this session. These artifacts include classes for UDFs that you define in this session and dependencies that you add when calling addDependency .

    returns

    The name of stage.

    Since

    0.1.0

  32. def hashCode () : Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native ()
  33. final def isInstanceOf [ T0 ] : Boolean
    Definition Classes
    Any
  34. def jdbcConnection : Connection

    Returns the JDBC Connection object used for the connection to the Snowflake database.

    Returns the JDBC Connection object used for the connection to the Snowflake database.

    returns

    JDBC Connection object

  35. def log () : Logger
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  36. def logDebug ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  37. def logDebug ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  38. def logError ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  39. def logError ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  40. def logInfo ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  41. def logInfo ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  42. def logTrace ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  43. def logTrace ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  44. def logWarning ( msg: String , throwable: Throwable ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  45. def logWarning ( msg: String ) : Unit
    Attributes
    protected[ internal ]
    Definition Classes
    Logging
  46. final def ne ( arg0: AnyRef ) : Boolean
    Definition Classes
    AnyRef
  47. final def notify () : Unit
    Definition Classes
    AnyRef
    Annotations
    @native ()
  48. final def notifyAll () : Unit
    Definition Classes
    AnyRef
    Annotations
    @native ()
  49. def range ( start: Long , end: Long ) : DataFrame

    Creates a new DataFrame from a range of numbers.

    Creates a new DataFrame from a range of numbers. The resulting DataFrame has the column name "ID" and a row for each number in the sequence.

    start

    Start of the range.

    end

    End of the range.

    returns

    A DataFrame

    Since

    0.1.0

  50. def range ( end: Long ) : DataFrame

    Creates a new DataFrame from a range of numbers starting from 0.

    Creates a new DataFrame from a range of numbers starting from 0. The resulting DataFrame has the column name "ID" and a row for each number in the sequence.

    end

    End of the range.

    returns

    A DataFrame

    Since

    0.1.0

  51. def range ( start: Long , end: Long , step: Long ) : DataFrame

    Creates a new DataFrame from a range of numbers.

    Creates a new DataFrame from a range of numbers. The resulting DataFrame has the column name "ID" and a row for each number in the sequence.

    start

    Start of the range.

    end

    End of the range.

    step

    Step function for producing the numbers in the range.

    returns

    A DataFrame

    Since

    0.1.0

  52. def read : DataFrameReader

    Returns a DataFrameReader that you can use to read data from various supported sources (e.g.

    Returns a DataFrameReader that you can use to read data from various supported sources (e.g. a file in a stage) as a DataFrame.

    returns

    A DataFrameReader

    Since

    0.1.0

  53. def removeDependency ( path: String ) : Unit

    Removes a path from the set of dependencies.

    Removes a path from the set of dependencies.

    path

    Path to a local directory, local file, or file in a stage.

    Since

    0.1.0

  54. def setQueryTag ( queryTag: String ) : Unit

    Sets a query tag for this session.

    Sets a query tag for this session. You can use the query tag to find all queries run for this session.

    If not set, the default value of query tag is the Snowpark library call and the class and method in your code that invoked the query (e.g. com.snowflake.snowpark.DataFrame.collect Main$.main(Main.scala:18) ).

    queryTag

    String to use as the query tag for this session.

    Since

    0.1.0

  55. def sql ( query: String ) : DataFrame

    Returns a new DataFrame representing the results of a SQL query.

    Returns a new DataFrame representing the results of a SQL query.

    You can use this method to execute an arbitrary SQL statement.

    query

    The SQL statement to execute.

    returns

    A DataFrame

    Since

    0.1.0

  56. final def synchronized [ T0 ] ( arg0: ⇒ T0 ) : T0
    Definition Classes
    AnyRef
  57. def table ( multipartIdentifier: Array [ String ] ) : Updatable

    Returns an Updatable that points to the specified table.

    Returns an Updatable that points to the specified table.

    multipartIdentifier

    An array of strings that specify the database name, schema name, and table name.

    Since

    0.7.0

  58. def table ( multipartIdentifier: List [ String ] ) : Updatable

    Returns an Updatable that points to the specified table.

    Returns an Updatable that points to the specified table.

    multipartIdentifier

    A list of strings that specify the database name, schema name, and table name.

    returns

    A Updatable

    Since

    0.2.0

  59. def table ( multipartIdentifier: Seq [ String ] ) : Updatable

    Returns an Updatable that points to the specified table.

    Returns an Updatable that points to the specified table.

    multipartIdentifier

    A sequence of strings that specify the database name, schema name, and table name (e.g. Seq("database_name", "schema_name", "table_name") ).

    returns

    A Updatable

    Since

    0.1.0

  60. def table ( name: String ) : Updatable

    Returns an Updatable that points to the specified table.

    Returns an Updatable that points to the specified table.

    name can be a fully qualified identifier and must conform to the rules for a Snowflake identifier.

    name

    Table name that is either a fully qualified name or a name in the current database/schema.

    returns

    A Updatable

    Since

    0.1.0

  61. def tableFunction ( func: TableFunction , args: Map [ String , Column ] ) : DataFrame

    Creates a new DataFrame from the given table function and arguments.

    Creates a new DataFrame from the given table function and arguments.

    Example

    import com.snowflake.snowpark.functions._
    import com.snowflake.snowpark.tableFunctions._
    
    session.tableFunction(
      flatten,
      Map("input" -> parse_json(lit("[1,2]")))
    )
    func

    Table function object, can be created from TableFunction class or referred from the built-in list from tableFunctions.

    args

    function arguments map of the given table function. Some functions, like flatten, have named parameters. use this map to assign values to the corresponding parameters.

    Since

    0.4.0

  62. def tableFunction ( func: TableFunction , args: Seq [ Column ] ) : DataFrame

    Creates a new DataFrame from the given table function and arguments.

    Creates a new DataFrame from the given table function and arguments.

    Example

    import com.snowflake.snowpark.functions._
    import com.snowflake.snowpark.tableFunctions._
    
    session.tableFunction(
      split_to_table,
      Seq(lit("split by space"), lit(" "))
    )
    func

    Table function object, can be created from TableFunction class or referred from the built-in list from tableFunctions.

    args

    function arguments of the given table function.

    Since

    0.4.0

  63. def tableFunction ( func: TableFunction , firstArg: Column , remaining: Column * ) : DataFrame

    Creates a new DataFrame from the given table function and arguments.

    Creates a new DataFrame from the given table function and arguments.

    Example

    import com.snowflake.snowpark.functions._
    import com.snowflake.snowpark.tableFunctions._
    
    session.tableFunction(
      split_to_table,
      lit("split by space"),
      lit(" ")
    )
    func

    Table function object, can be created from TableFunction class or referred from the built-in list from tableFunctions.

    firstArg

    the first function argument of the given table function.

    remaining

    all remaining function arguments.

    Since

    0.4.0

  64. def toString () : String
    Definition Classes
    AnyRef → Any
  65. lazy val udf : UDFRegistration

    Returns a UDFRegistration object that you can use to register UDFs.

    Returns a UDFRegistration object that you can use to register UDFs. For example:

    session.udf.registerTemporary("mydoubleudf", (x: Int) => 2 * x)
    session.sql(s"SELECT mydoubleudf(c) FROM table")
    Since

    0.1.0

  66. lazy val udtf : UDTFRegistration

    Returns a UDTFRegistration object that you can use to register UDTFs.

    Returns a UDTFRegistration object that you can use to register UDTFs. For example:

    class MyWordSplitter extends UDTF1[String] {
      override def process(input: String): Iterable[Row] = input.split(" ").map(Row(_))
      override def endPartition(): Iterable[Row] = Array.empty[Row]
      override def outputSchema(): StructType = StructType(StructField("word", StringType))
    }
    val tableFunction = session.udtf.registerTemporary(new MyWordSplitter)
    session.tableFunction(tableFunction, lit("My name is Snow Park")).show()
    Annotations
    @ PublicPreview ()
    Since

    1.2.0

  67. def unsetQueryTag () : Unit

    Unset query_tag parameter for this session.

    Unset query_tag parameter for this session.

    If not set, the default value of query tag is the Snowpark library call and the class and method in your code that invoked the query (e.g. com.snowflake.snowpark.DataFrame.collect Main$.main(Main.scala:18) ).

    Since

    0.10.0

  68. final def wait () : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... )
  69. final def wait ( arg0: Long , arg1: Int ) : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... )
  70. final def wait ( arg0: Long ) : Unit
    Definition Classes
    AnyRef
    Annotations
    @throws ( ... ) @native ()
  71. object implicits extends Implicits with Serializable

    Provides implicit methods for convert Scala objects to Snowpark DataFrame and Column objects.

    Provides implicit methods for convert Scala objects to Snowpark DataFrame and Column objects.

    To use this, import session.implicits._ :

    val session = Session.builder.configFile(..).create
    import session.implicits._

    After you import this, you can call the toDF method of a Seq to convert a sequence to a DataFrame:

    // Create a DataFrame from a local sequence of integers.
    val df = (1 to 10).toDF("a")
    val df = Seq((1, "one"), (2, "two")).toDF("a", "b")

    You can also refer to columns in DataFrames by using $"colName" and 'colName :

    // Refer to a column in a DataFrame by using $"colName".
    val df = session.table("T").filter($"a" > 1)
    // Refer to columns by using 'colName.
    val df = session.table("T").filter('a > 1)
    Since

    0.1.0

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped