Large Language Model (LLM) Functions (Snowflake Cortex)¶

Required privileges¶

The CORTEX_USER database role in the SNOWFLAKE database includes the privileges that allow users to call Snowflake Cortex LLM functions. By default, the CORTEX_USER role is granted to the PUBLIC role. The PUBLIC role is automatically granted to all users and roles, so this allows all users in your account to use the Snowflake Cortex LLM functions.

If you don’t want all users to have this privilege, you can revoke access to the PUBLIC role and grant access to specific roles.

To revoke the CORTEX_USER database role from the PUBLIC role, run the following command using the ACCOUNTADMIN role:

REVOKE DATABASE ROLE SNOWFLAKE.CORTEX_USER
  FROM ROLE PUBLIC;

You can then selectively provide access to specific roles. The SNOWFLAKE.CORTEX_USER database role cannot be granted directly to a user. For more information, see Using SNOWFLAKE database roles. A user with the ACCOUNTADMIN role can grant this role to a custom role in order to allow users to access Cortex LLM Functions. In the following example, use the ACCOUNTADMIN role and grant the user some_user the CORTEX_USER database role via the account role cortex_user_role, which you create for this purpose.

USE ROLE ACCOUNTADMIN;

CREATE ROLE cortex_user_role;
GRANT DATABASE ROLE SNOWFLAKE.CORTEX_USER TO ROLE cortex_user_role;

GRANT ROLE cortex_user_role TO USER some_user;

You can also grant access to Snowflake Cortex LLM functions through existing roles commonly used by specific groups of users. (See User roles.) For example, if you have created an analyst role that is used as a default role by analysts in your organization, you can easily grant these users access to Snowflake Cortex LLM functions with a single GRANT statement.

GRANT DATABASE ROLE SNOWFLAKE.CORTEX_USER TO ROLE analyst;

Availability¶

Snowflake Cortex LLM functions are currently available in the following regions.

Cost considerations¶

Snowflake Cortex LLM functions incur compute cost based on the number of tokens processed. Refer to the consumption table for each function’s cost in credits per million tokens.

A token is the smallest unit of text processed by Snowflake Cortex LLM functions, approximately equal to four characters of text. The equivalence of raw input or output text to tokens can vary by model.

• For functions that generate new text in the response (COMPLETE, SUMMARIZE, and TRANSLATE), both input and output tokens are counted.

• For CORTEX GUARD, only input tokens are counted. The number of input tokens is based on the number of output tokens per LLM model used in the COMPLETE function.

• For the EMBED_TEXT_* functions, only input tokens are counted.

• For functions that only extract information from the input (EXTRACT_ANSWER and SENTIMENT), only input tokens are counted.

• For EXTRACT_ANSWER, the number of billable tokens is the sum of the number of tokens in the from_text and question fields.

• SUMMARIZE, TRANSLATE, EXTRACT_ANSWER, and SENTIMENT add a prompt to the input text in order to generate the response. As a result, the input token count is slightly higher than the number of tokens in the text you provide.

• TRY_COMPLETE does not incur costs for error handling. This means that if the TRY_COMPLETE function returns NULL, no cost is incurred.

• COUNT_TOKENS incurs only compute cost to run the function. No additional token based costs are incurred.

Snowflake recommends executing queries that call a Snowflake Cortex LLM Function with a smaller warehouse (no larger than MEDIUM) because larger warehouses do not increase performance. The cost associated with keeping a warehouse active will continue to apply when executing a query that calls a Snowflake Cortex LLM Function. For general information on compute costs, see Understanding compute cost.

SELECT *
  FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_DAILY_HISTORY
  WHERE SERVICE_TYPE='AI_SERVICES';

SELECT *
  FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_FUNCTIONS_USAGE_HISTORY;

Usage quotas¶

To ensure that all Snowflake customers can access LLM capabilities, Snowflake Cortex LLM functions may be subject to throttling during periods of high utilization. Usage quotas are not applied at the account level.

Throttled requests will receive an error response and should be retried later.

Managing costs and throttling¶

Snowflake recommends using a warehouse size no larger than MEDIUM when calling Snowflake Cortex LLM functions. Using a larger warehouse than necessary does not increase performance, but can result in unnecessary costs and a higher risk of throttling. This recommendation may not apply in the future due to upcoming product updates.

Model restrictions¶

Models used by Snowflake Cortex have limitations on size as described in the table below. Sizes are given in tokens. Tokens generally correspond to words, but not all tokens are words, so the number of words corresponding to a limit is slightly less than the number of tokens. Inputs that exceed the limit result in an error.

Choosing a model¶

The Snowflake Cortex COMPLETE function supports multiple models of varying capability, latency, and cost. These models have been carefully chosen to align with common customer use cases. To achieve the best performance per credit, choose a model that’s a good match for the content size and complexity of your task. Here are brief overviews of the available models.

LLM functions overview¶

Cortex LLM Functions can be grouped into the following categories:

• COMPLETE function: General purpose function that can perform a wide range of user-specified tasks, such as aspect-based sentiment classification, synthetic data generation, and customized summaries. You can also use this function with your fine-tuned models.

• Task-specific functions: Purpose-built and managed functions that automate routine-tasks, like simple summaries and quick translations, that don’t require any customization.

• Helper functions: Purpose-built and managed functions that reduce cases of failures when running other LLM functions, for example by getting the count of tokens in an input prompt to ensure the call doesn’t exceed a model limit.

Error conditions¶

Snowflake Cortex LLM functions can produce the following error messages.

Using Snowflake Cortex LLM functions with Python¶

Snowflake Cortex LLM functions are available in Snowpark ML version 1.1.2 and later. See Installing Snowpark ML for instructions on setting up Snowpark ML.

If you run your Python script outside of Snowflake, you must create a Snowpark session to use these functions. See Connecting to Snowflake for instructions.

The following Python example illustrates calling Snowflake Cortex LLM functions on single values:

from snowflake.cortex import Complete, ExtractAnswer, Sentiment, Summarize, Translate

text = """
    The Snowflake company was co-founded by Thierry Cruanes, Marcin Zukowski,
    and Benoit Dageville in 2012 and is headquartered in Bozeman, Montana.
"""

print(Complete("llama2-70b-chat", "how do snowflakes get their unique patterns?"))
print(ExtractAnswer(text, "When was snowflake founded?"))
print(Sentiment("I really enjoyed this restaurant. Fantastic service!"))
print(Summarize(text))
print(Translate(text, "en", "fr"))

You can also call an LLM function on a table column, as shown below. This example requires a session object (stored in session) and a table articles containing a text column abstract_text, and creates a new column abstract_summary containing a summary of the abstract.

from snowflake.cortex import Summarize
from snowflake.snowpark.functions import col

article_df = session.table("articles")
article_df = article_df.withColumn(
    "abstract_summary",
    Summarize(col("abstract_text"))
)
article_df.collect()

Using Snowflake Cortex LLM functions with Snowflake CLI¶

Snowflake Cortex LLM functions are available in Snowflake CLI version 2.4.0 and later. See Introducing Snowflake CLI for more information about using Snowflake CLI.

The following examples illustrate using the snow cortex commands on single values. The -c parameter specifies which connection to use.

snow cortex complete "Is 5 more than 4? Please answer using one word without a period." -c "snowhouse"

snow cortex extract-answer "what is snowflake?" "snowflake is a company" -c "snowhouse"

snow cortex sentiment "Mary had a little Lamb" -c "snowhouse"

snow cortex summarize "John has a car. John's car is blue. John's car is old and John is thinking about buying a new car. There are a lot of cars to choose from and John cannot sleep because it's an important decision for John."

snow cortex translate herb --to pl

You can also use files that contain the text you want to use for the commands. For this example, assume that the file about_cortex.txt contains the following content:

Snowflake Cortex gives you instant access to industry-leading large language models (LLMs) trained by researchers at companies like Mistral, Reka, Meta, and Google, including Snowflake Arctic, an open enterprise-grade model developed by Snowflake.

Since these LLMs are fully hosted and managed by Snowflake, using them requires no setup. Your data stays within Snowflake, giving you the performance, scalability, and governance you expect.

Snowflake Cortex features are provided as SQL functions and are also available in Python. The available functions are summarized below.

COMPLETE: Given a prompt, returns a response that completes the prompt. This function accepts either a single prompt or a conversation with multiple prompts and responses.
EMBED_TEXT_768: Given a piece of text, returns a vector embedding that represents that text.
EXTRACT_ANSWER: Given a question and unstructured data, returns the answer to the question if it can be found in the data.
SENTIMENT: Returns a sentiment score, from -1 to 1, representing the detected positive or negative sentiment of the given text.
SUMMARIZE: Returns a summary of the given text.
TRANSLATE: Translates given text from any supported language to any other.

You can then execute the snow cortex summarize command by passing in the filename using the --file parameter, as shown:

snow cortex summarize --file about_cortex.txt

Snowflake Cortex offers instant access to industry-leading language models, including Snowflake Arctic, with SQL functions for completing prompts (COMPLETE), text embedding (EMBED\_TEXT\_768), extracting answers (EXTRACT\_ANSWER), sentiment analysis (SENTIMENT), summarizing text (SUMMARIZE), and translating text (TRANSLATE).

For more information about these commands, see snow cortex commands.

Legal notices¶

Refer to Snowflake AI and ML.