Snowflake Cortex AISQL (including LLM functions)¶

Available functions¶

Snowflake Cortex features are provided as SQL functions and are also available in Python. Cortex AISQL functions can be grouped into the following categories:

• AISQL functions

• Helper functions

Performance considerations¶

Cortex AISQL Functions are optimized for throughput. We recommend using these functions to process numerous inputs such as text from large SQL tables. Batch processing is typically better suited for AISQL Functions. For more interactive use cases where latency is important, use the REST API. These are available for simple inference (Complete API), embedding (Embed API) and agentic applications (Agents API).

Required privileges¶

The CORTEX_USER database role in the SNOWFLAKE database includes the privileges that allow users to call Snowflake Cortex AI functions. By default, the CORTEX_USER role is granted to the PUBLIC role. The PUBLIC role is automatically granted to all users and roles, so this allows all users in your account to use the Snowflake Cortex AI functions.

If you don’t want all users to have this privilege, you can revoke access to the PUBLIC role and grant access to specific roles.

To revoke the CORTEX_USER database role from the PUBLIC role, run the following commands using the ACCOUNTADMIN role:

REVOKE DATABASE ROLE SNOWFLAKE.CORTEX_USER
  FROM ROLE PUBLIC;

REVOKE IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE
  FROM ROLE PUBLIC;

You can then selectively provide access to specific roles. The SNOWFLAKE.CORTEX_USER database role cannot be granted directly to a user. For more information, see Using SNOWFLAKE database roles. A user with the ACCOUNTADMIN role can grant this role to a custom role in order to allow users to access Cortex AI functions. In the following example, use the ACCOUNTADMIN role and grant the user some_user the CORTEX_USER database role via the account role cortex_user_role, which you create for this purpose.

USE ROLE ACCOUNTADMIN;

CREATE ROLE cortex_user_role;
GRANT DATABASE ROLE SNOWFLAKE.CORTEX_USER TO ROLE cortex_user_role;

GRANT ROLE cortex_user_role TO USER some_user;

You can also grant access to Snowflake Cortex AI functions through existing roles commonly used by specific groups of users. (See User roles.) For example, if you have created an analyst role that is used as a default role by analysts in your organization, you can easily grant these users access to Snowflake Cortex AISQL functions with a single GRANT statement.

GRANT DATABASE ROLE SNOWFLAKE.CORTEX_USER TO ROLE analyst;

Control model access¶

There are two methods to control access to models in Snowflake Cortex. You can use one or both methods together for a mix of broad and fine-grained access control:

• Model allowlist

• Role-based access control

The model allowlist provides a default level of access to models for all users in the account, which can be customized using the CORTEX_MODELS_ALLOWLIST parameter. Role-based access control allows fine-grained access management by granting or revoking privileges to specific model objects through application roles.

Model access control is available for the following services:

• AISQL functions

• Cortex REST API

• TRY_COMPLETE (SNOWFLAKE.CORTEX)

• Cortex Playground

Fine-grained access control methods aren’t available for the following services:

• Cortex Search

• Cortex Analyst

• Cortex Copilot

• Cortex Agents

• Document AI

CALL SNOWFLAKE.MODELS.CORTEX_BASE_MODELS_REFRESH();

SHOW MODELS IN SNOWFLAKE.MODELS;

SHOW APPLICATION ROLES IN APPLICATION SNOWFLAKE;

GRANT APPLICATION ROLE SNOWFLAKE."CORTEX-MODEL-ROLE-LLAMA3.1-70B" TO ROLE MY_ROLE;

SELECT AI_COMPLETE('SNOWFLAKE.MODELS."LLAMA3.1-70B"', 'Hello');

Availability¶

Snowflake Cortex AI functions are currently available in the following regions. To access LLMs from regions that aren’t listed, use cross-region inference parameter.

The following Snowflake Cortex AI functions are currently available in the following extended regions.

The following table lists legacy models. If you’re just getting started, start with models in the previous tables:

Cost considerations¶

Snowflake Cortex AI functions incur compute cost based on the number of tokens processed. Refer to the Snowflake Service Consumption Table for each function’s cost in credits per million tokens.

A token is the smallest unit of text processed by Snowflake Cortex AI functions, approximately equal to four characters. The equivalence of raw input or output text to tokens can vary by model.

• For functions that generate new text in the response (AI_COMPLETE, AI_CLASSIFY, AI_FILTER, AI_AGG, AI_SUMMARIZE, and TRANSLATE), both input and output tokens are counted.

• For CORTEX GUARD, only input tokens are counted. The number of input tokens is based on the number of output tokens per LLM model used in the COMPLETE function.

• For AI_SIMILARITY the EMBED_* functions, only input tokens are counted.

• For EXTRACT_ANSWER, the number of billable tokens is the sum of the number of tokens in the from_text and question fields.

• AI_CLASSIFY, AI_FILTER, AI_AGG, AI_SUMMARIZE_AGG, SUMMARIZE, TRANSLATE, EXTRACT_ANSWER, and ENTITY_SENTIMENT, and SENTIMENT add a prompt to the input text in order to generate the response. As a result, the input token count is higher than the number of tokens in the text you provide.

• AI_CLASSIFY labels, descriptions, and examples are counted as input tokens for each record processed, not just once for each AI_CLASSIFY call.

• For PARSE_DOCUMENT (SNOWFLAKE.CORTEX), billing is based on the number of document pages processed.

• TRY_COMPLETE(SNOWFLAKE.CORTEX) does not incur costs for error handling. This means that if the TRY_COMPLETE(SNOWFLAKE.CORTEX) function returns NULL, no cost is incurred.

• COUNT_TOKENS(SNOWFLAKE.CORTEX) incurs only compute cost to run the function. No additional token based costs are incurred.

Snowflake recommends executing queries that call a Snowflake Cortex AISQL function or a Cortex PARSE_DOCUMENT function with a smaller warehouse (no larger than MEDIUM) because larger warehouses do not increase performance. The cost associated with keeping a warehouse active will continue to apply when executing a query that calls a Snowflake Cortex LLM Function. For general information on compute costs, see Understanding compute cost.

SELECT *
  FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_DAILY_HISTORY
  WHERE SERVICE_TYPE='AI_SERVICES';

SELECT *
  FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_FUNCTIONS_USAGE_HISTORY;

SELECT * FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_FUNCTIONS_QUERY_USAGE_HISTORY;

SELECT * FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_FUNCTIONS_QUERY_USAGE_HISTORY
WHERE query_id='<query-id>';

SELECT AI_COMPLETE('mistral-7b', 'Is a hot dog a sandwich'), AI_COMPLETE('mistral-large', 'Is a hot dog a sandwich');

Usage quotas¶

To ensure that all Snowflake customers can access LLM capabilities, Snowflake Cortex AI functions may be subject to throttling during periods of high utilization. Usage quotas are not applied at the account level.

Throttled requests will receive an error response and should be retried later.

Managing costs and throttling¶

Snowflake recommends using a warehouse size no larger than MEDIUM when calling Snowflake Cortex AISQL functions. Using a larger warehouse than necessary does not increase performance, but can result in unnecessary costs and a higher risk of throttling. This recommendation might not apply in the future due to upcoming product updates.

Model restrictions¶

Models used by Snowflake Cortex have limitations on size as described in the table below. Sizes are given in tokens. Tokens generally represent about four characters of text, so the number of words corresponding to a limit is less than the number of tokens. Inputs that exceed the limit result in an error.

The maximum size of the output that a model can produce is limited by the following:

• The model’s output token limit.

• The space available in the context window after the model consumes the input tokens.

For example, claude-3-5-sonnet has a context window of 200,000 tokens. If 100,000 tokens are used for the input, the model can generate up to 8,192 tokens. However, if 195,000 tokens are used as input, then the model can only generate up to 5,000 tokens for a total of 200,000 tokens.

Choosing a model¶

The Snowflake Cortex COMPLETE function supports multiple models of varying capability, latency, and cost. These models have been carefully chosen to align with common customer use cases. To achieve the best performance per credit, choose a model that’s a good match for the content size and complexity of your task. Here are brief overviews of the available models.

Previous model versions¶

The Snowflake Cortex COMPLETE function also supports the following older model versions. We recommend using the latest model versions instead of the versions listed in this table.

Error conditions¶

Snowflake Cortex AI functions can produce the following error messages.

Using Snowflake Cortex AISQL with Python¶

You can use Snowflake Cortex AISQL functions in the Snowpark Python API. Within the API, you can use the functions to classify, summarize, and filter both text and image data.

These functions include the following:

• ai_agg() - Aggregates a column of text using natural language instructions. For more information, see AI_AGG.

• ai_classify() - Classifies text or images into custom categories. For more information, see AI_CLASSIFY.

• ai_filter()– Filters data using natural language. For more information, see AI_FILTER.

The ai_agg() function aggregates a column of text using natural language instructions in a similar manner to how you would ask an analyst to summarize or extract findings from grouped or ungrouped data.

The following example summarizes customer reviews for each product using the ai_agg() function. The function takes a column of text and a natural language instruction to summarize the reviews.

from snowflake.snowpark.functions import ai_agg, col

df = session.create_dataframe([
    [1, "Excellent product!"],
    [1, "Great battery life."],
    [1, "A bit expensive but worth it."],
    [2, "Terrible customer service."],
    [2, "Won’t buy again."],
], schema=["product_id", "review"])

# Summarize reviews per product
summary_df = df.group_by("product_id").agg(
    ai_agg(col("review"), "Summarize the customer reviews in one sentence.")
)
summary_df.show()

The ai_classify() function takes a text or image and classifies it into the categories that you define.

The following example classifies travel reviews into categories such as “travel” and “cooking”. The function takes a column of text and a list of categories to classify the text into.

from snowflake.snowpark.functions import ai_classify, col

df = session.create_dataframe([
    ["I dream of backpacking across South America."],
    ["I made the best pasta yesterday."],
], schema=["sentence"])

df = df.select(
    "sentence",
    ai_classify(col("sentence"), ["travel", "cooking"]).alias("classification")
)
df.show()

The ai_filter() function evaluates a natural language condition and returns TRUE or FALSE. You can use it to filter or tag rows.

from snowflake.snowpark.functions import ai_filter, prompt, col

df = session.create_dataframe(["Canada", "Germany", "Japan"], schema=["country"])

filtered_df = df.select(
    "country",
    ai_filter(prompt("Is {0} in Asia?", col("country"))).alias("is_in_asia")
)
filtered_df.show()

Existing Snowpark ML functions are still supported in Snowpark ML version 1.1.2 and later. See Using Snowflake ML Locally for instructions on setting up Snowpark ML.

If you run your Python script outside of Snowflake, you must create a Snowpark session to use these functions. See Connecting to Snowflake for instructions.

The following Python example illustrates calling Snowflake Cortex AI functions on single values:

from snowflake.cortex import Complete, ExtractAnswer, Sentiment, Summarize, Translate

text = """
    The Snowflake company was co-founded by Thierry Cruanes, Marcin Zukowski,
    and Benoit Dageville in 2012 and is headquartered in Bozeman, Montana.
"""

print(Complete("llama2-70b-chat", "how do snowflakes get their unique patterns?"))
print(ExtractAnswer(text, "When was snowflake founded?"))
print(Sentiment("I really enjoyed this restaurant. Fantastic service!"))
print(Summarize(text))
print(Translate(text, "en", "fr"))

You can pass options that affect the model’s hyperparameters when using the COMPLETE function. The following Python example illustrates calling the COMPLETE function with a modification of the maximum number of output tokens that the model can generate:

from snowflake.cortex import Complete,CompleteOptions

model_options1 = CompleteOptions(
    {'max_tokens':30}
)

print(Complete("llama3.1-8b", "how do snowflakes get their unique patterns?", options=model_options1))

You can also call an AI function on a table column, as shown below. This example requires a session object (stored in session) and a table articles containing a text column abstract_text, and creates a new column abstract_summary containing a summary of the abstract.

from snowflake.cortex import Summarize
from snowflake.snowpark.functions import col

article_df = session.table("articles")
article_df = article_df.withColumn(
    "abstract_summary",
    Summarize(col("abstract_text"))
)
article_df.collect()

Using Snowflake Cortex AI functions with Snowflake CLI¶

Snowflake Cortex AISQL is available in Snowflake CLI version 2.4.0 and later. See Introducing Snowflake CLI for more information about using Snowflake CLI.

The following examples illustrate using the snow cortex commands on single values. The -c parameter specifies which connection to use.

snow cortex complete "Is 5 more than 4? Please answer using one word without a period." -c "snowhouse"

snow cortex extract-answer "what is snowflake?" "snowflake is a company" -c "snowhouse"

snow cortex sentiment "Mary had a little Lamb" -c "snowhouse"

snow cortex summarize "John has a car. John's car is blue. John's car is old and John is thinking about buying a new car. There are a lot of cars to choose from and John cannot sleep because it's an important decision for John."

snow cortex translate herb --to pl

You can also use files that contain the text you want to use for the commands. For this example, assume that the file about_cortex.txt contains the following content:

Snowflake Cortex gives you instant access to industry-leading large language models (LLMs) trained by researchers at companies like Anthropic, Mistral, Reka, Meta, and Google, including Snowflake Arctic, an open enterprise-grade model developed by Snowflake.

Since these LLMs are fully hosted and managed by Snowflake, using them requires no setup. Your data stays within Snowflake, giving you the performance, scalability, and governance you expect.

Snowflake Cortex features are provided as SQL functions and are also available in Python. The available functions are summarized below.

COMPLETE: Given a prompt, returns a response that completes the prompt. This function accepts either a single prompt or a conversation with multiple prompts and responses.
EMBED_TEXT_768: Given a piece of text, returns a vector embedding that represents that text.
EXTRACT_ANSWER: Given a question and unstructured data, returns the answer to the question if it can be found in the data.
SENTIMENT: Returns a sentiment score, from -1 to 1, representing the detected positive or negative sentiment of the given text.
SUMMARIZE: Returns a summary of the given text.
TRANSLATE: Translates given text from any supported language to any other.

You can then execute the snow cortex summarize command by passing in the filename using the --file parameter, as shown:

snow cortex summarize --file about_cortex.txt

Snowflake Cortex offers instant access to industry-leading language models, including Snowflake Arctic, with SQL functions for completing prompts (COMPLETE), text embedding (EMBED\_TEXT\_768), extracting answers (EXTRACT\_ANSWER), sentiment analysis (SENTIMENT), summarizing text (SUMMARIZE), and translating text (TRANSLATE).

For more information about these commands, see snow cortex commands.

Legal notices¶

The data classification of inputs and outputs are as set forth in the following table.

For additional information, refer to Snowflake AI and ML.