Large Language Model (LLM) Functions (Snowflake Cortex)¶

COMPLETE function¶

The COMPLETE function is a general purpose function that can perform a wide range of user-specified tasks, such as aspect-based sentiment classification, synthetic data generation, and customized summaries. Cortex Guard is a safety parameter available within the COMPLETE function designed to filter possible unsafe and harmful responses from a language model. You can also use this function with your fine-tuned models.

Task-specific functions¶

Task-specific functions are purpose-built and managed functions that automate routine tasks, like simple summaries and quick translations, that don’t require any customization.

• CLASSIFY_TEXT: Given a piece of text, classifies it into one of the categories that you define.

• EXTRACT_ANSWER: Given a question and unstructured data, returns the answer to the question if it can be found in the data.

• PARSE_DOCUMENT: Given an internal or external stage with documents, returns an object that contains a JSON-formatted string with extracted text content using OCR mode, or the extracted text and layout elements using LAYOUT mode.

• SENTIMENT: Returns a sentiment score, from -1 to 1, representing the detected positive or negative sentiment of the given text.

• SUMMARIZE: Returns a summary of the given text.

• TRANSLATE: Translates given text from any supported language to any other.

• EMBED_TEXT_768: Given a piece of text, returns a vector embedding of 768 dimensions that represents that text.

• EMBED_TEXT_1024: Given a piece of text, returns a vector embedding of 1024 dimensions that represents that text.

Helper functions¶

Helper functions are purpose-built and managed functions that reduce cases of failures when running other LLM functions, for example by getting the count of tokens in an input prompt to ensure the call doesn’t exceed a model limit.

• COUNT_TOKENS: Given an input text, returns the token count based on the model or Cortex function specified.

• TRY_COMPLETE: Works like the COMPLETE function, but returns NULL when the function could not execute instead of an error code.

Track costs for AI services¶

To track credits used for AI Services including LLM Functions in your account, use the METERING_HISTORY view:

SELECT *
  FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_DAILY_HISTORY
  WHERE SERVICE_TYPE='AI_SERVICES';

Track credit consumption for LLM functions¶

To view the credit and token consumption for each LLM function call, use the CORTEX_FUNCTIONS_USAGE_HISTORY view:

SELECT *
  FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_FUNCTIONS_USAGE_HISTORY;

Large models¶

If you’re not sure where to start, try the most capable models first to establish a baseline to evaluate other models. reka-core and mistral-large2 are the most capable models offered by Snowflake Cortex, and will give you a good idea what a state-of-the-art model can do.

• reka-core is Reka AI’s most advanced large language model with strong reasoning abilities, code generation, and multilingual fluency.

• mistral-large2 is Mistral AI’s most advanced large language model with top-tier reasoning capabilities. Compared to mistral-large, it’s significantly more capable in code generation, mathematics, reasoning, and provides much stronger multilingual support. It’s ideal for complex tasks that require large reasoning capabilities or are highly specialized, such as synthetic text generation, code generation, and multilingual text analytics.

• llama3.1-405b is an open source model from the llama3.1 model family from Meta with a large 128K context window. It excels in long document processing, multilingual support, synthetic data generation and model distillation.

Medium models¶

• llama3.1-70b is an open source model that demonstrates state-of-the-art performance ideal for chat applications, content creation, and enterprise applications. It is a highly performant, cost effective model that enables diverse use cases with a context window of 128K. llama3-70b is still supported and has a context window of 8K.

• snowflake-arctic is Snowflake’s top-tier enterprise-focused LLM. Arctic excels at enterprise tasks such as SQL generation, coding and instruction following benchmarks.

• reka-flash is a highly capable multilingual language model optimized for fast workloads that require high quality, such as writing product descriptions or blog posts, coding, and extracting answers from documents with hundreds of pages.

• mixtral-8x7b is ideal for text generation, classification, and question answering. Mistral models are optimized for low latency with low memory requirements, which translates into higher throughput for enterprise use cases.

• The jamba-Instruct model is built by AI21 Labs to efficiently meet enterprise requirements. It is optimized to offer a 256k token context window with low cost and latency, making it ideal for tasks like summarization, Q&A, and entity extraction on lengthy documents and extensive knowledge bases.

• The AI21 Jamba 1.5 family of models is state-of-the-art, hybrid SSM-Transformer instruction following foundation models. The jamba-1.5-mini and jamba-1.5-large with a context length of 256K supports use cases such as structured output (JSON), and grounded generation.

Small models¶

• The llama3.2-1b and llama3.2-3b models support context length of 128K tokens and are state-of-the-art in their class for use cases like summarization, instruction following, and rewriting tasks. The Llama 3.2 models deliver multilingual capabilities, with support for English, German, French, Italian, Portuguese, Hindi, Spanish and Thai.

• llama3.1-8b is ideal for tasks that require low to moderate reasoning. It’s a light-weight, ultra-fast model with a context window of 128K. llama3-8b and llama2-70b-chat are still supported models that provide a smaller context window and relatively lower accuracy.

• mistral-7b is ideal for your simplest summarization, structuration, and question answering tasks that need to be done quickly. It offers low latency and high throughput processing for multiple pages of text with its 32K context window.

• gemma-7b is suitable for simple code and text completion tasks. It has a context window of 8,000 tokens but is surprisingly capable within that limit, and quite cost-effective.

The following table provides information on how popular models perform on various benchmarks, including the models offered by Snowflake Cortex COMPLETE as well as a few other popular models.

^*Provided for comparison; not available in Snowflake Cortex COMPLETE.

COMPLETE¶

Given a prompt, the instruction-following COMPLETE function generates a response using your choice of language model. In the simplest use case, the prompt is a single string. You may also provide a conversation including multiple prompts and responses for interactive chat-style usage, and in this form of the function you can also specify hyperparameter options to customize the style and size of the output. In order to implement safeguards, you can also enable the Cortex Guard parameter that filters potentially unsafe and harmful responses from a LLM.

To implement safeguards, you can enable the Cortex Guard parameter that filters unsafe and harmful responses from an LLM.

The COMPLETE function supports the following models. Different models can have different costs.

• gemma-7b

• jamba-1.5-mini

• jamba-1.5-large

• jamba-instruct

• llama2-70b-chat

• llama3-8b

• llama3-70b

• llama3.1-8b

• llama3.1-70b

• llama3.1-405b

• llama3.2-1b

• llama3.2-3b

• mistral-large

• mistral-large2

• mistral-7b

• mixtral-8x7b

• reka-core

• reka-flash

• snowflake-arctic

See COMPLETE (SNOWFLAKE.CORTEX) for syntax and examples.

Task-specific functions¶

Task-specific functions are managed by the Snowflake AI team to deliver high-quality results, without requiring you to develop your own prompts. Under the hood, the Snowflake AI team might implement various prompt optimizations and fine-tuning to deliver high-quality results for each task. Task-specific functions provide the benefits of ease of use and efficiency, without any compromise on quality.

Helper functions¶

Helper functions are managed functions that are built to help reduce errors when running other Cortex LLM functions.