Fine-tuning (Snowflake Cortex)¶

Overview¶

Cortex Fine-tuning allows users to leverage parameter-efficient fine-tuning (PEFT) to create customized adaptors for use with pre-trained models on more specialized tasks. If you don’t want the high cost of training a large model from scratch but need better latency and results than you’re getting from prompt engineering or even retrieval augmented generation (RAG) methods, fine-tuning an existing large model is an option. Fine-tuning allows you to use examples to adjust the behavior of the model and improve the model’s knowledge of domain-specific tasks.

Cortex Fine-tuning is a fully managed service that lets you fine-tune popular LLMs using your data, all within Snowflake.

Cortex Fine-tuning features are provided as a Snowflake Cortex function, FINETUNE, with the following arguments:

• CREATE: Creates a fine-tuning job with the given training data.

• SHOW: Lists all the fine-tuning jobs that the current role has access to.

• DESCRIBE: Describes the progress and status of a particular fine-tuning job.

• CANCEL: Cancels a given fine-tuning job.

Cost considerations¶

The Snowflake Cortex Fine-tuning function incurs compute cost based on the number of tokens used in training. In addition, running the COMPLETE function on a fine-tuned model incurs compute costs based on the number of tokens processed. Refer to the Snowflake Service Consumption Table for each cost in credits per million tokens.

A token is the smallest unit of text processed by the Snowflake Cortex Fine-tuning function, approximately equal to four characters of text. The equivalence of raw input or output text to tokens can vary by model.

• For the COMPLETE function, which generates new text in the response, both input and output tokens are counted.

• Fine-tuning trained tokens are calculated as follows:

  Fine-tuning trained tokens = number of input tokens * number of epochs trained
  

  Copy

  Use the FINETUNE ('DESCRIBE') (SNOWFLAKE.CORTEX) to see the number of trained tokens for your fine-tuning job.

Other considerations¶

• In addition to the charges incurred for tuning and inference, there are normal storage and warehouse costs for storing the output customized adaptors, as well as for running any SQL commands.

• Fine-tuning jobs are often long running and are not attached to a worksheet session.

• The following limits apply to training/validation dataset size for each model:

  Model	Training data limit
  llama3-8b	1 GB
  llama3-70b	250 MB
  llama3.1-8b	1 GB
  llama3.1-70b	250 MB
  mistral-7b	1 GB
  mixtral-8x7b	250 MB

Access control requirements¶

To run a fine-tuning job, the role that creates the fine-tuning job needs the following privileges:

The following SQL is an example of granting the CREATE MODEL privilege to a role, my_role, on my_schema.

GRANT CREATE MODEL ON SCHEMA my_schema TO ROLE my_role;

Additionally, to use the FINETUNE function, the ACCOUNTADMIN role must grant the SNOWFLAKE.CORTEX_USER database role to the user who will call the function. See LLM Functions required privileges topic for details.

To give other roles access to use the fine-tuned model, you must grant usage on the model. For details, see Model Privileges.

Models available to fine-tune¶

You have the following base models that you can fine-tune. Models available for fine-tuning may be added or removed in the future:

How to fine-tune a model¶

The overall workflow for tuning a model is as follows:

1. Prepare the training data.

2. Start the fine-tuning job with the required parameters.

3. Monitor training job.

Once training is complete, you can use the model name provided by Cortex Fine-tuning to run inference on your model.

SELECT SNOWFLAKE.CORTEX.FINETUNE(
  'CREATE',
  'my_tuned_model',
  'mistral-7b',
  'SELECT a AS prompt, d AS completion FROM train',
  'SELECT a AS prompt, d AS completion FROM validation'
);

USE DATABASE mydb;
USE SCHEMA myschema;

SELECT SNOWFLAKE.CORTEX.FINETUNE(
  'CREATE',
  'my_tuned_model',
  'mistral-7b',
  'SELECT prompt, completion FROM my_training_data',
  'SELECT prompt, completion FROM my_validation_data'
);

SELECT SNOWFLAKE.CORTEX.FINETUNE(
  'CREATE',
  'mydb.myschema.my_tuned_model',
  'mistral-7b',
  'SELECT prompt, completion FROM mydb2.myschema2.my_training_data',
  'SELECT prompt, completion FROM mydb2.myschema2.my_validation_data'
);

Use your fine-tuned model for inference¶

Use the COMPLETE LLM function with the name your fine-tuned model to make inferences.

This example shows a call to the COMPLETE function with the name of your fine-tuned model.

SELECT SNOWFLAKE.CORTEX.COMPLETE(
  'my_tuned_model',
  'How to fine-tune mistral models'
);

The following is a snippet of the output from the example call:

Mistral models are a type of deep learning model used for image recognition and classification. Fine-tuning a Mistral model involves adjusting the model's parameters to ...

Limitations and known issues¶

• Fine-tuning jobs are listable at the account-level only.

• The fine-tuning jobs returned from FINETUNE ('SHOW') (SNOWFLAKE.CORTEX) are not permanent and may be garbage collected periodically.

• If a base model is removed from the Cortex LLM Functions, your fine-tuned model will no longer work.

Sharing models¶

Fine-tuned models can be shared to other accounts with the USAGE privilege via Data Sharing.

Replicating models¶

Cross-region inference does not support fine-tuned models. Inference must take place in the same region where the model object is located. You can use database replication to replicate the fine-tuned model object to a region you want to make inference from if it’s different than the region the model was trained in.

For example, if you create a fine-tuned model based on mistral-7b in your account in the AWS US West 2 region, you can use data sharing to share it with another account in this region, or you can use database replication to replicate the model to another account in your organization in a different region that supports the mistral-7b model, such as AWS Europe West. For details on replicating objects, see Replicating databases and account objects across multiple accounts.

Legal notices¶

The data classification of inputs and outputs are as set forth in the following table.

For additional information, refer to Snowflake AI and ML.