Snowflake Model Registry¶

Required privileges¶

To create a model, you must either own the schema where the model is created or have the CREATE MODEL privilege on it. To use a model, you must either own the model or have the USAGE privilege on it. The USAGE privilege allows grantees to use the model for inference without being able to see any of its internals. To give users access to all existing models in a schema, use GRANT USAGE ON ALL MODELS IN SCHEMA <schema> TO ROLE <role>; You can also give users access to future models created in a schema automatically via GRANT USAGE ON FUTURE MODELS IN SCHEMA <schema> TO ROLE <role>;. .

If a user’s role has USAGE on a model, it appears in the Snowsight model registry page. For details about how privileges work in Snowflake, see Access control privileges.

Current limitations¶

The following limits apply to models and model versions:

Opening the Snowflake Model Registry¶

Models are first-class Snowflake objects and can be organized within a database and schema along with other Snowflake objects. The Snowflake Model Registry provides a Python class for managing models within a schema. Thus, any Snowflake schema can be used as a registry. It is not necessary to initialize or otherwise prepare a schema for this purpose. Snowflake recommends creating one or more dedicated schemas for this purpose, such as ML.REGISTRY. You can create the schema using CREATE SCHEMA.

Before you can create or modify models in the registry, you must open the registry. Opening the registry returns a reference to it, which you can then use to add new models and obtain references to existing models.

from snowflake.ml.registry import Registry

reg = Registry(session=sp_session, database_name="ML", schema_name="REGISTRY")

Registering models and versions¶

Adding a model to the registry is called logging the model. Log a model by calling the registry’s log_model method. This method serializes the model — a Python object — and creates a Snowflake model object from it. This method also adds metadata, such as a description, to the model as specified in the log_model call.

Each model can have unlimited versions. To log additional versions of the model, call log_model again with the same model_name but a different version_name.

You cannot add tags to a model when it is added to the registry, because tags are attributes of the model, and log_model adds a specific model version, only creating a model when adding its first version. You can update the model’s tags after logging the first version of the model.

In the following example, clf, short for “classifier,” is the Python model object, which was already created elsewhere in your code. You can add a comment at registration time, as shown here. The combination of name and version must be unique in the schema. You may specify conda_dependencies lists; the specified packages will be deployed with the model.

from snowflake.ml.model import type_hints
mv = reg.log_model(clf,
                   model_name="my_model",
                   version_name="v1",
                   conda_dependencies=["scikit-learn"],
                   comment="My awesome ML model",
                   metrics={"score": 96},
                   sample_input_data=train_features,
                   task=type_hints.Task.TABULAR_BINARY_CLASSIFICATION)

The arguments of log_model are described here.

Required arguments

Optional arguments

log_model returns a snowflake.ml.model.ModelVersion object, which represents the version of the model that was added to the registry.

After registration, the model itself cannot be modified (although you can change its metadata). To delete a model and all its versions, use the registry’s delete_model method.

Working with model artifacts¶

After a model has been logged, its artifacts (the files backing the model, including its serialized Python objects and various metadata files such as its manifest) are available on an internal stage. Artifacts cannot be modified, but you can view or download the artifacts of models you own.

You can access model artifacts from a stage using, for example, the GET command or its equivalent in Snowpark Python, FileOperation.get.

However, you cannot address model artifacts using the usual stage path syntax. Instead, use a snow:// URL, a more general way to specify the location of objects in Snowflake. For example, a version inside a model can be specified by a URL of the form snow://model/<model_name>/versions/<version_name>/.

Knowing the of name of the model and the version you want, you can use the LIST command to view the artifacts of the model as follows:

LIST 'snow://model/my_model/versions/V3/';

The output resembles:

name                                      size                  md5                      last_modified
versions/V3/MANIFEST.yml           30639    2f6186fb8f7d06e737a4dfcdab8b1350        Thu, 18 Jan 2024 09:24:37 GMT
versions/V3/functions/apply.py      2249    e9df6db11894026ee137589a9b92c95d        Thu, 18 Jan 2024 09:24:37 GMT
versions/V3/functions/predict.py    2251    132699b4be39cc0863c6575b18127f26        Thu, 18 Jan 2024 09:24:37 GMT
versions/V3/model.zip             721663    e92814d653cecf576f97befd6836a3c6        Thu, 18 Jan 2024 09:24:37 GMT
versions/V3/model/env/conda.yml          332        1574be90b7673a8439711471d58ec746        Thu, 18 Jan 2024 09:24:37 GMT
versions/V3/model/model.yaml       25718    33e3d9007f749bb2e98f19af2a57a80b        Thu, 18 Jan 2024 09:24:37 GMT

To retrieve one of these artifacts, use the SQL GET command:

GET 'snow://model/model_my_model/versions/V3/MANIFEST.yml'

Or the equivalent with Snowpark Python:

session.file.get('snow://model/my_model/versions/V3/MANIFEST.yml', 'model_artifacts')

Deleting models¶

Use the registry’s delete_model method to delete a model and all its versions:

reg.delete_model("mymodel")

Getting models from the registry¶

To get information about each model, use the show_models method:

model_df = reg.show_models()

The result of show_models is a pandas DataFrame. The available columns are listed here:

To get a list of the models in the registry instead, each as a Model instance, use the models method:

model_list = reg.models()

To get a reference to a specific model from the registry by name, use the registry’s get_model method:

m = reg.get_model("MyModel")

After you have a reference to a model, either one from the list returned by the models method or one retrieved using get_model, you can work with its metadata and its versions.

Viewing and updating a model’s metadata¶

You can view and update a model’s metadata attributes in the registry, including its name, comment, tags, and metrics.

print(m.comment)
m.comment = "A better description than the one I provided originally"

print(m.description)
m.description = "A better description than the one I provided originally"

print(m.show_tags())

m.set_tag("live_version", "v1")

m.get_tag("live_version")

m.unset_tag("live_version")

m.rename("MY_MODEL_TOO")

Working with model versions¶

A model can have unlimited versions, each identified by a string. You can use any version naming convention that you like. Logging a model actually logs a specific version of the model. To log additional versions of a model, call log_model again with the same model_name but a different version_name.

A version of a model is represented by an instance of the snowflake.ml.model.ModelVersion class.

To get a list of all the versions of a model, call the model object’s versions method. The result is a list of ModelVersion instances:

version_list = m.versions()

To get information about each model as a DataFrame instead, call the model’s show_versions method:

version_df = m.show_versions()

The resulting DataFrame contains the following columns:

m.delete_version("rc1")

default_version = m.default
m.default = "v2"

m = reg.get_model("MyModel")

mv = m.version("v1")
mv = m.default

print(mv.comment)
print(mv.description)

mv.comment = "A model version comment"
mv.description = "Same as setting the comment"

from sklearn import metrics

test_accuracy = metrics.accuracy_score(test_labels, prediction)

test_confusion_matrix = metrics.confusion_matrix(test_labels, prediction)

# scalar metric
mv.set_metric("test_accuracy", test_accuracy)

# hierarchical (dictionary) metric
mv.set_metric("evaluation_info", {"dataset_used": "my_dataset", "accuracy": test_accuracy, "f1_score": f1_score})

# multivalent (matrix) metric
mv.set_metric("confusion_matrix", test_confusion_matrix)

metrics = mv.show_metrics()

mv.delete_metric("test_accuracy")

mv.export("~/mymodel/")

mv.export("~/mymodel/", export_mode=ExportMode.FULL)

clf = mv.load()

conda_env = session.file.get("snow://model/<modelName>/versions/<versionName>/runtimes/python_runtime/env/conda.yml", ".")
open("~/conda.yml", "w").write(conda_env)

conda env create --name newenv --file=~/conda.yml
conda activate newenv

clf = mv.load(options={"use_gpu": True})

Calling model methods¶

Model versions can have methods, which are attached functions that can be executed to perform inference or other model operations. The versions of a model can have different methods, and the signatures of these methods can also differ.

To call a method of a model version, use mv.run, where mv is a ModelVersion object. Specify the name of the function to be called and pass a Snowpark or pandas DataFrame that contains the inference data, along with any required parameters. The method is executed in a Snowflake warehouse.

The return value of the method is a Snowpark or pandas DataFrame, matching the type of DataFrame passed in. Snowpark DataFrames are evaluated lazily, so the method is run only when the DataFrame’s collect, show, or to_pandas method is called.

The following example illustrates running the predict method of a model. This model’s predict method does not require any parameters besides the inference data (test_features here). If it did, they would be passed as additional arguments after the inference data.

remote_prediction = mv.run(test_features, function_name="predict")
remote_prediction.show()   # assuming test_features is Snowpark DataFrame

To see what methods can be called on a given model, call mv.show_functions. The return value of this method is a list of ModelFunctionInfo objects. Each of these objects includes the following attributes:

• name: The name of the function that can be called from Python or SQL.

• target_method: The name of the Python method in the original logged model.

Sharing models¶

The model registry can store two types of models. You can distinguish them using the MODEL_TYPE column in the output of SHOW MODELS.

• CORTEX_FINETUNED: Models generated with Cortex Fine-tuning, which do not contain user code. To share this type of model, use Data Sharing.

• USER_MODEL: Models that contain user code, such as models developed using Snowpark ML modeling classes. These models cannot currently be shared. The ability to share models that contain user code will be available in a future release.

Cost considerations¶

Using the Snowflake Model Registry incurs standard Snowflake consumption-based costs. These include:

• Cost of storing model artifacts, metadata, and functions. For general information about storage costs, see Exploring storage cost.

• Cost of copying files between stages to Snowflake. See COPY FILES.

• Cost of serverless model object operations through the Snowsight UI or the SQL or Python interface, such as showing models and model versions and altering model comments, tags, and metrics.

• Warehouse compute costs, which vary depending on the type of model and the quantity of data used in inference. For general information about Snowflake compute costs, see Understanding compute cost. Warehouse compute costs are incurred for:

  • Model and version creation operations

  • Invoking a model’s methods