CUDA-X Libraries in Snowflake ML¶

Large-Scale topic modeling¶

Topic modeling on large and feature-rich data sets requires:

• Using embedding models

• Applying dimensionality reduction at scale

• Using clustering and visualization to extract accurate and relevant topics

GPU parallelism can help you accomplish the preceding workflows more efficiently. By accelerating your processing with cuML, you can transform millions of product reviews from raw text to well-defined topic clusters that can be reduced from hours on CPU to minutes on GPU with no modifications to existing Python code. This highlights the seamless drop-in acceleration for UMAP and HDBSCAN libraries.

For more information about performing topic modeling over GPUs on Snowflake, see https://quickstarts.snowflake.com/guide/accelerate-topic-modeling-with-gpus-in-snowflake-ml/#0

Computational Genomics Workflows¶

Use Snowflake’s CUDA-X integrations to significantly accelerate the processing of biological sequences. You can convert DNA sequences into feature vectors for scalable classification tasks, such as predicting gene families.

Executing pandas and scikit-learn code directly on GPUs with cuDF and cuML speeds up data loading, preprocessing, and ensemble model training. This GPU acceleration for existing workflows, without code changes, allows researchers to prioritize biological insights and model design over low-level GPU programming.

Applied Use Case: Topic Modeling at Scale¶

Computational efficiency is crucial for large scale text analysis and topic modeling. GPUs use parallel processing to reduce processing time from hours to minutes. This section demonstrates how to accelerate ML models on a dataset of 200,000 beauty product reviews using GPU acceleration with CUDA-X.

You can use CUDA-X to do the following:

• Transform raw text into numerical representations (embeddings) for machine learning.

• Accelerate dimensionality reduction

To utilize the CUDA libraries, add %load_ext cuml.accel at the beginning of your code. This reduces your processing time from hours to minutes.

The following example code uses the SentenceTransformer class to create embeddings.

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
embeddings = model.encode(texts, show_progress_bar=True)

The following example code uses HDBSCAN to reduce high-dimensional data. It retains the cluster topics.

from umap import UMAP
from hdbscan import HDBSCAN
umap_model = UMAP(n_components=15, n_neighbors=15, min_dist=0.0)
hdbscan_model = HDBSCAN(min_cluster_size=100, gen_min_span_tree=True, prediction_data=True)

Applied Use Case: Running complex genomics workflows¶

Gene family organization, which includes paralogs and orthologs, is crucial for understanding gene evolution, function, and conserved biological processes.

With the CUDA-X libraries, you can create a classification model to predict gene families from DNA sequences. This model can accelerate genomic annotation, identify novel gene functions, and provide insights into evolutionary pathways.

The dataset has a series of plain text nucleotide sequences and their corresponding gene family class labels. The classes correspond to seven distinct human gene families.

The following code uses the nucleotide transformer from Hugging Face to convert the DNA sequences into vectors. The transformer tokenizes and batches the sequences to transform each gene sequence into a 1280-feature vector.

%load_ext cudf.pandas
%load_ext cuml.accel

from transformers import AutoTokenizer, AutoModelForMaskedLM
import torch

def get_dna_embeddings(sequences, classes):
    tokens_ids = tokenizer.batch_encode_plus(sequences, return_tensors="pt", padding="longest")["input_ids"].to('cuda:0')

    attention_mask = tokens_ids != tokenizer.pad_token_id
    try:
        torch_outs = model(
            tokens_ids,
            attention_mask=attention_mask,
            encoder_attention_mask=attention_mask,
            output_hidden_states=True
        )
    except:
        return []

    embeddings = torch_outs['hidden_states'][-1].detach()
    attention_mask = torch.unsqueeze(attention_mask, dim=-1)
    mean_sequence_embeddings = torch.sum(attention_mask*embeddings, axis=-2)/torch.sum(attention_mask, axis=1)
    return list(zip(mean_sequence_embeddings.numpy(), classes))

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("InstaDeepAI/nucleotide-transformer-500m-human-ref")
model = AutoModelForMaskedLM.from_pretrained("InstaDeepAI/nucleotide-transformer-500m-human-ref")

# Example of obtaining embeddings (simplified)
sequences = ["ATGCCCCAACTAAATACTACCGTATGGCCCACCATAATTACCCCCA", ...]
classes = [0, ...]


genes = []
batch_size=10

emb = get_dna_embeddings(human_genes[i], human_classes[i])
genes += emb

You can use the following code to evaluate two ensemble classification models:

• A Random Forest Classifier

• An XGBoost classifier

%load_ext cudf.pandas
%load_ext cuml.accel

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

human_dna = pd.read_table(human_url) # This would now run on GPU

genes = []
batch_size=10

human_genes = human_dna['sequence'].tolist()
human_classes = human_dna['class'].tolist()

human_genes = [human_genes[i:i + batch_size] for i in range(0, len(human_genes), batch_size)]
human_classes = [human_classes[i:i + batch_size] for i in range(0, len(human_classes), batch_size)]

# Create the embeddings
for i in tqdm(range(len(human_genes)), desc='Producing embeddings...'):
    emb = get_dna_embeddings(human_genes[i], human_classes[i])
    genes += emb

genes_df = pd.DataFrame(genes, columns=['embeddings', 'class'])
genes_df[[f'emb_{i}' for i in range(1280)]] = pd.DataFrame(genes_df['embeddings'].tolist(), index=genes_df.index) # the embeddings generated above

X, y = genes_df[[f'emb_{i}' for i in range(1280)]], genes_df['class']
X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                   test_size = 0.20,
                                                   random_state=42)

classifier = RandomForestClassifier(n_estimators=200, max_depth=20, max_features=1.0, n_jobs=-1)
classifier.fit(X_train, y_train)