Cortex Training API reference

This reference covers CortexTrainingClient in the customer-facing cortex_training Python namespace. Install the package, then import the client:

uv pip install git+https://github.com/snowflakedb/cortex-training.git
from cortex_training import CortexTrainingClient, SubJobConfig, JobType
from cortex_training import (
    serialize_forward_backward_args,
    build_forward_backward_kwargs,
    build_forward_backward_payload,
)

Parameter descriptions below are taken from the SDK docstrings and method signatures in CortexTrainingClient and SubJobConfig.

CortexTrainingClient

HTTP client for the Cortex Training API.

Recommended construction for Snowflake accounts:

client = CortexTrainingClient.from_pat(
    host="myaccount.snowflakecomputing.com",
    pat="...",
    database="MY_DB",
    schema="PUBLIC",
)

Base URL path used for requests:

https://{host}/api/v2/databases/{database}/schemas/{schema}/cortex-training

from_pat

Deprecated: use from_configs. Same arguments as before (host, pat, …).

CortexTrainingClient.from_pat(*args, **kwargs)
ParameterTypeDefaultDescription
*argsAny()
**kwargsAny{}

Constructor options

from_pat forwards optional constructor kwargs to __init__. You can also construct a client directly for local/mock use by passing base_url.

CortexTrainingClient(
    base_url,
    database,
    schema,
    endpoint='cortex-training',
    poll_interval=0.5,
    poll_timeout=1800.0,
    poll_backoff_multiplier=1.25,
    poll_max_interval=6.0,
    pool_maxsize=1024,
    max_retries=10,
    request_timeout=_DEFAULT_REQUEST_TIMEOUT,
)
ParameterTypeDefaultDescription
base_urlstrrequired
databasestrrequired
schemastrrequired
endpointstr'cortex-training'
poll_intervalfloat0.5
poll_timeoutfloat1800.0
poll_backoff_multiplierfloat1.25
poll_max_intervalfloat6.0
pool_maxsizeint1024
max_retriesint10
request_timeoutfloat | tuple[float, float]_DEFAULT_REQUEST_TIMEOUT

Job lifecycle

create_job

Create a job from a list of sub-jobs and return its server job_id.

Each SubJobConfig is validated client-side before the request is sent (see SubJobConfig.validate). A job supports zero or one training sub-job and any number of sampling sub-jobs; log_probability sub-jobs are rejected here. job_id is optional; when omitted the server generates one. experiment_name is optional; when omitted the server auto-creates an experiment for the job. hardware is optional (Hardware.H200 / .B200 / .B300); when set, all sub-jobs in the job run on that GPU type. Omitted defaults to H200. idle_timeout_seconds bounds how long the job may sit idle before the server reclaims it: 0 disables reclamation, any other value must be between 300 and 604,800. pending_timeout_seconds bounds how long the job may wait for capacity before failing with pending_timeout: between 300 and 604,800, with no disable value. Omit either to use the server default, currently 30 minutes for idle and 24 hours for pending.

The calling role needs USE CORTEX TRAINING on the account, plus CREATE EXPERIMENT on the schema when experiment_name is omitted or MODIFY on that experiment when it is given. Without USE CORTEX TRAINING the service answers 403 and this raises requests.HTTPError; an ACCOUNTADMIN can grant it with GRANT USE CORTEX TRAINING ON ACCOUNT TO ROLE <role>.

client.create_job(
    sub_jobs,
    job_id=None,
    experiment_name=None,
    hardware=None,
    idle_timeout_seconds=None,
    pending_timeout_seconds=None,
)
ParameterTypeDefaultDescription
sub_jobslist[SubJobConfig]required
job_idstr | NoneNone
experiment_namestr | NoneNone
hardwareHardware | str | NoneNone
idle_timeout_secondsint | NoneNone
pending_timeout_secondsint | NoneNone

create_job_from_body

Create a job from a raw REST CreateJob request body.

This is useful for tooling that already has the REST JSON payload, while create_job remains the typed path for Python callers.

A job supports zero or one training sub-job and any number of sampling sub-jobs; a second training sub-job or any log_probability sub-job raises ValueError before the request is sent.

Every peft_config present in the body is canonicalized in place and an invalid one raises ValueError before the request is sent. The rest of the body, including optimizer, is forwarded as given.

client.create_job_from_body(body)
ParameterTypeDefaultDescription
bodydictrequired

wait_for_job

Poll until the job reaches RUNNING. Raises on terminal states or timeout.

client.wait_for_job(job_id)
ParameterTypeDefaultDescription
job_idstrrequired

get_job

client.get_job(job_id, retry_on=_is_transient)
ParameterTypeDefaultDescription
job_idstrrequired
retry_on—_is_transient

list_jobs

client.list_jobs(status=None)
ParameterTypeDefaultDescription
statusstr | NoneNone

cancel_job

client.cancel_job(job_id)
ParameterTypeDefaultDescription
job_idstrrequired

get_capacity

Return the calling account’s GPU ceiling and current usage.

Backed by the account-scoped endpoint /cortex-training/capacity (not under /{job_id}). The account is resolved server-side from the caller’s session — never from a request field — so a caller can only ever read its own account’s capacity. Optional hardware (Hardware.H200 / .B200 / .B300) is sent as a query parameter and scopes the numbers to that GPU type; omitted defaults to H200.

The returned dict always carries all six fields. The server emits proto3 JSON, which omits zero/false fields (an account holding nothing under a zero ceiling is literally {}), so we fill in the documented defaults:

  • has_reservation (bool): whether max_total_gpus is a guaranteed commitment rather than a best-effort per-account cap. When False the account draws on the shared pool, so available_gpus is an upper bound.
  • max_total_gpus (int): the account’s ceiling — its commitment when has_reservation, else its per-account cap. -1 means no ceiling, 0 a real quota of zero, > 0 the limit. This is the canonical ceiling; prefer it over reserved_gpus.
  • reserved_gpus (int, deprecated): mirrors max_total_gpus when has_reservation is True, else 0.
  • in_use_gpus (int): GPUs the account holds — jobs in placing, initializing, or running. Queued work is not counted here; it is in pending_gpus.
  • pending_gpus (int): GPUs requested by jobs still waiting for capacity (pending). It claims quota, so it reduces available_gpus.
  • available_gpus (int): what a new job could start with right now, max_total_gpus - in_use_gpus - pending_gpus, never below 0. When has_reservation is True, it is also capped at what the hardware can currently schedule, so a shortfall in your reserved capacity shows here.

See docs/reference/rest-api.md section 5.5 for the authoritative field list, and get_capacity_breakdown to split in_use_gpus and pending_gpus by sub-job type or by submitting user.

client.get_capacity(hardware=None)
ParameterTypeDefaultDescription
hardwareHardware | str | NoneNone

get_experiment_run

Return {experiment_name, experiment_run_name} for job_id.

Backed by the endpoint /cortex-training/{job_id}/experiment-run.

client.get_experiment_run(job_id)
ParameterTypeDefaultDescription
job_idstrrequired

Training

These methods submit async requests. Each returns a request_id you pass to poll_request.

forward_backward

Submit a forward+backward pass. Returns request_id.

client.forward_backward(job_id, data)
ParameterTypeDefaultDescription
job_idstrrequired
databytesrequired

step

Submit an optimizer step. Returns request_id.

client.step(job_id, learning_rate=None)
ParameterTypeDefaultDescription
job_idstrrequired
learning_ratefloat | NoneNone

poll_request

Poll until the async request completes. Returns the result dict.

client.poll_request(job_id, request_id)
ParameterTypeDefaultDescription
job_idstrrequired
request_idstrrequired

get_request_status

client.get_request_status(
    job_id,
    request_id,
    max_events=None,
    cursor=None,
)
ParameterTypeDefaultDescription
job_idstrrequired
request_idstrrequired
max_eventsint | NoneNone
cursorstr | NoneNone

Checkpoints

save

Submit a checkpoint save. Returns request_id.

client.save(job_id, checkpoint_id=None, checkpoint_type=None)
ParameterTypeDefaultDescription
job_idstrrequired
checkpoint_idstr | NoneNone
checkpoint_typestr | NoneNone

load

Submit a checkpoint load into an existing job. Returns request_id.

client.load(
    job_id,
    checkpoint_id,
    source_job_id=None,
    target_sub_job_id=None,
)
ParameterTypeDefaultDescription
job_idstrrequiredThe job to load the checkpoint into.
checkpoint_idstrrequiredCheckpoint identifier (from a prior save).
source_job_idstr | NoneNoneOptional. Load from another job’s checkpoint store.
target_sub_job_idstr | NoneNoneOptional. Route the load to a specific training sub-job. Format: "{job_id}:training:{index}". Omit to use the session’s default training sub-job.

Returns str: request_id for polling via poll_request().

list_checkpoints

client.list_checkpoints(job_id)
ParameterTypeDefaultDescription
job_idstrrequired

export_checkpoint

client.export_checkpoint(job_id, checkpoint_id)
ParameterTypeDefaultDescription
job_idstrrequired
checkpoint_idstrrequired

delete_checkpoint

Delete a checkpoint. Returns None on success.

client.delete_checkpoint(job_id, checkpoint_id)
ParameterTypeDefaultDescription
job_idstrrequired
checkpoint_idstrrequired

Inference

generate

Submit a generate request to a sampling sub-job.

Returns a GenerateHandle when wait_seconds is not None; with None (the default) it returns the request_id.

The typed fields are serialized as a DSSST1 safetensors frame and sent as application/octet-stream. Oversized frames are byte-chunked across multiple POSTs; the final POST returns the pollable request id. Poll with poll_request.

Prompt-length validation is asymmetric by input type:

  • Pre-tokenized prompts are checked client-side against the sampling sub-job’s max_seq_len and fail fast with a ValueError when len(prompt) >= max_seq_len — the token count is exact, so this never rejects a prompt vLLM would accept. A flat list[int], and a bare 1-D tensor, each count as one prompt.
  • String prompts are not validated here. Counting their tokens would require replicating the server tokenizer (including special tokens), so an over-long string instead fails server-side in vLLM with a maximum model length error rather than client-side.

wait_seconds asks the server to hold the call for up to that many whole seconds and reply with the result if the generation ends in time. The server caps the wait, and holds the call only if it has waiting enabled; if not, it answers right away with just the request id, and the retry rule below still applies. A finished result in the reply makes the handle’s result what poll_request would return; it is None when the generation was still running, the server did not wait, or the reply could not be read whole, and the caller polls request_id as usual. A chunked result counts as not whole: the chunks that came with the reply are dropped, and poll_request fetches them all again starting with the first. A generation that failed or was cancelled within the wait raises the error poll_request raises. 0 sends no wait and returns a handle with no result. Use a result that is set rather than polling again: like a second poll of any generate, a re-poll leaves the tensors under results as tensors, and the server may have dropped the result by then.

A waiting submit is retried only on failures that come before any generation starts: a connection that could not be opened (refused or connect timeout; a TLS failure does not count), and HTTP 429, 404, 409, 307 or 308. A job-not-found 404 is still retried only within 120 seconds of this client creating the job, and a chunk-group 409 is not retried. A 5xx, a read timeout, or a reset after send is raised, because the generation may already be running and a retry would run it twice.

client.generate(
    job_id,
    prompts,
    sampling_params=None,
    routing_key=None,
    strict=None,
    sub_job_id=None,
    wait_seconds=None,
)
ParameterTypeDefaultDescription
job_idstrrequired
promptslist[str | list[int]]required
sampling_paramsdict | list[dict | None] | NoneNone
routing_keystr | list[str | None] | NoneNone
strictbool | NoneNone
sub_job_idstr | NoneNoneOptional. Which sampling sub-job runs the request, e.g. "{job_id}:sampling:1"; discover the ids from get_job’s sub_jobs. Sent as a query parameter, since the body is an opaque frame. If omitted, no sub_job_id is sent and the server chooses the sub-job; currently that is the job’s only sampling sub-job, and a job with several rejects the request and lists the valid ids.
wait_secondsint | NoneNone

weight_sync

Sync model weights from a training sub-job into one or more sampling sub-jobs (used in RL training loops). Returns request_id.

Multi-sub-job sessions require the operation envelope to include a routing hint; by default the operation is routed through the source training sub-job.

client.weight_sync(
    job_id,
    source_sub_job_id,
    target_sub_job_ids,
    weight_format=None,
    bucket_size=None,
    sub_job_id=None,
    sub_job_type=None,
)
ParameterTypeDefaultDescription
job_idstrrequiredThe job owning both sub-jobs.
source_sub_job_idstrrequiredTraining sub-job to read weights from, e.g. f"{job_id}:training:0".
target_sub_job_idslist[str]requiredSampling sub-jobs to update, e.g. [f"{job_id}:sampling:0"].
weight_formatstr | NoneNone"vllm" (server default) or "hf" for full weights, or "lora" to broadcast only the adapter tensors.
bucket_sizeint | NoneNoneOptional transfer-bucket size in bytes. Increase this when one model tensor exceeds the server default bucket.
sub_job_idstr | NoneNone
sub_job_typestr | NoneNone

Model deployments

A model deployment serves a base model or a weights-only checkpoint as an OpenAI-compatible chat completions endpoint. These methods manage the deployment; call the endpoint itself with any OpenAI-compatible client. Only EVALUATION deployments are supported, and a deployment holds its GPUs until you drop it.

A deployment is identified by an ID of the form sf-ct-<UUID>. Pass it as model when you call the endpoint.

Deployment spec

create_model_deployment takes a spec with exactly three top-level fields:

{
  "deployment": {"class": "EVALUATION", "min_replicas": 1, "gpus_per_replica": 1, "gpu_type": "H200"},
  "source": {"type": "base_model", "model": "Qwen/Qwen3-1.7B"},
  "inference": {"max_seq_len": 8192}
}
FieldRule
deployment.class"EVALUATION". No other class is supported.
deployment.gpu_typeH200 or B300, case-insensitive.
deployment.gpus_per_replica1, 2, 4, or 8. Each replica’s tensor-parallel size.
deployment.min_replicasPositive integer. min_replicas * gpus_per_replica must be at most 8 or a multiple of 8.
source.typebase_model (with model) or checkpoint (with job_id and checkpoint_id).
inference.max_seq_lenMaximum sequence length the deployment serves.

A checkpoint source must name a completed weights-only checkpoint and doesn’t name a model: the client reads the model from the source training job. Which models can be deployed, and the minimum GPUs per replica each needs, is decided by the server.

The checkpoint can come from a full fine-tune or a LoRA training job. For a LoRA checkpoint, the deployment serves the base model with your trained adapter, using the LoRA settings from the training job. You can’t set LoRA settings in the spec.

Deployment statuses

A deployment reports one of these statuses:

StatusMeaning
PENDINGWaiting for GPU capacity.
INITIALIZINGGPUs are assigned and the model is loading.
READYThe endpoint is serving requests.
FAILEDThe deployment failed. The metadata includes failure: {code, message}.
DROPPINGThe deployment is being dropped.
DROPPEDThe deployment was dropped and holds no GPUs.
TERMINATEDThe deployment was torn down and holds no GPUs.

Calling the endpoint

The endpoint is POST {base_url}/chat/completions, where base_url is https://{host}/api/v2/databases/{database}/schemas/{schema}/cortex-training/inference/v1. The client exposes it as the model_deployment_base_url property, and wait_for_model_deployment returns it as base_url. The endpoint behaves as follows:

ItemBehavior
AuthenticationAuthorization: Bearer <PAT> and X-Snowflake-Authorization-Token-Type: PROGRAMMATIC_ACCESS_TOKEN.
modelThe deployment ID, sf-ct-<UUID>.
Length limitmax_completion_tokens. The endpoint rejects max_tokens with HTTP 400 (unsupported_parameter).
StreamingWith "stream": true, the response is server-sent events: one data: line per chat.completion.chunk, then data: [DONE]. Add "stream_options": {"include_usage": true} to get a final chunk with token counts.
Not readyHTTP 503 (endpoint_failed) while the deployment is PENDING or INITIALIZING, and briefly after it’s dropped.
Not foundHTTP 404 (model_not_found) for an unknown deployment, or one that failed or was dropped.
Not enabledHTTP 403 or 405 while inference isn’t enabled for the account.

Without "stream": true, the response is one chat.completion object.

create_model_deployment

Create a model deployment from a deployment spec and return its metadata, including the deployment ID. The deployment starts in PENDING; call wait_for_model_deployment to wait until it’s READY.

client.create_model_deployment(spec)
ParameterTypeDefaultDescription
specModelDeploymentSpec | Mapping[str, Any]required

wait_for_model_deployment

Poll until the deployment is READY and return its metadata, including base_url. Raises ModelDeploymentWaitError if the deployment reaches FAILED, TERMINATED, or DROPPED, and ModelDeploymentWaitTimeout when the client’s poll_timeout passes. A timeout doesn’t drop the deployment, which keeps holding GPUs until you drop it.

client.wait_for_model_deployment(deployment_id)
ParameterTypeDefaultDescription
deployment_idstrrequired

get_model_deployment

Return the metadata for one deployment, including deployment, status, model_source, and the capacity fields. base_url is included unless the deployment is FAILED, DROPPING, DROPPED, or TERMINATED, and failure is included for a FAILED or TERMINATED deployment.

client.get_model_deployment(deployment_id)
ParameterTypeDefaultDescription
deployment_idstrrequired

describe_model_deployment

Return one deployment’s status, endpoint, and configuration. The configuration is grouped under spec in the same deployment, source, and inference shape as the create spec, but it’s a read view and isn’t guaranteed to be a valid create_model_deployment input. A deployment created from a checkpoint currently reports a base_model source.

client.describe_model_deployment(deployment_id)
ParameterTypeDefaultDescription
deployment_idstrrequired

list_model_deployments

Return the metadata for every deployment in the client’s database and schema. Pass status (for example, "READY" or ["PENDING", "INITIALIZING"], case-insensitive) to keep only deployments in those statuses. Entries don’t include base_url; use get_model_deployment to get it.

client.list_model_deployments(status=None)
ParameterTypeDefaultDescription
statusstr | Iterable[str] | NoneNone

drop_model_deployment

Drop a deployment and release its GPUs. Safe to retry: a deployment that’s already DROPPING or DROPPED is reported as dropped. A FAILED or TERMINATED deployment holds no GPUs and can’t be dropped, so the method returns {"dropped": False, "status": ...} instead of raising an error.

client.drop_model_deployment(deployment_id)
ParameterTypeDefaultDescription
deployment_idstrrequired

download_model_deployment_logs

Download a deployment’s logs into output_dir. log_type is "execution" (the default) or "stdout". Returns {"deployment", "logs"}, where logs lists each saved file’s filename and saved_path (plus chunk_count for stdout). Use these logs to find out why a deployment FAILED.

client.download_model_deployment_logs(deployment_id, output_dir, log_type='execution')
ParameterTypeDefaultDescription
deployment_idstrrequired
output_dirstr | os.PathLike[str]required
log_typestr'execution'

download_model_deployment_metrics

Download a deployment’s metrics into output_dir. Returns {"deployment", "metrics"}, where metrics lists each saved file’s filename, saved_path, and chunk_count.

client.download_model_deployment_metrics(deployment_id, output_dir)
ParameterTypeDefaultDescription
deployment_idstrrequired
output_dirstr | os.PathLike[str]required

Logs

fetch_execution_logs

Download every log file under the job’s experiment run stage.

Returns a list of {"sub_job_id", "filename", "artifact_uri", "content"} entries, one per object found under a _logs/<sub_job_id>/ subtree in the run’s stage. Filenames such as execution.jsonl and server.log are both included; bodies are decoded as UTF-8.

sub_job_id and filename, when given, download only the files of that sub-job and with that name (for example one sub-job’s execution.jsonl); the rest are listed but not fetched.

artifact_uri is the logical snow:// URI. Files are retrieved through Snowflake’s supported experiment artifact LIST/GET interface; no object-store credentials are exposed to the caller.

client.fetch_execution_logs(job_id, sub_job_id=None, filename=None)
ParameterTypeDefaultDescription
job_idstrrequired
sub_job_idstr | NoneNone
filenamestr | NoneNone

stream_logs

Yield a sub-job’s log entries, polling with backoff.

Drains the log then, with follow=True (the default), keeps polling for new entries until the caller stops iterating — suitable for a live tail. With follow=False it stops at the first empty page, once it has caught up with the log (whether or not the server reports eof).

The stream also ends, without an error, once the log can grow no more: at an empty page that reports eof, or when the job ends after the stream has read from it. A job that had already ended before the first read raises, as tail_logs does.

client.stream_logs(
    job_id,
    sub_job_id=None,
    sub_job_type=None,
    cursor=None,
    max_lines=None,
    poll_interval=None,
    follow=True,
)
ParameterTypeDefaultDescription
job_idstrrequired
sub_job_idstr | NoneNone
sub_job_typestr | NoneNone
cursorstr | NoneNone
max_linesint | NoneNone
poll_intervalfloat | NoneNone
followboolTrue

Returns Iterator[dict].

SubJobConfig

One sub-job within a CreateJob request.

training_job

Build a training SubJobConfig. All training fields are explicit.

SubJobConfig.training_job(
    model_name,
    optimizer,
    max_seq_len,
    train_batch_size,
    n_gpus,
    gradient_clipping=None,
    multiplex_job_id=None,
    load_optimizer_states=None,
    extra_training=None,
    global_batch_size=None,
    dtype=None,
    seed=None,
    model_post_init=None,
    source_checkpoint_info=None,
)
ParameterTypeDefaultDescription
model_namestrrequiredModel identifier (e.g., “Qwen/Qwen3-1.7B”).
optimizerdictrequiredOptimizer config dict (e.g., {“name”: “AdamW”, “lr”: 0.0001}).
max_seq_lenintrequiredMaximum sequence length for training.
train_batch_sizeintrequiredPer-device batch size.
n_gpusintrequiredNumber of GPUs (DP size).
gradient_clippingfloat | NoneNoneOptional gradient clipping threshold.
multiplex_job_idstr | NoneNoneOptional multiplex job identifier.
load_optimizer_statesbool | NoneNoneWhether to load optimizer states from checkpoints. Set to False when loading across different DP sizes (n_gpus) since optimizer states are DP-sharded and cannot be resized. When None, uses server default (True). This is a creation-time setting and cannot be changed later.
extra_trainingdict | NoneNoneAdditional training config keys (passthrough).
global_batch_sizeint | NoneNoneGlobal batch size across all GPUs.
dtypestr | NoneNoneModel dtype (e.g., “bfloat16”).
seedint | NoneNoneRandom seed.
model_post_initlist[str] | NoneNoneList of post-initialization operations.
source_checkpoint_infodict | NoneNoneOptional checkpoint to resume from at creation time (use {"checkpoint_id": "...", "source_job_id": "..."}).

sampling_job

Build a sampling SubJobConfig.

job_type remains for source compatibility, but only SAMPLING is currently accepted.

SubJobConfig.sampling_job(
    model_name,
    max_seq_len,
    n_gpus,
    multiplex_job_id=None,
    extra_sampling=None,
    job_type=JobType.SAMPLING,
    global_batch_size=None,
    dtype=None,
    seed=None,
    model_post_init=None,
    source_checkpoint_info=None,
)
ParameterTypeDefaultDescription
model_namestrrequired
max_seq_lenintrequired
n_gpusintrequired
multiplex_job_idstr | NoneNone
extra_samplingdict | NoneNone
job_typeJobTypeJobType.SAMPLING
global_batch_sizeint | NoneNone
dtypestr | NoneNone
seedint | NoneNone
model_post_initlist[str] | NoneNone
source_checkpoint_infodict | NoneNone

JobType

class JobType(str, Enum):
    TRAINING = "training"
    SAMPLING = "sampling"
    LOG_PROBABILITY = "log_probability"

Payload helpers

Helpers that build the forward and backward pass payload in the client’s wire format.

serialize_forward_backward_args

Serialize {"args": args, "kwargs": kwargs} for Cortex Training fwd-bwd.

serialize_forward_backward_args(args=(), kwargs=None)
ParameterTypeDefaultDescription
argstuple()
kwargsdict[str, Any] | NoneNone

build_forward_backward_kwargs

Build tensor kwargs for a forward-backward training batch.

payload is intentionally JSON-shaped so CLI examples can stay readable: callers may provide either tokenizable texts plus a tokenizer or direct tensor data under input_ids / labels / position_ids.

build_forward_backward_kwargs(payload)
ParameterTypeDefaultDescription
payloaddict[str, Any]required

build_forward_backward_payload

Build a serialized fwd-bwd byte payload from readable JSON.

build_forward_backward_payload(spec)
ParameterTypeDefaultDescription
specdict[str, Any]required