Cortex Training API reference¶
This reference covers CortexTrainingClient in the customer-facing
cortex_training Python namespace. Install the package, then import the client:
Parameter descriptions below are taken from the SDK docstrings and method signatures in
CortexTrainingClient and SubJobConfig.
CortexTrainingClient¶
HTTP client for the Cortex Training API.
Recommended construction for Snowflake accounts:
Base URL path used for requests:
from_ pat¶
Deprecated: use from_configs. Same arguments as before (host, pat, …).
| Parameter | Type | Default | Description |
|---|---|---|---|
*args | Any | () | |
**kwargs | Any | {} |
Constructor options¶
from_pat forwards optional constructor kwargs to __init__. You can also construct a client directly for local/mock use by passing base_url.
| Parameter | Type | Default | Description |
|---|---|---|---|
base_url | str | required | |
database | str | required | |
schema | str | required | |
endpoint | str | 'cortex-training' | |
poll_interval | float | 0.5 | |
poll_timeout | float | 1800.0 | |
poll_backoff_multiplier | float | 1.25 | |
poll_max_interval | float | 6.0 | |
pool_maxsize | int | 1024 | |
max_retries | int | 10 | |
request_timeout | float | tuple[float, float] | _DEFAULT_REQUEST_TIMEOUT |
Job lifecycle¶
create_ job¶
Create a job from a list of sub-jobs and return its server job_id.
Each SubJobConfig is validated client-side before the request
is sent (see SubJobConfig.validate). A job supports zero or one
training sub-job and any number of sampling sub-jobs;
log_probability sub-jobs are rejected here. job_id is optional;
when omitted the server generates one. experiment_name is optional;
when omitted the server auto-creates an experiment for the job.
hardware is optional (Hardware.H200 / .B200 / .B300); when
set, all sub-jobs in the job run on that GPU type. Omitted defaults to
H200.
idle_timeout_seconds bounds how long the job may sit idle before the
server reclaims it: 0 disables reclamation, any other value must be
between 300 and 604,800. pending_timeout_seconds bounds how long the
job may wait for capacity before failing with pending_timeout: between
300 and 604,800, with no disable value. Omit either to use the server
default, currently 30 minutes for idle and 24 hours for pending.
The calling role needs USE CORTEX TRAINING on the account, plus
CREATE EXPERIMENT on the schema when experiment_name is omitted
or MODIFY on that experiment when it is given. Without
USE CORTEX TRAINING the service answers 403 and this raises
requests.HTTPError; an ACCOUNTADMIN can grant it with
GRANT USE CORTEX TRAINING ON ACCOUNT TO ROLE <role>.
| Parameter | Type | Default | Description |
|---|---|---|---|
sub_jobs | list[SubJobConfig] | required | |
job_id | str | None | None | |
experiment_name | str | None | None | |
hardware | Hardware | str | None | None | |
idle_timeout_seconds | int | None | None | |
pending_timeout_seconds | int | None | None |
create_ job_ from_ body¶
Create a job from a raw REST CreateJob request body.
This is useful for tooling that already has the REST JSON payload,
while create_job remains the typed path for Python callers.
A job supports zero or one training sub-job and any number of
sampling sub-jobs; a second training sub-job or any
log_probability sub-job raises ValueError before the
request is sent.
Every peft_config present in the body is canonicalized in place and
an invalid one raises ValueError before the request is sent.
The rest of the body, including optimizer, is forwarded as given.
| Parameter | Type | Default | Description |
|---|---|---|---|
body | dict | required |
wait_ for_ job¶
Poll until the job reaches RUNNING. Raises on terminal states or timeout.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required |
get_ job¶
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
retry_on | — | _is_transient |
list_ jobs¶
| Parameter | Type | Default | Description |
|---|---|---|---|
status | str | None | None |
cancel_ job¶
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required |
get_ capacity¶
Return the calling account’s GPU ceiling and current usage.
Backed by the account-scoped endpoint /cortex-training/capacity
(not under /{job_id}). The account is resolved server-side from the
caller’s session — never from a request field — so a caller can only
ever read its own account’s capacity. Optional hardware
(Hardware.H200 / .B200 / .B300) is sent as a query parameter and
scopes the numbers to that GPU type; omitted defaults to H200.
The returned dict always carries all six fields. The server emits
proto3 JSON, which omits zero/false fields (an account holding nothing
under a zero ceiling is literally {}), so we fill in the documented
defaults:
has_reservation(bool): whethermax_total_gpusis a guaranteed commitment rather than a best-effort per-account cap. WhenFalsethe account draws on the shared pool, soavailable_gpusis an upper bound.max_total_gpus(int): the account’s ceiling — its commitment whenhas_reservation, else its per-account cap.-1means no ceiling,0a real quota of zero,> 0the limit. This is the canonical ceiling; prefer it overreserved_gpus.reserved_gpus(int, deprecated): mirrorsmax_total_gpuswhenhas_reservationisTrue, else 0.in_use_gpus(int): GPUs the account holds — jobs inplacing,initializing, orrunning. Queued work is not counted here; it is inpending_gpus.pending_gpus(int): GPUs requested by jobs still waiting for capacity (pending). It claims quota, so it reducesavailable_gpus.available_gpus(int): what a new job could start with right now,max_total_gpus - in_use_gpus - pending_gpus, never below0. Whenhas_reservationisTrue, it is also capped at what the hardware can currently schedule, so a shortfall in your reserved capacity shows here.
See docs/reference/rest-api.md section 5.5 for the authoritative
field list, and get_capacity_breakdown to split in_use_gpus
and pending_gpus by sub-job type or by submitting user.
| Parameter | Type | Default | Description |
|---|---|---|---|
hardware | Hardware | str | None | None |
get_ experiment_ run¶
Return {experiment_name, experiment_run_name} for job_id.
Backed by the endpoint /cortex-training/{job_id}/experiment-run.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required |
Training¶
These methods submit async requests. Each returns a request_id you pass to poll_request.
forward_ backward¶
Submit a forward+backward pass. Returns request_id.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
data | bytes | required |
step¶
Submit an optimizer step. Returns request_id.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
learning_rate | float | None | None |
poll_ request¶
Poll until the async request completes. Returns the result dict.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
request_id | str | required |
get_ request_ status¶
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
request_id | str | required | |
max_events | int | None | None | |
cursor | str | None | None |
Checkpoints¶
save¶
Submit a checkpoint save. Returns request_id.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
checkpoint_id | str | None | None | |
checkpoint_type | str | None | None |
load¶
Submit a checkpoint load into an existing job. Returns request_id.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | The job to load the checkpoint into. |
checkpoint_id | str | required | Checkpoint identifier (from a prior save). |
source_job_id | str | None | None | Optional. Load from another job’s checkpoint store. |
target_sub_job_id | str | None | None | Optional. Route the load to a specific training sub-job. Format: "{job_id}:training:{index}". Omit to use the session’s default training sub-job. |
Returns str: request_id for polling via poll_request().
list_ checkpoints¶
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required |
export_ checkpoint¶
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
checkpoint_id | str | required |
delete_ checkpoint¶
Delete a checkpoint. Returns None on success.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
checkpoint_id | str | required |
Inference¶
generate¶
Submit a generate request to a sampling sub-job.
Returns a GenerateHandle when wait_seconds is not
None; with None (the default) it returns the request_id.
The typed fields are serialized as a DSSST1 safetensors frame and sent
as application/octet-stream. Oversized frames are byte-chunked
across multiple POSTs; the final POST returns the pollable request id.
Poll with poll_request.
Prompt-length validation is asymmetric by input type:
- Pre-tokenized prompts are checked client-side against the
sampling sub-job’s
max_seq_lenand fail fast with aValueErrorwhenlen(prompt) >= max_seq_len— the token count is exact, so this never rejects a prompt vLLM would accept. A flatlist[int], and a bare 1-D tensor, each count as one prompt. - String prompts are not validated here. Counting their tokens
would require replicating the server tokenizer (including special
tokens), so an over-long string instead fails server-side in vLLM
with a
maximum model lengtherror rather than client-side.
wait_seconds asks the server to hold the call for up to that many
whole seconds and reply with the result if the generation ends in
time. The server caps the wait, and holds the call only if it has
waiting enabled; if not, it answers right away with just the request
id, and the retry rule below still applies. A finished result in the
reply makes the handle’s result what poll_request would
return; it is None when the generation was still running, the
server did not wait, or the reply could not be read whole, and the
caller polls request_id as usual. A chunked result counts as not
whole: the chunks that came with the reply are dropped, and
poll_request fetches them all again starting with the first.
A generation that failed or was cancelled
within the wait raises the error poll_request raises. 0
sends no wait and returns a handle with no result. Use a result
that is set rather than polling again: like a second poll of any
generate, a re-poll leaves the tensors under results as tensors,
and the server may have dropped the result by then.
A waiting submit is retried only on failures that come before any generation starts: a connection that could not be opened (refused or connect timeout; a TLS failure does not count), and HTTP 429, 404, 409, 307 or 308. A job-not-found 404 is still retried only within 120 seconds of this client creating the job, and a chunk-group 409 is not retried. A 5xx, a read timeout, or a reset after send is raised, because the generation may already be running and a retry would run it twice.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
prompts | list[str | list[int]] | required | |
sampling_params | dict | list[dict | None] | None | None | |
routing_key | str | list[str | None] | None | None | |
strict | bool | None | None | |
sub_job_id | str | None | None | Optional. Which sampling sub-job runs the request, e.g. "{job_id}:sampling:1"; discover the ids from get_job’s sub_jobs. Sent as a query parameter, since the body is an opaque frame. If omitted, no sub_job_id is sent and the server chooses the sub-job; currently that is the job’s only sampling sub-job, and a job with several rejects the request and lists the valid ids. |
wait_seconds | int | None | None |
weight_ sync¶
Sync model weights from a training sub-job into one or more sampling sub-jobs (used in RL training loops). Returns request_id.
Multi-sub-job sessions require the operation envelope to include a routing hint; by default the operation is routed through the source training sub-job.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | The job owning both sub-jobs. |
source_sub_job_id | str | required | Training sub-job to read weights from, e.g. f"{job_id}:training:0". |
target_sub_job_ids | list[str] | required | Sampling sub-jobs to update, e.g. [f"{job_id}:sampling:0"]. |
weight_format | str | None | None | "vllm" (server default) or "hf" for full weights, or "lora" to broadcast only the adapter tensors. |
bucket_size | int | None | None | Optional transfer-bucket size in bytes. Increase this when one model tensor exceeds the server default bucket. |
sub_job_id | str | None | None | |
sub_job_type | str | None | None |
Model deployments¶
A model deployment serves a base model or a weights-only checkpoint as an
OpenAI-compatible chat completions endpoint. These methods manage the
deployment; call the endpoint itself with any OpenAI-compatible client. Only
EVALUATION deployments are supported, and a deployment holds its GPUs until
you drop it.
A deployment is identified by an ID of the form sf-ct-<UUID>. Pass it as
model when you call the endpoint.
Deployment spec¶
create_model_deployment takes a spec with exactly three top-level fields:
| Field | Rule |
|---|---|
deployment.class | "EVALUATION". No other class is supported. |
deployment.gpu_type | H200 or B300, case-insensitive. |
deployment.gpus_per_replica | 1, 2, 4, or 8. Each replica’s tensor-parallel size. |
deployment.min_replicas | Positive integer. min_replicas * gpus_per_replica must be at most 8 or a multiple of 8. |
source.type | base_model (with model) or checkpoint (with job_id and checkpoint_id). |
inference.max_seq_len | Maximum sequence length the deployment serves. |
A checkpoint source must name a completed weights-only checkpoint and doesn’t
name a model: the client reads the model from the source training job. Which
models can be deployed, and the minimum GPUs per replica each needs, is
decided by the server.
The checkpoint can come from a full fine-tune or a LoRA training job. For a LoRA checkpoint, the deployment serves the base model with your trained adapter, using the LoRA settings from the training job. You can’t set LoRA settings in the spec.
Deployment statuses¶
A deployment reports one of these statuses:
| Status | Meaning |
|---|---|
PENDING | Waiting for GPU capacity. |
INITIALIZING | GPUs are assigned and the model is loading. |
READY | The endpoint is serving requests. |
FAILED | The deployment failed. The metadata includes failure: {code, message}. |
DROPPING | The deployment is being dropped. |
DROPPED | The deployment was dropped and holds no GPUs. |
TERMINATED | The deployment was torn down and holds no GPUs. |
Calling the endpoint¶
The endpoint is POST {base_url}/chat/completions, where base_url is
https://{host}/api/v2/databases/{database}/schemas/{schema}/cortex-training/inference/v1.
The client exposes it as the model_deployment_base_url property, and
wait_for_model_deployment returns it as base_url. The endpoint behaves as follows:
| Item | Behavior |
|---|---|
| Authentication | Authorization: Bearer <PAT> and X-Snowflake-Authorization-Token-Type: PROGRAMMATIC_ACCESS_TOKEN. |
model | The deployment ID, sf-ct-<UUID>. |
| Length limit | max_completion_tokens. The endpoint rejects max_tokens with HTTP 400 (unsupported_parameter). |
| Streaming | With "stream": true, the response is server-sent events: one data: line per chat.completion.chunk, then data: [DONE]. Add "stream_options": {"include_usage": true} to get a final chunk with token counts. |
| Not ready | HTTP 503 (endpoint_failed) while the deployment is PENDING or INITIALIZING, and briefly after it’s dropped. |
| Not found | HTTP 404 (model_not_found) for an unknown deployment, or one that failed or was dropped. |
| Not enabled | HTTP 403 or 405 while inference isn’t enabled for the account. |
Without "stream": true, the response is one chat.completion object.
create_ model_ deployment¶
Create a model deployment from a deployment spec and return its metadata, including the deployment ID. The deployment starts in PENDING; call wait_for_model_deployment to wait until it’s READY.
| Parameter | Type | Default | Description |
|---|---|---|---|
spec | ModelDeploymentSpec | Mapping[str, Any] | required |
wait_ for_ model_ deployment¶
Poll until the deployment is READY and return its metadata, including base_url. Raises ModelDeploymentWaitError if the deployment reaches FAILED, TERMINATED, or DROPPED, and ModelDeploymentWaitTimeout when the client’s poll_timeout passes. A timeout doesn’t drop the deployment, which keeps holding GPUs until you drop it.
| Parameter | Type | Default | Description |
|---|---|---|---|
deployment_id | str | required |
get_ model_ deployment¶
Return the metadata for one deployment, including deployment, status, model_source, and the capacity fields. base_url is included unless the deployment is FAILED, DROPPING, DROPPED, or TERMINATED, and failure is included for a FAILED or TERMINATED deployment.
| Parameter | Type | Default | Description |
|---|---|---|---|
deployment_id | str | required |
describe_ model_ deployment¶
Return one deployment’s status, endpoint, and configuration. The configuration is grouped under spec in the same deployment, source, and inference shape as the create spec, but it’s a read view and isn’t guaranteed to be a valid create_model_deployment input. A deployment created from a checkpoint currently reports a base_model source.
| Parameter | Type | Default | Description |
|---|---|---|---|
deployment_id | str | required |
list_ model_ deployments¶
Return the metadata for every deployment in the client’s database and schema. Pass status (for example, "READY" or ["PENDING", "INITIALIZING"], case-insensitive) to keep only deployments in those statuses. Entries don’t include base_url; use get_model_deployment to get it.
| Parameter | Type | Default | Description |
|---|---|---|---|
status | str | Iterable[str] | None | None |
drop_ model_ deployment¶
Drop a deployment and release its GPUs. Safe to retry: a deployment that’s already DROPPING or DROPPED is reported as dropped. A FAILED or TERMINATED deployment holds no GPUs and can’t be dropped, so the method returns {"dropped": False, "status": ...} instead of raising an error.
| Parameter | Type | Default | Description |
|---|---|---|---|
deployment_id | str | required |
download_ model_ deployment_ logs¶
Download a deployment’s logs into output_dir. log_type is "execution" (the default) or "stdout". Returns {"deployment", "logs"}, where logs lists each saved file’s filename and saved_path (plus chunk_count for stdout). Use these logs to find out why a deployment FAILED.
| Parameter | Type | Default | Description |
|---|---|---|---|
deployment_id | str | required | |
output_dir | str | os.PathLike[str] | required | |
log_type | str | 'execution' |
download_ model_ deployment_ metrics¶
Download a deployment’s metrics into output_dir. Returns {"deployment", "metrics"}, where metrics lists each saved file’s filename, saved_path, and chunk_count.
| Parameter | Type | Default | Description |
|---|---|---|---|
deployment_id | str | required | |
output_dir | str | os.PathLike[str] | required |
Logs¶
fetch_ execution_ logs¶
Download every log file under the job’s experiment run stage.
Returns a list of
{"sub_job_id", "filename", "artifact_uri", "content"}
entries, one per object found under a _logs/<sub_job_id>/ subtree
in the run’s stage. Filenames such as execution.jsonl and
server.log are both included; bodies are decoded as UTF-8.
sub_job_id and filename, when given, download only the files
of that sub-job and with that name (for example one sub-job’s
execution.jsonl); the rest are listed but not fetched.
artifact_uri is the logical snow:// URI. Files are retrieved
through Snowflake’s supported experiment artifact LIST/GET
interface; no object-store credentials are exposed to the caller.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
sub_job_id | str | None | None | |
filename | str | None | None |
stream_ logs¶
Yield a sub-job’s log entries, polling with backoff.
Drains the log then, with follow=True (the default), keeps polling
for new entries until the caller stops iterating — suitable for a live
tail. With follow=False it stops at the first empty page, once it
has caught up with the log (whether or not the server reports eof).
The stream also ends, without an error, once the log can grow no more:
at an empty page that reports eof, or when the job ends after the
stream has read from it. A job that had already ended before the first
read raises, as tail_logs does.
| Parameter | Type | Default | Description |
|---|---|---|---|
job_id | str | required | |
sub_job_id | str | None | None | |
sub_job_type | str | None | None | |
cursor | str | None | None | |
max_lines | int | None | None | |
poll_interval | float | None | None | |
follow | bool | True |
Returns Iterator[dict].
SubJobConfig¶
One sub-job within a CreateJob request.
training_ job¶
Build a training SubJobConfig. All training fields are explicit.
| Parameter | Type | Default | Description |
|---|---|---|---|
model_name | str | required | Model identifier (e.g., “Qwen/Qwen3-1.7B”). |
optimizer | dict | required | Optimizer config dict (e.g., {“name”: “AdamW”, “lr”: 0.0001}). |
max_seq_len | int | required | Maximum sequence length for training. |
train_batch_size | int | required | Per-device batch size. |
n_gpus | int | required | Number of GPUs (DP size). |
gradient_clipping | float | None | None | Optional gradient clipping threshold. |
multiplex_job_id | str | None | None | Optional multiplex job identifier. |
load_optimizer_states | bool | None | None | Whether to load optimizer states from checkpoints. Set to False when loading across different DP sizes (n_gpus) since optimizer states are DP-sharded and cannot be resized. When None, uses server default (True). This is a creation-time setting and cannot be changed later. |
extra_training | dict | None | None | Additional training config keys (passthrough). |
global_batch_size | int | None | None | Global batch size across all GPUs. |
dtype | str | None | None | Model dtype (e.g., “bfloat16”). |
seed | int | None | None | Random seed. |
model_post_init | list[str] | None | None | List of post-initialization operations. |
source_checkpoint_info | dict | None | None | Optional checkpoint to resume from at creation time (use {"checkpoint_id": "...", "source_job_id": "..."}). |
sampling_ job¶
Build a sampling SubJobConfig.
job_type remains for source compatibility, but only SAMPLING is
currently accepted.
| Parameter | Type | Default | Description |
|---|---|---|---|
model_name | str | required | |
max_seq_len | int | required | |
n_gpus | int | required | |
multiplex_job_id | str | None | None | |
extra_sampling | dict | None | None | |
job_type | JobType | JobType.SAMPLING | |
global_batch_size | int | None | None | |
dtype | str | None | None | |
seed | int | None | None | |
model_post_init | list[str] | None | None | |
source_checkpoint_info | dict | None | None |
JobType¶
Payload helpers¶
Helpers that build the forward and backward pass payload in the client’s wire format.
serialize_ forward_ backward_ args¶
Serialize {"args": args, "kwargs": kwargs} for Cortex Training fwd-bwd.
| Parameter | Type | Default | Description |
|---|---|---|---|
args | tuple | () | |
kwargs | dict[str, Any] | None | None |
build_ forward_ backward_ kwargs¶
Build tensor kwargs for a forward-backward training batch.
payload is intentionally JSON-shaped so CLI examples can stay readable:
callers may provide either tokenizable texts plus a tokenizer or
direct tensor data under input_ids / labels / position_ids.
| Parameter | Type | Default | Description |
|---|---|---|---|
payload | dict[str, Any] | required |
build_ forward_ backward_ payload¶
Build a serialized fwd-bwd byte payload from readable JSON.
| Parameter | Type | Default | Description |
|---|---|---|---|
spec | dict[str, Any] | required |