snowflake.ml.model.batch.JobSpec¶

class snowflake.ml.model.batch.JobSpec(*, image_repo: Optional[str] = None, job_name: Optional[str] = None, job_name_prefix: Optional[str] = None, num_workers: Optional[int] = None, function_name: Optional[str] = None, force_rebuild: bool = False, max_batch_rows: Optional[int] = None, warehouse: Optional[str] = None, cpu_requests: Optional[str] = None, memory_requests: Optional[str] = None, gpu_requests: Optional[str] = None, replicas: Optional[int] = None, block: bool = False)¶

Bases: BaseModel

Specification for batch inference job execution.

Defines the compute resources, job settings, and execution parameters for running batch inference jobs in Snowflake.

image_repo¶

Container image repository for the inference job. If not specified, uses the default repository.

Type:

Optional[str]

job_name¶

Custom name for the batch inference job. If not provided, a name will be auto-generated in the form of “BATCH_INFERENCE_<UUID>”. Mutually exclusive with job_name_prefix.

Type:

Optional[str]

job_name_prefix¶

Prefix for auto-generated job names. When set, the job name will be generated as “<PREFIX>_<UUID>”. This is useful for task integration where each repeated execution needs a unique name with a recognizable prefix. Mutually exclusive with job_name.

Type:

Optional[str]

num_workers¶

The number of workers to run the inference service for handling requests in parallel within an instance of the service. By default, it is set to 2*vCPU+1 of the node for CPU based inference and 1 for GPU based inference. For GPU based inference, please see best practices before playing with this value.

Type:

Optional[int]

function_name¶

Name of the specific function to call for inference. Required when the model has multiple inference functions.

Type:

Optional[str]

force_rebuild¶

Whether to force rebuilding the container image even if it already exists. Defaults to False.

Type:

bool

max_batch_rows¶

Maximum number of rows to process in a single batch. Auto determined if None. Larger values may improve throughput.

Type:

Optional[int]

warehouse¶

Snowflake warehouse to use for the batch inference job. If not specified, uses the session’s current warehouse.

Type:

Optional[str]

cpu_requests¶

The cpu limit for CPU based inference. Can be an integer, fractional or string values. If None, we attempt to utilize all the vCPU of the node.

Type:

Optional[str]

memory_requests¶

The memory limit for inference. Can be an integer or a fractional value, but requires a unit (GiB, MiB). If None, we attempt to utilize all the memory of the node.

Type:

Optional[str]

gpu_requests¶

The gpu limit for GPU based inference. Can be integer or string values. Use CPU if None.

Type:

Optional[str]

replicas¶

Number of SPCS job nodes used for distributed inference. If not specified, defaults to 1 replica.

Type:

Optional[int]

block¶

Whether the SPCS batch inference job runs synchronously or asynchronously. When True, the call blocks until the job completes. When False, the call returns immediately after job creation. Defaults to False.

Type:

bool

Example

CopyExpand

>>> job_spec = JobSpec(
...     job_name="my_inference_job",
...     num_workers=4,
...     cpu_requests="2",
...     memory_requests="8Gi",
...     max_batch_rows=2048
... )

Show lessSee more

Scroll to top

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Attributes

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}¶

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}¶

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_extra¶

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

model_fields: ClassVar[dict[str, FieldInfo]] = {'block': FieldInfo(annotation=bool, required=False, default=False), 'cpu_requests': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'force_rebuild': FieldInfo(annotation=bool, required=False, default=False), 'function_name': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'gpu_requests': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'image_repo': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'job_name': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'job_name_prefix': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_batch_rows': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'memory_requests': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'num_workers': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'replicas': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'warehouse': FieldInfo(annotation=Union[str, NoneType], required=False, default=None)}¶

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

model_fields_set¶

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

image_repo: Optional[str]¶

job_name: Optional[str]¶

job_name_prefix: Optional[str]¶

num_workers: Optional[int]¶

function_name: Optional[str]¶

force_rebuild: bool¶

max_batch_rows: Optional[int]¶

warehouse: Optional[str]¶

cpu_requests: Optional[str]¶

memory_requests: Optional[str]¶

gpu_requests: Optional[str]¶

replicas: Optional[int]¶

block: bool¶