snowflake.ingest.streaming.StreamingIngestElasticChannel¶

ChannelErrorHandler¶

ChannelSuccessHandler¶

class StreamingIngestElasticChannel(channel: snowflake.ingest.streaming._python_ffi.PyChannel, callback: snowflake.ingest.streaming._mark_complete_callback.MarkCompleteCallback, *, binary_input_format: snowflake.ingest.streaming._python_ffi.PyBinaryInputFormat, _internal: bool = False)¶

Elastic channel for Snowflake Streaming Ingest.

Unlike regular channels, elastic channels have no offset token concepts and their lifecycle is tied to the client (no close method). The same instance is returned on repeated calls to get_elastic_channel().

Note

This class should not be instantiated directly. Use get_elastic_channel() to obtain the elastic channel instance.

set_error_handler(handler: ChannelErrorHandler) → None¶

Optional and replaceable (last one set wins); set it before appending for full coverage. This is the only asynchronous-failure signal for the fire-and-forget append_row/ append_rows; the Futures returned by append_row_with_wait/append_rows_with_wait still complete exceptionally as well. See ChannelErrorHandler for the threading contract.

Parameters:: handler – Callable taking the ErrorDetail for the failure event, which bundles the failing append tokens with the StreamingIngestError.
Raises:: ValueError – If handler is None.

set_success_handler(handler: ChannelSuccessHandler) → None¶

Optional and replaceable (last one set wins); set it before appending for full coverage. This is the only success signal for the fire-and-forget append_row/append_rows; the Futures returned by append_row_with_wait/append_rows_with_wait still complete successfully as well. Independent of set_error_handler() – registering one does not require the other. See ChannelSuccessHandler for the threading contract.

Parameters:: handler – Callable taking the SuccessDetail for the acknowledgement batch, which carries the acknowledged append tokens.
Raises:: ValueError – If handler is None.

append_row(row: Dict[str, Any], append_token: object) → None¶

Append a single row into the elastic channel without waiting for acknowledgement.

Fire-and-forget: returns as soon as the row is handed to the SDK, with no Future to await. The append is still tracked for the handler registered via set_error_handler(), so an asynchronous failure is still reported there, keyed by append_token – that handler is the only way to learn about one. Use append_row_with_wait() when you need a Future.

Parameters:

row – Dictionary representing the row data to append.
append_token – Required caller-supplied opaque token (any object) echoed back in the ErrorDetail handed to the handler registered via set_error_handler() if the append fails asynchronously, or in the SuccessDetail handed to the handler registered via set_success_handler() once it is acknowledged. Required but nullable: pass None explicitly to opt out, which leaves this append untracked for either handler. The token is retained in memory until the append is acknowledged, so a large object raises the SDK’s memory footprint – prefer a small id.

Raises:

ValueError, TypeError – If the row cannot be serialized to JSON.
StreamingIngestError – If the row appending fails.

append_rows(rows: List[Dict[str, Any]], append_token: object) → None¶

Append multiple rows into the elastic channel without waiting for acknowledgement.

Fire-and-forget: returns as soon as the rows are handed to the SDK, with no Future to await. The append is still tracked for the handler registered via set_error_handler(), so an asynchronous failure is still reported there, keyed by append_token – that handler is the only way to learn about one. Use append_rows_with_wait() when you need a Future.

Parameters:

rows – List of dictionaries representing the row data to append.
append_token – Required caller-supplied opaque token (any object) echoed back in the ErrorDetail handed to the handler registered via set_error_handler() if the append fails asynchronously, or in the SuccessDetail handed to the handler registered via set_success_handler() once it is acknowledged. Required but nullable: pass None explicitly to opt out, which leaves this append untracked for either handler. The token is retained in memory until the append is acknowledged, so a large object raises the SDK’s memory footprint – prefer a small id.

Raises:

ValueError, TypeError – If the rows cannot be serialized to JSON.
StreamingIngestError – If the rows appending fails.

append_row_with_wait(row: Dict[str, Any], append_token: object) → concurrent.futures.Future¶

Append a single row into the elastic channel and return a Future to await.

Parameters:

row – Dictionary representing the row data to append.
append_token – Required caller-supplied opaque token (any object) echoed back in the ErrorDetail handed to the handler registered via set_error_handler() if the append fails asynchronously, or in the SuccessDetail handed to the handler registered via set_success_handler() once it is acknowledged. Required but nullable: pass None explicitly to opt out, which leaves this append untracked for either handler. The token is retained in memory until the append is acknowledged, so a large object raises the SDK’s memory footprint – prefer a small id.

Returns:

Completes when the row is acknowledged by Snowflake, or completes exceptionally if it fails. A registered error handler also fires on failure.

Return type:

Future

Raises:

ValueError, TypeError – If the row cannot be serialized to JSON.
StreamingIngestError – If the row appending fails.

append_rows_with_wait(rows: List[Dict[str, Any]], append_token: object) → concurrent.futures.Future¶

Append multiple rows into the elastic channel and return a Future to await.

Parameters:

rows – List of dictionaries representing the row data to append.
append_token – Required caller-supplied opaque token (any object) echoed back in the ErrorDetail handed to the handler registered via set_error_handler() if the append fails asynchronously, or in the SuccessDetail handed to the handler registered via set_success_handler() once it is acknowledged. Required but nullable: pass None explicitly to opt out, which leaves this append untracked for either handler. The token is retained in memory until the append is acknowledged, so a large object raises the SDK’s memory footprint – prefer a small id.

Returns:

Completes when the rows are acknowledged by Snowflake, or completes exceptionally if they fail. A registered error handler also fires on failure.

Return type:

Future

Raises:

ValueError, TypeError – If the rows cannot be serialized to JSON.
StreamingIngestError – If the rows appending fails.

initiate_flush() → None¶

Initiate a flush of all buffered data in this channel without waiting for completion.

Raises:: StreamingIngestError – If initiating the flush fails.

wait_for_flush(timeout_seconds: int | None = None) → None¶

Wait for the elastic channel to flush all buffered data.

Waits for all buffered data in this channel to be flushed to the Snowflake server side. This method triggers a flush of all pending data and waits for the flush operation to complete. If the timeout is reached, a TimeoutError is raised.

Parameters:

timeout_seconds – Optional timeout in seconds for the flush operation. Defaults to None if no timeout is desired.

Raises:

ValueError – If timeout_seconds is negative
TimeoutError – If the timeout is reached
StreamingIngestError – If waiting for the flush fails

get_channel_status() → ChannelStatus¶

Get the status of the elastic channel.

Returns:: The status of the channel.
Return type:: ChannelStatus
Raises:: StreamingIngestError – If getting the channel status fails.

is_closed() → bool¶

Check if the elastic channel is closed (because the client was closed).

Returns:: True if the channel is closed, False otherwise.
Return type:: bool

property channel_name: str¶: Get the channel name (always “ELASTIC”).

property db_name: str¶: Get the database name.

property schema_name: str¶: Get the schema name.

property pipe_name: str¶: Get the pipe name.