snowflake::ingest::streaming::StreamingIngestElasticChannel Class Referencefinal

Elastic channel for Snowflake Streaming Ingest. More...

#include <snowflake/ingest/streaming/elastic_channel.h>

Public Member Functions

 StreamingIngestElasticChannel (const StreamingIngestElasticChannel &)=delete
StreamingIngestElasticChannel & operator= (const StreamingIngestElasticChannel &)=delete
 StreamingIngestElasticChannel (StreamingIngestElasticChannel &&)=delete
StreamingIngestElasticChannel & operator= (StreamingIngestElasticChannel &&)=delete
Result<void> AppendRow (std::string_view serialized_row,std::optional<AppendToken> token=std::nullopt)
 Insert one row into the elastic channel without waiting for acknowledgement (fire-and-forget): returns as soon as the row is buffered.
Result<void> AppendRows (std::string_view ndjson,int64_t num_rows,std::optional<AppendToken> token=std::nullopt)
 Insert a batch of rows into the elastic channel without waiting for acknowledgement (fire-and-forget): returns as soon as the rows are buffered.
Result<std::future<Result<void>>> AppendRowWithWait (std::string_view serialized_row,std::optional<AppendToken> token=std::nullopt)
 Insert one row into the elastic channel and return a future that completes when Snowflake acknowledges the row (or completes with an error if the append fails).
Result<std::future<Result<void>>> AppendRowsWithWait (std::string_view ndjson,int64_t num_rows,std::optional<AppendToken> token=std::nullopt)
 Insert a batch of rows into the elastic channel and return a future that completes when Snowflake acknowledges the batch (or completes with an error if the append fails).
Result<void> SetErrorHandler (ErrorHandler handler)
 Register a handler for appends that fail asynchronously, after AppendRow()/AppendRows() has returned.
Result<void> SetSuccessHandler (SuccessHandler handler)
 Register a handler for appends that are acknowledged by Snowflake, after AppendRow()/AppendRows() has returned.
Result<void> InitiateFlush ()
 Initiates a flush of all buffered data in this channel without waiting for completion.
Result<std::future<Result<void>>> WaitForFlush (std::chrono::milliseconds timeout=std::chrono::milliseconds(0))
 Asynchronously waits for all buffered data in this channel to be flushed to the Snowflake server side.
Result<ChannelStatus> GetChannelStatus () const
 Get the channel status from Snowflake.
bool IsClosed () const
 Check if the elastic channel is closed (because the client was closed).
const std::string & channel_name () const noexcept
 Get the channel name (always "ELASTIC").
const std::string & db_name () const noexcept
 Get the database name.
const std::string & schema_name () const noexcept
 Get the schema name.
const std::string & pipe_name () const noexcept
 Get the pipe name.

Friends

struct internal::ElasticChannelAccess

Detailed Description

Elastic channel for Snowflake Streaming Ingest.

Unlike regular channels, elastic channels have no offset token concepts and their lifecycle is tied to the client (no close method). The same instance is returned on repeated calls to StreamingIngestClient::GetElasticChannel(). Not copyable or movable.

Warning
Handlers must not capture the channel's shared_ptr.

Member Function Documentation

AppendRow

Result<void>
StreamingIngestElasticChannel::AppendRow(
std::string_view serialized_row,
std::optional<AppendToken> token = std::nullopt ))

Insert one row into the elastic channel without waiting for acknowledgement (fire-and-forget): returns as soon as the row is buffered.

The append is still tracked for the error handler and the success handler — the outcome is surfaced through whichever is registered, keyed by token, just not through a future. Use AppendRowWithWait() when you need a future to await.

Parameters
serialized_rowdata to write: a caller-serialized JSON-formatted string that maps Column Name to Column Value for the columns in the table, e.g. R"({"c1": 1, "c2": "abc"})". The string should:
  • hold exactly one row.
  • encode BINARY values following the pipe's BINARY_INPUT_FORMAT.
  • not be modified concurrently while this call is in progress.
tokencaller-supplied opaque append token, delivered in the ErrorDetail to a registered error handler if this append fails asynchronously, or in the SuccessDetail to a registered success handler once it is acknowledged. If token is std::nullopt, neither handler is invoked — and, with no future either (fire-and-forget), the outcome is not reported at all. The token is retained in memory until the append is acknowledged by the service, so attaching a large object raises the client's memory footprint — prefer a small id.
Returns
an empty Result on success, or a StreamingIngestError on failure.

AppendRows

Result<void>
StreamingIngestElasticChannel::AppendRows(
std::string_view ndjson,
int64_t num_rows,
std::optional<AppendToken> token = std::nullopt ))

Insert a batch of rows into the elastic channel without waiting for acknowledgement (fire-and-forget): returns as soon as the rows are buffered.

The append is still tracked for the error handler and the success handler — the outcome is surfaced through whichever is registered, keyed by token, just not through a future. Use AppendRowsWithWait() when you need a future to await.

Parameters
ndjsondata to write: a single string holding one or more rows of JSON, with rows separated by a newline character (\n). Each row is a caller-serialized JSON object that maps Column Name to Column Value for the columns in the table. For example, for two rows: "{\"c1\":1,\"c2\":\"abc\"}\\n{\"c1\":2,\"c2\":\"def\"}". The string should:
  • hold exactly num_rows rows
  • encode BINARY values following the pipe's BINARY_INPUT_FORMAT.
  • not be modified concurrently while this call is in progress.
num_rowsnumber of rows in ndjson.
tokencaller-supplied opaque append token, delivered in the ErrorDetail to a registered error handler if this append fails asynchronously, or in the SuccessDetail to a registered success handler once it is acknowledged. If token is std::nullopt, neither handler is invoked — and, with no future either (fire-and-forget), the outcome is not reported at all. The token is retained in memory until the append is acknowledged by the service, so attaching a large object raises the client's memory footprint — prefer a small id.
Returns
an empty Result on success, or a StreamingIngestError on failure.

AppendRowsWithWait

Result<std::future<Result<void>>>
StreamingIngestElasticChannel::AppendRowsWithWait(
std::string_view ndjson,
int64_t num_rows,
std::optional<AppendToken> token = std::nullopt ))

Insert a batch of rows into the elastic channel and return a future that completes when Snowflake acknowledges the batch (or completes with an error if the append fails).

A registered error handler also fires on failure, and a registered success handler on acknowledgement — dual delivery either way.

Parameters
ndjsondata to write; see AppendRows().
num_rowsnumber of rows in ndjson.
tokencaller-supplied opaque append token, delivered in the ErrorDetail to a registered error handler if this append fails asynchronously, or in the SuccessDetail to a registered success handler once it is acknowledged. If token is std::nullopt, neither handler is invoked; the returned future is then the only signal. The token is retained in memory until the append is acknowledged by the service, so attaching a large object raises the client's memory footprint — prefer a small id.
Returns
a std::future that completes with an empty Result when the rows are acknowledged by Snowflake, or with the error if the append fails. Returns a StreamingIngestError directly in case of error.

AppendRowWithWait

Result<std::future<Result<void>>>
StreamingIngestElasticChannel::AppendRowWithWait(
std::string_view serialized_row,
std::optional<AppendToken> token = std::nullopt ))

Insert one row into the elastic channel and return a future that completes when Snowflake acknowledges the row (or completes with an error if the append fails).

A registered error handler also fires on failure, and a registered success handler on acknowledgement — dual delivery either way.

Parameters
serialized_rowdata to write; see AppendRow().
tokencaller-supplied opaque append token, delivered in the ErrorDetail to a registered error handler if this append fails asynchronously, or in the SuccessDetail to a registered success handler once it is acknowledged. If token is std::nullopt, neither handler is invoked; the returned future is then the only signal. The token is retained in memory until the append is acknowledged by the service, so attaching a large object raises the client's memory footprint — prefer a small id.
Returns
a std::future that completes with an empty Result when the row is acknowledged by Snowflake, or with the error if the append fails. Returns a StreamingIngestError directly in case of error.

channel_name

const std::string & StreamingIngestElasticChannel::channel_name() const
inlinenoexcept

Get the channel name (always "ELASTIC").

Returns
the channel name

db_name

const std::string & StreamingIngestElasticChannel::db_name() const
inlinenoexcept

Get the database name.

Returns
name of the database

GetChannelStatus

Result<ChannelStatus>
StreamingIngestElasticChannel::GetChannelStatus() const

Get the channel status from Snowflake.

Returns
the channel status on success, or a StreamingIngestError on failure.

InitiateFlush

Result<void> StreamingIngestElasticChannel::InitiateFlush()

Initiates a flush of all buffered data in this channel without waiting for completion.

Returns
an empty Result on success, or a StreamingIngestError on failure.

IsClosed

bool StreamingIngestElasticChannel::IsClosed() const

Check if the elastic channel is closed (because the client was closed).

Returns
true if the channel is closed

pipe_name

const std::string & StreamingIngestElasticChannel::pipe_name() const
inlinenoexcept

Get the pipe name.

Returns
name of the pipe

schema_name

const std::string & StreamingIngestElasticChannel::schema_name() const
inlinenoexcept

Get the schema name.

Returns
name of the schema

SetErrorHandler

Result<void>
StreamingIngestElasticChannel::SetErrorHandler(ErrorHandler handler)

Register a handler for appends that fail asynchronously, after AppendRow()/AppendRows() has returned.

The handler is invoked once per failure event with an ErrorDetail carrying the append tokens that failed together under a common error. Optional, and replaceable (last one set wins) — set it before appending for full coverage. Failed futures still complete with the error either way.

The handler is only invoked for appends made with a token; an append given std::nullopt still fails its future but is never reported to the handler.

The handler runs inline on this channel's internal acknowledgement task. It must therefore return quickly and do no heavy or thread-blocking work: no blocking I/O, no network or disk calls, no waiting on locks or futures, and no synchronous re-entry into the SDK. Do only cheap bookkeeping inline (for example, record the failed tokens) and hand any real work to your own executor or queue. Blocking delays this channel's own acknowledgements — and therefore its append and flush completion — and, because every channel's acknowledgement task shares one process-wide worker pool, enough blocked handlers can starve acknowledgements elsewhere too. The SDK logs a warning when a handler runs slowly. A thrown exception is caught and logged so one bad handler cannot poison the acknowledgement path.

Parameters
handlerthe error handler to register. Must not be empty.
Returns
an empty Result on success, or a StreamingIngestError on failure.

SetSuccessHandler

Result<void>
StreamingIngestElasticChannel::SetSuccessHandler(SuccessHandler handler)

Register a handler for appends that are acknowledged by Snowflake, after AppendRow()/AppendRows() has returned.

The handler is invoked once per successful acknowledgement batch with a SuccessDetail carrying the append tokens that were acknowledged together. Optional, and replaceable (last one set wins) — set it before appending for full coverage. Independent of the error handler: either can be registered without the other, and a given append is reported to exactly one of them. Successful futures still complete either way.

The handler runs inline on this channel's internal acknowledgement task. It must therefore return quickly and do no heavy or thread-blocking work: no blocking I/O, no network or disk calls, no waiting on locks or futures, and no synchronous re-entry into the SDK. Do only cheap bookkeeping inline (for example, record the tokens) and hand any real work to your own executor or queue. Blocking delays this channel's own acknowledgements — and therefore its append and flush completion — and, because every channel's acknowledgement task shares one process-wide worker pool, enough blocked handlers can starve acknowledgements elsewhere too. The SDK logs a warning when a handler runs slowly. A thrown exception is caught and logged so one bad handler cannot poison the acknowledgement path.

Parameters
handlerthe success handler to register. Must not be empty.
Returns
an empty Result on success, or a StreamingIngestError on failure.

WaitForFlush

Result<std::future<Result<void>>>
StreamingIngestElasticChannel::WaitForFlush(
std::chrono::milliseconds timeout = std::chrono::milliseconds(0))

Asynchronously waits for all buffered data in this channel to be flushed to the Snowflake server side.

This method triggers a flush of all pending data and waits for the flush operation to complete.

Parameters
timeoutThe maximum time (non-negative) to wait for the flush to complete. If 0, the operation will wait indefinitely.
Returns
a std::future that completes with an empty Result when the flush is done, or with kChannelWaitForFlushTimeout if the timeout is reached. Returns a StreamingIngestError directly in case of error.