snowflake::ingest::streaming::StreamingIngestClient Class Referencefinal
A class that is the starting point for using the Streaming Ingest client APIs, a single client maps to exactly one account/database/schema/pipe in Snowflake; however, multiple clients can point to the same account/database/schema/pipe. More...
#include <snowflake/ingest/streaming/client.h>
Public Member Functions | |
| ~StreamingIngestClient () | |
| Close the client if it is still open. | |
| StreamingIngestClient (const StreamingIngestClient &)=delete | |
| StreamingIngestClient & | operator= (const StreamingIngestClient &)=delete |
| Result< | GetElasticChannel () |
| Get the elastic channel for this client. | |
| Result< | OpenChannel (std::string_view channel_name,std::optional<std::string_view> offset_token=std::nullopt) |
| Open a channel with a given channel name. | |
| Result< | Close (bool wait_for_flush=true,std::chrono::milliseconds timeout=std::chrono::milliseconds(0)) |
| Close the client. | |
| Result< | DropChannel (std::string_view channel_name) |
| Drop the specified channel on the server using the channel name. | |
| Result< | GetChannelStatuses (const std::vector<std::string> &channel_names) const |
| Return the channel status for the given list of channels from Snowflake. | |
| Result< | GetLatestCommittedOffsetTokens (const std::vector<std::string> &channel_names) const |
| Return the latest committed/persisted offset token for the given list of channels in Snowflake. | |
| Result< | InitiateFlush () |
| Initiates a flush by the Client which causes all outstanding buffered data to be flushed to Snowflake. | |
| Result< | WaitForFlush (std::chrono::milliseconds timeout=std::chrono::milliseconds(0)) |
| Asynchronously waits for all buffered data in all channels managed by this client to be flushed to the Snowflake server side. | |
| bool | IsClosed () const |
| Check if the client is closed. | |
| const std:: | client_name () const noexcept |
| Get the client name. | |
| const std:: | db_name () const noexcept |
| Get the database name. | |
| const std:: | schema_name () const noexcept |
| Get the schema name. | |
| const std:: | pipe_name () const noexcept |
| Get the pipe name. | |
Static Public Member Functions | |
| static Result< | Create (const ClientOptions &options) |
| Create a new client. | |
| static Result< | CreateFromTable (const TableClientOptions &options) |
| Create a table-mode client. | |
Detailed Description
A class that is the starting point for using the Streaming Ingest client APIs, a single client maps to exactly one account/database/schema/pipe in Snowflake; however, multiple clients can point to the same account/database/schema/pipe.
Each client will contain information for Snowflake authentication and authorization, and it will be used to create one or more StreamingIngestChannel.
The client is created by calling StreamingIngestClient::Create and closed by calling StreamingIngestClient::Close. Not copyable or movable.
- Warning
- Thread safety: All other operations on this object must complete before it is destroyed, except
WaitForFlushfuture.
Constructor & Destructor Documentation
~StreamingIngestClient¶
Close the client if it is still open.
It waits for all the data in all channels managed by this client to be flushed to the Snowflake server side, for at most 60 seconds. Any error is ignored.
- Warning
- There is no guarantee that all the data is flushed to the Snowflake server side: if the flush fails or the timeout is reached, the client is closed without the remaining data being flushed. To be sure all data is flushed, or to get the error or use a different timeout, call
StreamingIngestClient::Closebefore the client is destroyed. - Destroyed inside an elastic channel handler, the client is closed without flushing, because the flush would wait on the task running the handler; buffered rows in all its channels are lost.
Member Function Documentation
client_name¶
|
const std::string & StreamingIngestClient::client_name() const
|
inlinenoexcept |
Get the client name.
- Returns
- the client name
Close¶
StreamingIngestClient::Close(
bool wait_for_flush = true,
std::chrono::milliseconds timeout = std::chrono::milliseconds(0) ))
Close the client.
If wait_for_flush=true this function will make sure all the data in all channels managed by this client is flushed to the Snowflake server side or return an error if the timeout is reached.
- Parameters
-
wait_for_flush whether to wait for all channels to be flushed timeout The maximum time (non-negative) for close to wait. If 0, the operation will wait indefinitely.
- Returns
- an empty
Resulton success, or aStreamingIngestErroron failure.
Create¶
|
Result<std::unique_ptr<StreamingIngestClient>>
StreamingIngestClient::Create(const ClientOptions & options) |
static |
Create a new client.
- Parameters
-
options the client configuration; see ClientOptions
- Returns
- the client on success, or a
StreamingIngestErroron failure
CreateFromTable¶
|
Result<std::unique_ptr<StreamingIngestClient>>
StreamingIngestClient::CreateFromTable(const TableClientOptions & options) |
static |
Create a table-mode client.
The client writes to the table's default TABLE-STREAMING pipe.
- Parameters
-
options the table-mode client configuration
- Returns
- the client on success, or a
StreamingIngestErroron failure.
db_name¶
|
const std::string & StreamingIngestClient::db_name() const
|
inlinenoexcept |
Get the database name.
- Returns
- name of the database
DropChannel¶
Drop the specified channel on the server using the channel name.
Note that this call will blindly drop the latest version of the channel and any pending data will be lost. It will also delete Offset Token and other state from Snowflake servers. So only use it if you are completely done ingesting data for this channel. If you open a channel with the same name in the future, it will behave like a new channel.
Also see StreamingIngestChannel::Close() to close channels. That approach will drop the local version of the channel and if the channel has been concurrently reopened by another client, that version of the channel won't be affected.
- Parameters
-
channel_name the channel name to drop
- Returns
- an empty
Resulton success, or aStreamingIngestErroron failure.
GetChannelStatuses¶
StreamingIngestClient::GetChannelStatuses(
const std::vector<std::string> & channel_names) const
Return the channel status for the given list of channels from Snowflake.
- Parameters
-
channel_names the list of channel names
- Returns
- the
ChannelStatusfor each channel, keyed by channel name, or aStreamingIngestErroron failure.
GetElasticChannel¶
Get the elastic channel for this client.
Elastic channels have no offset token concepts and their lifecycle is tied to the client. The same instance is returned on repeated calls (singleton).
- Returns
- the elastic channel for this client on success, or a
StreamingIngestErroron failure
GetLatestCommittedOffsetTokens¶
StreamingIngestClient::GetLatestCommittedOffsetTokens(
const std::vector<std::string> & channel_names) const
Return the latest committed/persisted offset token for the given list of channels in Snowflake.
- Parameters
-
channel_names the list of channel names
- Returns
- the latest committed offset token for each channel, keyed by channel name, or a
StreamingIngestErroron failure.
InitiateFlush¶
Initiates a flush by the Client which causes all outstanding buffered data to be flushed to Snowflake.
Note that data can still be accepted by the Client - this is an asynchronous call and will return after flush is initiated for all Channels opened by this Client
- Returns
- an empty
Resulton success, or aStreamingIngestErroron failure.
IsClosed¶
Check if the client is closed.
- Returns
- a boolean which indicates whether the client is closed
OpenChannel¶
StreamingIngestClient::OpenChannel(
std::string_view channel_name,
std::optional<std::string_view> offset_token = std::nullopt ))
Open a channel with a given channel name.
The channel is opened on the account/database/schema/pipe defined by the current client
- Parameters
-
channel_name the name of the channel to open offset_token the optional offset token to set on the channel to be opened. If not provided and this reopens an existing channel, the latest persisted offset token will be reused.
- Returns
- the channel and its
ChannelStatuson success, or aStreamingIngestErroron failure
pipe_name¶
|
const std::string & StreamingIngestClient::pipe_name() const
|
inlinenoexcept |
Get the pipe name.
- Returns
- name of the pipe
schema_name¶
|
const std::string & StreamingIngestClient::schema_name() const
|
inlinenoexcept |
Get the schema name.
- Returns
- name of the schema
WaitForFlush¶
Asynchronously waits for all buffered data in all channels managed by this client to be flushed to the Snowflake server side.
This method triggers a flush of all pending data across all channels and waits for the flush operations to complete.
- Parameters
-
timeout The maximum time (non-negative) to wait for all channels to flush. If 0, the operation will wait indefinitely.
- Returns
- a
std::futurethat completes with an emptyResultwhen the flush is done, or withkClientWaitForFlushTimeoutif the timeout is reached. Returns aStreamingIngestErrordirectly in case of error.