Manter o Openflow Connector for Amazon Kinesis Data Streams

Nota

O conector está sujeito aos Termos do conector Snowflake.

Este tópico descreve como manter o conector Openflow Connector for Amazon Kinesis Data Streams, incluindo como gerenciar e redefinir o estado do conector.

Gerenciar o estado do conector

The Openflow Connector for Amazon Kinesis Data Streams connector uses DynamoDB to store the consumer application state.

DynamoDB table created by the connector

The connector creates a DynamoDB table with the name specified in AWS Kinesis Application Name. The table stores the checkpointed sequence number for each shard in the stream. This tracks which records have been processed.

If multiple processors use the same application name, they cooperate to consume data from the stream and share this table. If processors have different application names, each creates its own table to independently track consumed records.

Redefinir o estado do conector

Se o estado do conector no DynamoDB se tornar corrompido ou inconsistente, talvez seja necessário redefini-lo. Há duas abordagens para redefinir o estado do conector.

Reset by changing the application name

The simplest way to reset the connector state is to change the AWS Kinesis Application Name parameter:

  1. Pare o conector.

  2. Navegue até o contexto de parâmetro do conector.

  3. Change the AWS Kinesis Application Name parameter value to a new value.

  4. Inicie o conector.

The connector creates a new DynamoDB table with the new application name and begins consuming records from the position specified by the AWS Kinesis Initial Stream Position parameter.

Nota

  • When you change the application name, the connector doesn’t delete the old DynamoDB table. You must manually delete it through the AWS Console or the AWS CLI.

  • If your IAM policy restricts DynamoDB access to a specific table name, you must update the policy to allow access to the new table name. For more information on configuring IAM permissions, see Configuração de Openflow Connector for Amazon Kinesis Data Streams.

Reset by deleting the DynamoDB table

Alternatively, you can delete the existing DynamoDB table to reset the state:

  1. Pare o conector.

  2. In the AWS Console or using the AWS CLI, delete the DynamoDB table associated with the application name.

  3. Inicie o conector.

The connector recreates the table and begins consuming records from the position specified by the AWS Kinesis Initial Stream Position parameter.

Aviso

Resetting the connector state causes the connector to reprocess records from the position specified by the initial stream position. Depending on your AWS Kinesis Initial Stream Position setting, this may result in duplicate data being ingested into Snowflake or data not being ingested at all.