Openflow Connector for Amazon Kinesis Data Streams 유지 관리

참고

이 커넥터에는 `Snowflake Connector 약관<https://www.snowflake.com/legal/snowflake-connector-terms/>`_이 적용됩니다.

이 항목에서는 커넥터 상태를 관리하고 재설정하는 방법을 포함하여 Openflow Connector for Amazon Kinesis Data Streams 커넥터를 유지 관리하는 방법에 대해 설명합니다.

커넥터 상태 관리

The Openflow Connector for Amazon Kinesis Data Streams connector uses DynamoDB to store the consumer application state.

DynamoDB table created by the connector

The connector creates a DynamoDB table with the name specified in AWS Kinesis Application Name. The table stores the checkpointed sequence number for each shard in the stream. This tracks which records have been processed.

If multiple processors use the same application name, they cooperate to consume data from the stream and share this table. If processors have different application names, each creates its own table to independently track consumed records.

커넥터 상태 재설정하기

DynamoDB의 커넥터 상태가 손상되거나 일관되지 않으면 재설정해야 할 수 있습니다. 커넥터 상태를 재설정하는 방법에는 두 가지가 있습니다.

Reset by changing the application name

The simplest way to reset the connector state is to change the AWS Kinesis Application Name parameter:

  1. 커넥터를 중지합니다.

  2. 커넥터의 매개 변수 컨텍스트로 이동합니다.

  3. Change the AWS Kinesis Application Name parameter value to a new value.

  4. 커넥터를 시작합니다.

The connector creates a new DynamoDB table with the new application name and begins consuming records from the position specified by the AWS Kinesis Initial Stream Position parameter.

참고

  • When you change the application name, the connector doesn’t delete the old DynamoDB table. You must manually delete it through the AWS Console or the AWS CLI.

  • If your IAM policy restricts DynamoDB access to a specific table name, you must update the policy to allow access to the new table name. For more information on configuring IAM permissions, see Set up Openflow Connector for Amazon Kinesis Data Streams.

Reset by deleting the DynamoDB table

Alternatively, you can delete the existing DynamoDB table to reset the state:

  1. 커넥터를 중지합니다.

  2. In the AWS Console or using the AWS CLI, delete the DynamoDB table associated with the application name.

  3. 커넥터를 시작합니다.

The connector recreates the table and begins consuming records from the position specified by the AWS Kinesis Initial Stream Position parameter.

경고

Resetting the connector state causes the connector to reprocess records from the position specified by the initial stream position. Depending on your AWS Kinesis Initial Stream Position setting, this may result in duplicate data being ingested into Snowflake or data not being ingested at all.