SnowConvert AI: Data migration

SnowConvert AI, as part of the end-to-end migration experience, provides the capability to migrate your actual data from source tables to Snowflake after the database structure is deployed. This data migration feature ensures that your data is transferred efficiently and accurately to complete the data migration process. This data migration feature is available for SQL Server and Amazon Redshift databases.

Data migration process overview

The data migration process varies depending on your source database platform.

Amazon RedshiftからSnowflakeへ

SnowConvert AI は、S3バケット内の PARQUET ファイルにデータをアンロードし、そのファイルからSnowflakeのデプロイ済みテーブルに直接データをコピーすることで、Redshiftのテーブルからデータを移行します。

SQL ServerからSnowflakeへ

SQL Serverからのデータ移行では、最適化されたデータ転送方法を利用して、テーブルデータを対応するSnowflakeテーブルに効率的に移行します。

前提条件

Ensure that you meet the following general prerequisites:

  • You completed the deployment process with database structure in Snowflake.

  • You have active connections to both a source database and Snowflake account.

  • You have sufficient permissions for data operations on both source and target systems.

In addition, ensure that the prerequisites are complete for your source database platform:

Prerequisites for Amazon Redshift sources

Before executing data migration from Redshift, ensure that you meet the following prerequisites:

S3 bucket requirements

  • An S3 bucket in AWS in the same region as your Redshift cluster.

  • Empty bucket path; the process fails if files exist in the specified path.

IAM role for Redshift

S3バケットにデータをアンロードできる、Redshiftクラスターに関連する IAM ロールを作成します。

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:GetBucketLocation",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<your_bucket_name>/*",
        "arn:aws:s3:::<your_bucket_name>"
      ]
    }
  ]
}
Copy

IAM user for S3 access

S3バケットからオブジェクトを読み取り、削除する権限を持つ IAM ユーザーを作成します。

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:DeleteObject",
        "s3:DeleteObjectVersion",
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::<your_bucket_name>/*",
        "arn:aws:s3:::<your_bucket_name>"
      ]
    }
  ]
}
Copy

警告

If you don't provide s3:DeleteObject and s3:DeleteObjectVersion permissions, the temporary data files won't be deleted from the S3 bucket even if the migration succeeds.

Prerequisites for SQL Server sources

  • Valid connection to your SQL Server source database.

  • Appropriate permissions to read data from source tables.

  • Network connectivity between the migration tool and both source and target systems.

Migrate your data

Complete the end-to-end data migration process by transferring your actual data to the deployed database structure in Snowflake:

Migrate Amazon Redshift data

  1. In SnowConvert AI, open the project, and then select Migrate data.

  2. On the Connect to source database page, complete the fields with the connection information for your source database, and then select Continue.

  3. On the Connect to Snowflake page, complete the fields with your connection information, and then select Continue.

  4. On the Set up S3 bucket page, specify the following information:

    • Database: Select the database that contains the tables that you want to migrate.

    • Schema: Select the schema that contains the tables that you want to migrate.

    • Stage: Select the stage where data files will be stored.

    • S3 bucket URI: Enter the Uniform Resource Identifier for the Amazon S3 bucket that contains the source data. The URL must end with /.

    • Data unloading IAM role ARN: Enter the Amazon Resource Name (ARN) for the IAM role that you used to unload the data to Amazon S3.

    The following image is an example of the page:

  5. On the Select tables to migrate data page, select the tables that you want to migrate.

    The following image is an example of the page:

  6. Select Migrate data.

    When the migration completes successfully, the Results page appears.

    If the results don't look correct, you can select Retry data migration to run the data migration process again.

  7. When the results look correct on the Results page, select Go to data validation to start the validation process.

Migrate SQL Server data

  1. In SnowConvert AI, open the project, and then select Migrate data.

  2. On the Connect to source database page, complete the fields with the connection information for your source database, and then select Continue.

  3. On the Connect to Snowflake page, complete the fields with your connection information, and then select Continue.

  4. On the Select tables to migrate data page, select the tables that you want to migrate.

    The following image is an example of the page:

  5. Select Migrate data.

    When the migration completes successfully, the Results page appears.

    If the results don't look correct, you can select Retry data migration to run the data migration process again.

  6. When the results look correct on the Results page, select Go to data validation to start the validation process.

言語: 日本語