Masquer les informations personnelles identifiables (PII)¶

Disponibilité régionale¶

Voir Disponibilité régionale.

Limitations¶

• Le masquage est effectué à l’aide de modèles d’AI et peut ne pas trouver toutes les informations personnelles identifiables. Examinez toujours la sortie pour vous assurer de la conformité avec les politiques de confidentialité des données de votre organisation. Veuillez contacter le support Snowflake si AI_REDACT ne parvient pas à masquer certaines PII.

• Les fonctions COUNT_TOKENS et AI_COUNT_TOKENS ne prennent pas encore en charge AI_REDACT.

• Pour l’instant, AI_REDACT fonctionne mieux avec des textes en anglais bien formés. Les performances peuvent varier avec d’autres langues ou du texte présentant de nombreuses fautes d’orthographe, de ponctuation ou de grammaire.

• AI_REDACT currently redacts only US PII and some UK and Canadian PII, where noted in Catégories de PII détectées.

• AI_REDACT is currently limited in the number of tokens it can input and output. Input and output together can be up to 4,096 tokens. Output is limited to 1,024 tokens. If the input text is longer, split it into smaller chunks and redact each chunk separately, perhaps using SPLIT_TEXT_RECURSIVE_CHARACTER. See Chunking example for an example of redacting text that exceeds token limits.

  Note

  Un jeton est la plus petite unité de données traitée par le modèle d’AI. Pour les textes en anglais, les directives de l’industrie considèrent qu’un jeton équivaut à environ quatre caractères ou 0,75 mot.

Catégories de PII détectées¶

AI_REDACT supports redacting the following categorise of PII. The values in the Category column are the strings supported in the optional categories argument.

Catégorie	Remarques
NAME	Recognizes full name, first name, middle name, and last name
EMAIL
PHONE_NUMBER
DATE_OF_BIRTH
GENDER	Recognizes male, female, and nonbinary
AGE
ADDRESS	Identifies: complete postal address (US, UK, CA) street address (US, UK, CA) postal code (US, UK, CA) city (US, UK, CA) state (US) or province (CA) county, borough, or township (US)
NATIONAL_ID	Identifies Social Security numbers (US)
PASSPORT	Identifies passport numbers (US, UK, CA)
TAX_IDENTIFIER	Identifie les numéros d’identification fiscale (ITNs).
PAYMENT_CARD_DATA	Identifies complete card information, card number, expiration date, and CVV
DRIVERS_LICENSE	Supported US, UK, CA
IP_ADDRESS

Error handling¶

Ordinarily, AI_REDACT raises an error if it cannot process the input text. When a query redacts multiple rows, an error causes the entire query to fail. To allow processing to continue with other rows, you can set the session parameter AI_SQL_ERROR_HANDLING_USE_FAIL_ON_ERROR to FALSE. Errors then return NULL instead of stopping the query.

ALTER SESSION SET AI_SQL_ERROR_HANDLING_USE_FAIL_ON_ERROR=FALSE;

With this parameter set to FALSE, you can also pass TRUE as the final argument to AI_REDACT, which causes the return value to be an OBJECT that contains separate fields for the redacted text and any error message. One of these fields is NULL depending on whether the AI_REDACT call processed successfully.

Considérations relatives aux clients¶

AI_REDACT incurs costs based on the number of input and output tokens processed, as with other Cortex AI Functions. See the Snowflake Pricing Guide for details.

Exemples¶

SELECT AI_REDACT(
    input => 'My name is John Smith and I live at twenty third street, San Francisco.'
);

My name is [NAME] and I live at [ADDRESS]

SELECT AI_REDACT(
    input => 'My name is John and I live at twenty third street, San Francisco.',
    categories => ['NAME', 'EMAIL']
);

My name is [NAME] and I live at twenty third street, San Francisco.

-- Create a table with unredacted text
CREATE OR REPLACE TABLE raw_table AS
  SELECT 'My previous manager, Washington, used to live in Kirkland. His first name was Mike.' AS my_column
  UNION ALL
  SELECT 'My name is William and I live in San Francisco. You can reach me at (415).450.0973';

-- view unredacted data
SELECT * FROM raw_table;

-- Create a redaction table
CREATE OR REPLACE TABLE redaction_table (
  value VARCHAR
);

-- Redact PII from raw_table and insert into redaction_table
INSERT INTO redaction_table
SELECT AI_REDACT(my_column) AS value FROM raw_table;

-- view redacted results
SELECT * FROM redaction_table;

-- Run AI_SENTIMENT on redacted text
SELECT
    value AS redacted_text,
    AI_SENTIMENT(value) AS summary_sentiment
FROM redaction_table;

ALTER SESSION SET AI_SQL_ERROR_HANDLING_USE_FAIL_ON_ERROR=FALSE;

-- Create a redaction table with columns for value and error message
CREATE OR REPLACE TABLE redaction_table (
  value VARCHAR,
  error VARCHAR
);

-- Redact PII from raw_table and insert into redaction_table
-- Both the redacted text and any error message are stored
INSERT INTO redaction_table
SELECT
  result:value::STRING AS value,
  result:error::STRING AS error
  FROM (SELECT AI_REDACT(my_column, TRUE) AS result FROM raw_table);

CREATE OR REPLACE TABLE patients (
    patient_id INT PRIMARY KEY,
    patient_notes text
);

CREATE OR REPLACE TABLE final_temp_table AS
WITH chunked_data AS (
    -- Step 1: Split text into chunks
    SELECT
        patient_id,
        chunk.value AS chunk_text,
        chunk.index AS chunk_index
    FROM
        patients,
        LATERAL FLATTEN(
            input => SNOWFLAKE.CORTEX.SPLIT_TEXT_RECURSIVE_CHARACTER(
                patient_notes,
                'none',
                1000
            )
        ) AS chunk
    WHERE
        patient_notes IS NOT NULL
        AND LENGTH(patient_notes) > 0
),
redacted_chunks AS (
    -- Step 2: Apply AI_REDACT to each chunk
  SELECT
  patient_id,
        chunk_index,
        chunk_text,
        TO_VARIANT(results:value) AS redacted_chunk,
        TO_VARIANT(results:error) AS error_string
        from (
    SELECT
        patient_id,
        chunk_index,
        chunk_text,
        AI_REDACT(chunk_text,TRUE) AS results
    FROM
        chunked_data
)
),

-- Step 3: Concatenate redacted chunks
final AS (
SELECT
chunk_text as original,
IFF(error_string IS NOT NULL, chunk_text, redacted_chunk) AS redacted_text,
patient_id,
chunk_index
FROM
    redacted_chunks
)
SELECT * FROM final;

SELECT
  patient_id,
  LISTAGG(redacted_text, '') WITHIN GROUP (ORDER BY chunk_index) AS full_output
  FROM final_temp_table
  GROUP BY patient_id;

Avis juridiques¶

La classification des données d’entrées et de sorties est présentée dans la table suivante.

Pour plus d’informations, reportez-vous à Snowflake AI et ML.