Emitting Trace Events in Python

You can use the Snowflake telemetry package to emit trace events from a function or procedure handler written in Python. The package is available from the Anaconda Snowflake channel.

You can access stored trace event data by executing a SELECT command on the event table. For more information, refer to Accessing Trace Data.

Note

For guidelines to keep in mind when adding trace events, refer to General Guidelines for Adding Trace Events.

For general information about setting up logging and retrieving messages in Snowflake, refer to Logging Messages from Functions and Procedures.

Before logging from code, you must:

  • Set up an event table to collect messages logged from handler code.

    For more information, refer to Setting up an Event Table.

  • Be sure you have the trace level set so that the data you want are stored in the event table.

    For more information, refer to Setting Trace Level.

Note

For guidelines to keep in mind when adding trace events, refer to General Guidelines for Adding Trace Events.

Adding Support for the Telemetry Package

To use telemetry package, you must make the open source Snowflake telemetry package, which is included with Snowflake, available to your handler code. The package is available from the Anaconda Snowflake channel.

  • In the PACKAGES clause in your CREATE PROCEDURE or CREATE FUNCTION statement, include the snowflake-telemetry-python package. The PACKAGES clause makes the included Snowflake telemetry package available to your code.

    Code in the following example uses the PACKAGES clause to reference the telemetry package as well as the Snowpark library (which is required for stored procedures written in Python – for more information, refer to Writing Stored Procedures in Python).

    CREATE OR REPLACE FUNCTION my_function(...)
      RETURNS ...
      LANGUAGE PYTHON
      ...
      PACKAGES = ('snowflake-telemetry-python')
      ...
    
    Copy
  • Import the telemetry package in your handler code.

    from snowflake import telemetry
    
    Copy

Adding Trace Events

You can add trace events by calling the telemetry.add_event method, passing a name for the event. You can also optionally associate attributes – key-value pairs – with an event.

The add_event method is available in the following form:

telemetry.add_event(<name>, <attributes>)
Copy

where

  • name is a Python string that specifies the name of the trace event.

  • attributes is an OpenTelemetry Attributes object that specifies the attributes for this trace event. This argument is optional. Omit the argument if you do not have any attributes to specify for this trace event.

Handler code in the following example adds two events, FunctionEmptyEvent and FunctionEventWithAttributes. With FunctionEventWithAttributes, the code also adds two attributes: key1 and key2.

telemetry.add_event("FunctionEmptyEvent")
telemetry.add_event("FunctionEventWithAttributes", {"key1": "value1", "key2": "value2"})
Copy

Adding these events results in two rows in the event table, each with a different value in the RECORD column:

{
  "name": "FunctionEmptyEvent"
}
Copy
{
  "name": "FunctionEventWithAttributes"
}
Copy

The FunctionEventWithAttributes event row includes the following attributes in the row’s RECORD_ATTRIBUTES column:

{
  "key1": "value1",
  "key2": "value2"
}
Copy

Adding Span Attributes

You can set attributes – key-value pairs – associated with spans by calling the telemetry.set_span_attribute method.

For details on spans, see How Snowflake Represents Trace Events.

The set_span_attribute method is available in the following form:

telemetry.set_span_attribute(<key>, <value>)
Copy

where:

Code in the following example creates four attributes and sets their values:

// Setting span attributes.
telemetry.set_span_attribute("example.boolean", true);
telemetry.set_span_attribute("example.long", 2L);
telemetry.set_span_attribute("example.double", 2.5);
telemetry.set_span_attribute("example.string", "testAttribute");
Copy

Setting these attributes results in the following in the event table’s RECORD_ATTRIBUTES column:

{
  "example.boolean": true,
  "example.long": 2,
  "example.double": 2.5,
  "example.string": "testAttribute"
}
Copy

Python Examples

The next sections provide examples of adding trace events in Python stored procedures and functions.

Stored Procedure Example

CREATE OR REPLACE PROCEDURE do_tracing()
RETURNS VARIANT
LANGUAGE PYTHON
PACKAGES=('snowflake-snowpark-python', 'snowflake-telemetry-python')
RUNTIME_VERSION=3.8
HANDLER='run'
AS $$
from snowflake import telemetry
def run(session):
  telemetry.set_span_attribute("example.proc.do_tracing", "begin")
  telemetry.add_event("event_with_attributes", {"example.key1": "value1", "example.key2": "value2"})
  return "SUCCESS"
$$;
Copy

UDF Example

CREATE OR REPLACE FUNCTION times_two(x number)
RETURNS NUMBER
LANGUAGE PYTHON
PACKAGES=('snowflake-telemetry-python')
RUNTIME_VERSION=3.8
HANDLER = 'times_two'
AS $$
from snowflake import telemetry
def times_two(x):
  telemetry.set_span_attribute("example.func.times_two", "begin")
  telemetry.add_event("event_without_attributes")
  telemetry.add_event("event_with_attributes", {"example.key1": "value1", "example.key2": "value2"})

  response = 2 * x

  telemetry.set_span_attribute("example.func.times_two.response", response)

  return response
$$;
Copy

When you call the trace event API from a Python function that processes an input row, the API will be called for every row processed by the UDF.

For example, the following statement calls the Python function defined in the previous example for 50 rows, resulting in 100 trace events (two for each row):

select count(times_two(seq8())) from table(generator(rowcount => 50));
Copy

UDTF Example

CREATE OR REPLACE FUNCTION digits_of_number(input number)
RETURNS TABLE(result number)
LANGUAGE PYTHON
PACKAGES=('snowflake-telemetry-python')
RUNTIME_VERSION=3.8
HANDLER = 'TableFunctionHandler'
AS $$
from snowflake import telemetry

class TableFunctionHandler:

  def __init__(self):
    telemetry.add_event("test_udtf_init")

  def process(self, input):
    telemetry.add_event("test_udtf_process", {"input": str(input)})
    response = input

    while input > 0:
      response = input % 10
      input /= 10
      yield (response,)

  def end_partition(self):
    telemetry.add_event("test_udtf_end_partition")
$$;
Copy

When you call the trace event API in the process() method of a UDTF handler class, the API will be called for every row processed.

For example, the following statement calls the process() method defined in the previous example for 50 rows, resulting in 100 trace events (two for each row) added by the process() method:

select * from table(generator(rowcount => 50)), table(digits_of_number(seq8())) order by 1;
Copy