Logging messages in Python¶

You can log messages from a function or procedure handler written in Python by using logging, the logging module in Python’s standard library. When you’ve set up an event table to store log entries, Snowflake stores log entries generated by your handler code in the table.

For more information about logging levels supported by Python, refer to the logging levels documentation. Note that Snowflake treats two of the Python logging levels in a particular way:

The Python CRITICAL level will be treated as FATAL.
The Python NOTSET level will be treated as TRACE.

For general information about setting up logging and retrieving messages in Snowflake, refer to Logging Messages from Functions and Procedures.

Before logging from code, you must:

Set up an event table to collect messages logged from handler code.

For more information, refer to Setting up an Event Table.
Be sure you have the logging level set so that the messages you want are stored in the event table.

For more information, refer to Setting log level.

Overriding log threshold levels with Python¶

You can use Python handler code to override the log threshold levels that have been log level set with SQL. When you set the log level with Python, log entries will use the logging levels defined by Python.

By setting log levels in Python, you can do the following:

Override the threshold set for the Snowflake session or for objects such as the procedure or UDF.
Set thresholds scoped to specified Python packages.

For example, you can use the logger name you set (and which is stored in the event table) to set a threshold for that logger with Python.

Python code in the following example sets the log level for the Snowpark session package to DEBUG.

session_logger = logging.getLogger('snowflake.snowpark.session')
session_logger.setLevel(logging.DEBUG)

Copy

Using the logger name to set logging level¶

You can use the logger name recorded in the event table to set a threshold for log entries from that logger. This can be useful when you want to set a logger’s threshold so that it filters out unwanted log entries above a particular level.

To do that, you’d first query the event table to discover the logger name associated with the entries for which you want to capture a different logging level. Then, using that logger name, you’d set the log level to the threshold you want.

Code in the following example queries for log entries, including the logger name in the returned data. You can get the name as the value of the Scope column.

SET event_table_name='my_db.public.my_event_table';

SELECT
  TIMESTAMP as time,
  RECORD['severity_text'] as log_level,
  SCOPE['name'] as logger_name,
  VALUE as message
FROM
  IDENTIFIER($event_table_name)
WHERE
  RECORD_TYPE = 'LOG';

Copy

This query might return many entries from several loggers. If, after looking through the results, you decide that you’re getting many INFO messages that you don’t want from the numpy logger, you can use Python to set that logger’s threshold to capture log entries at the ERROR level and above.

numpy_logger = logging.getLogger('numpy_logs')
numpy_logger.setLevel(logging.ERROR)

Copy

For more about querying the event table, see Accessing Logged Message Data.

Python examples¶

The following sections provide examples of adding support for logging from Python code.

Stored procedure example¶

Code in the following example imports the logging module, gets a logger, and logs a message at the INFO level.

For more information about logging levels supported by Python, refer to the logging levels documentation.

CREATE OR REPLACE PROCEDURE do_logging()
RETURNS VARCHAR
LANGUAGE PYTHON
PACKAGES=('snowflake-snowpark-python')
RUNTIME_VERSION=3.8
HANDLER='do_things'
AS $$
import logging

logger = logging.getLogger("python_logger")
logger.info("Logging from Python module.")

def do_things(session):
  logger.info("Logging from Python function start.")

  try:
    throw_exception()
  except Exception:
    logger.error("Logging an error from Python handler: ")
    return "ERROR"

  return "SUCCESS"

def throw_exception():
  raise Exception("Something went wrong.")

$$;

Copy

You can access log messages by executing a SELECT command on the event table. For more information, refer to Accessing Logged Message Data.

Code in the following example queries the event table where the log messages are stored. The query reports on the severity and message of each log entry from the handler class.

SET event_table_name='my_db.public.my_event_table';

SELECT
  RECORD['severity_text'] AS SEVERITY,
  VALUE AS MESSAGE
FROM
  IDENTIFIER($event_table_name)
WHERE
  SCOPE['name'] = 'python_logger'
  AND RECORD_TYPE = 'LOG';

Copy

The preceding example generates the following output.

---------------------------------------------------------------------------
| SEVERITY | MESSAGE                                                      |
---------------------------------------------------------------------------
| "INFO"   | "Logging from Python module."                                |
---------------------------------------------------------------------------
| "INFO"   | "Logging from Python function start."                        |
---------------------------------------------------------------------------
| "ERROR"  | "Logging an error from Python handler."                      |
---------------------------------------------------------------------------

Streamlit example¶

Code in the following example imports the logging module, gets a logger, and logs a message at the INFO level.

For more information about logging levels supported by Python, refer to the logging levels documentation.

import streamlit as st
import logging

logger = logging.getLogger('app_logger')

st.title("Streamlit logging example")

hifives_val = st.slider("Number of high-fives", min_value=0, max_value=90, value=60)

if st.button("Submit"):
    logger.info(f"Submitted with high-fives: {hifives_val}")

Copy