Monitor Openflow using telemetry data¶

This topic describes how to monitor the state of Openflow and troubleshoot problems.

Accessing Openflow logs¶

Snowflake sends Openflow logs to the event table you configured when you set up Openflow (BYOC | Snowflake deployment).

Snowflake recommends that you include a timestamp in the WHERE clause of event table queries. This is particularly important because of the potential volume of data generated by various Snowflake components. By applying filters, you can retrieve a smaller subset of data, which improves query performance.

To get started quickly with Openflow’s telemetry, see Example Queries below.

Openflow Telemetry Schema¶

For information about the event table columns, see Event table columns.

The following sections describe how Openflow structures telemetry in an Event Table.

Resource Attributes¶

Describes the event metadata set by Openflow. For general information on other types of resource attributes see RESOURCE_ATTRIBUTES column in the Event Table columns documentation.

Resource Attributes Example:

Copy codeExpand code block

{
  "application": "openflow",
  "cloud.service.provider": "aws",
  "container.id": "a1b2c3d4e5f6",
  "container.image.name": "example-openflow-prod.registry-internal.snowflakecomputing.com/openflow/openflow/openflow_repo/runtime-server",
  "container.image.tag": "2026.3.17.13",
  "deployment.version": "1.35.0",
  "k8s.container.name": "pg-dev-server",
  "k8s.container.restart_count": "0",
  "k8s.namespace.name": "runtime-pg-dev",
  "k8s.node.name": "ip-10-10-62-36.us-east-2.compute.internal",
  "k8s.pod.name": "pg-dev-0",
  "k8s.pod.start_time": "2025-04-25T22:14:29Z",
  "k8s.pod.uid": "94610175-1685-4c8f-b0a1-42898d1058e6",
  "openflow.dataplane.id": "abeddb4f-95ae-45aa-95b1-b4752f30c64a"
}

Show lessSee more

Scroll to top

Scope¶

Scope Example:

Copy codeExpand code block

{
  "name": "runtime"
}

Show lessSee more

Scroll to top

Record Type¶

Depending on the type of Openflow telemetry represented by this row, this will be one of:

• LOG
• METRIC

Openflow does not collect TRACE records, but that is also a valid type for this column in Snowflake Event Tables.

Record¶

Optional. This JSON object describes the type of metric represented by this row.

Record Example:

Copy codeExpand code block

{
  "metric": {
    "name": "connection.queued.duration.max",
    "unit": "millisecond"
  },
  "metric_type": "gauge",
  "value_type": "INT"
}

Show lessSee more

Scroll to top

Record Attributes¶

Logs¶

Record attributes for Logs will typically indicate where this log was sourced. For example, logs from an Openflow Runtime named testruntime could have Record Attributes of:

Copy codeExpand code block

{
  "log.file.path": "/var/log/pods/runtime-testruntime_testruntime-0_66d80cdb-9484-40a4-bdba-f92eb0af14c7/testruntime-server/0.log",
  "log.iostream": "stdout",
  "logtag": "F"
}

Show lessSee more

Scroll to top

System Metrics¶

System metrics like CPU usage will typically not set Record Attributes, so this will be null.

Openflow Application Metrics¶

Record Attributes for Application or “Flow” metrics provide details about the component in the data pipeline that produced the metric. This will vary based on the type of component. See Application Metrics

Copy codeExpand code block

{
  "component": "PutSnowpipeStreaming",
  "execution.node": "ALL",
  "group.id": "c052f9d7-7f76-3013-a2c5-d3b064fa7326",
  "id": "c69e2913-22a9-36bb-a159-6a5ed1fb9d63",
  "name": "PutSnowpipeStreaming",
  "type": "processor"
}

Show lessSee more

Scroll to top

Value¶

This column contains the raw value of the telemetry. For metrics, this will be a numeric value (integer or double). For logs, this will either be a semi-structured string value or a well-formatted JSON string.

Openflow Runtime Logs¶

Openflow Runtimes emit most logs as JSON, so applying Snowflake’s TRY_PARSE_JSON to the VALUE column allows you to further break this value into the following structured fields:

{
  "processGroupId": "6dc1d98f-019d-1000-ffff-ffffa3ba8a09",
  "processGroupIdPath": "/58385a8b-019d-1000-2a52-9ef1c34b0e5f/6dc1d98f-019d-1000-ffff-ffffa3ba8a09",
  "processGroupName": "latency targets",
  "processGroupNamePath": "/Openflow/latency targets",
  "registeredFlowIdentifier": "sqlserver-multidatabase",
  "registeredFlowVersion": "0.29.0-ebb7a257"
}

Application Metrics¶

Connection Metrics¶

Connection Record Attributes¶

Each Connection metric includes the following Record Attributes:

Input and Output Port Metrics¶

Input Port and Output Ports are technically two separate types of components. For consistency, metrics and attributes for Input and Output Ports are the same, with the exception of the type attribute that indicates whether it is an input port or an output port.

Input and Output Port Record Attributes¶

Each Port metric includes the following Record Attributes:

Process Group Metrics¶

Process Group Record Attributes¶

Each Process Group metric includes the following Record Attributes:

Processor Metrics¶

Processor Record Attributes¶

Each Processor metric includes the following Record Attributes:

Additional Attributes for Counters​¶

In addition to the standard Processor attributes above, processor.counter metrics include the following:

Remote Process Group Metrics¶

Remote Process Group Record Attributes¶

Each Remote Process Group metric includes the following Record Attributes:

JVM Metrics¶

JVM Record Attributes¶

JVM metrics do not provide Record Attributes.

CPU Metrics¶

CPU Record Attributes¶

Storage Metrics¶

Storage Record Attributes¶

Example Queries¶

The following queries are examples to get you started with Openflow Telemetry.

All queries assume that Openflow is configured to send telemetry to the default Event Table of SNOWFLAKE.TELEMETRY.EVENTS. If your Snowflake Account or Openflow Deployment is configured with a different Event Table, substitute that table name where you see SNOWFLAKE.TELEMETRY.EVENTS.

Find Stuck FlowFiles¶

This query returns connections with FlowFiles that have been queued for more than some threshold, indicating that they may be stuck and require intervention. Adjust the 30 minute threshold as needed for your use case.

SELECT * FROM (
  SELECT
    resource_attributes:"openflow.dataplane.id" as Deployment_ID,
    resource_attributes:"k8s.namespace.name" as Runtime_Key,
    record_attributes:name as Connection_Name,
    record_attributes:id as Connection_ID,
    MAX(TO_NUMBER(value / 60 / 1000)) as Max_Queued_File_Minutes
  FROM snowflake.telemetry.events
  WHERE true
    AND record_type = 'METRIC'
    AND record:metric:name = 'connection.queued.duration.max'
    AND timestamp > dateadd(minutes, -30, sysdate())
  GROUP BY 1, 2, 3, 4
  ORDER BY Max_Queued_File_Minutes DESC
) WHERE Max_Queued_File_Minutes > 30;

Find Error Logs for Openflow Runtimes¶

SELECT
  timestamp,
  Deployment_ID,
  Runtime_Key,
  parsed_log:level as log_level,
  parsed_log:loggerName as logger,
  parsed_log:formattedMessage as message,
  parsed_log
FROM (
  SELECT
    timestamp,
    resource_attributes:"openflow.dataplane.id" as Deployment_ID,
    resource_attributes:"k8s.namespace.name" as Runtime_Key,
    TRY_PARSE_JSON(value) as parsed_log
  FROM snowflake.telemetry.events
  WHERE true
    AND timestamp > dateadd('minutes', -30, sysdate())
    AND record_type = 'LOG'
    AND resource_attributes:"k8s.namespace.name" like 'runtime-%'
  ORDER BY timestamp DESC
) WHERE log_level = 'ERROR';

Find Running and Non-Running Processors¶

Some flows expect that all processors are in a “running” state, even if they are not actively processing data.

This query helps you find any processors that are running or in another state, such as:

• stopped
• invalid
• disabled

SELECT
  timestamp,
  resource_attributes:"openflow.dataplane.id" as Deployment_ID,
  resource_attributes:"k8s.namespace.name" as Runtime_Key,
  record_attributes:component as Processor,
  record_attributes:id as Processor_ID,
  TO_NUMBER(value) as Running
FROM snowflake.telemetry.events
WHERE true
  AND record:metric:name = 'processor.run.status.running'
  AND record_type = 'METRIC'
  AND timestamp > dateadd(minutes, -30, sysdate());

Find High CPU Usage for Openflow Runtimes¶

Slow data flows or reduced throughput may be the result of a bottleneck on the CPU. Openflow Runtimes scale up automatically, based on the number of minimum and maximum nodes you have configured.

If an Openflow Runtime is using its maximum number of nodes and still CPU usage remains high, consider:

1. Increasing the maximum number of nodes allocated to the Runtime
2. Troubleshoot the Connector or flow to identify the bottleneck

Snowsight Charts provide an easy way to visualize query results for CPU usage over time.

SELECT
  timestamp,
  resource_attributes:"openflow.dataplane.id" as Deployment_ID,
  resource_attributes:"k8s.namespace.name" as Runtime_Key,
  resource_attributes:"k8s.pod.name" as Runtime_Pod,
  TO_NUMBER(value, 10, 3) * 100 as CPU_Usage_Percentage
FROM snowflake.telemetry.events
WHERE true
  AND timestamp > dateadd(minute, -30, sysdate())
  AND record_type = 'METRIC'
  AND record:metric:name ilike 'container.cpu.usage'
  AND resource_attributes:"k8s.namespace.name" ilike 'runtime-%'
  AND resource_attributes:"k8s.container.name" ilike '%-server'
ORDER BY timestamp desc, CPU_Usage_Percentage desc;