Surveiller Openflow¶

Accès aux connexions Openflow¶

Snowflake envoie les journaux Openflow vers la table d’événements que vous avez configurée lors de la configuration d’Openflow. Snowflake recommande d’inclure un horodatage dans la clause WHERE des requêtes de tables d’événements. Ceci est particulièrement important en raison du volume potentiel de données générées par les différents composants de Snowflake. En appliquant des filtres, vous pouvez extraire un sous-ensemble de données plus restreint, ce qui améliore les performances de la requête.

La table des événements comprend les colonnes suivantes, qui fournissent des informations utiles concernant les journaux connectés par Snowflake à partir d’Openflow :

• TIMESTAMP : indique quand Snowflake a collecté le journal.

• RESOURCE_ATTRIBUTES : fournit un objet JSON qui identifie le service Snowflake qui a généré le message de journal. Par exemple, il fournit des détails tels que l’application et l’ID de plan de données pour Openflow.

  {
  "application": "openflow",
  "cloud.service.provider": "aws",
  "k8s.container.name": "pg-dev-server",
  "k8s.container.restart_count": "0",
  "k8s.namespace.name": "runtime-pg-dev",
  "k8s.node.name": "ip-10-10-62-36.us-east-2.compute.internal",
  "k8s.pod.name": "pg-dev-0",
  "k8s.pod.start_time": "2025-04-25T22:14:29Z",
  "k8s.pod.uid": "94610175-1685-4c8f-b0a1-42898d1058e6",
  "k8s.statefulset.name": "pg-dev",
  "openflow.dataplane.id": "abeddb4f-95ae-45aa-95b1-b4752f30c64a"
  }
  

  Copy

• RECORD_ATTRIBUTES : pour un service Snowflake, il identifie une source d’erreur (sortie standard ou erreur standard).

  {
  "log.file.path": "/var/log/pods/runtime-pg-dev_pg-dev-0_94610175-1685-4c8f-b0a1-42898d1058e6/pg-dev-server/0.log",
  "log.iostream": "stdout",
  "logtag": "F"
  }
  

  Copy

• VALUE : la sortie standard et l’erreur standard sont divisées en lignes, chaque ligne générant son propre enregistrement dans la table des événements.

  "{\"timestamp\":1746655642080,\"nanoseconds\":80591397,\"level\":\"INFO\",\"threadName\":\"Clustering Tasks Thread-2\",\"loggerName\":\"org.apache.nifi.controller.cluster.ClusterProtocolHeartbeater\",\"formattedMessage\":\"Heartbeat created at 2025-05-07T22:07:22.071Z and sent to pg-dev-0.pg-dev.runtime-pg-dev.svc.cluster.local:8445 at 2025-05-07T22:07:22.080590784Z; determining Cluster Coordinator took 7 millis; DNS lookup for coordinator took 0 millis; connecting to coordinator took 0 millis; sending heartbeat took 1 millis; receiving first byte from response took 1 millis; receiving full response took 1 millis; total time was 9 millis\",\"throwable\":null}"
  

SELECT
    timestamp,
    resource_attributes:"k8s.namespace.name" AS runtime_key,
    parse_json(value::string):loggerName AS logger,
    parse_json(value::string):formattedMessage AS log_value
FROM openflow.telemetry.EVENTS_<account-id>
WHERE true
AND timestamp < dateadd('days', -1, sysdate())
AND record_type = 'LOG'
AND resource_attributes:"k8s.namespace.name" LIKE 'runtime-%'
AND resource_attributes:"k8s.container.name" LIKE '%-server'
AND parse_json(value::string):level = 'ERROR'
ORDER BY timestamp desc
LIMIT 5;

SELECT
    timestamp,
    RESOURCE_ATTRIBUTES:"k8s.namespace.name" AS Namespace,
    RESOURCE_ATTRIBUTES:"k8s.pod.name" AS Pod,
    RESOURCE_ATTRIBUTES:"k8s.container.name" AS Container,
    value
FROM openflow.telemetry.EVENTS_<account-id>
WHERE true
AND record_type = 'LOG'
AND timestamp > dateadd(minute, -5, sysdate())
AND value LIKE '%Caused By%'
ORDER BY timestamp desc
LIMIT 10;

SELECT
    timestamp,
    RECORD_ATTRIBUTES:component AS Processor,
    RECORD_ATTRIBUTES:id AS Processor_ID,
    value AS Running
FROM openflow.telemetry.EVENTS_<account-id>
WHERE true
AND RECORD:metric:name = 'processor.run.status.running'
AND RECORD_TYPE='METRIC'
AND timestamp > dateadd(minute, -5, sysdate());

SELECT
    timestamp,
    RESOURCE_ATTRIBUTES:"k8s.namespace.name" AS Namespace,
    RESOURCE_ATTRIBUTES:"k8s.pod.name" AS Pod,
    RESOURCE_ATTRIBUTES:"k8s.container.name" AS Container,
    value AS CPU_Usage
FROM openflow.telemetry.EVENTS_<account-id>
WHERE TIMESTAMP > dateadd(minute, -1, sysdate())
AND RECORD_TYPE = 'METRIC'
AND RECORD:metric:name ilike 'container.cpu.usage'
AND RESOURCE_ATTRIBUTES:"k8s.namespace.name" ilike 'runtime%'
AND RESOURCE_ATTRIBUTES:"k8s.container.name" ilike '%server'
AND RESOURCE_ATTRIBUTES:"k8s.namespace.name" NOT IN ('runtime-infra', 'runtime-operator')
ORDER BY TIMESTAMP desc, CPU_Usage desc;

Mesures disponibles¶

SELECT * from events
WHERE
1 = 1
AND record_type = 'METRIC'
AND resource_attributes:application = 'openflow'
AND record:metric.name IN ('cores.load', 'cores.available')
ORDER BY timestamp desc
LIMIT 1000;