Skip to content

Query System Observability

This page illustrates data flow for logging and metrics from the query system and where these are collected.

Observability data flow⚓︎

Most nodes in the diagram link to relevant documentation.

---
config:
  flowchart:
    padding: 20
---
flowchart TB

subgraph client-software[Client Applications]
    clickhouse-http-clients[ClickHouse HTTP
                            Clients]
    mysql-clients[MySQL
                  Clients]
    clickhouse-clients[ClickHouse
                       Clients]
    http-client[HTTP-capable
                Clients]
end

subgraph query-system[Hydrolix Query System]
  http-query-api[HTTP Query
                 API server]
  query-head-h[ClickHouse
               HTTP]
  query-head-c[ClickHouse
               Native]
  query-head-m[MySQL
               Server]
end

clickhouse-clients --> query-head-c
mysql-clients --> query-head-m
clickhouse-http-clients --> query-head-h

http-client -- HTTP header hdx_query_debug=true --> http-query-api

query-system -- sends system
                  logs (vector) --> hydrologs
query-system -- Prometheus
                  scrapes --> metrics
query-system -- records recent
                  queries in a table --> active-queries[ active queries ]
query-system -. optionally writes
                  logs (vector) .-> storage
http-query-api -- HTTP header
                  X-Hdx-Query-Stats --> http-client

subgraph observability-systems[Observability Systems]
  storage[Primary Storage]
  hydrologs[Hydrologs]
  active-queries[Active Queries]
  metrics[Prometheus Metrics]
end


click hydrologs href "https://docs.hydrolix.io/latest/self-managed/observability/hydrologs/"
click active-queries href "https://docs.hydrolix.io/latest/exploring-data/query-troubleshooting/active-queries/"
click metrics href "https://docs.hydrolix.io/latest/exploring-data/query-troubleshooting/query-metrics/"

click http-query-api href "https://docs.hydrolix.io/latest/exploring-data/query-interfaces/http-query-api/"
click query-head-h href "https://docs.hydrolix.io/latest/exploring-data/query-interfaces/clickhouse/clickhouse-http/"
click query-head-c href "https://docs.hydrolix.io/latest/exploring-data/query-interfaces/clickhouse/"
click query-head-m href "https://docs.hydrolix.io/latest/exploring-data/query-interfaces/mysql-client/"

click mysql-clients href "https://docs.hydrolix.io/latest/exploring-data/query-interfaces/mysql-client/"
click clickhouse-clients href "https://docs.hydrolix.io/latest/exploring-data/query-interfaces/clickhouse/clickhouse-client/"

Where to find observability data⚓︎