Skip to content

Query Head Pools

This feature was introduced in Hydrolix version 5.11.

Query head pools provide parallel, fully isolated query infrastructure in a single cluster. Before version 5.11, query heads were shared across all clients and resource pools. With query head pooling, both query head and query peer pools can be created independently.

Connection routing⚓︎

Query head pooling routes incoming ClickHouse connections through a proxy layer before they reach a query head pool:

  • ClickHouse HTTP connections: Pass through http_proxy, an HTTP reverse proxy.
  • ClickHouse native (TCP) connections: Pass through ClickProx, a native TCP proxy.

The proxy layer reads the database or username parameter from the connection and applies routing rules defined in the HydrolixCluster spec to direct traffic to the appropriate pool or the default pool.

Clients must set the database or username parameter

Routing rules take effect only when the client sets the database or username parameter in its ClickHouse connection. The ClickHouse database parameter is a connection-level setting in the ClickHouse protocol and is separate from Hydrolix's project.table table notation. Connections that don't match any routing rule are sent to the default pool.

ClickProx⚓︎

ClickProx was introduced in Hydrolix version 5.11.

ClickProx is a Python asyncio-based native TCP proxy. It routes ClickHouse native protocol connections between Traefik and query head pools based on the database or username parameter in the client's Hello packet. Because it operates at the TCP level, it's transparent to ClickHouse clients and doesn't require any client configuration changes.

The HTTP proxy serves the same routing role for ClickHouse HTTP connections, reading the database parameter from the HTTP query parameters.

Connection flow with and without query head pooling⚓︎

---
title: Query Head Pooling Enabled
---
graph TB
  httpclient
  clickhouse

  subgraph traefik[Traefik]
    clickprox
    http_proxy[http_proxy]
  end

  httpclient == "`HTTP, tcp/8080`" ==> http_proxy
  clickhouse -- "`native, tcp/9440`" --> clickprox

  subgraph defaulthead
    direction TB
    qh_default[Default Query Heads]
    qp_default[Default Query Peer Pool]
    qh_default -. "internal" .-> qp_default
  end
  subgraph analyst
    direction TB
    qh_analyst[Analyst Query Heads]
    qp_analyst[Analyst Query Peer Pool]
    qh_analyst -. "internal" .-> qp_analyst
  end

  clickprox -- "`routing rules`" --> analyst
  clickprox -- "`default`" --> defaulthead

  http_proxy == "`routing rules`" ==> analyst
  http_proxy == "`default`" ==> defaulthead
---
title: No Query Head Pooling
---
graph TB
  httpclient
  clickhouse
  subgraph traefik[Traefik]
    qh_default[Default Query Heads]
  end

  clickhouse -- "`native, tcp/9440`" --> qh_default
  httpclient == "`HTTP, tcp/8080`" ==> qh_default

  qp_default[Default Query Peer Pool]
  qp_analyst[Analyst Query Peer Pool]
  qh_default -. "`internal`" .-> qp_default
  qh_default -. "`internal`" .-> qp_analyst