Query System Features

The Hydrolix query system supports the following features.

Name of feature	Brief description
Query head pools	Manage separate query head resources by routing ClickHouse clients by database parameter
Query peer resource pools	Manage independent, scalable query peer pools for different client populations or purposes
Authentication and Authorization	Verify client identity and authorize access
Account permissions	Enforce role-based access controls on connected clients
Data access controls	Allow data administrators to apply granular row- and column-level security controls
Spread list	Distribute partitions over multiple object storage locations to increase overall available object storage I/O requests per second
Shard key	Split data into separate partitions for a single specified column in addition to the primary timestamp
Column value mapping	Retrieve partitions from different object storage locations depending on a row's value
Query partition locality	Use consistent hashing on partition filenames to assign partitions to the same query peer in a resource pool when possible
Query caching	Store and return cached results for identical queries (not enabled by default)
Latest N rows	Return data early during ORDER BY LIMIT queries after collecting enough results, cancelling outstanding requests to query peers
Column aliases	Allow multiple names to refer to a single column
Custom views	Define a subset of columns available to querying clients
Calculated columns	Compute expressions from stored data before returning results to clients

System-specific features

Query partition locality

For every query received, the query head assigns the partitions by filename to a query peer.

Each query peer retrieves the partition manifest and index files and writes them to local disk. The much larger data files aren't cached.

The query head assigns partitions to peers using a consistent hash of the partition name. This increases the likelihood that a query peer will already have the necessary files in its local disk. This optimization minimizes the data transfer from object store when multiple queries require the same partition.

Peer discovery

The query system uses a service, either Zookeeper or HDX query discovery, to track available query peers in a cluster. Hydrolix version 6.2 introduced HDX Query Discovery, a service that replaces Zookeeper and provides Kubernetes-native tracking of query peers and query heads.

This feature was introduced in Hydrolix version 6.2

The hdx-query-discovery container tracks the query peers and query heads available in a cluster, replacing Zookeeper as the active peer registry for each query head. It runs as a sidecar container in each query head pod using the Kubernetes API to monitor and write cluster membership to a JSON file called pods.txt. Each query head polls the JSON file at a 1-second interval and reads its contents when it detects a change.

After any cluster change, HDX query discovery refreshes its known pod and cluster state, then writes changes to the JSON file. Each entry in the file holds:

• Cluster internal IP and port
• Count of available CPU cores
• Query service type: peer or head
• An optional pool name

Newly arrived peers are eligible to receive work immediately. Peers that are no longer present don't receive new work and are expected to complete, whether to success or failure, any assignment they already hold. With each update, the service replaces the file contents with a complete snapshot of current membership. This prevents each query head having to read a backlog of changes.

Configuration

Two tunables transition a cluster away from Zookeeper to using the HDX Query Discovery Service. Set both in the Hydrolix Cluster Spec:

spec:
  disable_zookeeper: true
  hdx_query_discovery_service:
    enabled: true

Set hdx_query_discovery_service.enabled: true before setting disable_zookeeper: true if you want to validate the service in a running cluster. With disable_zookeeper: false, query head continues to read peer membership from Zookeeper and ignores the file written by HDX query discovery, so the service runs with no effect on query scheduling.

See disable_zookeeper and hdx_query_discovery_service for more details.

Verification

The hdx-query-discovery-service container logs the following when updating the pods.txt file contents:

{"level":"info","ts":1787175232.8444688,"caller":"cmd/main.go:149","msg":"wrote discovery file","path":"/var/lib/turbine/db/pods.txt","entries":3}