Query API Options
For HTTP POST requests, the following can be added to the body of the request to alter how a query is executed or how a response is formulated at run-time.
For HTTP GET requests, the following can be appended to the end of a query URL to alter how a query is executed or how a response is formulated at run-time.
By default, Hydrolix Query API returns results as formatted JSON. Many other formats are supported; for more information visit https://clickhouse.tech/docs/en/interfaces/formats/
|query.output_format||too many to list||JSON|
The following are “rate” limiters for how the query components execute a query and retrieve partitions from distributed storage. It is strongly suggested that should a user wish to change these away from the defaults they talk to Hydrolix support first.
By default, query processing is distributed across all available query peers (to maximize massively parallel processing). If fewer peers are desired, this flag will instruct the query head to use a subset of available peers.
By default, each query peer will run one process (aka, a “stream”) per CPU core, and this is the recommended configuration. In the unlikely situation where a different number of processes is desired, that number may be specified here.
Hydrolix query processing generally requires each query process to extract data from many HDX partitions. This flag sets a limit on the number of partitions which may be “open” at the same time. Note that this setting is applied per process, so, for example, a 4 core query peer running 4 processes, each opening 25 partitions, will result in 100 open partitions per peer. Decreasing this setting may slow down query performance. Depending on the size of each partition, there’s some risk that opening more than 25 will result in excessive memory pressure on the peer. Tread carefully.
Depending on the number of columns required and predicates applied to a particular query, the processing of each HDX partition may generate many independent HTTP range requests from the query peer to cloud storage to extract data. This flag sets a limit on the number of in-flight downloads which may be “open” at the same time. Please note that this setting is applied per partition, so, if there are 100 open partitions (as in the example above), and each partition is allowed to have 2 concurrent iops, a single box may have 200 simultaneous HTTP requests awaiting data. Tread carefully.
|Query Option||Min Value||Max Value||Default Value|
|query.max_peers||1||total # available query peers||null (all peers)|
|storage.max_streams||1||2x # available CPU cores||null (1 per core)|