Skip to content

Options Precedence

Overview of precedence⚓︎

Queries to the Hydrolix system use flexible query options to maximize control. This page describes the order of resolution and precedence when the query engine processes options from multiple sources.

Categories of query options⚓︎

See Table of Query Options for a listing of the several categories of query options.

Inheritance hierarchy⚓︎

Every query processes the options hierarchy, gathering options that will apply for the query execution.

In the case of duplicate settings, options encountered later completely replace prior settings, and there is no composition of settings.

Therefore, the SQL SETTINGS clause has the highest precedence.

System defaults⚓︎

System defaults are listed in Table of Query Options.

Organization, project, and table query options⚓︎

Query options can be set at any level of the cluster hierarchy.

See also Projects and Tables for more details on the relationships between org, project, and table.

HTTP query headers⚓︎

The HTTP Query API accepts query options with the header X-HDX-query-settings. In the case of duplicate HTTP headers (allowed by specification) later settings will take precedence.

In this GET example, hdx_query_max_streams is set to 10.

Example of Last HTTP Header Taking Precedence
1
2
3
4
GET <YOUR-HYDROLIX-HOST>/query/?query=....
X-HDX-query-settings: hdx_query_max_streams=12
...
X-HDX-query-settings: hdx_query_max_streams=10

Each X-HDX-query-settings header may contain multiple comma separated key=value query options. Don't add a space after each comma separator. For example:

Example of Multiple Query Options in a Single HTTP Header
1
2
GET <YOUR-HYDROLIX-HOST>/query/?query=....
X-HDX-query-settings: hdx_query_pool_name=mypool,hdx_query_max_streams=12

HTTP query parameters⚓︎

Attach one or more optional parameters to your query API requests, to convey query options to the Hydrolix query engine for a specific request. These options take the form of additional HTTP parameters alongside the required query parameter.

Example: Parameters take precedence⚓︎

When a single HTTP request includes the same query option setting in both HTTP headers and parameters, the latter will win.

In this GET example, demonstrating both query parameters and headers, hdx_query_pool_name is set to the value alternatepool.

Example of a Query Parameter Taking Precedence Over an HTTP Header
1
2
GET <YOUR-HYDROLIX-HOST>/query/?query=....&hdx_query_pool_name=alternatepool
X-HDX-query-settings: hdx_query_pool_name=mypool

SQL SETTINGS clause⚓︎

For SQL dialects that support the SETTINGS clause, all Hydrolix query options are available. Multiple query options should be separated by a comma ,.

Use single quotes around string arguments, for example hdx_query_output_format = 'JSON'.

Example: Set query admin comment using SQL⚓︎

In this example, we're writing the results of the query into the storage bucket and providing a comment that the user generating this query is Rita Miller.

Example Usage of Query Admin Comment Field in SQL Statement
1
2
3
4
5
6
7
SELECT COUNT() FROM
sample.cts
WHERE timestamp >= toDateTime(1636289714)
AND timestamp <= toDateTime(1636376114)
AND arrayJoin(data.leaf_cert.all_domains) LIKE '%hydrolix.live%'
SETTINGS hdx_query_output_file_enabled='true',
         hdx_query_admin_comment='User: Rita Miller'

Query options SETTINGS only work for SELECT and INSERT_INTO statements

Support across interfaces⚓︎

The precedence hierarchy is the same regardless of query interface used, but the HTTP header and query parameters are unsupported via ClickHouse or MySQL interfaces, as these are transported over a non-HTTP protocol.

HTTP Query API⚓︎

  • System defaults
  • Organization, project, table query options
  • HTTP query headers
  • HTTP query parameters
  • SQL SETTINGS

ClickHouse native and MySQL Clients⚓︎

Both ClickHouse native and MySQL dialects support the SETTINGS option. Query options precedence for clients using these interfaces is as follows:

  • System defaults
  • Organization, project, table query options
  • SQL SETTINGS

The ClickHouse SQL dialect also supports the SQL FORMAT clause which understands any ClickHouse formats. If a FORMAT clause is supplied, it will take precedence over any setting of query_hdx_output_format.

Hydrolix Connector for Apache Spark⚓︎

The Hydrolix Connector for Apache Spark embeds the Hydrolix engine into the client software, which natively reads Hydrolix partitions.

The Spark SQL dialect doesn't support SETTINGS so the system defaults apply.

Set query options using the API⚓︎

Use the following Config API calls, one for each level of the hierarchy

Set query options using the UI⚓︎

The Query options flyout menu is found in different places, depending on the place in the hierarchy.

For organization, visit Data > Organization Settings.

UI: Query options for organizations found under Data > Organization Settings

For project, visit Data > Tables and use the + to expand a project.

UI: Query options for projects found under Data > Tables

For table, visit Data > Tables, select a table, and scroll to the Advanced options section of the table detail page.

UI: Query options for tables found under Data > Tables

Adjust the query options values

  1. Click the vertical ellipsis (⋮) and select Edit for an org or table. Select Query options if it's a project.
  2. Change query option values in the left-side flyout.
  3. Scroll to the bottom of the flyout and select Save changes.

UI: Query options can be modified using left-side flyout menu

The changes to the query options go into effect immediately.