Skip to content

HTTP Query API

Each cluster runs the Hydrolix HTTP query API. The API endpoint is https://hostname.hydrolix.live/query.

The API provides both a recommended POST and alternate GET endpoint.

This interface supports

Clients may authenticate with credentials or present an authorization token.

Streaming results⚓︎

Use query option hdx_query_streaming_result to request a streamed response as soon as any data become available. By default this setting is false.

The server only accepts this query option as an HTTP header or query parameter. It chooses a buffer allocation strategy for streaming before parsing the SQL, so it doesn't accept hdx_query_streaming_result in an SQL SETTINGS clause.

Response data from the server begins streaming to the client as data is available. This reduces the time before the client application can begin reading data.

When beginning a streaming response, the X-Hdx-Query-Stats aren't complete, so the response header is omitted. To analyze query performance stats, correlate the X-Clickhouse-Query-Id with logs, for example Hydrologs or active queries.

These examples demonstrate an HTTP client sending a query and receiving a response. The logs are timestamped with the ts -i "%.T" tool to record incremental time passing between lines.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
00:00:00.000007 X-Hdx-Query-Settings: hdx_query_streaming_result=true
00:00:00.000006
00:00:00.425689 SELECT
00:00:00.000052   *
00:00:00.000011 FROM hydro.logs
00:00:00.000007 WHERE (timestamp > (NOW() - INTERVAL 58 MINUTES))
00:00:00.000000
00:00:01.818280  # (1)!
00:00:00.000071
00:00:00.000010 HTTP/1.1 200 OK

00:00:00.000006 X-Clickhouse-Format: TSV
00:00:00.000006 X-Clickhouse-Query-Id: 94cfb9e2-1184-48b5-84f8-e461cbe9c21d
  1. Example time differential is 1.8 seconds.

In this example using streaming, the elapsed time to client's receipt of initial data after sending the query is 1.8 seconds.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
00:00:00.000007 X-Hdx-Query-Settings: hdx_query_streaming_result=false
00:00:00.000006
00:00:00.155948 SELECT
00:00:00.000062   *
00:00:00.000015 FROM hydro.logs
00:00:00.000011 WHERE (timestamp > (NOW() - INTERVAL 58 MINUTES))
00:00:00.000000
00:00:11.676516  # (1)!
00:00:00.000063
00:00:00.000014 HTTP/1.1 200 OK

00:00:00.000008 X-Clickhouse-Format: TSV
00:00:00.000008 X-Clickhouse-Query-Id: 4e2b6264-49ef-4cd0-8bae-68c684744001
00:00:00.000009 X-Hdx-Query-Stats: exec_time=11574 head_rows_read=549045 head_bytes_read=2976353124 peer_rows_read=668471 peer>
  1. Example query time differential is 11.6 seconds.

In this example, without streaming, the elapsed time to initial data is 11.6 seconds after the client sends the query. The server used the time to collect the entire result set.

Present an authorization token⚓︎

The HTTP query API supports auth tokens and authentication via local user credentials.

Get the bearer token, which is good for the next 24 hours, to authenticate future API calls. This command assumes you've set the $HDX_HOSTNAME, $HDX_USER, and $HDX_PASSWORD environment variables:

1
2
3
4
5
6
7
8
export HDX_TOKEN=$(
  curl -v -X POST -H "Content-Type: application/json" \
  https://$HDX_HOSTNAME/config/v1/login/ \
  -d "{
    \"username\":\"$HDX_USER\",
    \"password\":\"$HDX_PASSWORD\"  
  }" | jq -r ".auth_token.access_token"
)

If the token is valid, the HTTP Query API returns an HTTP 200 and the response.

1
2
3
4
5
curl -X POST \
  --header "Authorization: Bearer ${HDX_TOKEN}" \
  --url https://hostname.hydrolix.live/query \
  --data "SELECT COUNT(*) FROM hydro.logs WHERE timestamp > NOW() - INTERVAL 1 HOUR;" \
2354732

If the token is invalid, the HTTP Query API returns an HTTP 400 and the following error.

1
2
3
4
{
    "error": "Code: 516. DB::Exception: username@example.com: Authentication failed: password is incorrect, or there is no user with such name. (AUTHENTICATION_FAILED)",
    "query": "SELECT COUNT(*) FROM hydro.logs WHERE timestamp > NOW() - INTERVAL 1 HOUR;"
}

If the token is empty, the HTTP Query API returns an HTTP 400 and the following error.

1
2
3
4
{
    "error": "std::exception. Code: 1001, type: h::db::ContextError, e.what() = <ContextError no authentication token provided> (version 24.8.6.1)",
    "query": "SELECT COUNT(*) FROM hydro.logs WHERE timestamp > NOW() - INTERVAL 1 HOUR;"
}

Authenticate with credentials⚓︎

If the credentials are valid, the HTTP Query API returns an HTTP 200 and the response.

1
2
3
4
5
curl -X POST \
  --user "${HDX_USERNAME}:${HDX_PASSWORD}" \
  --url https://hostname.hydrolix.live/query \
  --data "SELECT COUNT(*) FROM hydro.logs WHERE timestamp > NOW() - INTERVAL 1 HOUR;" \
2354732

If the credentials are invalid, the HTTP Query API returns an HTTP 400 and the following error.

1
2
3
{
    "error": "<TurbineApiAuthenticatorError api login failed with provided username/password 'username@example.com'. <HttpPermanentResponseError error=request_failed status_code=401 path=/config/v1/login {\"detail\":\"Could not login\"}>>",
    "query": "SELECT COUNT(*) FROM hydro.logs WHERE timestamp > NOW() - INTERVAL 1 HOUR;"