Bindplane
Bindplane manages pipelines that collect, process, and route telemetry. Its webhook destination can send logs to the Hydrolix HTTP Stream API.
For Bindplane setup, sources, and collectors, see the Bindplane documentation.
Native OpenTelemetry ingest
Collectors that Bindplane manages can also send OTLP data, including metrics and traces, to Hydrolix directly. See OpenTelemetry ingest.
Prerequisites⚓︎
- A Hydrolix project and table. To create a table, see Create a table using the API.
- An authorization token for the HTTP Stream API. A long-running pipeline should use a service account token. To create one, see Manage Service Accounts: Service account tokens. Give the service account a role with the
ingest_tablepermission on the table. Service account tokens expire after 365 days by default. Replace the token in the destination before then, or requests fail with401and data is dropped. - A Bindplane configuration with at least one log source.
Create a transform⚓︎
The webhook destination sends only the body of each log record. Timestamps, attributes, and resource fields aren't sent. A body that holds JSON arrives as an object, and Hydrolix maps its keys to columns with a transform. Any other body arrives as a string, which Hydrolix rejects. To send plain-text logs, see Parse plain-text logs.
With the default json_array payload format, each request holds an array of log bodies. For the other format, see Configure the Bindplane webhook destination.
| Webhook Request Body | |
|---|---|
This transform maps those keys. is_default: true makes it the table's default transform. To publish it to the table, see Publish a Transform.
To store more keys, add an output column for each. Set format to match the timestamps in the logs.
Parse plain-text logs⚓︎
A log line that isn't JSON needs a processor that parses it into fields and writes them to the body. In the Bindplane configuration, add a processor to the source, such as Parse with Regex. Keep Source Field Type and Target Field Type set to Body, the defaults for logs, and use a named capture group for each field.
This pattern parses a line such as 2026-01-01T00:00:00Z WARN disk 85 percent full into the timestamp, level, and message keys that the transform maps:
| Regex Pattern | |
|---|---|
Bindplane also has parsers for Grok, key-value, CSV, and XML formats.
Configure the Bindplane webhook destination⚓︎
- In the Bindplane UI, open the configuration, and add a Webhook destination or edit an existing one.
-
Set the connection settings:
Parameter Value Scheme httpsHostname hostname.hydrolix.livePort 443Path /ingest/eventHTTP Method POSTContent Type application/jsonPayload Format json_array, the defaultjson_arraysends each batch of log records in one request.singlesends one request for each record, which means many more requests from a busy source. Its benefit is that each rejected record gets its own400response, which the collector logs as a warning. Withjson_array, rejected records are part of a207response, which Bindplane counts as a success, and nothing is logged. -
Under Additional Headers, add these headers:
Header Value Required AuthorizationBearer {token}Yes X-Hdx-Table{project_name}.{table_name}, such asmy_project.bindplane_logsYes X-Hdx-Transform{transform_name}No, if the table has a default transform X-Hdx-Token{table_access_token}No, unless the table requires a table access token To set up a table access token, see Table Access Tokens.
-
Save the destination, and apply the configuration to its collectors.
Verify⚓︎
-
Query the table in the Hydrolix query interface. If the transform's primary column isn't named
timestamp, use its name in the filter. -
Compare the row count with the number of log lines the source read. Bindplane reports a partial success as a success, so the destination can show no errors while Hydrolix rejects some records. See Some log records are missing.
-
On a self-managed cluster, the
intake-headlogs show each request it processes:Intake Head Log Line
Troubleshoot⚓︎
Something not working?
Start at Troubleshooting Symptoms and Fixes. Find the symptom, confirm it with the signal listed there, and follow the link to the procedure.
Some log records are missing⚓︎
Hydrolix rejects a log body that isn't a JSON object. When a request mixes objects and strings, Hydrolix stores the objects and returns 207 with an error for each rejected record:
| Partial Success Response | |
|---|---|
Bindplane treats any 2xx response as a success, so it doesn't retry or report the rejected records. A request that holds only strings fails with 400. To fix both, parse the logs into fields. See Parse plain-text logs.
To confirm that records are being rejected, set Payload Format to single for a short time. The collector then logs a warning for each rejected record.
Requests fail with an HTTP error⚓︎
Required access: User
Confirm: The status code and response body name the problem: 401: Unauthorized (the Authorization header), 403 with missing or bad auth token (a table access token), 403: Forbidden (the ingest_table permission), 400 with a message such as no table ‘x’ found or unhandled content-type ‘x’ (the table, transform, or format), or 404 with not found (the path). No response at all, such as a timeout, means the sender can't reach the cluster.
The status code shows which part of the request was refused.
Fix: correct the part the status code points to.
- 401: the
Authorizationheader is missing, malformed, or not valid, such as an expired token. Get a new token. An integration that runs unattended should use a service account token. See Manage Service Accounts: Service account tokens. - 403 with
missing or bad auth token: the table requires a table access token, in addition to theAuthorizationheader. Send one from its token list in theX-Hdx-Tokenheader or thetokenquery string parameter. See Table Access Tokens. - 403 with
403: Forbidden: the cluster checks ingest permissions, and the token's account doesn't have theingest_tablepermission for the table. Give the account a role with that permission at the table, project, or organization scope. A token that's already in use keeps its old permissions for up to an hour. To use the new role right away, create a new token. See HTTP Stream API: Ingest authorization. - 400: the response body names the problem. See Ingest returns HTTP 400.
- 404: send requests to
https://hostname.hydrolix.live/ingest/event. - No response: check the hostname, and that the sender can reach the cluster over HTTPS.
Test the connection with curl⚓︎
Send one test event with the same values as the webhook destination. Replace {token}, {project_name}.{table_name}, {transform_name}, and hostname.hydrolix.live.
| Send a Test Event | |
|---|---|
A response of {"code":200,"message":"success"} means the token, table, and transform are correct. Fields in the event that the transform doesn't map aren't stored.