Skip to content

Bindplane

Bindplane manages pipelines that collect, process, and route telemetry. Its webhook destination can send logs to the Hydrolix HTTP Stream API.

For Bindplane setup, sources, and collectors, see the Bindplane documentation.

Native OpenTelemetry ingest

Collectors that Bindplane manages can also send OTLP data, including metrics and traces, to Hydrolix directly. See OpenTelemetry ingest.

Prerequisites⚓︎

  • A Hydrolix project and table. To create a table, see Create a table using the API.
  • An authorization token for the HTTP Stream API. A long-running pipeline should use a service account token. To create one, see Manage Service Accounts: Service account tokens. Give the service account a role with the ingest_table permission on the table. Service account tokens expire after 365 days by default. Replace the token in the destination before then, or requests fail with 401 and data is dropped.
  • A Bindplane configuration with at least one log source.

Create a transform⚓︎

The webhook destination sends only the body of each log record. Timestamps, attributes, and resource fields aren't sent. A body that holds JSON arrives as an object, and Hydrolix maps its keys to columns with a transform. Any other body arrives as a string, which Hydrolix rejects. To send plain-text logs, see Parse plain-text logs.

With the default json_array payload format, each request holds an array of log bodies. For the other format, see Configure the Bindplane webhook destination.

Webhook Request Body
1
2
3
4
[
  {"level": "info", "message": "user signed in", "timestamp": "2026-01-01T00:00:00Z"},
  {"level": "warn", "message": "disk 85 percent full", "timestamp": "2026-01-01T00:00:01Z"}
]

This transform maps those keys. is_default: true makes it the table's default transform. To publish it to the table, see Publish a Transform.

Bindplane Transform
{
  "name": "bindplane_json",
  "type": "json",
  "settings": {
    "is_default": true,
    "output_columns": [
      { "name": "timestamp", "datatype": { "type": "datetime", "primary": true, "format": "2006-01-02T15:04:05Z", "resolution": "seconds" } },
      { "name": "level", "datatype": { "type": "string" } },
      { "name": "message", "datatype": { "type": "string" } }
    ]
  }
}

To store more keys, add an output column for each. Set format to match the timestamps in the logs.

Parse plain-text logs⚓︎

A log line that isn't JSON needs a processor that parses it into fields and writes them to the body. In the Bindplane configuration, add a processor to the source, such as Parse with Regex. Keep Source Field Type and Target Field Type set to Body, the defaults for logs, and use a named capture group for each field.

This pattern parses a line such as 2026-01-01T00:00:00Z WARN disk 85 percent full into the timestamp, level, and message keys that the transform maps:

Regex Pattern
^(?P<timestamp>\S+) (?P<level>\S+) (?P<message>.*)$

Bindplane also has parsers for Grok, key-value, CSV, and XML formats.

Configure the Bindplane webhook destination⚓︎

  1. In the Bindplane UI, open the configuration, and add a Webhook destination or edit an existing one.
  2. Set the connection settings:

    Parameter Value
    Scheme https
    Hostname hostname.hydrolix.live
    Port 443
    Path /ingest/event
    HTTP Method POST
    Content Type application/json
    Payload Format json_array, the default

    json_array sends each batch of log records in one request. single sends one request for each record, which means many more requests from a busy source. Its benefit is that each rejected record gets its own 400 response, which the collector logs as a warning. With json_array, rejected records are part of a 207 response, which Bindplane counts as a success, and nothing is logged.

  3. Under Additional Headers, add these headers:

    Header Value Required
    Authorization Bearer {token} Yes
    X-Hdx-Table {project_name}.{table_name}, such as my_project.bindplane_logs Yes
    X-Hdx-Transform {transform_name} No, if the table has a default transform
    X-Hdx-Token {table_access_token} No, unless the table requires a table access token

    To set up a table access token, see Table Access Tokens.

  4. Save the destination, and apply the configuration to its collectors.

Verify⚓︎

  1. Query the table in the Hydrolix query interface. If the transform's primary column isn't named timestamp, use its name in the filter.

    Query Recent Bindplane Logs
    1
    2
    3
    4
    SELECT *
    FROM my_project.bindplane_logs
    WHERE timestamp > NOW() - INTERVAL 1 HOUR
    LIMIT 10
    
  2. Compare the row count with the number of log lines the source read. Bindplane reports a partial success as a success, so the destination can show no errors while Hydrolix rejects some records. See Some log records are missing.

  3. On a self-managed cluster, the intake-head logs show each request it processes:

    Intake Head Log Line
    2026-03-12T17:50:57.115898558Z {"file":"http_event.go:353","level":"info","message":"Processing event for table bindplane_logs","timestamp":"2026-03-12T17:50:57.115+00:00"}
    

Troubleshoot⚓︎

Something not working?

Start at Troubleshooting Symptoms and Fixes. Find the symptom, confirm it with the signal listed there, and follow the link to the procedure.

Some log records are missing⚓︎

Hydrolix rejects a log body that isn't a JSON object. When a request mixes objects and strings, Hydrolix stores the objects and returns 207 with an error for each rejected record:

Partial Success Response
{"code":207,"message":{"success_count":1,"errors":["unexpected data row format string"]}}

Bindplane treats any 2xx response as a success, so it doesn't retry or report the rejected records. A request that holds only strings fails with 400. To fix both, parse the logs into fields. See Parse plain-text logs.

To confirm that records are being rejected, set Payload Format to single for a short time. The collector then logs a warning for each rejected record.

Requests fail with an HTTP error⚓︎

Required access: User

Confirm: The status code and response body name the problem: 401: Unauthorized (the Authorization header), 403 with missing or bad auth token (a table access token), 403: Forbidden (the ingest_table permission), 400 with a message such as no table ‘x’ found or unhandled content-type ‘x’ (the table, transform, or format), or 404 with not found (the path). No response at all, such as a timeout, means the sender can't reach the cluster.

The status code shows which part of the request was refused.

Fix: correct the part the status code points to.

  • 401: the Authorization header is missing, malformed, or not valid, such as an expired token. Get a new token. An integration that runs unattended should use a service account token. See Manage Service Accounts: Service account tokens.
  • 403 with missing or bad auth token: the table requires a table access token, in addition to the Authorization header. Send one from its token list in the X-Hdx-Token header or the token query string parameter. See Table Access Tokens.
  • 403 with 403: Forbidden: the cluster checks ingest permissions, and the token's account doesn't have the ingest_table permission for the table. Give the account a role with that permission at the table, project, or organization scope. A token that's already in use keeps its old permissions for up to an hour. To use the new role right away, create a new token. See HTTP Stream API: Ingest authorization.
  • 400: the response body names the problem. See Ingest returns HTTP 400.
  • 404: send requests to https://hostname.hydrolix.live/ingest/event.
  • No response: check the hostname, and that the sender can reach the cluster over HTTPS.

Test the connection with curl⚓︎

Send one test event with the same values as the webhook destination. Replace {token}, {project_name}.{table_name}, {transform_name}, and hostname.hydrolix.live.

Send a Test Event
1
2
3
4
5
6
curl -X POST "https://hostname.hydrolix.live/ingest/event" \
  -H "Authorization: Bearer {token}" \
  -H "Content-Type: application/json" \
  -H "X-Hdx-Table: {project_name}.{table_name}" \
  -H "X-Hdx-Transform: {transform_name}" \
  -d '{"timestamp": "2026-01-01T00:00:00Z", "message": "test event"}'

A response of {"code":200,"message":"success"} means the token, table, and transform are correct. Fields in the event that the transform doesn't map aren't stored.