Skip to content

Cribl

Cribl manages telemetry data for IT and security teams. A Cribl webhook destination can send that data to Hydrolix for storage and querying.

Sending OpenTelemetry data?

Hydrolix also accepts native OpenTelemetry data over OTLP/gRPC. If Cribl already routes OpenTelemetry data, its OpenTelemetry destination can send OTLP over gRPC directly to a Hydrolix cluster, without the webhook destination. To send OpenTelemetry logs, metrics, and traces directly, see OpenTelemetry Ingest.

Prerequisites⚓︎

  • The cluster hostname, such as hostname.hydrolix.live. On a self-managed cluster, the hostname is the hydrolix_url value in the Hydrolix cluster manifest, without the https:// scheme.
  • A Hydrolix table for the Cribl data, named as project_name.table_name, such as cribl_project.cribl_table. To create one, see Create a table using the API.
  • A service account token for the HTTP Stream API. To create one, see Manage Service Accounts: Service account tokens.

Register a Hydrolix transform⚓︎

A transform maps the Cribl data to table columns. Publish the Cribl transform to the table, as described in Publish a Transform.

Registering the transform in the UI requires the project name, table name, and output_columns property. Registering it with the API requires the project ID, table ID, and the whole transform JSON.

This transform stores every incoming field as JSON in one column, _raw, and the timestamp in _time. To map fields to their own columns, see Data Source Configuration.

Configure Cribl⚓︎

Create a pipeline and a webhook destination in Cribl, then route source data through the pipeline to the destination.

  1. Create a hydrolix pipeline in Cribl using the cribl-hydrolix-pipeline.json file.
  2. Create a webhook destination in Cribl. In Configure > General Settings, set:

    Setting Value
    Output ID hydrolix
    Webhook URL https://hostname.hydrolix.live/ingest/event?table={my_project.my_table}
    Method POST
    Format JSON Array
    Backpressure behavior Block

    In Configure > Authentication, set:

    Setting Value
    Authentication Type Auth token
    Token {service_account_token}

    To keep the token in the Cribl secret store, choose Auth token (text secret) and select the secret.

    If the table requires a table access token, add it in Advanced Settings > Extra HTTP headers with the name X-Hdx-Token. To set one up, see Table Access Tokens.

  3. Connect the source data to the hydrolix pipeline.

  4. To check that data is flowing, query the destination table in Cribl:

    Query Recent Cribl Data
    SELECT _time, _raw FROM my_project.my_table LIMIT 10
    

    To learn more about querying in Cribl, see Write Your First Query.

Dashboards⚓︎

A starter Grafana dashboard shows data flow and high-level statistics. To use it, import cribl-default-dashboard.json into Grafana.

Troubleshoot⚓︎

Something not working?

Start at Troubleshooting Symptoms and Fixes. Find the symptom, confirm it with the signal listed there, and follow the link to the procedure.

Requests fail with an HTTP error⚓︎

Required access: User

Confirm: The status code and response body name the problem: 401: Unauthorized (the Authorization header), 403 with missing or bad auth token (a table access token), 403: Forbidden (the ingest_table permission), 400 with a message such as no table ‘x’ found or unhandled content-type ‘x’ (the table, transform, or format), or 404 with not found (the path). No response at all, such as a timeout, means the sender can't reach the cluster.

The status code shows which part of the request was refused.

Fix: correct the part the status code points to.

  • 401: the Authorization header is missing, malformed, or not valid, such as an expired token. Get a new token. An integration that runs unattended should use a service account token. See Manage Service Accounts: Service account tokens.
  • 403 with missing or bad auth token: the table requires a table access token, in addition to the Authorization header. Send one from its token list in the X-Hdx-Token header or the token query string parameter. See Table Access Tokens.
  • 403 with 403: Forbidden: the cluster checks ingest permissions, and the token's account doesn't have the ingest_table permission for the table. Give the account a role with that permission at the table, project, or organization scope. A token that's already in use keeps its old permissions for up to an hour. To use the new role right away, create a new token. See HTTP Stream API: Ingest authorization.
  • 400: the response body names the problem. See Ingest returns HTTP 400.
  • 404: send requests to https://hostname.hydrolix.live/ingest/event.
  • No response: check the hostname, and that the sender can reach the cluster over HTTPS.