Cribl
Cribl manages telemetry data for IT and security teams. A Cribl webhook destination can send that data to Hydrolix for storage and querying.
Sending OpenTelemetry data?
Hydrolix also accepts native OpenTelemetry data over OTLP/gRPC. If Cribl already routes OpenTelemetry data, its OpenTelemetry destination can send OTLP over gRPC directly to a Hydrolix cluster, without the webhook destination. To send OpenTelemetry logs, metrics, and traces directly, see OpenTelemetry Ingest.
Prerequisites⚓︎
- The cluster hostname, such as
hostname.hydrolix.live. On a self-managed cluster, the hostname is thehydrolix_urlvalue in the Hydrolix cluster manifest, without thehttps://scheme. - A Hydrolix table for the Cribl data, named as
project_name.table_name, such ascribl_project.cribl_table. To create one, see Create a table using the API. - A service account token for the HTTP Stream API. To create one, see Manage Service Accounts: Service account tokens.
Register a Hydrolix transform⚓︎
A transform maps the Cribl data to table columns. Publish the Cribl transform to the table, as described in Publish a Transform.
Registering the transform in the UI requires the project name, table name, and output_columns property. Registering it with the API requires the project ID, table ID, and the whole transform JSON.
This transform stores every incoming field as JSON in one column, _raw, and the timestamp in _time. To map fields to their own columns, see Data Source Configuration.
Configure Cribl⚓︎
Create a pipeline and a webhook destination in Cribl, then route source data through the pipeline to the destination.
- Create a
hydrolixpipeline in Cribl using the cribl-hydrolix-pipeline.json file. -
Create a webhook destination in Cribl. In Configure > General Settings, set:
Setting Value Output ID hydrolixWebhook URL https://hostname.hydrolix.live/ingest/event?table={my_project.my_table}Method POSTFormat JSON ArrayBackpressure behavior BlockIn Configure > Authentication, set:
Setting Value Authentication Type Auth tokenToken {service_account_token}To keep the token in the Cribl secret store, choose Auth token (text secret) and select the secret.
If the table requires a table access token, add it in Advanced Settings > Extra HTTP headers with the name
X-Hdx-Token. To set one up, see Table Access Tokens. -
Connect the source data to the
hydrolixpipeline. -
To check that data is flowing, query the destination table in Cribl:
Query Recent Cribl Data To learn more about querying in Cribl, see Write Your First Query.
Dashboards⚓︎
A starter Grafana dashboard shows data flow and high-level statistics. To use it, import cribl-default-dashboard.json into Grafana.
Troubleshoot⚓︎
Something not working?
Start at Troubleshooting Symptoms and Fixes. Find the symptom, confirm it with the signal listed there, and follow the link to the procedure.
Requests fail with an HTTP error⚓︎
Required access: User
Confirm: The status code and response body name the problem: 401: Unauthorized (the Authorization header), 403 with missing or bad auth token (a table access token), 403: Forbidden (the ingest_table permission), 400 with a message such as no table ‘x’ found or unhandled content-type ‘x’ (the table, transform, or format), or 404 with not found (the path). No response at all, such as a timeout, means the sender can't reach the cluster.
The status code shows which part of the request was refused.
Fix: correct the part the status code points to.
- 401: the
Authorizationheader is missing, malformed, or not valid, such as an expired token. Get a new token. An integration that runs unattended should use a service account token. See Manage Service Accounts: Service account tokens. - 403 with
missing or bad auth token: the table requires a table access token, in addition to theAuthorizationheader. Send one from its token list in theX-Hdx-Tokenheader or thetokenquery string parameter. See Table Access Tokens. - 403 with
403: Forbidden: the cluster checks ingest permissions, and the token's account doesn't have theingest_tablepermission for the table. Give the account a role with that permission at the table, project, or organization scope. A token that's already in use keeps its old permissions for up to an hour. To use the new role right away, create a new token. See HTTP Stream API: Ingest authorization. - 400: the response body names the problem. See Ingest returns HTTP 400.
- 404: send requests to
https://hostname.hydrolix.live/ingest/event. - No response: check the hostname, and that the sender can reach the cluster over HTTPS.
Related information⚓︎
- More Ways to Stream Your Data with Hydrolix and Cribl - Overview of integrating Cribl Stream with Hydrolix for telemetry pipeline routing and storage
- Cribl Search with Hydrolix - How to use Cribl Search to query data stored in Hydrolix