Skip to content

Set Up OpenTelemetry Ingest

Native OpenTelemetry ingest is available in Hydrolix version 6.3 and later.

This guide covers sending OpenTelemetry logs, metrics, and traces to Hydrolix from end to end:

  1. Create the tables.
  2. Define the transforms.
  3. Point a producer at the cluster.
  4. Confirm the data arrives.

For more about the ingest path, see OpenTelemetry ingest.

Before you begin⚓︎

Set up the following before sending data:

  • A running Hydrolix cluster with a native OpenTelemetry endpoint reachable. This guide uses OTLP/gRPC on port 4317. See Protocol for the OTLP/HTTP endpoints.
  • A service account token. Every request carries this token as a bearer token.

Tables and Transforms Must Exist First

Hydrolix schemas are user-defined, so each target table and its transform must exist before the matching data arrives. The endpoint doesn't create them.

Create a table for each signal⚓︎

Create one table for each signal type you plan to send, named in project.table format. A typical setup uses a separate table for logs, metrics, and traces. See Projects and tables.

Create a transform for each table⚓︎

Each table needs a default transform. It maps each flattened OpenTelemetry field to a table column, so the endpoint knows where to write incoming data.

Start from the reference schema for the signal type in Reference transforms, then adjust it for your data.

Send the data⚓︎

Choose how to send telemetry:

  • From an application with an OpenTelemetry SDK. The quickest path for a single application, local development, and early testing. See the SDK Setup Guide.
  • Through an OpenTelemetry Collector. A shared pipeline that batches, retries, filters, and gives many applications one egress path. The recommended option for production. See the Collector Setup Guide.

Both paths target the same endpoint, authenticate with a bearer token, and set two routing headers: x-hdx-table (the target table in project.table format) and, optionally, x-hdx-transform.

Verify the data arrived⚓︎

Query a target table to confirm records are landing.

Confirm Data Arrived
1
SELECT count() FROM my_project.logs

A non-zero count confirms the producer reached the endpoint and the transform accepted the data.

Troubleshooting⚓︎

Most failures return an error to the producer: a gRPC error status, or a 401 when the token is missing or invalid. The exception is an unset primary timestamp, which returns success and drops the record silently. For more on the native-endpoint behavior, see Reference transforms.

Symptom Likely cause What to do
Sends succeed, but no rows appear The record's primary timestamp is unset. Send from an instrumentation SDK, which sets the timestamp. Test tools such as telemetrygen leave it unset.
table not found or no default transform right after setup The new table or transform isn't routable yet Wait up to a minute, then retry the send
A column is empty on every row On the native endpoint, a column name with uppercase letters ingests as NULL Rename the column to lowercase
A whole batch fails to ingest A from_json_pointers column extracts a key that's absent from every record in the batch Extract only always-present keys, and read optional attributes from a map column
Connection or protocol errors The client is sending to a port or path the cluster doesn't serve Check the transport against Protocol. gRPC uses port 4317; HTTP uses the /v1 routes.
Authentication errors The bearer token is missing or invalid Send a valid service account token in the authorization header