Native OpenTelemetry ingest is available in Hydrolix version 6.3 and later.
A transform maps incoming OpenTelemetry fields to the columns of a Hydrolix table. Hydrolix schemas are user-defined. Every table needs a transform before it can accept data, and Hydrolix doesn't create one. This page provides a reference transform for each of the three OpenTelemetry signal types. Copy one as a starting schema and adjust it for your data.
When Hydrolix receives OpenTelemetry data, it flattens the nested structure (resources, scopes, and records) into flat records with snake_case field names. The flattened fields are the same over gRPC and over HTTP. A transform's output columns read from those flattened fields. For an overview of how data reaches Hydrolix, see OpenTelemetry ingest. To create the destination table, see Projects and tables.
To adapt a reference transform, follow these conventions specific to the OpenTelemetry ingest path:
Use snake_case column names that match the flattened input field names.
Use a timestamp primary column: epoch from a nanosecond field such as time_unix_nano, or datetime from an automatic value.
Use map columns for attributes to keep the whole object queryable by key.
Use from_json_pointers to extract a single attribute into its own column.
Use sql_expression to compute a column, such as a span's duration.
Store array-valued fields (events, links, exemplars, quantiles) as string columns and read them with JSON functions. On the native endpoint, mapping one of these to a json or nested column prevents the records from landing.
Output column names match the flattened input fields, which are snake_case. Name output columns in lowercase. On the native endpoint, a column name that contains uppercase letters ingests as NULL. Lowercase names are required for data to land.
The primary column is a timestamp. When the data already carries one, use type: epoch with a source.from_input_field that points to a nanosecond field such as time_unix_nano. When it doesn't, use type: datetime with from_automatic_value set to receive_time or current_time to generate one at ingest.
The source field must carry a value on every record. On the native endpoint, a record whose primary timestamp field is unset is dropped for logs and rejected for traces, because the endpoint has no timestamp to place the record on. Instrumentation SDKs populate these fields, which keeps normal application traffic unaffected. Some test tools, such as telemetrygen, leave them unset. When a source might omit the timestamp, set the primary from an automatic value (receive_time or current_time) instead of the input field.
resource_attributes, log_attributes, span_attributes, and data_point_attributes arrive as key-value objects.
OTLP origin
Hydrolix field
Logs
Metrics
Traces
Resource.attributes
resource_attributes
✓
✓
✓
LogRecord.attributes
log_attributes
✓
data point attributes
data_point_attributes
✓
Span.attributes
span_attributes
✓
To store an entire object, use a map column. A map column preserves every key it contains, including keys not given their own column. Query it by key, for example resource_attributes['service.name'].
To pull a single attribute into its own column, use from_json_pointers. Extract only keys that reliably appear on the data, such as service.name. On the native endpoint, if an extracted key is absent from every record in an ingest batch, that batch fails. A sparse or optional attribute can break ingestion whenever a batch contains none of it. Read optional or sparse attributes from the map column at query time instead.
Use sql_expression to derive a column, such as duration_ms from a span's start and end times. Transform expressions run through Apache DataFusion and use ANSI SQL. Standard functions such as CASE, COALESCE, and CAST work, along with the Hydrolix functions shard_key_hash, normalize_uuid, normalize_ip, and cast_array_elements. ClickHouse-specific functions such as if and sipHash64 are available at query time; in a transform, use their ANSI equivalents, such as CASE in place of if.
A log record is a message, typically with a timestamp and a severity level. The body is usually text, but it can be structured data such as a JSON object. OpenTelemetry log records also carry structured attributes and can reference the trace_id and span_id of the operation that produced them, which connects a log to the request that generated it.
The receiver flattens each log record into these fields.
Field
Type
Description
time_unix_nano
uint64
When the application emitted the log
observed_time_unix_nano
uint64
When the OpenTelemetry pipeline collected the log
severity_number
int32
Numeric severity from 1 (TRACE) to 24 (FATAL)
severity_text
string
Human-readable severity, such as INFO, WARN, or ERROR
body
string
The log message
trace_id
string (hex)
The trace this log belongs to
span_id
string (hex)
The span that produced this log
log_attributes
object
Key-value metadata on the log record
resource_attributes
object
Key-value metadata about the service that produced the log
This transform extracts service_name, keeps the attribute objects as map columns, and writes the rest to typed columns. service_name becomes its own indexed column because it's a common filter and grouping key. The map columns store every attribute key without a fixed schema. Promote a frequently queried attribute to its own indexed column. The reference transform stores body as a string, so a structured body is captured as JSON text.
A metric is a numeric measurement over time. OpenTelemetry groups its metric types into three shapes: gauges and sums (single scalar values), histograms and exponential histograms (value distributions), and summaries (precomputed quantiles). A single metric produces one record per unique combination of attributes. The data_point_attributes field distinguishes the data points of one metric.
The receiver flattens each metric data point into a common set of fields plus a set of value fields that varies by metric_type.
Field
Type
Description
time_unix_nano
uint64
When the data point was recorded
start_time_unix_nano
uint64
Start of the measurement window for cumulative metrics
metric_name
string
What's being measured, such as system.cpu.usage
metric_description
string
Human-readable description of the metric
metric_unit
string
The unit of the measurement
metric_type
string
gauge, sum, histogram, exponential_histogram, or summary
aggregation_temporality
string
How values accumulate: CUMULATIVE, DELTA, or UNSPECIFIED
is_monotonic
bool
For sums, whether the counter only increases
data_point_attributes
object
Dimensions that distinguish this data point
resource_attributes
object
Key-value metadata about the service
exemplars
array
Sample measurements that link a data point to example trace spans
The value fields depend on metric_type. One table holds columns for all of them, and each row populates only the columns for its own type.
This transform covers gauge, sum, and histogram metrics. Its output columns drop the metric_ prefix, so metric_name becomes name and metric_type becomes type, to match common OpenTelemetry and Grafana field names. To store exponential-histogram or summary detail, or to keep exemplars and quantiles, start from the complete metrics transform. Most data points carry an exemplars array, where each element has time_unix_nano, value, span_id, trace_id, and filtered_attributes; summaries instead carry a quantiles array of quantile and value pairs. Store both as string columns, as in the complete transform.
A trace represents the path of a request through a system. It's made up of spans, where each span is a single operation with a parent-child relationship to other spans. The receiver produces one record per span.
The receiver flattens each span into these fields.
Field
Type
Description
trace_id
string (hex)
Identifier shared by all spans in the same request
span_id
string (hex)
Identifier for this span
parent_span_id
string (hex)
The span that started this one, empty for a root span
name
string
What the span represents, such as GET /api/users
kind
string
The span role: SERVER, CLIENT, INTERNAL, PRODUCER, or CONSUMER
start_time_unix_nano
uint64
When the operation started
end_time_unix_nano
uint64
When the operation finished
status_code
string
Whether the operation succeeded: OK, ERROR, or UNSET
status_message
string
Error description when the status is ERROR
span_attributes
object
Key-value metadata specific to this span
resource_attributes
object
Key-value metadata about the service that produced the span
events
array
Timestamped annotations that happened during the span
This transform derives duration_ms from the span's start and end times, extracts service_name, and keeps span_attributes as a map. Deriving duration_ms at ingest stores span latency as a column that queries and dashboards read directly, instead of subtracting the two timestamps on every query.