Batch metrics overview

Prometheus in Hydrolix

The Hydrolix stack includes Prometheus, an open-source metrics database. Hydrolix continuously updates its Prometheus instance with metrics information.

You can query, view, and actively monitor this information using a stack's Grafana instance, or you can access it with your own monitoring platform. See Prometheus Integration for more information about setting up Prometheus with an external server.

Use multiple components with a metric

If more than one component uses a given metric, querying it will return results from all relevant components. You can restrict results to a specific component by adding a service keyword to your query.

For example, "process_open_fds{service="stream-peer"}".

For a list of all Hydrolix metrics, see All Metrics.

Each ingest method's peers has multiple containers. One container runs message acquisition, and the other is the indexer which indexes and completes enrichment jobs.

Batch metrics

These metrics track activity specific to batch ingestions.

Metric Name	Type	Components	Purpose
`processed_count`	Counter	Batch peer	Count of items processed.
`processed_failure`	Counter	Batch peer	Count of processing failures.
`rows_read`	Counter	Batch peer	Count of rows read.
`processing_duration_histo`	Histogram	Batch peer	Histogram of batch processing durations in milliseconds.
`processing_duration_summary`	Summary	Batch peer	Summary of batch processing durations in milliseconds.

Age metrics

Metric Name	Type	Components
`query_count`	Counter	Decay/Reaper calls to the catalog.
`query_failure`	Counter	Decay/Reaper failed catalog calls.
`query_latency_summary`	Counter	Latency in calls to catalog.
`query_latency_summary_count/sum`	Counter	Latency in calls to catalog.

Indexer Metrics

Indexer metrics cover the indexing and enrichment of ingested data. These are available across all of the peer components.

Metric Name	Type	Components	Purpose
`hdx_sink_row_count`	Counter	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Count of rows processed by the indexer and uploaded to storage. Includes Hot and Cold reporting.
`hdx_sink_byte_count`	Counter	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Count of bytes processed by the indexer and uploaded to storage. Includes Hot and Cold reporting.
`hdx_sink_value_count`	Counter	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Count of values processed by the indexer and uploaded to storage. Includes Hot and Cold reporting.
`hdx_sink_error_count`	Counter	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Count of errors in indexing and uploading to storage.
`indexer_rows_written_count/bucket/sum`	Histogram	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Total rows indexed (written to partitions)
`indexer_bytes_written_count/bucket/sum`	Histogram	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Total bytes indexed (written to partitions)
`indexer_partitions_rejected_count/bucket/sum`	Histogram	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Histogram of partitions not able to written. If value is 0=raw data parsing failed, 1=raw data / transform schema mismatch, 3=Error writing partition file, 4= Other Error during indexing
`indexer_partitions_written_count/bucket/sum`	Histogram	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Total partitions created
`indexer_partition_write_seconds_count/bucket/sum`	Histogram	Batch (inc. Autoingest), Kafka, Kinesis, Stream HTTP, Intake Head	Time from receiving indexing query to writing partition file (seconds)

For a complete list of the metrics used by Hydrolix, including Prometheus, RabbitMQ, and others, see All Metrics.