Merge Metrics

The Hydrolix stack includes Prometheus, an open-source metrics database. While the Stack runs, Hydrolix continuously updates its Prometheus instance with metrics information. You can query, view, and actively monitor this information through the use of a stack's Grafana instance or you can access it via your own monitoring platform.

Using Prometheus directly

Prometheus has its own web-based UI, available by visiting in your web browser.

This view is far more basic than Grafana's, suitable for quickly entering queries and seeing simple, graphed results. Hydrolix does make this feature available immediately, without any additional setup.

Hydrolix's metrics

This table lists the metrics available for merge.

For more information about metric types, refer to Prometheus's documentation.

Merge metrics

These metrics correspond to Hydrolix's merge service.

merge_duration_summarySummaryMerge peerMerge processing duration, in milliseconds.
merge_duration_histoHistogramMerge peerMerge processing duration, in milliseconds.
merge_sdk_duration_summarySummaryMerge peerMerge SDK processing duration, in milliseconds.
merge_sdk_duration_histoHistogramMerge peerMerge SDK processing duration, in milliseconds.
merge_candidate_histoHistogramMerge peerPartitions per merge candidate.
merge_successCounterMerge peerCount of merge successes.
merge_failureCounterMerge peerCount of merge successes.

General metrics

These metrics track various counters and statistics regarding data ingestion.

bytes_writtenCounterBatch peer, Stream peerBytes written to the indexer.
partitions_createdCounterBatch peer, Stream peerCount of partitions created.
process_cpu_seconds_totalCounterBatch peer, Stream head, Stream peerTotal user and system CPU time spent in seconds.
process_max_fdsGaugeBatch peer, Stream head, Stream peerMaximum number of open file descriptors.
process_open_fdsGaugeBatch peer, Stream head, Stream peerNumber of open file descriptors.
process_resident_memory_bytesGaugeBatch peer, Stream head, Stream peerResident memory size in bytes.
process_start_time_secondsGaugeBatch peer, Stream head, Stream peerStart time of the process since unix epoch in seconds.
process_virtual_memory_bytesGaugeBatch peer, Stream head, Stream peerVirtual memory size in bytes.
process_virtual_memory_max_bytesGaugeBatch peer, Stream head, Stream peerMaximum amount of virtual memory available in bytes.
promhttp_metric_handler_requests_in_flightGaugeBatch peer, Stream head, Stream peerCurrent number of scrapes being served.
promhttp_metric_handler_requests_totalCounterBatch peer, Stream head, Stream peerTotal number of scrapes by HTTP status code.

Did this page help you?