Merge metrics overview

Prometheus in Hydrolix

The Hydrolix stack includes Prometheus, an open-source metrics database. Hydrolix continuously updates its Prometheus instance with metrics information.

You can query, view, and actively monitor this information using a stack's Grafana instance, or you can access it with your own monitoring platform. See Prometheus Integration for more information about setting up Prometheus with an external server.

Use the Prometheus UI

Prometheus has its own web-based UI.

This view is a basic metric view, suitable for quickly entering queries and seeing simple, graphed results. This feature is available in Hydrolix without any additional setup.

Navigate to https://<YourHostname>/prometheus to view dashboards.

For more information about Prometheus metric types, refer to the Prometheus documentation.

Merge metrics

Metric Name	Type	Components
`merge_success`	Counter	Count of merge successes.
`merge_failure`	Counter	Count of merge successes.
`query_count`	Counter	Merge calls to the catalog.
`query_failure`	Counter	Merge failed catalog calls.
`query_latency_summary`	Counter	Latency in calls to catalog.
`query_latency_summary_count/sum`	Counter	Latency in calls to catalog.
`merge_duration_histo_bucket/count/sum`	Histogram	Merge processing duration, in milliseconds.
`merge_duration_summary`	Summary	Merge processing duration, in milliseconds.

Merge controller (v5.3.0+)

Metric Name	Type	Components	Description	Labels
`partition_distribution`	Histogram	Merge Controller	Histogram of partition memory size	`project_id` `project_name` `table_id` `table_name` `target`
`segments`	Gauge	Merge Controller	Current count of known segments. A segment is a group of buckets that share a merge target.	`project_id` `project_name` `table_id` `table_name` `target`
`active_merges`	Gauge	Merge Controller	Current count of active merge operations	`project_id` `project_name` `table_id` `table_name` `target`
`partitions`	Gauge	Merge Controller	Current count of partitions awaiting segmentation	`project_id` `project_name` `table_id` `table_name` `target`
`partitions_dispatched`	Counter	Merge Controller	Count of partitions dispatched for merging. Can be used in combination with the rate function to determine “partitions dispatched for merge per second”	`project_id` `project_name` `table_id` `table_name` `target`
`candidates`	Gauge	Merge Controller	Current count of constructed candidates waiting dispatch.	`project_id` `project_name` `table_id` `table_name` `target`
`candidates_dispatched`	Counter	Merge Controller	Count of candidates dispatched for merging. Can be used in combination with the rate function to determine “candidates dispatched for merge per second”	`project_id` `project_name` `table_id` `table_name` `target`
`partitions_per_candidate`	Histogram	Merge Controller	Distribution of partitions within a dispatched candidate.	`project_id` `project_name` `table_id` `table_name` `target`
`candidate_mem_size`	Histogram	Merge Controller	Distribution of calculated memory requirements for dispatched candidates	`project_id` `project_name` `table_id` `table_name` `target`
`duplicate_partitions`	Counter	Merge Controller	Count of partitions sourced that are already being tracked.	`project_id` `project_name` `table_id` `table_name` `target`
`memory_coefficient`	Gauge	Merge Controller	Current value of the memory coefficient being applied to sourced partitions.	`project_id` `project_name` `table_id` `table_name` `target`
`expired_segments`	Counter	Merge Controller	Count of segments expired for moving out of the target range	`project_id` `project_name` `table_id` `table_name` `target`
`bucket_duration`	Histogram	Merge Controller	Distribution of time in milliseconds of a bucket’s age upon closing. The basis label captures the reason for closing. Options are: `full`: The bucket reached max size `idle_ttl`: The bucket wasn't added to within the idle timeout `age_ttl`: The bucket was open longer than the max age `segment_ttl`: The bucket was part of a segment that fell out of the target range	`project_id` `project_name` `table_id` `table_name` `target` `basis`
`connected_clients`	Gauge	Merge Controller	Count of currently connected merge-peers	`pool_id`
`query_latency`	Histogram	Merge Controller	Distribution of response times in milliseconds for catalog operations	`project_id` `table_id` `method`

Catalog

You can find more information about merge operations in your tables in the Catalog Metadata.

For a complete list of the metrics used by Hydrolix, including Prometheus, RabbitMQ, and others, see All Metrics.