Merge Metrics

Merge metrics overview⚓︎

Prometheus in Hydrolix⚓︎

The Hydrolix stack includes Prometheus, an open-source metrics database. Hydrolix continuously updates its Prometheus instance with metrics information.

You can query, view, and actively monitor this information using a stack's Grafana instance, or you can access it with your own monitoring platform. See Prometheus Integration for more information about setting up Prometheus with an external server.

Use the Prometheus UI⚓︎

Prometheus has its own web-based UI.

This view is a basic metric view, suitable for quickly entering queries and seeing simple, graphed results. This feature is available in Hydrolix without any additional setup.

Navigate to https://<YourHostname>/prometheus to view dashboards.

For more information about Prometheus metric types, refer to the Prometheus documentation.

Merge metrics⚓︎

Metric Name	Type	Components
`merge_success`	Counter	Count of merge successes.
`merge_failure`	Counter	Count of merge successes.
`query_count`	Counter	Merge calls to the catalog.
`query_failure`	Counter	Merge failed catalog calls.
`query_latency_summary`	Counter	Latency in calls to catalog.
`query_latency_summary_count/sum`	Counter	Latency in calls to catalog.
`merge_duration_histo_bucket/count/sum`	Histogram	Merge processing duration, in milliseconds.
`merge_duration_summary`	Summary	Merge processing duration, in milliseconds.

Merge controller (v5.3.0+)⚓︎

Metric Name	Type	Components	Description	Labels
`partition_distribution`	Histogram	Merge Controller	Histogram of partition memory size	`project_id` `project_name` `table_id` `table_name` `target`
`segments`	Gauge	Merge Controller	Current count of known segments. A segment is a group of buckets that share a merge target.	`project_id` `project_name` `table_id` `table_name` `target`
`active_merges`	Gauge	Merge Controller	Current count of active merge operations	`project_id` `project_name` `table_id` `table_name` `target`
`partitions`	Gauge	Merge Controller	Current count of partitions awaiting segmentation	`project_id` `project_name` `table_id` `table_name` `target`
`partitions_dispatched`	Counter	Merge Controller	Count of partitions dispatched for merging. Can be used in combination with the rate function to determine “partitions dispatched for merge per second”	`project_id` `project_name` `table_id` `table_name` `target`
`candidates`	Gauge	Merge Controller	Current count of constructed candidates waiting dispatch.	`project_id` `project_name` `table_id` `table_name` `target`
`candidates_dispatched`	Counter	Merge Controller	Count of candidates dispatched for merging. Can be used in combination with the rate function to determine “candidates dispatched for merge per second”	`project_id` `project_name` `table_id` `table_name` `target`
`partitions_per_candidate`	Histogram	Merge Controller	Distribution of partitions within a dispatched candidate.	`project_id` `project_name` `table_id` `table_name` `target`
`candidate_mem_size`	Histogram	Merge Controller	Distribution of calculated memory requirements for dispatched candidates	`project_id` `project_name` `table_id` `table_name` `target`
`duplicate_partitions`	Counter	Merge Controller	Count of partitions sourced that are already being tracked.	`project_id` `project_name` `table_id` `table_name` `target`
`memory_coefficient`	Gauge	Merge Controller	Current value of the memory coefficient being applied to sourced partitions.	`project_id` `project_name` `table_id` `table_name` `target`
`expired_segments`	Counter	Merge Controller	Count of segments expired for moving out of the target range	`project_id` `project_name` `table_id` `table_name` `target`
`bucket_duration`	Histogram	Merge Controller	Distribution of time in milliseconds of a bucket’s age upon closing. The basis label captures the reason for closing. Options are: `full`: The bucket reached max size `idle_ttl`: The bucket wasn't added to within the idle timeout `age_ttl`: The bucket was open longer than the max age `segment_ttl`: The bucket was part of a segment that fell out of the target range	`project_id` `project_name` `table_id` `table_name` `target` `basis`
`connected_clients`	Gauge	Merge Controller	Count of currently connected merge-peers	`pool_id`
`query_latency`	Histogram	Merge Controller	Distribution of response times in milliseconds for catalog operations	`project_id` `table_id` `method`

Catalog⚓︎

You can find more information about merge operations in your tables in the Catalog Metadata.

For a complete list of the metrics used by Hydrolix, including Prometheus, RabbitMQ, and others, see All Metrics.