Skip to content

Merge Metrics

Merge metrics overview⚓︎

Prometheus in Hydrolix⚓︎

The Hydrolix stack includes Prometheus, an open-source metrics database. Hydrolix continuously updates its Prometheus instance with metrics information.

You can query, view, and actively monitor this information using a stack's Grafana instance, or you can access it with your own monitoring platform. See Prometheus Integration for more information about setting up Prometheus with an external server.

Use the Prometheus UI⚓︎

Prometheus has its own web-based UI.

This view is a basic metric view, suitable for quickly entering queries and seeing simple, graphed results. This feature is available in Hydrolix without any additional setup.

Navigate to https://<YourHostname>/prometheus to view dashboards.

For more information about Prometheus metric types, refer to the Prometheus documentation.

Merge metrics⚓︎

Metric Name Type Components Purpose
merge_success (deprecated) Counter Merge peer Count of merge successes.
merge_failure (deprecated) Counter Merge peer Count of merge failures.
query_count (deprecated) Counter Merge peer Count of Merge calls to the catalog.
query_failure (deprecated) Counter Merge peer Count of failed Merge catalog calls.
query_latency_summary (deprecated) Summary Merge peer Latency of calls to the catalog.
merge_duration_summary (deprecated) Summary Merge peer Merge processing duration, in milliseconds.

Merge controller⚓︎

Metric Name Type Components Description Labels
partition_distribution Histogram Merge Controller Histogram of partition memory size. project_id project_name table_id table_name target
segments Gauge Merge Controller Current count of known segments. A segment is a group of buckets that share a merge target. project_id project_name table_id table_name target
active_merges Gauge Merge Controller Current count of active merge operations. project_id project_name table_id table_name target
partitions Gauge Merge Controller Current count of partitions awaiting segmentation. project_id project_name table_id table_name target
partitions_dispatched Counter Merge Controller Count of partitions dispatched for merging. Can be used in combination with the rate function to determine "partitions dispatched for merge per second." project_id project_name table_id table_name target
candidates Gauge Merge Controller Current count of constructed candidates waiting dispatch. project_id project_name table_id table_name target
candidates_dispatched Counter Merge Controller Count of candidates dispatched for merging. Can be used in combination with the rate function to determine "candidates dispatched for merge per second." project_id project_name table_id table_name target
partitions_per_candidate Histogram Merge Controller Distribution of partitions within a dispatched candidate. project_id project_name table_id table_name target
candidate_mem_size Histogram Merge Controller Distribution of calculated memory requirements for dispatched candidates. project_id project_name table_id table_name target
duplicate_partitions Counter Merge Controller Count of partitions sourced that are already being tracked. project_id project_name table_id table_name target
memory_coefficient Gauge Merge Controller Current value of the memory coefficient being applied to sourced partitions. project_id project_name table_id table_name target
expired_segments Counter Merge Controller Count of segments expired for moving out of the target range. project_id project_name table_id table_name target
bucket_duration Histogram Merge Controller Distribution of time in milliseconds of a bucket's age upon closing. The basis label captures the reason for closing. Options are:

full: The bucket reached max size

idle_ttl: The bucket wasn't added to within the idle timeout

age_ttl: The bucket was open longer than the max age

segment_ttl: The bucket was part of a segment that fell out of the target range		 project_id project_name table_id table_name target basis
connected_clients Gauge Merge Controller Count of currently connected merge-peers. pool_id
query_latency Histogram Merge Controller Distribution of response times in milliseconds for catalog operations. project_id table_id method
admin_query_latency Histogram Merge Controller Distribution of response times in milliseconds for admin catalog operations. project_id table_id method
state_duration Histogram Merge Controller Distribution of time in milliseconds spent in each reactor state. project_id table_id state
merge_duty_cycle Gauge Merge Controller Overall work rate of the merge-controller, as a value between 0 and 1. project_id project_name table_id table_name target

Catalog⚓︎

For more information about merge operations in your tables, see Catalog Metadata.

For a complete list of the metrics used by Hydrolix, including Prometheus, RabbitMQ, and others, see All Metrics.