Working with Metrics

Working with Metrics

Timeseries data are measurements or events that are monitored, where time is a critical dimension to the data. This could be server metrics, application performance monitoring metrics, network data, logs, sensor data, events, clicks, IOT, and many other types of analytics data.

Timeseries analytics is the measurement of change over time.

All fields in Hydrolix are indexed, however, time is the primary index. Time as the primary key is one of the reasons why Hydrolix can get query results from object storage quickly. Internally, a table consists of many time partitioned data files in a columnar format. Each file represents sequentially time ordered data and associated metadata. This is all handled behind the scenes and invisible to the user or administrator.

![Figure1][1]

Working with time

Since Hydrolix uses time as a primary key, queries that reference tables should contain a time range in the WHERE clause. This helps maximize performance be reducing the scope of which portions of a table need to be included in a particular search.



[1]:  https://dev-hdx-doc-images.s3.us-east-2.amazonaws.com/current/images/Time-seriesSQL-Functions.png

Ingest Schema (Transform)

Click arrow to expand

{
    "name": "metrics",
    "type": "csv",
    "table": "{{table_uuid}}",
    "settings": {
        "compression": "gzip",
        "format-detail": {
            "delimiter": 9
        },
        "output_columns": [{
                "name": "timestamp",
                "type": "datetime",
                "format": "20060102150405",
                "treatment": "primary",
                "position": 0
            },
            {
                "name": "hostname",
                "type": "string",
                "treatment": "tag",
                "position": 1
            },
            {
                "name": "region",
                "type": "string",
                "treatment": "tag",
                "position": 2
            },
            {
                "name": "datacenter",
                "type": "string",
                "treatment": "tag",
                "position": 3
            },
            {
                "name": "rack",
                "type": "string",
                "treatment": "tag",
                "position": 4
            },
            {
                "name": "os",
                "type": "string",
                "treatment": "tag",
                "position": 5
            },
            {
                "name": "arch",
                "type": "string",
                "treatment": "tag",
                "position": 6
            },
            {
                "name": "team",
                "type": "string",
                "treatment": "tag",
                "position": 7
            },
            {
                "name": "service",
                "type": "string",
                "treatment": "tag",
                "position": 8
            },
            {
                "name": "service_version",
                "type": "string",
                "treatment": "tag",
                "position": 9
            },
            {
                "name": "service_environment",
                "type": "string",
                "treatment": "tag",
                "position": 10
            },
            {
                "name": "usage_user",
                "type": "uint64",
                "treatment": "metric",
                "position": 11
            },
            {
                "name": "usage_system",
                "type": "uint64",
                "treatment": "metric",
                "position": 12
            },
            {
                "name": "usage_idle",
                "type": "uint64",
                "treatment": "metric",
                "position": 13
            },
            {
                "name": "usage_nice",
                "type": "uint64",
                "treatment": "metric",
                "position": 14
            },
            {
                "name": "usage_iowait",
                "type": "uint64",
                "treatment": "metric",
                "position": 15
            },
            {
                "name": "usage_irq",
                "type": "uint64",
                "treatment": "metric",
                "position": 16
            },
            {
                "name": "usage_softirq",
                "type": "uint64",
                "treatment": "metric",
                "position": 17
            },
            {
                "name": "usage_steal",
                "type": "uint64",
                "treatment": "metric",
                "position": 18
            },
            {
                "name": "usage_guest",
                "type": "uint64",
                "treatment": "metric",
                "position": 19
            },
            {
                "name": "usage_guest_nice",
                "type": "uint64",
                "treatment": "metric",
                "position": 20
            }
        ]
    }
}

Query Schema (View)

Click arrow to expand

{
    "name": "metrics_view",
    "description": "metrics sample data set",
    "table": "{{table_uuid}}",
    "settings": {
        "is_default": true,
        "output_columns": [{
                "name": "timestamp",
                "type": "datetime",
                "treatment": "primary"
            },
            {
                "name": "hostname",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "region",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "datacenter",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "rack",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "os",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "arch",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "team",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "service",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "service_version",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "service_environment",
                "type": "string",
                "treatment": "tag"
            },
            {
                "name": "usage_user",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_system",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_idle",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_nice",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_iowait",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_irq",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_softirq",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_steal",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_guest",
                "type": "uint64",
                "treatment": "metric"
            },
            {
                "name": "usage_guest_nice",
                "type": "uint64",
                "treatment": "metric"
            }
        ]
    }
}


Did this page help you?