Complex Data Types

Hydrolix currently supports arrays and maps. These types have an additional object within datatype called elements. The following are supported:

Elements

The elements object defines the structure of the map or array and the subsequent datatypes that are contained with them. It uses the same settings as the parent object ( type, format, index fields etc) to define how data should be treated.

Maps

Hydrolix supports the use of Maps as a data-type. Maps are defined as { key : string } definitions with the Key and the Value requiring their own configurations. An example is below where the Key is an indexed string and the value is a epoch millisecond timestamp.

{
    "name": "map_column_name",
    "datatype": {
        "type": "map",
        "index": false,
        "elements": [
            {
                "type": "string",
                "index": true
            },
            {
                "type": "epoch",
                "index": true,
                "format": "us",
                "resolution": "ms"
            }
        ]
    }
}

When being used in a query you would use something like the following for a uint64:

select mymap['uint64'] .... from ....... where mymap['uint64'] = 6288

Array of Maps

Hydrolix supports the use of Array of Maps data-type.

Array of maps are defined as:

"array":
[
  {
    "key": "string"
  },
  {
    "key2": "other_string"
  }
]

🚧

Array of maps only support the same datatype

We don't support mix datatype in array of maps, the value needs to always be the same type.
In the previous example it's a string.

The transform for array of maps is the following:

{
    "name": "array",
    "datatype": {
        "type": "array",
        "elements": 
        [
            {
                "type": "map",
                "elements":
                [
                    {
                        "type": "string"
                    },
                    {
                        "type": "string"
                    }
                ]
            }
        ]
    }
}

Indexing

Hydrolix has been specifically designed to index by default, without the traditional penalties found in older data platforms. It is therefore strongly suggested that the user should only turn off indexing of a column when absolutely necessary. By default Hydrolix indexes all data types except doubles.

Indexing is turned (on and) off within the data type descriptor in Output Columns.

"settings": {
   "output_columns": [
        {
            "name": "indexed_string_column",
            "datatype": {
                "type": "string",
                "index": true
            }
        },
        {
            "name": "not_indexed_string_column",
            "datatype": {
                "type": "string",
                "index": false
            }
        }
    ]
}