Complex Data Types
Hydrolix currently supports arrays and maps. These types have an additional object within datatype
called elements
. The following are supported:
Type | Indexed | Nullable | Description | Stored as |
---|---|---|---|---|
array | Yes (Default, unless Double) | Yes | An array of any one of the primitive [or complex] data types that Hydrolix supports. | array |
map | Yes (Default, unless double) | Yes | A map containing a pair of primitive [or complex] data types | map |
Elements
The elements
object defines the structure of the map or array and the subsequent datatypes that are contained with them. It uses the same settings as the parent object ( type
, format
, index
fields etc) to define how data should be treated.
Maps
Hydrolix supports the use of Maps as a data-type. Maps are defined as { key : string }
definitions with the Key and the Value requiring their own configurations. An example is below where the Key is an indexed string and the value is a epoch millisecond timestamp.
{
"name": "map_column_name",
"datatype": {
"type": "map",
"index": false,
"elements": [
{
"type": "string",
"index": true
},
{
"type": "epoch",
"index": true,
"format": "us",
"resolution": "ms"
}
]
}
}
When being used in a query you would use something like the following for a uint64:
select mymap['uint64'] .... from ....... where mymap['uint64'] = 6288
Array of Maps
Hydrolix supports the use of Array of Maps data-type.
Array of maps are defined as:
"array":
[
{
"key": "string"
},
{
"key2": "other_string"
}
]
Array of maps only support the same datatype
We don't support mix datatype in array of maps, the value needs to always be the same type.
In the previous example it's a string.
The transform for array of maps is the following:
{
"name": "array",
"datatype": {
"type": "array",
"elements":
[
{
"type": "map",
"elements":
[
{
"type": "string"
},
{
"type": "string"
}
]
}
]
}
}
Indexing
Hydrolix has been specifically designed to index by default, without the traditional penalties found in older data platforms. It is therefore strongly suggested that the user should only turn off indexing of a column when absolutely necessary. By default Hydrolix indexes all data types except doubles.
Indexing is turned (on and) off within the data type descriptor in Output Columns.
"settings": {
"output_columns": [
{
"name": "indexed_string_column",
"datatype": {
"type": "string",
"index": true
}
},
{
"name": "not_indexed_string_column",
"datatype": {
"type": "string",
"index": false
}
}
]
}
Updated 11 months ago