Hydrolix supports storing and indexing arrays and maps. The following table contains additional information on these data types:

Within the transform schema, these types have an additional elements object nested under the datatype property. You can see examples of this within our example transform schemas for Maps and Arrays.

Elements

The elements object defines the structure of the map or array and the subsequent datatypes that are contained within them. It uses the same settings as the parent object ( type, format, index fields etc) to define how data should be treated.

Maps

🚧

Map keys and values must be homogenous types

While key type and value type can differ within a map, all keys and all values stored in a map must be of the same basic or complex type. The following are examples of valid and invalid transform schema and input data based on this concept:

Invalid

Invalid Transform SchemaExample input data

// cannot create a map which accepts both (String, String) and (String, Int32) data
[
  {
    "name": "invalid_map_heterogenous",
    "datatype": {
      "type": "map",
      "elements": [
        {
          "type": "string",
          "index": true
        },
        {
          "type": "string",
          "index": true
        },
        {
          "type": "string",
          "index": true
        },
        {
          "type": "int32",
          "index": true
        }
      ]
    }
  }
]

[
  {
    "invalid_map_heterogenous":
      {"key1": "value1",
       "key2": 2}
  }
]

Valid

Valid Transform SchemaExample input data

// can instead create two maps within the transform, one which accepts (String, String) 
// and another that accepts (String, Int32) data
[
  {
    "name": "map_string_string",
    "datatype": {
      "type": "map",
      "elements": [
        {
          "type": "string"
        },
        {
          "type": "string"
        }
      ]
    }
  },
  {
    "name": "map_string_int32",
    "datatype": {
      "type": "map",
      "elements": [
        {
          "type": "string"
        },
        {
          "type": "int32"
        }
      ]
    }
  }
]

[
  {
    "map_string_string":
      {"key1": "value1"},
    "map_string_int32": 
      {"key2": 2}
  }
]

Maps are defined as structures containing {key:value} pairs with the key and the value requiring discrete configurations. The following example transform schemas define columns of type map.

Transform Example Schema

[
  {
    "name": "map_string_epoch",
    "datatype": {
      "type": "map",
      "elements": [
        {
          "name": "string_key",
          "type": "string"
        },
        {
          "name": "epoch_value",
          "type": "epoch",  
          "format": "us",
          "resolution": "ms"
        }
      ]
    }
  }
]

[
  {
    "name": "map_string_array",
    "datatype": {
      "type": "map",
      "elements": [
        {
          "name": "string_key",
          "type": "string"
        },
        {
          "name": "array_value",
          "type": "array",
          "elements": [
            {
              "type": "string"
            }
          ]
        }
      ]
    }
  }
]

Query

The following is an example query against a map object of type map(string,uint64):

select mymap['{key}'] from {column} where mymap['{key}'] = 6288

Arrays

🚧

Arrays can only store one type

Values stored within an array must all be of the same basic or complex type.

Array of Maps

Hydrolix supports the use of the Array of Maps data type. The following example transform schemas define columns of type array.

Transform Example Schema

[
  {
    "name": "array_map_string_string",
    "datatype": {
      "type": "array",
      "elements": [
        {
          "name": "map_string_string",
          "type": "map",
          "elements": [
            {
              "type": "string"
            },
            {
              "type": "string"
            }
          ]
        }
      ]
    }
  }
]

[
  {
    "name": "array_map_int_string",
    "datatype": {
      "type": "array",
      "elements": [
        {
          "name": "map_int_string",
          "type": "map",
          "elements": [
            {
              "type": "int32"
            },
            {
              "type": "string"
            }
          ]
        }
      ]
    }
  }
]

Indexing

Hydrolix has been designed to index by default in most cases. Turn off indexing of a column only when necessary. You can read more about indexing at Hydrolix in this blog post.

Enable/Disable

Enabling and disabling indexing is done by setting "index": true or "index": false within the output columns of the transform schema. You can read more about output columns within our Transform Structure documentation. The following example demonstrates how to enable or disable indexing:

"settings": {
   "output_columns": [
        {
            "name": "indexed_array_column",
            "datatype": {
                "type": "array",
                "index": true
            }
        },
        {
            "name": "not_indexed_array_column",
            "datatype": {
                "type": "array",
                "index": false
            }
        }
    ]
}

Array

Supported Index Lookup Operations