Create a Transform (Write Schema)

Hydrolix needs to know how to transform your data into our data types on ingestion. It's a simple mapping from field name to type. Also called a write schema in other systems.

Hydrolix has a more expressive type system than json (or csv which has none). We have signed and unsigned ints of various widths for example along with arrays of all the types. We can capture timestamps as date time format or unix epoch to millisecond precession. This small effort up front will mean more compression, faster computations and a happier data footprint.

Here is a simple example with a few fields. We specify datetime in golang format, if you're not used to seeing this format check here for examples.

curl -X POST 'https://my-domain.hydrolix.live/config/v1/orgs/{{org_uuid}}/projects/{{project_uuid}}/tables/{{table_uuid}}/transforms/' \
-H 'Authorization: Bearer thebearertoken1234567890abcdefghijklmnopqrstuvwxyz' \
-H 'Content-Type: application/json' \
-d '{
  "name": "mytransformname",
  "description": " a description of transform",
  "type": "json",
  "settings": {
    "is_default": true,
    "output_columns": [
      {
        "name": "timestamp",
        "datatype": {
          "type": "datetime",
          "format": "2006-01-02 15:04:05",
          "primary": true,
          "resolution": "seconds"
        }
      },
      {
        "name": "imabignumber",
        "datatype": {
          "type": "uint64"
        }
      },
      {
        "name": "stringoftext",
        "datatype": {
          "type": "string"
        }
      },
      {
        "name": "arrayofstrings",
        "datatype": {
          "type": "array",
          "elements": [
            {
              "type": "string"
            }
          ]
        }
      }
    ]
  }
}

As with all API responses, you will find a transform uuid in the response. Allowing you to resubmit a PATCH or PUT request if you make any mistakes or want to extent the transform.

More information on the full range of Data Types and Transforms (Write Schema).


Did this page help you?