Create Project and Table

Hydrolix has a RESTful JSON API that uses JWT authentication. You can login, create a project or table, and other actions. The full API is described in Config API. It's recommended that you use a standard HTTP client like Postman or Insomnia , which makes working with APIs much easier, especially for capturing and using environment variables.

Your API endpoints are specific to your deployment base URI: https://hostname.hydrolix.live/config/v1. You'll need to call three API endpoints:

• POST /login with username & password. The response contains the bearer token and org uuid.
• POST /{{org uuid}}/projects with a name to create a project. The response contains the project uuid.
• POST /{{org uuid}}/project/{{project uuid}}/tables with a name to create a table. The response contains the table uuid.
  

Security

Hydrolix will check your IP has permission to access API: Enabling Access to your platform. If your IP hasn't been enabled, your request will timeout. Hydrolix API users will receive an email asking them to set a password. If you don't have an email, ask your admin to invite you to the project. The API uses a bearer token based on your permissions.

Login

The first step is to login to the API

curl -X POST 'https://hostname.hydrolix.live/config/v1/login' \
-H 'Content-Type: application/json' \
-d '{
    "username": "myusername",
    "password": "mypassword"
}'

{
   "uuid": "cc96f3d0-7f15-4608-9d28-613ab5b6c780",
   "email": "myusername@mycompany.com",
   "orgs": [
      {
         "uuid": "d1234567-1234-1234-abcd-defgh123456",
         "name": "Hydrolix",
         "type": "singletenant"
      }
   ],
   "groups": [
      "org.d1234567-1234-1234-abcd-defgh123456.Hydrolix.admin",
      "org.d1234567-1234-1234-abcd-defgh123456.Hydrolix.operator",
      "org.d1234567-1234-1234-abcd-defgh123456.Hydrolix.publisher",
      "org.d1234567-1234-1234-abcd-defgh123456.Hydrolix.reader"
   ],
   "auth_token": {
      "access_token": "thebearertoken1234567890abcdefghijklmnopqrstuvwxyz",
      "expires_in": 3600,
      "token_type": "Bearer"
   }
}

You will need the orgs.uuid and the auth_token.access_token from the response for subsequent steps. These are typically defined as environment variables in standard HTTP client API tools.

Create a Project

A project is a container for one or more tables. Every table must be within a project. To create a project, use the following API endpoint.

curl -X POST 'https://hostname.hydrolix.live/config/v1/orgs/{{org_uuid}}/projects/' \
-H 'Authorization: Bearer thebearertoken1234567890abcdefghijklmnopqrstuvwxyz' \
-H 'Content-Type: application/json' \
-d '{
    "name": "website",
    "description": "A description of my project"
}'

Quick Tip

Use a project and table name that you know will be easy to use in your queries. Underscores, hyphens, and long names are painful to use if you will be writing lots of queries!

[
   {
      "name": "website",
      "org": "d1234567-1234-1234-abcd-defgh123456",
      "description": "A description of my project",
      "uuid": "myprojectuuid-1324-abcd-efgh-132465789",
      "url": "https://hostname.hydrolix.live/config/v1/orgs/d1234567-1234-1234-abcd-defgh123456/projects/myprojectuuid-1324-abcd-efgh-132465789",
      "created": "2021-05-11T13:27:59.258016Z",
      "modified": "2021-05-11T13:27:59.258037Z"
   }
 ]

You will need the uuid for the next step.

Create a Table

A table is where the data is stored. A single table can store one or more data sets. Data sets that are searched together should be stored together because they have strong context. To create a table, use the following endpoint.

curl -X POST 'https://hostname.hydrolix.live/config/v1/orgs/{{org_uuid}}/projects/{{project_uuid}}/tables/' \
-H 'Authorization: Bearer thebearertoken1234567890abcdefghijklmnopqrstuvwxyz' \
-H 'Content-Type: application/json' \
-d '{
    "name": "events",
    "description": "My new table"
}'

[
   {
      "project": "myprojectuuid-1324-abcd-efgh-132465789",
      "name": "events",
      "description": "My new table",
      "uuid": "mytableuuid-9876-9876-4567-zyxwv1234",
      "created": "2021-08-12T10:32:44.747749Z",
      "modified": "2021-08-12T11:14:21.206759Z",
      "settings": {
         "stream": {
            "hot_data_max_age_minutes": 3,
            "hot_data_max_active_partitions": 3,
            "hot_data_max_rows_per_partition": 12288000,
            "hot_data_max_minutes_per_partition": 1,
            "hot_data_max_open_seconds": 60,
            "hot_data_max_idle_seconds": 30,
            "cold_data_max_age_days": 3650,
            "cold_data_max_active_partitions": 50,
            "cold_data_max_rows_per_partition": 12288000,
            "cold_data_max_minutes_per_partition": 60,
            "cold_data_max_open_seconds": 30,
            "cold_data_max_idle_seconds": 60
         },
         "age": {
            "max_age_days": 0
         },
         "reaper": {
            "max_age_days": 1
         },
        "merge": {
            "enabled": true,
            "sql": ""
        },
        "autoingest": [
        {
            "enabled": false,
            "source": "",
            "pattern": "",
            "max_rows_per_partition": 12288000,
            "max_minutes_per_partition": 60,
            "max_active_partitions": 50,
            "dry_run": false
        }
        ],
        "sort_keys": [],
        "shard_key": null,
        "max_future_days": 0
    },
      "url": "https://hostname.hydrolix.live/config/v1/orgs/d1234567-1234-1234-abcd-defgh123456/projects/myprojectuuid-1324-abcd-efgh-132465789/tables/mytableuuid-9876-9876-4567-zyxwv1234"
   }
]

Make sure to keep the table uuid along with the other uuids you have used previously. Your next job is to create the write schema or transform.

👍 Configuring Settings

This tutorial includes settings that should cover the majority of cases. You can learn more about these settings in the following docs:

• Merge Settings
• Age Settings
• Autoingest Settings