HKT Command Reference

hkt

The hkt CLI (Hydrolix Kubernetes Tools) is purely used for Kubernetes deployments and provides the templates needed to deploy and manage Hydrolix within Kubernetes.

Usage

$ hkt [OPTIONS] COMMAND [ARGS]

Options

Option

Purpose

--help

Displays summary documentation for using hdxctl and exits.

Commands

Summary

The following commands are available with hkt.

Command

Purpose

get-resources

Retrieves the YAML for the Kubernetes Cluster and components.

hydrolix-cluster

Creates the Hydrolix Cluster resource definition YAML

operator-resources

Creates the Operator resource definition YAML

scale-defaults

Provides a list of scale-defaults for replicas, cpu, memory, storage and data_strorage for the 3 predefined scale profiles default, dev and prod

version

Retrieves the version of the hkt tool.

config-report

Report all the different options specified in the cluster

get-resources

Retrieves the YAML for the Kubernetes Cluster, jobs and components.

Usage

hkt get-resources [OPTIONS] {secrets|data|init-cluster|keycloak|init-turbine-api|hydrolix|load-sample-project|ingress}

Options

Option

Purpose

--version TEXT

Version override. Default uses the value found in the --cluster-file if not set.

-c, --cluster-file TEXT [required]

Required: The Hydrolix Cluster resource definition file.

--help

Displays summary documentation for using hdxctl and exits.

Flags [required]

Flag

Description

secrets

Describes the secrets of the Kubernetes cluster. Outputs YAML.

data

Describes the data components for the cluster. Outputs YAML.

init-cluster

Describes the initialisation Job that is executed on creation of a cluster. Outputs YAML.

keycloak

Describes the service and deployment of the keycloak component. Outputs YAML.

init-turbine-api

Describes the turbine api initialisation Job that is executed on creation of a cluster. Outputs YAML.

hydrolix

Describes the bulk of the Hydrolix cluster components. Outputs YAML.

load-sample-project

Describes a load Job for example data into the cluster that is executed on creation of a cluster. Outputs YAML.

ingress

Describes ingress (Traefik) components of the cluster. Outputs YAML.

Example

$ hkt get-resources -c hydrolixcluster.yaml init-cluster
apiVersion: batch/v1
kind: Job
metadata:
  name: init-cluster-v1-2-3
  namespace: hdxcli-abc456
spec:
  backoffLimit: 0
  template:
    spec:
      containers:
      - env: []
        envFrom:
        - configMapRef:
            name: general
        - configMapRef:
            name: curated
            optional: true
        - secretRef:
            name: general
        - secretRef:
            name: curated
            optional: true
        image: 123456.abc.def.us-east-2.amazonaws.com/turbine/init-cluster:1234-abcde
        name: init-cluster-v1-2-3
        resources:
          limits: &id001
            cpu: 500m
            ephemeral-storage: 5Gi
            memory: 1Gi
          requests: *id001
      imagePullSecrets:
      - name: hdx-registry-creds
      restartPolicy: Never

hydrolix-cluster

Provides the hydrolix cluster resources for a Hydrolix Kubernetes deployment.

Usage

$ hkt hydrolix-cluster [OPTIONS]

Options

Option

Description

--client-id TEXT [required]

Required. The Hydrolix licence/client_id.

--admin-email TEXT [required]

Required. Email address to be used as an admin email.

--host TEXT [required]

Required. Hostname to be used to access the Hydrolix Cluster

--domain TEXT [required]

Required. Domain name to be used to access the Hydrolix Cluster

--owner TEXT [required]

Required. A named owner for the system.

--cloud TEXT [required]

Required. The cloud the Kubernetes cluster will be deployed upon.

--region TEXT [required]

Required. The Region to be used

--env TEXT

Use an Environment variable within the command

--scale-profile TEXT

Optional. The default scale-profile to use in bringing up the cluster. See scale-defaults below for more information.

--ip-allowlist TEXT

Required. An IP allowlist of IP's / CIDR block to access the cluster. More detailed IP access control can be defined if needed.

--help

Displays summary documentation for this command, and exits.

Example

hkt hydrolix-cluster --ip-allowlist 0.0.0.0/0 [email protected] --client-id=hdxcli-1234abc --host=myhost --owner=imadmin --region=us-central1 --cloud=gcp --domain=mydomain.net --scale-profile=prod
apiVersion: hydrolix.io/v1
kind: HydrolixCluster
metadata:
  name: hdxcli-1234abc
  namespace: hdxcli-1234abc
spec:
  admin_email: [email protected]
  client_id: hdxcli-1234abc
  cloud: gcp
  domain: mydomain.net
  env:
  host: myhost
  ip_allowlist:
  - source: 0.0.0.0/0
  owner: imadmin
  region: us-central1
  scale: {}
  scale_profile: prod

operator-resources

Provides the operator YAML required for a Hydrolix Kubernetes deployment.

Usage

hkt operator-resources [OPTIONS]

Options

Option

Description

--client-id TEXT [required]

The Hydrolix licence/client_id. Required field.

--version

Version of Hydrolix that should be used with this operator. If not set the default is the current version of the HKT Tool.

--help

Displays summary documentation for this command, and exits.

Example

$ hkt operator-resources --client-id hdxcli-1234abc
apiVersion: v1
kind: ServiceAccount
metadata:
  name: operator
  namespace: hdxcli-1234abc
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: operator-abcd1234
rules:
- apiGroups:
  - apiextensions.k8s.io
  resources:
  - customresourcedefinitions
   .....

scale-defaults

Provides a list of scale-defaults for replicas, cpu, memory, storage and data_strorage for the 3 predefined scale profiles default, dev and prod

Usage

$ hkt scale-defaults [OPTIONS]

Options

Option

Description

-p, --profile TEXT

The scale profile to be viewed ( default, dev and prod). If none are specified the default is shown.

--help

Displays summary documentation for this command, and exits.

Example

$ hkt scale-defaults -p dev
service                replicas    cpu  memory    storage    data_storage
-------------------  ----------  -----  --------  ---------  --------------
alter-turbine                 0    0.2  1Gi       1Gi        -
alter-head                    0    0.8  1Gi       1Gi        -
alter-peer                    0    0.8  1Gi       1Gi        -
batch-head                    1    0.5  1Gi       1Gi        -
batch-peer                    1    1    4Gi       1Gi        -
decay                         1    0.5  512Mi     1Gi        -
ui                            1    0.5  1Gi       1Gi        -
traefik                       1    0.5  1Gi       1Gi        -
init-cluster                  1    0.5  1Gi       1Gi        -
init-turbine-api              1    0.5  1Gi       1Gi        -
load-sample-project           1    0.5  1Gi       1Gi        -
intake-api                    1    0.5  1Gi       1Gi        -
kafka-peer                    1    0.5  1Gi       1Gi        -
keycloak                      1    1    1Gi       1Gi        -
kinesis-peer                  1    0.5  1Gi       1Gi        -
merge-head                    1    0.5  1Gi       1Gi        -
merge-peer                    1    0.5  1Gi       1Gi        -
autoingest                    0    0.5  512Mi     1Gi        -
operator                      1    0.5  1Gi       1Gi        -
postgres                      1    1    1Gi       1Gi        5Gi
prometheus                    0    1    1Gi       1Gi        5Gi
query-head                    1    1    1Gi       1Gi        -
query-peer                    1    1    1Gi       1Gi        -
rabbitmq                      1    0.5  1Gi       1Gi        -
reaper                        1    0.5  512Mi     1Gi        -
redpanda                      1    1    1Gi       1Gi        5Gi
stream-head                   1    0.5  1Gi       1Gi        -
stream-peer                   1    0.5  1Gi       1Gi        -
turbine-api                   1    1    1Gi       1Gi        -
version                       1    0.5  1Gi       1Gi        -
zookeeper                     1    0.5  1Gi       1Gi        -

version

Retrieves the version of the hkt tool.

Usage

hkt version [OPTIONS]

Options

Option

Description

--help

Displays summary documentation for this command, and exits.

Example

$ hkt version
v3.10.1

config-report

Display the different variables available in the configuration.

Usage

hkt config-report [OPTIONS]

Options

Option

Description

-c, --cluster-file

Generate a report of the current variable definition for the cluster

Example

hkt config-report
SETTING                             VALUE
----------------------------------  ---------------------------------------
admin_email                         "[email protected]"
batch_peer_threads                  1
client_id                           "hdxcli-12345"
cloud                               "gcp"
domain                              "example-domin.tld"
enable_manifest_cache               true
enable_query_auth                   false
heartbeat_timeout                   600
host                                "example-host"
job_purge_age                       7
job_purge_period                    120
kafka_careful_mode                  false
kafka_tls_ca                        ""
kafka_tls_cert                      ""
merge_batch_size                    10000
merge_interval                      "15s"
merge_max_partitions_per_candidate  100
merge_queue_limit                   500
namespace                           "hdxcli-12345"
overcommit                          false
owner                               "example-user"
per_pool_merge_queues               false
pools.0.name                        "query-peer"
pools.0.service                     "query-peer"
premium_storage_class               "premium-rwo"
query_otlp_exporter_enabled         false
region                              "us-west1"
sdk_timeout_sec                     300
stale_job_batch_size                300
stale_job_limit                     3000
stale_job_period                    120
standard_storage_class              "standard-rwo"
stream_partition_count              50
stream_replication_factor           1
task_start_timeout                  21600
turbine_api_init_pools              false
url                                 "http://example-host.example-domin.tld"
use_https_with_s3                   true
use_tls                             false
vacuum_rejects_dry_run              false
vacuum_rejects_max_age              "168h"
vacuum_rejects_period_mins          180

Environment Variables

It is recommended the following environment variables are used in the operation of hkt:

Name

Description

Example

CLIENT_ID

The Hydrolix licence you would have received when signing up to the platform.

hdxcli-abc456

ADMIN_EMAIL

An email account that will be used as the admin account of the cluster.

[email protected]

OWNER

Name of the owner for the cluster

admin

HYDROLIX_HOST

Hostname for the Hydrolix cluster

myhost

HYDROLIX_DOMAIN

Domain name for the Hydrolix cluster

mydomain.com

CLOUD

The cloud provider to deploy your cluster too.

gcp

REGION

The region in which the Hydrolix cluster will be deployed

us-central1

HKT_VERSION

The Hydrolix version to be used.

v1.2.3

PROJECT_ID

The project_id used for the cluster

Prod-12345


Did this page help you?