Commands

Profile

There is a 'default' profile that will be used if you omit --profile <profile-name> on each request to the HDXCLI.

List

hdxcli profile list

Add

One way is to do it interactively:

$ hdxcli profile add <profile-name>
Please, type the host name of your cluster: <host-name>  
Please, type the user name of your cluster: <user-name>  
Will you be using https  (Y/N): \<Y/N>

Or by passing the same information via options:

hdxcli add <profile-name> --hostname <hostname> --username <username> --scheme <http/s>

Show

If you omit --profile <profile-name>, 'default' profile will be shown.

hdxcli --profile <profile-name> profile show

Set/Unset

hdxcli allows you to save both project name and table name to use as default.

Set

hdxcli set <project-name> <table-name>

Unset

hdxcli unset

From now on, when you see --project <project-name> and --table <table-name>, you know that it can be omitted if the set command was previously used.

User

List

hdxcli user list

Delete

hdxcli user delete <user-email>

Show

hdxcli user --user <user-email> show

Assign roles

You can assign just one or multiple roles to a specific user. The option -r, --role can be used multiple times. It's a required setting.

hdxcli user assign-role <user-email> --role <role-name>

Remove roles

The same as the assign-role command, but instead it removes roles from a user.

hdxcli user remove-role <user-email> --role <role-name>

Invite

List

The -p or --pending option can be used to filter the invite list, displaying only pending invitations.

hdxcli user invite list

Send

The -r or --role option can be used multiple times. It's a required setting.

hdxcli user invite send <user-email> --role <role-name>

Resend

hdxcli user invite send <user-email>

Delete

hdxcli user invite delete <user-email>

Show

hdxcli user invite --user <user-email> show\

Role

List

hdxcli role list

Create

There are two ways to create a new role:

Using options. However, if you choose this method, the role can't have two or more policies.
Using the interactive method.

Create a role using options

-n, --name and -p, --permission are required options to create a new role, and the -p, --permission option can be used multiple times. While a role may or may not have a specific scope, these options are considered optional. The -t, --scope-type expects to receive the name of a resource, such as project, table, transform, etc., and the -i, --scope-id should be the UUID identifier for that specific resource.

Use the following command to create a new role:

hdxcli role create --name <role-name> --scope-type <scope-type> --scope-id <scope-id> --permission <permission-name>

Create a role using interactive method

To create a role interactively, use the following command:

$ hdxcli role create
Enter the name for the new role: <role-name>  
Adding a Policy, does it have a specific scope? [Y/n]\: \<y/n>  
Specify the type of scope for the role (e.g., project): <scope-type>  
Provide the 'uuid' for the specified scope: <scope-id>  
1 - commit_alterjob  
2 - change_kinesissource  
3 - ...  
n - All of them  
Enter the numbers corresponding to the permissions you'd want to add (comma-separated): \<i.e:1,2 or n>  
Do you want to add another Policy? [Y/n]\: \<y/n>

***

## Review Role Details

Role Name: <role-name>  
Policy 1:  
  Scope Type: <scope-type>  
  Scope ID: <scope-id>  
  Permissions: \<[permissions]>  
Confirm the creation of the new role? [Y/n]\: \<y/n>  
Created role role-name

Delete

hdxcli role delete <role-name>

Show

hdxcli role --role <role-name> show

Edit

This is an interactive command that allows you to modify the name of a role and add, modify, or delete policies associated with that role. Before finalizing the operation, it presents a detailed view of the modified role for confirmation or cancellation.

hdxcli role edit <role-name>

Add users

Required. This adds just one user or multiple users to a specific role. The option -u, --user can be used multiple times.

hdxcli role add-user <role-name> --user <user-email>

Remove users

The same as add-user command but to remove users from a role.

hdxcli role remove-user <role-name> --user <user-email>

Permissions list

This command displays a list of available permissions. Additionally, it allows filtering by type using the option -t, --type (for example -t function), showing only permissions related to that specific type.

hdxcli role permission list

Project

List

hdxcli project list

Create

hdxcli project create <project-name>

Delete

hdxcli project delete <project-name>

Display activity

hdxcli project --project <project-name> activity

Display statistics

hdxcli project --project <project-name> stats

Show

hdxcli project --project <project-name> show

Settings

Display all settings

hdxcli project --project <project-name> settings

Display single setting

hdxcli project --project <project-name> settings <setting-name>

Modify setting

hdxcli project --project <project-name> settings <setting-name> <new-value>

Migrate

hdxcli project --project <project-name> migrate <new-project-name> --target-profile <profile-name>

Table

List

hdxcli table --project <project-name> list

Create Raw Table (Regular)

If creating a regular table, no additional options are required. Use the table create command without specifying any options.

hdxcli table --project <project-name> create <table-name>

Additionally, an option (--settings-file) exists to set different table settings other than the default. This works for both regular tables and summary tables:

hdxcli table --project <project-name> create <table-name> --settings-file <settings-file>.json

Create Aggregation Table (Summary)

When creating summary tables, the following options are necessary:

--type or -t: Specify as summary.
--sql-query or -s: Provide the SQL query directly via the command line.

Alternatively, use --sql-query-file to specify a file containing the SQL query.

Example using a SQL query in the command line:

hdxcli table --project <project-name> create <table-name> --type summary --sql-query <sql-query>

Example using a file containing the SQL query:

hdxcli table --project <project-name> create <table-name> --type summary --sql-query-file <sql-query-file>.txt

Delete

hdxcli table --project <project-name> delete <table-name>

Display Activity

hdxcli table --project <project-name> --table <table-name> activity

Display Statistics

hdxcli table --project <project-name> --table <table-name> stats

Show

hdxcli table --project <project-name> --table <table-name> show

Settings

Display All Settings

hdxcli table --project <project-name> --table <table-name> settings

Display Single Setting

hdxcli table --project <project-name> --table <table-name> settings <setting-name>

Modify Setting

hdxcli table --project <project-name> --table <table-name> settings <setting-name> <new-value>

Migrate

hdxcli table --project <project-name> --table <table-name> migrate <target-project-name> <new-table-name> --target-profile <profile-name>

Truncate

hdxcli table --project <project-name> truncate <table-name>

Transform

List

hdxcli transform --project <project-name> --table <table-name> list

Create

The field ‘name’ in settings will be replaced by .

hdxcli transform --project <project-name> --table <table-name> create -f <transform-settings>.json <transform-name>

Delete

hdxcli transform --project <project-name> --table <table-name> delete <transform-name>

Show

hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> show

Settings

Display All Settings

hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings

Display Single Setting

hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings <setting-name>

Modify setting

hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings <setting-name> <new-value>

Migrate

hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> migrate <target-project-name> <target-table-name> <new-transform-name> --target-profile <profile-name>

Map-from

The command transform map-from consumes data languages such as SQL, Elastic and others and create a Hydrolix transform from them.

hdxcli transform map-from --ddl-custom-mapping \<sql_to_hdx_mapping>.json \<ddl_file> <transform-name>

To learn more about map-from command, see Mapping DDLs to Hydrolix transform.

Jobs

Batch

List

hdxcli job batch list

Ingest

It will use the default transform in <project-name>.<table-name> if you don't provide --transform <transform-name>.

hdxcli job batch --project <project-name> --table <table-name> --transform <transform-name> ingest <job-name> <job-settings>.json

Delete

hdxcli job batch delete <job-name>

Retry

hdxcli job batch retry <job-name>

Cancel

hdxcli job batch cancel <job-name>

Show

hdxcli job batch --job <job-name> show

Settings

Display all settings

hdxcli job batch --job <job-name> settings

Display single setting

hdxcli job batch --job <job-name> settings <setting-name>

Modify setting

hdxcli job batch --job <job-name> settings <setting-name> <new-value>

Alter

List

hdxcli job alter list

In addition, you can add the following options to filter the result of the alter job list command:

  --status TEXT   Filter alter jobs by status.
  --project TEXT  Filter alter jobs by project name.
  --table TEXT    Filter alter jobs by table name.

For example, to filter by status running, you would use:

hdxcli job alter list --status running

Create

There are two ways to create an alter job, each serving a different purpose. The update command allows you to modify the value of a column based on a specified where clause. On the other hand, as you might expect, the delete command provides a way to remove rows based on a where clause. After creating these alter jobs, you must perform a commit on the created alter job to apply the modifications.

Update

hdxcli job alter create update --table <project-name>.<table-name> --column <column-name> --value <value> --where <where-clause>

Delete

hdxcli job alter create delete --table <project-name>.<table-name> --where <where-clause>

Commit

hdxcli job alter commit <job-name>

Cancel

hdxcli job alter cancel <job-name>

Retry

hdxcli job alter retry <job-name>

Show

hdxcli job alter show <job-name>

Delete

hdxcli job alter delete <job-name>

Purge Jobs

This command purge all batch jobs in your org.

$ hdxcli job purgejobs
Please type 'purge all jobs' to proceed: purge all jobs  
All jobs purged

Stream

Ingest

hdxcli stream --project <project-name> --table <table-name> --transform <transform-name> ingest <data-file>

Sources

Kinesis / Kafka / SIEM

The command structure of these resources is the same, it's only necessary to replace ‘kinesis’ with ‘kafka’ or ‘siem’.

List

hdxcli sources kinesis --project <project-name> --table <table-name> list

Create

The field ‘name’ in settings will be replaced by <source-name>.

hdxcli sources kinesis --project <project-name> --table <table-name> create <source_settings>.json <source-name>

Delete

hdxcli sources kinesis --project <project-name> --table <table-name> delete <source-name>

Show

hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> show

Settings

Display All Settings

hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings

Display single setting

hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings <setting-name>

Modify setting

hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings <setting-name> <new-value>

Pool

List

hdxcli pool list

Create

pool-service needs to contain the name of the service (query-head, query-peer, etc. For more information see Hydrolix API Reference). pool-name is the name of the pool, that means the field ‘name’ in settings will be replaced by pool-name.
These options aren't required since they have default values. Override the defaults with these options:

Options

Option	Type	Description
`-r,` `--replicas`	integer	Number of replicas for the workload (default: 1)
`-c`, `--cpu`	float	Dedicated CPU allocation for each replica (default: 0.5)
`-m`, `--memory`	float	Dedicated memory allocation for each replica, expressed in Gi (default: 0.5)
`-s`, `--storage`	float	Storage capacity for each replica, expressed in Gi (default: 0.5)

hdxcli pool create <pool-service> <pool-name>

An example modifying the default options would be:

hdxcli pool create <pool-service> <pool-name> -r 5 -c 1 -m 1 -s 2

Delete

hdxcli pool delete <pool-name>

Show

hdxcli pool --pool <pool-name> show

Settings

Display all settings

hdxcli pool --pool <pool-name> settings

Display single setting

hdxcli pool --pool <pool-name> settings <setting-name>

Modify setting

hdxcli pool --pool <pool-name> settings <setting-name> <new-value>

Dictionary

List

hdxcli dictionary --project <project-name> list

Create

<dictionary-settings> must contain all required fields (for more information on the required specifications, see the Hydrolix API Reference). <dictionary-filename> is the name of the dictionary file previously uploaded to Hydrolix. The field ‘name’ in settings will be replaced by <dictionary-name>.

hdxcli dictionary --project <project-name> create <dictionary-settings>.json <dictionary-filename> <dictionary-name>

Delete

hdxcli dictionary --project <project-name> delete <dictionary-name>

Show

hdxcli dictionary --project <project-name> --dictionary <dictionary-name> show

Settings

Display all settings

hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings

Display single setting

hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings <setting-name>

Modify setting

hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings <setting-name> <new-value>

Migrate

hdxcli dictionary --project <project-name> --dictionary <dictionary-name> migrate <target-project-name> <new-dictionary-name> --target-profile <profile-name>

Dictionary Files

List

hdxcli dictionary --project <project-name> files list

Upload

hdxcli supports two formats for <dictionary-file-to-upload>: JSON and CSV.

If the format is ‘json’ you don’t need to specify it.

hdxcli dictionary --project <project-name> files upload <dictionary-file-to-upload>.json <dictionary-filename>

Otherwise, if the format is ‘csv’ you must use -t verbatim or --body-from-file-type verbatim.

hdxcli dictionary --project <project-name> files upload -t verbatim <dictionary-file-to-upload>.csv <dictionary-filename>

Delete

hdxcli dictionary --project <project-name> files delete <dictionary-filename>

Function

List

hdxcli function --project <project-name> list

Create

There are two ways to create a new function:

Passing the function on the command line using -s or --inline-sql option:

hdxcli function --project <project-name> create -s '<function>' <function-name>

Using a JSON file with the function settings via -f or --sql-from-file option. The field ‘name’ in settings will be replaced by <function-name>:

hdxcli function --project <project-name> create -f <function-settings>.json <function-name>

Delete

hdxcli function --project <project-name> delete <function-name>

Show

hdxcli function --project <project-name> --function <function-name> show

Settings

Display all settings

hdxcli function --project <project-name> --function <function-name> settings

Display single setting

hdxcli function --project <project-name> --function <function-name> settings <setting-name>

Modify setting

hdxcli function --project <project-name> --function <function-name> settings <setting-name> <new-value>

Migrate

hdxcli function --project <project-name> --function <function-name> migrate <target-project-name> <new-function-name> --target-profile <profile-name>

Storage

List

hdxcli storage list

Create

There are two available methods to create a new storage: using a file containing the storage configuration or passing the configuration via the command line.

Using settings file (-f, --settings-filename):

hdxcli storage create <storage-name> --settings-filename <storage-settings>.json

Passing the configuration via the command line:

hdxcli storage create <storage-name> --bucket-path <bucket-path> --bucket-name <bucket-name> --region <region> --cloud <cloud>

Delete

hdxcxli storage delete <storage-name>

Show

hdxcli storage --storage <storage-name> show

Settings

Display all settings

hdxcli storage --storage <storage-name> settings

Display single setting

hdxcli storage --storage <storage-name> settings <setting-name>

Modify setting

hdxcli storage --storage <storage-name> settings <setting-name> <new-value>

Credential

List

Displays a list of all credentials.

hdxcli credential list

List-types

Displays the available credential types.

hdxcli credential list-types

Create

Creates a new credential.

hdxcli credential create <credential-name> <credential-type>

Depending on the credential-type, the required credentials will be requested interactively.
Alternatively, you can provide all the necessary details in a single command line by passing a JSON string to the --details option.
For example:

hdxcli credential create <credential-name> <credential-type> --details '{"key1": "value1", "key2": "value2"}'

Delete

Deletes a credential.

hdxcxli credential delete <credential-name>

Show

Displays details of a specific credential.

hdxcli credential --credential <credential-name> show

Settings

Display all settings

Shows all settings associated with a specific credential.

hdxcli credential --credential <credential-name> settings

Display single setting

Shows the value of a specific setting for a credential.

hdxcli credential --credential <credential-name> settings <setting-name>

Query-option

Query options operate at org, project or table level. For more information, refer to Query Options Precedence.

List

hdxcli query-option list

Set

hdxcli query-option set <query-option-name> <query-option-value>

An option (--from-file) is also available to configure a bunch of query options at once.

hdxcli query-option set --from-file <query-option-file>.json

This query option file must be a JSON file with the following structure:

{
  '<query-option-name>': <query-option-value>,
  ...
}

Unset

To unset a query option, one only needs to use the following command:

hdxcli query-option unset <query-option-name>

Additionally, there is an option --all that unsets all query options.

Check-health

This command checks the integrity of the transforms. Upon completion, it will print a brief report to the standard output.

hdxcli check-health [PROJECT_NAME] [TABLE_NAME]

Both project_name and table_name are optional. If none of them are provided, the validation will run on every transform the user has access to. Otherwise, if project_name is specified, the validation will be limited to the transforms within that project. The same applies to table_name.

Repair

The --repair option attempts to automatically fix detected issues in transforms and autoviews. If a conflict is found, the tool will try to repair it when possible. Otherwise, it will notify the user about the issue.

Usage:

hdxcli check-health --repair [PROJECT_NAME] [TABLE_NAME]

Notes:

If no project or table is provided, the tool will analyze and attempt to repair all transforms accessible to the user.
Repairs are limited to certain known conflicts, such as datatype or indexing mismatches between the transform and the autoview.

Integration

List

Allows you to list all transforms available in the repository.

hdxcli integration transform list

Apply

With this command you can create a new transform using the settings of a transform hosted in the repository.

hdxcli integration transform --project <project-name> --table <table-name> apply <public-transform-name> <transform-name>

Show

hdxcli integration transform show <public-transform-name>

Migrate

Migrate data table within the same cluster:

hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name>

Migrate data table to another cluster, using --target-profile:

hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name> --target-profile <profile-name>

Or passing cluster information:

hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name> --target-hostname <hostname> --target-username <username> --target-password <password> --target-uri-scheme <http/s>

To learn more about migrate command, see Data Migration Command for Hydrolix Tables.

Textual User Interface

To activate the hdxcli user interface, do as follows:

hdxcli tui

Version

hdxcli version