Commands
Detailed reference for all the HDXCLI commands
Profile
There is a 'default' profile that will be used if you omit --profile <profile-name>
on each request to the HDXCLI.
List
hdxcli profile list
Add
One way is to do it interactively:
$ hdxcli profile add <profile-name>
Please, type the host name of your cluster: <host-name>
Please, type the user name of your cluster: <user-name>
Will you be using https (Y/N): \<Y/N>
Or by passing the same information via options:
hdxcli add <profile-name> --hostname <hostname> --username <username> --scheme <http/s>
Show
If you omit --profile <profile-name>
, 'default' profile will be shown.
hdxcli --profile <profile-name> profile show
Set/Unset
hdxcli
allows you to save both project name and table name to use as default.
Set
hdxcli set <project-name> <table-name>
Unset
hdxcli unset
From now on, when you see --project <project-name>
and --table <table-name>
, you know that it can be omitted if the set
command was previously used.
User
List
hdxcli user list
Delete
hdxcli user delete <user-email>
Show
hdxcli user --user <user-email> show
Assign roles
You can assign just one or multiple roles to a specific user. The option -r
, --role
can be used multiple times. It's a required setting.
hdxcli user assign-role <user-email> --role <role-name>
Remove roles
The same as the assign-role
command, but instead it removes roles from a user.
hdxcli user remove-role <user-email> --role <role-name>
Invite
List
The -p
or --pending
option can be used to filter the invite list, displaying only pending invitations.
hdxcli user invite list
Send
The -r
or --role
option can be used multiple times. It's a required setting.
hdxcli user invite send <user-email> --role <role-name>
Resend
hdxcli user invite send <user-email>
Delete
hdxcli user invite delete <user-email>
Show
hdxcli user invite --user <user-email> show
Service Account
This command manages Service Accounts. Service Accounts are typically used for programmatic access, automation, or to grant specific, limited permissions to applications or scripts without using individual user credentials. Operations include creating, listing, showing, deleting Service Accounts, managing their roles, and generating access tokens.
List
Displays a list of all accessible Service Accounts, showing their names and roles associated.
hdxcli service-account list
Create
Creates a new Service Account. You must provide a unique name for the new Service Account and specify at least one role to define its permissions.
hdxcli service-account create <service-account-name> [OPTIONS]
Options
Name | Type | Description |
---|---|---|
-r , --role | string | Role to assign to the Service Account. Use this option multiple times to assign several roles (e.g., -r role1 -r role2) (required). |
--gt , --generate-token | string | If set, a new access token is automatically generated for the Service Account immediately after creation and its details are displayed. |
Show
hdxcli service-account show <service-account-name> [OPTIONS]
Options
Name | Type | Description |
---|---|---|
-i , --indent | flag | Indents the JSON output for improved human readability. |
Delete
hdxcli service-account delete <service-account-name> [OPTIONS]
Options
Name | Type | Description |
---|---|---|
--disable-confirmation-prompt | flag | Skips the interactive confirmation prompt before deleting. |
Assign Role
Assigns one or more roles to an existing Service Account, granting it additional permissions.
hdxcli service-account assign-role <service-account-name> [OPTIONS]
Options
Name | Type | Description |
---|---|---|
-r , --role | string | Role to assign to the Service Account. Use this option multiple times to assign several roles (required). |
Remove Role
Removes one or more roles from an existing Service Account, revoking specific permissions.
hdxcli service-account remove-role <service-account-name> [OPTIONS]
Options
Name | Type | Description |
---|---|---|
-r , --role | string | Role to assign to the Service Account. Use this option multiple times to assign several roles (required). |
Generate Token
Generates a new access token for a specific Service Account. This token can then be used for authenticating API requests as that Service Account.
hdxcli service-account generate-token <service-account-name> [OPTIONS]
Options
Name | Type | Description |
---|---|---|
-j , --json | flag | Displays the generated token and its details in JSON format. |
Role
List
hdxcli role list
Create
There are two ways to create a new role:
- Using options. However, if you choose this method, the role can't have two or more policies.
- Using the interactive method.
Create a role using options
-n
, --name
and -p
, --permission
are required options to create a new role, and the -p
, --permission
option can be used multiple times. While a role may or may not have a specific scope, these options are considered optional. The -t
, --scope-type
expects to receive the name of a resource, such as project, table, transform, etc., and the -i
, --scope-id
should be the UUID identifier for that specific resource.
Use the following command to create a new role:
hdxcli role create --name <role-name> --scope-type <scope-type> --scope-id <scope-id> --permission <permission-name>
Create a role using interactive method
To create a role interactively, use the following command:
$ hdxcli role create
Enter the name for the new role: <role-name>
Adding a Policy, does it have a specific scope? [Y/n]\: \<y/n>
Specify the type of scope for the role (e.g., project): <scope-type>
Provide the 'uuid' for the specified scope: <scope-id>
1 - commit_alterjob
2 - change_kinesissource
3 - ...
n - All of them
Enter the numbers corresponding to the permissions you'd want to add (comma-separated): \<i.e:1,2 or n>
Do you want to add another Policy? [Y/n]\: \<y/n>
***
## Review Role Details
Role Name: <role-name>
Policy 1:
Scope Type: <scope-type>
Scope ID: <scope-id>
Permissions: \<[permissions]>
Confirm the creation of the new role? [Y/n]\: \<y/n>
Created role role-name
Delete
hdxcli role delete <role-name>
Show
hdxcli role --role <role-name> show
Edit
This is an interactive command that allows you to modify the name of a role and add, modify, or delete policies associated with that role. Before finalizing the operation, it presents a detailed view of the modified role for confirmation or cancellation.
hdxcli role edit <role-name>
Add users
Required. This adds just one user or multiple users to a specific role. The option -u
, --user
can be used multiple times.
hdxcli role add-user <role-name> --user <user-email>
Remove users
The same as add-user command but to remove users from a role.
hdxcli role remove-user <role-name> --user <user-email>
Permissions list
This command displays a list of available permissions. Additionally, it allows filtering by type using the option -t
, --type
(for example -t function
), showing only permissions related to that specific type.
hdxcli role permission list
Project
List
hdxcli project list
Create
hdxcli project create <project-name>
Delete
hdxcli project delete <project-name>
Display activity
hdxcli project --project <project-name> activity
Display statistics
hdxcli project --project <project-name> stats
Show
hdxcli project --project <project-name> show
Settings
Display all settings
hdxcli project --project <project-name> settings
Display single setting
hdxcli project --project <project-name> settings <setting-name>
Modify setting
hdxcli project --project <project-name> settings <setting-name> <new-value>
Migrate
hdxcli project --project <project-name> migrate <new-project-name> --target-profile <profile-name>
Table
List
hdxcli table --project <project-name> list
Create Raw Table (Regular)
If creating a regular table, no additional options are required. Use the table create command without specifying any options.
hdxcli table --project <project-name> create <table-name>
Additionally, an option (--settings-file
) exists to set different table settings other than the default. This works for both regular tables and summary tables:
hdxcli table --project <project-name> create <table-name> --settings-file <settings-file>.json
Create Aggregation Table (Summary)
When creating summary tables, the following options are necessary:
-
--type
or-t
: Specify as summary. -
--sql-query
or-s
: Provide the SQL query directly via the command line.
Alternatively, use --sql-query-file
to specify a file containing the SQL query.
Example using a SQL query in the command line:
hdxcli table --project <project-name> create <table-name> --type summary --sql-query <sql-query>
Example using a file containing the SQL query:
hdxcli table --project <project-name> create <table-name> --type summary --sql-query-file <sql-query-file>.txt
Delete
hdxcli table --project <project-name> delete <table-name>
Display Activity
hdxcli table --project <project-name> --table <table-name> activity
Display Statistics
hdxcli table --project <project-name> --table <table-name> stats
Show
hdxcli table --project <project-name> --table <table-name> show
Settings
Display All Settings
hdxcli table --project <project-name> --table <table-name> settings
Display Single Setting
hdxcli table --project <project-name> --table <table-name> settings <setting-name>
Modify Setting
hdxcli table --project <project-name> --table <table-name> settings <setting-name> <new-value>
Migrate
hdxcli table --project <project-name> --table <table-name> migrate <target-project-name> <new-table-name> --target-profile <profile-name>
Truncate
hdxcli table --project <project-name> truncate <table-name>
Transform
List
hdxcli transform --project <project-name> --table <table-name> list
Create
The field ‘name’ in settings will be replaced by .
hdxcli transform --project <project-name> --table <table-name> create -f <transform-settings>.json <transform-name>
Delete
hdxcli transform --project <project-name> --table <table-name> delete <transform-name>
Show
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> show
Settings
Display All Settings
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings
Display Single Setting
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings <setting-name>
Modify setting
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings <setting-name> <new-value>
Migrate
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> migrate <target-project-name> <target-table-name> <new-transform-name> --target-profile <profile-name>
Shadow
This command manages shadow
tables, which are used to test new or modified transform configurations with a sample of data from an original source table. This is useful to safely verify transform changes before applying them to the main production transform.
To work with shadow tables, you generally need to specify the source context using these global options with the sub-commands:
Name | Type | Description |
---|---|---|
--project | string | Name of the source table where the shadow table will be created. |
--table | string | Name of the source table. |
--transform | string | Name of the transform currently associated with the source table. |
Create
This command creates a new shadow table. It requires the path to a JSON file containing the settings for the new transform that will be associated with this shadow table.
hdxcli shadow --project SOURCE_PROJECTNAME --table SOURCE_TABLENAME --transform SOURCE_TRANSFORMNAME create TRANSFORM_SETTINGS_PATH [OPTIONS]
TRANSFORM_SETTINGS_PATH: Path to the JSON file with the new transform settings for the shadow table (required).
Options
Name | Type | Description |
---|---|---|
--sample-rate | integer (0-5) | Percentage of data from the source table to be ingested into the shadow table (required). |
--table-name | string | Name for the new shadow table. Default: shadow_ + source table name. |
--table-settings | path | Path to a file containing specific table settings for the shadow table itself. |
--transform-name | string | Name for the new transform that will be created for the shadow table. Default: shadow_ + source transform name. |
Delete
This command deletes a shadow table. It also removes the shadow table's configuration from the source transform.
hdxcli shadow --project SOURCE_PROJECTNAME delete SHADOW_TABLENAME
Start
The start command begins or resumes the ingestion of a data sample from the source table into its shadow table. You must specify the percentage of data to sample using the --sample-rate
option (a value between 1 and 5).
hdxcli shadow --project SOURCE_PROJECTNAME start SHADOW_TABLENAME --sample-rate RATE
Stop
The stop command halts the ingestion of data from the source table into its shadow table. It effectively sets the sampling rate to 0 on the source transform for that specific shadow table configuration.
hdxcli shadow --project SOURCE_PROJECTNAME stop SHADOW_TABLENAME
Jobs
Batch
List
hdxcli job batch list
Ingest
It will use the default transform in <project-name>.<table-name>
if you don't provide --transform <transform-name>
.
hdxcli job batch --project <project-name> --table <table-name> --transform <transform-name> ingest <job-name> <job-settings>.json
Delete
hdxcli job batch delete <job-name>
Retry
hdxcli job batch retry <job-name>
Cancel
hdxcli job batch cancel <job-name>
Show
hdxcli job batch --job <job-name> show
Settings
Display all settings
hdxcli job batch --job <job-name> settings
Display single setting
hdxcli job batch --job <job-name> settings <setting-name>
Modify setting
hdxcli job batch --job <job-name> settings <setting-name> <new-value>
Alter
List
hdxcli job alter list
In addition, you can add the following options to filter the result of the alter job list command:
--status TEXT Filter alter jobs by status.
--project TEXT Filter alter jobs by project name.
--table TEXT Filter alter jobs by table name.
For example, to filter by status running, you would use:
hdxcli job alter list --status running
Create
There are two ways to create an alter job, each serving a different purpose. The update
command allows you to modify the value of a column based on a specified where clause. On the other hand, as you might expect, the delete
command provides a way to remove rows based on a where clause. After creating these alter jobs, you must perform a commit on the created alter job to apply the modifications.
Update
hdxcli job alter create update --table <project-name>.<table-name> --column <column-name> --value <value> --where <where-clause>
Delete
hdxcli job alter create delete --table <project-name>.<table-name> --where <where-clause>
Commit
hdxcli job alter commit <job-name>
Cancel
hdxcli job alter cancel <job-name>
Retry
hdxcli job alter retry <job-name>
Show
hdxcli job alter show <job-name>
Delete
hdxcli job alter delete <job-name>
Purge Jobs
This command purge all batch jobs in your org.
$ hdxcli job purgejobs
Please type 'purge all jobs' to proceed: purge all jobs
All jobs purged
Stream
Ingest
hdxcli stream --project <project-name> --table <table-name> --transform <transform-name> ingest <data-file>
Sources
Kinesis / Kafka / SIEM
The command structure of these resources is the same, it's only necessary to replace ‘kinesis’ with ‘kafka’ or ‘siem’.
List
hdxcli sources kinesis --project <project-name> --table <table-name> list
Create
The field ‘name’ in settings will be replaced by <source-name>
.
hdxcli sources kinesis --project <project-name> --table <table-name> create <source_settings>.json <source-name>
Delete
hdxcli sources kinesis --project <project-name> --table <table-name> delete <source-name>
Show
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> show
Settings
Display All Settings
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings
Display single setting
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings <setting-name>
Modify setting
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings <setting-name> <new-value>
Pool
List
hdxcli pool list
Create
pool-service
needs to contain the name of the service (query-head, query-peer, etc. For more information see Hydrolix API Reference). pool-name
is the name of the pool, that means the field ‘name’ in settings will be replaced by pool-name
.
These options aren't required since they have default values. Override the defaults with these options:
Options
Name | Type | Description |
---|---|---|
-r, --replicas | integer | Number of replicas for the workload (default: 1) |
-c , --cpu | float | Dedicated CPU allocation for each replica (default: 0.5) |
-m , --memory | float | Dedicated memory allocation for each replica, expressed in Gi (default: 0.5) |
-s , --storage | float | Storage capacity for each replica, expressed in Gi (default: 0.5) |
hdxcli pool create <pool-service> <pool-name>
An example modifying the default options would be:
hdxcli pool create <pool-service> <pool-name> -r 5 -c 1 -m 1 -s 2
Delete
hdxcli pool delete <pool-name>
Show
hdxcli pool --pool <pool-name> show
Settings
Display all settings
hdxcli pool --pool <pool-name> settings
Display single setting
hdxcli pool --pool <pool-name> settings <setting-name>
Modify setting
hdxcli pool --pool <pool-name> settings <setting-name> <new-value>
Dictionary
List
hdxcli dictionary --project <project-name> list
Create
<dictionary-settings>
must contain all required fields (for more information on the required specifications, see the Hydrolix API Reference). <dictionary-filename>
is the name of the dictionary file previously uploaded to Hydrolix. The field ‘name’ in settings will be replaced by <dictionary-name>
.
hdxcli dictionary --project <project-name> create <dictionary-settings>.json <dictionary-filename> <dictionary-name>
Delete
hdxcli dictionary --project <project-name> delete <dictionary-name>
Show
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> show
Settings
Display all settings
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings
Display single setting
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings <setting-name>
Modify setting
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings <setting-name> <new-value>
Migrate
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> migrate <target-project-name> <new-dictionary-name> --target-profile <profile-name>
Dictionary Files
List
hdxcli dictionary --project <project-name> files list
Upload
hdxcli
supports two formats for <dictionary-file-to-upload>
: JSON and CSV.
If the format is ‘json’ you don’t need to specify it.
hdxcli dictionary --project <project-name> files upload <dictionary-file-to-upload>.json <dictionary-filename>
Otherwise, if the format is ‘csv’ you must use -t verbatim
or --body-from-file-type verbatim
.
hdxcli dictionary --project <project-name> files upload -t verbatim <dictionary-file-to-upload>.csv <dictionary-filename>
Delete
hdxcli dictionary --project <project-name> files delete <dictionary-filename>
Function
List
hdxcli function --project <project-name> list
Create
There are two ways to create a new function:
- Passing the function on the command line using
-s
or--inline-sql
option:
hdxcli function --project <project-name> create -s '<function>' <function-name>
- Using a JSON file with the function settings via
-f
or--sql-from-file
option. The field ‘name’ in settings will be replaced by<function-name>
:
hdxcli function --project <project-name> create -f <function-settings>.json <function-name>
Delete
hdxcli function --project <project-name> delete <function-name>
Show
hdxcli function --project <project-name> --function <function-name> show
Settings
Display all settings
hdxcli function --project <project-name> --function <function-name> settings
Display single setting
hdxcli function --project <project-name> --function <function-name> settings <setting-name>
Modify setting
hdxcli function --project <project-name> --function <function-name> settings <setting-name> <new-value>
Migrate
hdxcli function --project <project-name> --function <function-name> migrate <target-project-name> <new-function-name> --target-profile <profile-name>
Storage
List
hdxcli storage list
Create
There are two available methods to create a new storage: using a file containing the storage configuration or passing the configuration via the command line.
- Using settings file (
-f
,--settings-filename
):
hdxcli storage create <storage-name> --settings-filename <storage-settings>.json
Passing the configuration via the command line:
hdxcli storage create <storage-name> --bucket-path <bucket-path> --bucket-name <bucket-name> --region <region> --cloud <cloud>
Delete
hdxcxli storage delete <storage-name>
Show
hdxcli storage --storage <storage-name> show
Settings
Display all settings
hdxcli storage --storage <storage-name> settings
Display single setting
hdxcli storage --storage <storage-name> settings <setting-name>
Modify setting
hdxcli storage --storage <storage-name> settings <setting-name> <new-value>
Credential
List
Displays a list of all credentials.
hdxcli credential list
List-types
Displays the available credential types.
hdxcli credential list-types
Create
Creates a new credential.
hdxcli credential create <credential-name> <credential-type>
Depending on the credential-type
, the required credentials will be requested interactively.
Alternatively, you can provide all the necessary details in a single command line by passing a JSON string to the --details
option.
For example:
hdxcli credential create <credential-name> <credential-type> --details '{"key1": "value1", "key2": "value2"}'
Delete
Deletes a credential.
hdxcxli credential delete <credential-name>
Show
Displays details of a specific credential.
hdxcli credential --credential <credential-name> show
Settings
Display all settings
Shows all settings associated with a specific credential.
hdxcli credential --credential <credential-name> settings
Display single setting
Shows the value of a specific setting for a credential.
hdxcli credential --credential <credential-name> settings <setting-name>
Query-option
Query options operate at org, project or table level. For more information, refer to Query Options Precedence.
List
hdxcli query-option list
Set
hdxcli query-option set <query-option-name> <query-option-value>
An option (--from-file
) is also available to configure a bunch of query options at once.
hdxcli query-option set --from-file <query-option-file>.json
This query option file must be a JSON file with the following structure:
{
'<query-option-name>': <query-option-value>,
...
}
Unset
To unset a query option, one only needs to use the following command:
hdxcli query-option unset <query-option-name>
Additionally, there is an option --all
that unsets all query options.
Check-health
This command checks the integrity of the transforms. Upon completion, it will print a brief report to the standard output.
hdxcli check-health [PROJECT_NAME] [TABLE_NAME]
Both project_name
and table_name
are optional. If none of them are provided, the validation will run on every transform the user has access to. Otherwise, if project_name
is specified, the validation will be limited to the transforms within that project. The same applies to table_name
.
Repair
The --repair
option attempts to automatically fix detected issues in transforms and autoviews. If a conflict is found, the tool will try to repair it when possible. Otherwise, it will notify the user about the issue.
Usage:
hdxcli check-health --repair [PROJECT_NAME] [TABLE_NAME]
Notes:
- If no project or table is provided, the tool will analyze and attempt to repair all transforms accessible to the user.
- Repairs are limited to certain known conflicts, such as datatype or indexing mismatches between the transform and the autoview.
Integration
List
Allows you to list all transforms available in the repository.
hdxcli integration transform list
Apply
With this command you can create a new transform using the settings of a transform hosted in the repository.
hdxcli integration transform --project <project-name> --table <table-name> apply <public-transform-name> <transform-name>
Show
hdxcli integration transform show <public-transform-name>
Migrate
Migrate data table within the same cluster:
hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name>
Migrate data table to another cluster, using --target-profile
:
hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name> --target-profile <profile-name>
Or passing cluster information:
hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name> --target-hostname <hostname> --target-username <username> --target-password <password> --target-uri-scheme <http/s>
To learn more about migrate command, see Data Migration Command for Hydrolix Tables.
Textual User Interface
To activate the hdxcli
user interface, do as follows:
hdxcli tui
Version
hdxcli version
Updated 6 days ago