Commands
Detailed reference for all the HDXCLI commands
Profile
There is a 'default' profile that will be used if you omit --profile <profile-name>
on each request to the HDXCLI.
List
hdxcli profile list
Add
One way is to do it interactively:
hdxcli profile add <profile-name>
Please, type the host name of your cluster: <host-name>
Please, type the user name of your cluster: <user-name>
Will you be using https (Y/N): \<Y/N>
Or by passing the same information via options:
hdxcli add <profile-name> --hostname <hostname> --username <username> --scheme <http/s>
Show
If you omit --profile <profile-name>
, 'default' profile will be shown.
hdxcli --profile <profile-name> profile show
Set/Unset
hdxcli
allows you to save both project name and table name to use as default.
Set
hdxcli set <project-name> <table-name>
Unset
hdxcli unset
From now on, when you see --project <project-name>
and --table <table-name>
, you know that it can be omitted if the set
command was previously used.
User
List
hdxcli user list
Delete
hdxcli user delete <user-email>
Show
hdxcli user --user <user-email> show
Assign roles
You can assign just one or multiple roles to a specific user. The option -r
, --role
can be used multiple times. It's a required setting.
hdxcli user assign-role <user-email> --role <role-name>
Remove roles
The same as the assign-role
command, but instead it removes roles from a user.
hdxcli user remove-role <user-email> --role <role-name>
Invite
List
The -p
or --pending
option can be used to filter the invite list, displaying only pending invitations.
hdxcli user invite list
Send
The -r
or --role
option can be used multiple times. It is a required setting.
hdxcli user invite send <user-email> --role <role-name>
Resend
hdxcli user invite send <user-email>
Delete
hdxcli user invite delete <user-email>
Show
hdxcli user invite --user <user-email> show\
Role
List
hdxcli role list
Create
There are two ways to create a new role:
- Using options. However, if you choose this method, the role cannot have two or more policies.
- Using the interactive method.
Create a role using options
-n
, --name
and -p
, --permission
are required options to create a new role, and the -p
, --permission
option can be used multiple times. While a role may or may not have a specific scope, these options are considered optional. The -t
, --scope-type
expects to receive the name of a resource, such as project, table, transform, etc., and the -i
, --scope-id
should be the UUID identifier for that specific resource.
Use the following command to create a new role:
hdxcli role create --name <role-name> --scope-type <scope-type> --scope-id <scope-id> --permission <permission-name>
Create a role using interactive method
To create a role interactively, use the following command:
hdxcli role create
Enter the name for the new role: <role-name>
Adding a Policy, does it have a specific scope? [Y/n]\: \<y/n>
Specify the type of scope for the role (e.g., project): <scope-type>
Provide the 'uuid' for the specified scope: <scope-id>
1 - commit_alterjob
2 - change_kinesissource
3 - ...
n - All of them
Enter the numbers corresponding to the permissions you'd want to add (comma-separated): \<i.e:1,2 or n>
Do you want to add another Policy? [Y/n]\: \<y/n>
***
# Review Role Details
Role Name: <role-name>
Policy 1:
Scope Type: <scope-type>
Scope ID: <scope-id>
Permissions: \<[permissions]>
Confirm the creation of the new role? [Y/n]\: \<y/n>
Created role role-name
Delete
hdxcli role delete <role-name>
Show
hdxcli role --role <role-name> show
Edit
This is an interactive command that allows you to modify the name of a role and add, modify, or delete policies associated with that role. Before finalizing the operation, it presents a detailed view of the modified role for confirmation or cancellation.
hdxcli role edit <role-name>
Add users
Required. This adds just one user or multiple users to a specific role. The option -u
, --user
can be used multiple times.
hdxcli role add-user <role-name> --user <user-email>
Remove users
The same as add-user command but to remove users from a role.
hdxcli role remove-user <role-name> --user <user-email>
Permissions list
This command displays a list of available permissions. Additionally, it allows filtering by type using the option -t
, --type
(e.g., -t function
), showing only permissions related to that specific type.
hdxcli role permission list
Project
List
hdxcli project list
Create
hdxcli project create <project-name>
Delete
hdxcli project delete <project-name>
Display activity
hdxcli project --project <project-name> activity
Display statistics
hdxcli project --project <project-name> stats
Show
hdxcli project --project <project-name> show
Settings
Display all settings
hdxcli project --project <project-name> settings
Display single setting
hdxcli project --project <project-name> settings <setting-name>
Modify setting
hdxcli project --project <project-name> settings <setting-name> <new-value>
Migrate
hdxcli project migrate <project-name> -tp <profile-name>
Table
List
hdxcli table --project <project-name> list
Create Raw Table (Regular)
If creating a regular table, no additional options are required. Use the table create command without specifying any options.
hdxcli table --project <project-name> create <table-name>
Additionally, an option (--settings-file
) exists to set different table settings other than the default. This works for both regular tables and summary tables:
hdxcli table --project <project-name> create <table-name> --settings-file <settings-file>.json
Create Aggregation Table (Summary)
When creating summary tables, the following options are necessary:
-
--type
or-t
: Specify as summary. -
--sql-query
or-s
: Provide the SQL query directly via the command line.
Alternatively, use --sql-query-file
to specify a file containing the SQL query.
Example using a SQL query in the command line:
hdxcli table --project <project-name> create <table-name> --type summary --sql-query <sql-query>
Example using a file containing the SQL query:
hdxcli table --project <project-name> create <table-name> --type summary --sql-query-file <sql-query-file>.txt
Delete
hdxcli table --project <project-name> delete <table-name>
Display Activity
hdxcli table --project <project-name> --table <table-name> activity
Display Statistics
hdxcli table --project <project-name> --table <table-name> stats
Show
hdxcli table --project <project-name> --table <table-name> show
Settings
Display All Settings
hdxcli table --project <project-name> --table <table-name> settings
Display Single Setting
hdxcli table --project <project-name> --table <table-name> settings <setting-name>
Modify Setting
hdxcli table --project <project-name> --table <table-name> settings <setting-name> <new-value>
Migrate
hdxcli table --project <project-name> migrate <table-name> -tp <profile-name> -P <target-project-name>
Truncate
hdxcli table --project <project-name> truncate <table-name>
Transform
List
hdxcli transform --project <project-name> --table <table-name> list
Create
The field ‘name’ in settings will be replaced by .
hdxcli transform --project <project-name> --table <table-name> create -f <transform-settings>.json <transform-name>
Delete
hdxcli transform --project <project-name> --table <table-name> delete <transform-name>
Show
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> show
Settings
Display All Settings
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings
Display Single Setting
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings <setting-name>
Modify setting
hdxcli transform --project <project-name> --table <table-name> --transform <transform-name> settings <setting-name> <new-value>
Migrate
hdxcli transform --project <project-name> --table <table-name> migrate <transform-name> -tp <profile-name> -P <target-project-name> -T <target-table-name>
Map-from
The command transform map-from
consumes data languages such as SQL, Elastic and others and create a Hydrolix transform from them.
hdxcli transform map-from --ddl-custom-mapping \<sql_to_hdx_mapping>.json \<ddl_file> <transform-name>
To learn more about map-from
command, see Mapping DDLs to Hydrolix transform.
Jobs
Batch
List
hdxcli job batch list
Ingest
It will use the default transform in <project-name>.<table-name>
if you don't provide --transform <transform-name>
.
hdxcli job batch --project <project-name> --table <table-name> --transform <transform-name> ingest <job-name> <job-settings>.json
Delete
hdxcli job batch delete <job-name>
Retry
hdxcli job batch retry <job-name>
Cancel
hdxcli job batch cancel <job-name>
Show
hdxcli job batch --job <job-name> show
Settings
Display all settings
hdxcli job batch --job <job-name> settings
Display single setting
hdxcli job batch --job <job-name> settings <setting-name>
Modify setting
hdxcli job batch --job <job-name> settings <setting-name> <new-value>
Alter
List
hdxcli job alter list
In addition, you can add the following options to filter the result of the alter job list command:
--status TEXT Filter alter jobs by status.
--project TEXT Filter alter jobs by project name.
--table TEXT Filter alter jobs by table name.
For example, to filter by status running, you would use:
hdxcli job alter list --status running
Create
There are two ways to create an alter job, each serving a different purpose. The update
command allows you to modify the value of a column based on a specified where clause. On the other hand, as you might expect, the delete
command provides a way to remove rows based on a where clause. After creating these alter jobs, it is necessary to perform a commit on the created alter job to apply the modifications.
Update
hdxcli job alter create update --table <project-name>.<table-name> --column <column-name> --value <value> --where <where-clause>
Delete
hdxcli job alter create delete --table <project-name>.<table-name> --where <where-clause>
Commit
hdxcli job alter commit <job-name>
Cancel
hdxcli job alter cancel <job-name>
Retry
hdxcli job alter retry <job-name>
Show
hdxcli job alter show <job-name>
Delete
hdxcli job alter delete <job-name>
Purge Jobs
This command purge all batch jobs in your org.
hdxcli job purgejobs
Please type 'purge all jobs' to proceed: purge all jobs
All jobs purged
Stream
Ingest
hdxcli stream --project <project-name> --table <table-name> --transform <transform-name> ingest <data-file>
Sources
Kinesis / Kafka / SIEM
The command structure of these resources is the same, it's only necessary to replace ‘kinesis’ with ‘kafka’ or ‘siem’.
List
hdxcli sources kinesis --project <project-name> --table <table-name> list
Create
The field ‘name’ in settings will be replaced by <source-name>
.
hdxcli sources kinesis --project <project-name> --table <table-name> create <source_settings>.json <source-name>
Delete
hdxcli sources kinesis --project <project-name> --table <table-name> delete <source-name>
Show
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> show
Settings
Display All Settings
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings
Display single setting
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings <setting-name>
Modify setting
hdxcli sources kinesis --project <project-name> --table <table-name> --source <source-name> settings <setting-name> <new-value>
Pool
List
hdxcli pool list
Create
pool-service
needs to contain the name of the service (query-head, query-peer, etc. For more information see Hydrolix API Reference). pool-name
is the name of the pool, that means the field ‘name’ in settings will be replaced by pool-name
.
These options are not required since they have default values. Override the defaults with these options:
Options
-
-r
,--replicas
INTEGER
Number of replicas for the workload (default: 1) -
-c
,--cpu
FLOAT
Dedicated CPU allocation for each replica (default: 0.5) -
-m
,--memory
FLOAT
Dedicated memory allocation for each replica, expressed in Gi (default: 0.5) -
-s
,--storage
FLOAT
Storage capacity for each replica, expressed in Gi (default: 0.5)
hdxcli pool create <pool-service> <pool-name>
An example modifying the default options would be:
hdxcli pool create <pool-service> <pool-name> -r 5 -c 1 -m 1 -s 2
Delete
hdxcli pool delete <pool-name>
Show
hdxcli pool --pool <pool-name> show
Settings
Display all settings
hdxcli pool --pool <pool-name> settings
Display single setting
hdxcli pool --pool <pool-name> settings <setting-name>
Modify setting
hdxcli pool --pool <pool-name> settings <setting-name> <new-value>
Dictionary
List
hdxcli dictionary --project <project-name> list
Create
<dictionary-settings>
must contain all required fields (for more information on the required specifications, see the Hydrolix API Reference). <dictionary-filename>
is the name of the dictionary file previously uploaded to Hydrolix. The field ‘name’ in settings will be replaced by <dictionary-name>
.
hdxcli dictionary --project <project-name> create <dictionary-settings>.json <dictionary-filename> <dictionary-name>
Delete
hdxcli dictionary --project <project-name> delete <dictionary-name>
Show
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> show
Settings
Display all settings
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings
Display single setting
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings <setting-name>
Modify setting
hdxcli dictionary --project <project-name> --dictionary <dictionary-name> settings <setting-name> <new-value>
Migrate
hdxcli dictionary --project <project-name> migrate <dictionary-name> -tp <profile-name> -P <target-project-name>
Dictionary Files
List
hdxcli dictionary --project <project-name> files list
Upload
hdxcli
supports two formats for <dictionary-file-to-upload>
: JSON and CSV.
If the format is ‘json’ you don’t need to specify it.
hdxcli dictionary --project <project-name> files upload <dictionary-file-to-upload>.json <dictionary-filename>
Otherwise, if the format is ‘csv’ you must use -t verbatim
or --body-from-file-type verbatim
.
hdxcli dictionary --project <project-name> files upload -t verbatim <dictionary-file-to-upload>.csv <dictionary-filename>
Delete
hdxcli dictionary --project <project-name> files delete <dictionary-filename>
Function
List
hdxcli function --project <project-name> list
Create
There are two ways to create a new function:
- Passing the function on the command line using
-s
or--inline-sql
option:
hdxcli function --project <project-name> create -s '<function>' <function-name>
- Using a JSON file with the function settings via
-f
or--sql-from-file
option. The field ‘name’ in settings will be replaced by<function-name>
:
hdxcli function --project <project-name> create -f <function-settings>.json <function-name>
Delete
hdxcli function --project <project-name> delete <function-name>
Show
hdxcli function --project <project-name> --function <function-name> show
Settings
Display all settings
hdxcli function --project <project-name> --function <function-name> settings
Display single setting
hdxcli function --project <project-name> --function <function-name> settings <setting-name>
Modify setting
hdxcli function --project <project-name> --function <function-name> settings <setting-name> <new-value>
Migrate
hdxcli function --project <project-name> migrate <function-name> -tp <profile-name> -P <target-project-name>
Storage
List
hdxcli storage list
Create
There are two available methods to create a new storage: using a file containing the storage configuration or passing the configuration via the command line.
- Using settings file (
-f
,--settings-filename
):
hdxcli storage create <storage-name> --settings-filename <storage-settings>.json
Passing the configuration via the command line:
hdxcli storage create <storage-name> --bucket-path <bucket-path> --bucket-name <bucket-name> --region <region> --cloud <cloud>
Delete
hdxcxli storage delete <storage-name>
Show
hdxcli storage --storage <storage-name> show
Settings
Display all settings
hdxcli storage --storage <storage-name> settings
Display single setting
hdxcli storage --storage <storage-name> settings <setting-name>
Modify setting
hdxcli storage --storage <storage-name> settings <setting-name> <new-value>
Migrate
hdxcli storage migrate <storage-name> -tp <profile-name>
Query-option
Query options operate at org-level. For more information, refer to Query Options documentation.
List
hdxcli query-option list
Set
hdxcli query-option set <query-option-name> <query-option-value>
An option (--from-file
) is also available to configure a bunch of query options at once.
hdxcli query-option set --from-file <query-option-file>.json
This query option file must be a JSON file with the following structure:
{
'<query-option-name>': <query-option-value>,
...
}
Unset
To unset a query option, one only needs to use the following command:
hdxcli query-option unset <query-option-name>
Additionally, there is an option --all
that unsets all query options.
Integration
List
Allows you to list all transforms available in the repository.
hdxcli integration transform list
Apply
With this command you can create a new transform using the settings of a transform hosted in the repository.
hdxcli integration transform --project <project-name> --table <table-name> apply <public-transform-name> <transform-name>
Show
hdxcli integration transform show <public-transform-name>
Migrate
Migrate table within the same cluster:
hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name>
Migrate table to another cluster, using --target-profile
:
hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name> --target-profile <profile-name>
Or passing host information:
hdxcli migrate <source-project-name>.<source-table-name> <target-project-name>.<target-table-name> --target-hostname <hostname> --target-username <username> --target-password <password> --target-uri-scheme <http/s>
To learn more about migrate command, see Migration Command for Hydrolix Tables.
Textual User Interface
To activate the hdxcli
user interface, do as follows:
hdxcli tui
Version
hdxcli version
Updated 12 days ago