HDXCLI Commands
hdxcli
hdxcli
The hdxcli is a command line tool that can be used to manage certain aspects of your data platform. Primarily used as a CLI interface for the API, it can be used to script certain aspects as well as manage items such as project, table and transform settings.
The code is available on github, to install it you can use:
pip install hdxcli
Reference
hdxcli [OPTIONS] COMMAND [ARGS]
Options
Options | Purpose |
---|---|
--help | Displays summary documentation for using hdxcli and exits. |
--profile PROFILENAME | Perform operation with a different profile. (Default profile is 'default') |
--project PROJECTNAME | Explicitly pass the project name. If one was set it will be overridden. |
--table TABLENAME | Explicitly pass the table name. If one was set it will be overridden. |
--transform TRANSFORMNAME | Explicitly pass the transform name. If one was set it will be overridden, if none is given it'll use the default transform. |
--job JOBNAME | Perform operation on the passed jobname |
--function FUNCTIONNAME | Perform operation on the passed function |
--dictionary DICTIONARYNAME | Perform operation on the passed dictionary |
--password | Login password. If provided and the access token is expired, it will be used. |
Commands
Command | Purpose |
---|---|
dictionary | Dictionary-related operations |
function | Function-related operations |
job | Job-related operations |
profile | Profile-related operations |
project | Project-related operations |
purgejobs | Purge all batch jobs in your org |
set | Set project and or/table to apply subsequent commands on it |
table | Table-related operations |
transform | Transform-related operations |
unset | Remove any set projects/tables |
version | Print hdxcli version |
Configuration file
hdxcli
leverage a configuration file which reference your Hydrolix cluster environment.
The file is located here: ~/.hdx_cli/config.toml
[default]
username = "[email protected]"
hostname = "cluster.domain.net"
projectname = "hydro"
tablename = "logs"
[gcp]
username = "[email protected]"
hostname = "gcp-cluster.domain.net"
[aws]
username = "[email protected]"
hostname = "aws-cluster.domain.net"
The configuration has a default section, if nothing is set in your command hdxcli it'll use the default cluster, project and table. You can override the default by using --project
or --table
.
To use a specific profile table and project you can use:
hdxcli --profile gcp --project myproject --table mytable table settings
Usage
Using hdxcli command-line program
hdxcli
supports multiple profiles. You can use a default profile or use the --profile
option to operate on a non-default profile.
When trying to invoke a command, if a login to the server is necessary, a prompt will be shown and the token will be cached.
Command-line tool organization
The tool is organized, mostly with the general invocation form of:
hdxcli <resource> [<subresource...] <verb> [<resource_name>]
Table and project resources have defaults that depend on the profile you are working with, so they can be omitted if you used the set
command.
For all other resources, you can use --transform
, --dictionary
, etc. Please see the command line help for more information.
Projects, tables and transforms
The basic operations you can do with these resources are:
- list them
- create a new resource
- delete an existing resource
- modify an existing resource
- show a resource in raw json format
- show settings from a resource
- write a setting
- show a single setting
Working with transforms
In order to use a transforms, you need to:
- create a transform
hdxcli transform create -f localfile.json transformname
Where localfile.json is a local file and transformname is the name for the transform that will be uploaded to the cluster.
Remember that a transform is applied to a table in a project, so whatever you set
with the command-line tool will be the target of your transform.
If you want to override it, use:
hdxcli --project <the-project> --table <the-table> transform create -f localfile.json transformname
- get a specific transform settings
hdxcli --project hydro --table logs transform settings settings.format_details
settings.format_details: {'flattening': {'depth': 1, 'active': True, 'map_flattening_strategy': {'left': '.'}, 'slice_flattening_strategy': None}}
Listing and showing your profiles
Listing profiles:
hdxcli profile list
Showing default profile:
hdxcli profile show
Updated 7 months ago