`hdxcli`

The hdxcli is a command line tool that can be used to manage certain aspects of your data platform. Primarily used as a CLI interface for the API, it can be used to script certain aspects as well as manage items such as project, table and transform settings.
The code is available on github, to install it you can use:

pip install hdxcli

Reference

hdxcli [OPTIONS] COMMAND [ARGS]

Options

Options	Purpose
--help	Displays summary documentation for using `hdxcli` and exits.
--profile PROFILENAME	Perform operation with a different profile. (Default profile is 'default')
--project PROJECTNAME	Explicitly pass the project name. If one was set it will be overridden.
--table TABLENAME	Explicitly pass the table name. If one was set it will be overridden.
--transform TRANSFORMNAME	Explicitly pass the transform name. If one was set it will be overridden, if none is given it'll use the default transform.
--job JOBNAME	Perform operation on the passed jobname
--function FUNCTIONNAME	Perform operation on the passed function
--dictionary DICTIONARYNAME	Perform operation on the passed dictionary
--password	Login password. If provided and the access token is expired, it will be used.

Commands

Command	Purpose
dictionary	Dictionary-related operations
function	Function-related operations
job	Job-related operations
profile	Profile-related operations
project	Project-related operations
purgejobs	Purge all batch jobs in your org
set	Set project and or/table to apply subsequent commands on it
table	Table-related operations
transform	Transform-related operations
unset	Remove any set projects/tables
version	Print hdxcli version

Configuration file

hdxcli leverage a configuration file which reference your Hydrolix cluster environment.
The file is located here: ~/.hdx_cli/config.toml

[default]
username = "[email protected]"
hostname = "cluster.domain.net"
projectname = "hydro"
tablename = "logs"

[gcp]
username = "[email protected]"
hostname = "gcp-cluster.domain.net"

[aws]
username = "[email protected]"
hostname = "aws-cluster.domain.net"

The configuration has a default section, if nothing is set in your command hdxcli it'll use the default cluster, project and table. You can override the default by using --project or --table.
To use a specific profile table and project you can use:

hdxcli --profile gcp --project myproject --table mytable table settings

Usage

Using hdxcli command-line program

hdxcli supports multiple profiles. You can use a default profile or use the --profile option to operate on a non-default profile.

When trying to invoke a command, if a login to the server is necessary, a prompt will be shown and the token will be cached.

Command-line tool organization

The tool is organized, mostly with the general invocation form of:

hdxcli <resource> [<subresource...] <verb> [<resource_name>]

Table and project resources have defaults that depend on the profile you are working with, so they can be omitted if you used the set command.

For all other resources, you can use --transform, --dictionary, etc. Please see the command line help for more information.

Projects, tables and transforms

The basic operations you can do with these resources are:

list them
create a new resource
delete an existing resource
modify an existing resource
show a resource in raw json format
show settings from a resource
write a setting
show a single setting

Working with transforms

In order to use a transforms, you need to:

create a transform

hdxcli transform create -f localfile.json transformname

Where localfile.json is a local file and transformname is the name for the transform that will be uploaded to the cluster.
Remember that a transform is applied to a table in a project, so whatever you set with the command-line tool will be the target of your transform.

If you want to override it, use:

hdxcli --project <the-project> --table <the-table> transform create -f localfile.json transformname

get a specific transform settings

hdxcli --project hydro --table logs transform settings settings.format_details

settings.format_details: {'flattening': {'depth': 1, 'active': True, 'map_flattening_strategy': {'left': '.'}, 'slice_flattening_strategy': None}}

Listing and showing your profiles

Listing profiles:

hdxcli profile list

Showing default profile:

hdxcli profile show