User Permissions (RBAC)

Role Based Access Control (RBAC) is a system that manages permissions across multiple users and various roles. With RBAC, you can control which users can perform which actions upon each individual resource in your cluster. Hydrolix organizes resources into a hierarchy, allowing you to apply granular permissions for nested resources. Using this hierarchy, you can define exactly the permissions required for each role without spending hours fine tuning permissions for each resource.

RBAC controls access to API endpoints, the Hydrolix UI, and SQL queries.

Structure and Example

RBAC can become complex, so it's good to have a firm understanding of the structure involved. Here's a walkthrough with an example.

A short-sighted way of managing permissions would be to allow any permissions to be assigned directly to any user. This might work for very small groups of users, permissions, and projects, but it would soon become unmanageable.

Instead, Hydrolix provides roles and policies to help group users and permissions. Policies depend on scope information to help assign a group of permissions to the correct policy.

Imagine a user named Tessa who should have read access to the tables in a project named "Balloons." Tessa, who is able to read all the tables in the Balloons project, is a member of the ProjectReader role. That role has a policy that allows the read_table permission for all the tables in the Balloons project. Following the same structure as the image above:

Later, Tessa can be assigned more complicated roles by simply assigning roles to her, rather than giving her each permission related to a scope via a jumble of policies. For instance, if Tessa were later granted permissions to edit color dictionary in the database, she could simply be granted the ColorsEditor role. This assigns four permissions within the scope of the Colors dictionary. This configuration is pre-defined so it can be quickly assigned to other users as well.

Subsequently, if anybody in the ColorsEditor role needs additional permissions, additional policies can be added to the ColorsEditor role. For instance, if Tessa and all the other people in the ColorsEditor role needed permissions to edit a "Hues" dictionary, a policy similar to the one above for the Colors dictionary could be added to the ColorsEditor role.

Users

Every user with access to your Hydrolix cluster has an account. You can identify users by the email address used to activate the account.

Roles have unique names. Administrators associate users with roles by name.

You must associate each user with at least one role when you invite them to join your cluster.

Resources

Resources are the entities that organize data within a Hydrolix cluster.

Resource Hierarchy

The following chart shows how Hydrolix organizes resources. At the highest level, organizations contain all other resources.

Organization
    Project
        Table
            SQL
            Transform
        Dictionary
        Function

When a user has a permission to a resource, Hydrolix grants permissions for all resources below that resource in the hierarchy that belong to the higher-level resource. For instance, granting a user "view" permission to a table also grants transitive view permission to all rows, columns, and transforms in the table.

Positive Permissions

Hydrolix permissions can only be expressed in a "positive" way: roles can only grant permissions, never restrict permissions. For instance, consider a Hydrolix cluster that contains the following configuration:

Organization A
    Project X
        Table 1
        Table 2
        Table 3
    Project Y
        Table alpha
        Table beta
    Project Z
        Table canis
        Table felis

A user with view permissions for Project X has view access to tables 1, 2, and 3. There is no way to restrict a user with full project view permissions from viewing a table nested within that project. To grant a user permissions to only tables 1 and 3, but NOT 2, you must individually grant the user permissions to tables 1 and 3 instead of granting permissions at the project level.

Similarly, if you grant a user all permissions for organization A, that user gains access to projects X, Y, and Z, tables 1, 2, 3, alpha, beta, canis, and felis. This happens because Hydrolix only grants, and never restricts, permissions to users based on the hierarchy of resources. Restricted permissions are defined only by omission, so use the hierarchy of organizations, projects, and tables to simplify your permission scheme.

Roles

Roles determine whether or not an action executes within your Hydrolix cluster. Roles are comprised of a unique name, a description, and a collection of policies. You can reuse roles across multiple users.

Think of roles as a function with the following inputs:

user
action
resource

Hydrolix administrators write roles to define that function. When a user performs an action on a resource in your cluster, RBAC computes the function based on the user, action, and resource inputs. If the function says "yes", Hydrolix executes the action for the user. If the function says "no", Hydrolix skips the action and informs the user that they don't have access to that action on that resource.

For example,

user is "Tessa"
action is "read"
resource is "table"
the role name is "MyProjectReader"

Tessa can read a table. Another way to say this is that user "Tessa" has a "MyProjectReader" role, and that role has a policy in its collection that allows it to read tables, so Tessa is able to read tables.

Default Roles

The following are roles created by default for each Hydrolix cluster. These can be viewed by navigating within your cluster to Security > Roles.

Role	Permissions
super_admin	ALL
user_admin	delete_user, add_role, add_user_role, change_role, add_invite, view_user, remove_roles_user, view_role, permissions_role, add_roles_user, delete_role, remove_user_role
operator	view_summarysource, delete_kinesissource, change_transform, add_dictionary, change_dictionary, delete_kafkasource, add_batchjob, view_project, delete_dictionary, delete_summarysource, view_dictionary, delete_batchjob, change_kafkasource, add_kinesissource, status_batchjob, view_function, add_dictionaryfile, change_function, view_dictionaryfile, delete_dictionaryfile, add_summarysource, retry_batchjob, view_batchjob, add_transform, change_dictionaryfile, view_view, change_summarysource, cancel_batchjob, change_view, view_siemsource, delete_function, add_function, delete_view, view_kafkasource, add_view, add_siemsource, view_table, view_transform, change_kinesissource, change_siemsource, view_org, delete_siemsource, delete_transform, generate_table, view_kinesissource, add_kafkasource
read_only	show_columns_sql, select_metadata_sql, view_function, select_sql, dictGet_sql, select_catalog_sql

Policies

Policies combine a Scope and a set of Permissions. They are assigned to Roles. The scope determines which resources the policy applies to, e.g. a specific organization or project or table. The permissions set determines what actions users can take within that scope, e.g. editing or viewing data.

For example, can Tessa in the example above read any table in a project? The answer is yes, if there is a policy with a scope of 'project' that contains the select_sql permission in its permissions set. The "MyProjectReader" role in the previous example should have this policy in its policy collection, allowing Tessa to read all tables in this project.

Permissions

In Hydrolix, permissions control whether or not a user can perform a given action for a particular resource. Each permission corresponds to an action (denoted with prefixes at the beginning of the permission name) and a resource (denoted by suffixes at the end of the permission name). For instance, the view_table permission describes the view action executed on the table resource.

Hydrolix RBAC provides the following actions, each of which corresponds to multiple permission:

View

The view action allows users to list and retrieve information about a resource. Examples of permissions that require the view action include:

view_dictionary: see the contents of a custom dictionary
view_function: see the definition of a custom function
view_transform: see the definition of a transform

Intelligent list filtering

Note that Hydrolix doesn't have an explicit list action. This is because Hydrolix intelligently determines which objects a user has permission to list according to all their other permissions. An endpoint that returns a list will rarely return a 403 Forbidden error code -- instead, Hydrolix calculates permissions for parent, child, and sibling scope objects and returns only the relevant objects.

Add

The add action allows users to create new data in a resource. Examples of permissions that require the add action include:

add_dictionary: insert a new key-value pair into a custom dictionary
add_function: define a new function in a project
add_table: create a new table in a project
add_transform: create a new transform

Change

The change action allows users to alter existing data in a resource. Examples of permissions that require the change action include:

change_table: edit data in a column of an existing row in a table
alter_table_sql: alter the name of a table
change_dictionary: change the value of a key in a custom dictionary
change_transform: alter a transform definition

Delete

The delete action allows users to remove existing data from a resource. Examples of permissions that require the delete action include:

delete_function: remove a custom function
delete_transform: remove a transform
delete_table: remove a table from a project

ALL

The ALL permission grants a user every possible action for a given resource. A user with the ALL permission can perform any action on the specified resource.

SQL

SQL operations in Hydrolix require a model with more nuance than the basic view permission can provide. As a result, SQL operations have their own set of permissions in the Hydrolix RBAC system, indicated by the _sql suffix.

For example:

select_catalog_sql allows users to read the catalog, which contains metadata about table partitions
insert_sql allows users to insert data into a table with INSERT statements
select_sql allows users to read data from a table with SELECT statements, and also implicitly grants show_project_sql, show_table_sql, and show_columns_sql
alter_table_sql allows users to change data within a table using ALTER statements
show_columns_sql allows users to view column information within a table using SHOW COLUMNS
show_tables_sql allows users to view a list of tables within a project using SHOW TABLES
view_project_sql allows users to view a list of projects
select_metadata_sql allows users to query table metadata

Users must have a subset of these permissions to perform SQL queries on data stored within the Hydrolix cluster.

Other Permissions

Not all API endpoints perform operations that fit cleanly into the action categories of view, add, change, delete, or SQL operations. In these cases, Hydrolix provides additional permissions, including the following:

retry to attempt a failed action again
status to fetch the status of an action
verify to confirm the validity of a proposed action
commit to submit an action for later execution
cancel to stop a submitted action from executing
generate to create data using an automated process
stats to fetch general information about a system
activity to fetch a summary of activity within a resource
populate to generate metadata based on existing data
truncate to remove data based on some criteria

Examples

RBAC via the UI

Define a Role in the UI

You can use the Hydrolix UI to define roles.

Only users with administrator access can view, create, and edit roles.

Follow these steps to create a role in the Hydrolix:

Log in to the portal.
In the left sidebar, click Security to navigate to the security panel.
Navigate to the Roles tab to open the roles editor.
In the top right corner of the screen, click the + Add new button.
Select the Role option in the right sidebar.
Choose a unique name for your role.
Add a Description that explains the purpose of your role.
Select an existing policy from the list, or click Add New Policy to define a new policy.
Select a Scope Level, which defines the level of the resource hierarchy addressed by your new role.
Select a resource within that scope level, such as a specific project in your cluster.
In the Permissions box, enter the list of permissions to grant users with the role.
Click the Save changes button to add the policy to your role.
Repeat the previous five steps until you've defined the permissions needed for your role.
Click the Create Role button to save the role within your Hydrolix cluster configuration.

Assign a Role to a User in the UI

You can use the Hydrolix UI to assign and remove roles associated with a user.

Only users with administrator access can assign and remove roles from user accounts.

Users must have at least one role at all times.

Follow these steps to assign a role to a user in the Hydrolix UI:

Log in to the portal.
In the left sidebar, click Security to navigate to the security panel.
Navigate to the Users tab to open the roles editor.
Click on the user whose role assignment you would like to change.
In the Select role dropdown, add and remove roles as needed to combine permissions for the user.
Click the Save changes button to save the role assignment within your Hydrolix cluster configuration.

RBAC via the API

RBAC operations can be performed by the REST API, too.

For purposes of demonstration, we'll use the curl command due to its widespread availability. Other ways of interacting with the API are listed on the Query Data page. Piping the output of the curl command through the jq utility makes the results much more readable.

The first steps below position you to be able to use the API to modify RBAC settings.

Log in to the API

Get the bearer token, which is good for the next 24 hours, to authenticate future API calls. This command assumes you've set the $HDX_HOSTNAME, $HDX_USER and $HDX_PASSWORD environment variables:

export HDX_TOKEN=$(
  curl -v -X POST -H "Content-Type: application/json" \
  https://$HDX_HOSTNAME/config/v1/login/ \
  -d "{
    \"username\":\"$HDX_USER\",
    \"password\":\"$HDX_PASSWORD\"  
  }" | jq -r ".auth_token.access_token"
)

export HDX_TOKEN=$(
  curl -v -X POST -H "Content-Type: application/json" \
  https://$HDX_HOSTNAME/config/v1/login/ \
  -d "{
    \"username\":\"$HDX_USER\",
    \"password\":\"$HDX_PASSWORD\"  
  }" | jq -r ".auth_token.access_token"
)

List Orgs and Projects via the API

The rest of these examples use the $HDX_ORG and $HDX_PROJECT environment variables.

List and choose our Hydrolix installation's organizations by listing them with the following command:

curl -X GET -H "Content-Type: application/json" \
   -H "Authorization: Bearer $HDX_TOKEN" \
   https://$HDX_HOSTNAME/config/v1/orgs \
   | jq -r ".results.[] | [.uuid,.name] | @tsv"

From the output, find the organization's UUID you want to use and assign that to an environment variable called $HDX_ORG.

Similarly, use this command to find the list of projects available to the organization:

curl -X GET -H "Content-Type: application/json" \
   -H "Authorization: Bearer $HDX_TOKEN" \
   https://$HDX_HOSTNAME/config/v1/orgs/$HDX_ORG/projects \
   | jq -r ".[] | [.uuid,.name] | @tsv"

From the output, find the project you want to use and assign its UUID to an environment variable called $HDX_PROJECT.

Create a New Role via the API

This example creates a new role called test_role_1 that has read access to tables within that project:

curl -X POST -H "Content-Type: application/json" \
  -H "Authorization: Bearer $HDX_TOKEN" \
  https://$HDX_HOSTNAME/config/v1/roles -d "{
    \"name\": \"test_role_1\",
    \"policies\": [
        {
            \"permissions\": [
                \"view_org:metadata\"
            ],
            \"scope_type\": \"org\",
            \"scope_id\": \"$HDX_ORG\"
        },
        {
            \"permissions\": [
                \"dictGet_sql\",
                \"select_metadata_sql\",
                \"select_sql\",
                \"show_columns_sql\",
                \"view_function\"
            ],
            \"scope_type\": \"project\",
            \"scope_id\": \"$HDX_PROJECT\"
        }
    ]
}" | jq

Invite a New User via the API

This will start the process of creating a new user. The system will attempt to e-mail an invitation to the address supplied, using that as the name of the account. If the e-mail isn't successfully delivered to the e-mail account, the invite_url in the POST response can be used to confirm the new user.

curl -v -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $HDX_TOKEN" https://$HDX_HOSTNAME/config/v1/inviteurl -d "{
    \"email\": \"[email protected]\",
    \"org\": \"$HDX_ORG\",
    \"roles\": [
        \"test_role_1\"
    ]
}" | jq

Result:

{
  "invite_url": "https://$HDX_HOSTNAME/verifyaccount/ZWU4OWM2OWMtY2UxMi00YTAwLTlmMTYtZjhlMTQzYzZlYjJh/bzhb2o-8d08517022529381fb7125e008a28045"
}

This user has the test_role_1 role from the example above. Once the two examples have been configured, the result will look like this:

Permissions Reference

Hydrolix supports various permissions for different resources. Use the list of actions above and the following comprehensive lists of permissions to find the right collection of permissions for your roles.

This reference covers the major table, project, and organization resource type. Other resource types are available as well, and can retrieved using the UI and the API.

Table Permissions

Hydrolix supports the following permissions at the table scope:

add_alterjob
cancel_alterjob
commit_alterjob
delete_alterjob
retry_alterjob
status_alterjob
verify_alterjob
view_alterjob
add_batchjob
cancel_batchjob
delete_batchjob
retry_batchjob
status_batchjob
view_batchjob
add_kafkasource
change_kafkasource
delete_kafkasource
view_kafkasource
add_kinesissource
change_kinesissource
delete_kinesissource
view_kinesissource
add_siemsource
change_siemsource
delete_siemsource
view_siemsource
alter_table_sql
insert_sql
select_sql
select_catalog_sql
select_metadata_sql
show_columns_sql
show_tables_sql
add_summarysource
change_summarysource
delete_summarysource
view_summarysource
activity_table
add_table
change_table
delete_table
generate_table
populate_catalog_table
stats_table
truncate_table
view_table
add_transform
change_transform
delete_transform
view_transform
add_view
change_view
delete_view
view_view
ALL

Project Permissions

Hydrolix supports the following permissions at the project scope:

add_alterjob
cancel_alterjob
commit_alterjob
delete_alterjob
retry_alterjob
status_alterjob
verify_alterjob
view_alterjob
add_batchjob
cancel_batchjob
delete_batchjob
retry_batchjob
status_batchjob
view_batchjob
add_dictionary
change_dictionary
delete_dictionary
view_dictionary
add_dictionaryfile
change_dictionaryfile
delete_dictionaryfile
view_dictionaryfile
add_function
change_function
delete_function
view_function
add_kafkasource
change_kafkasource
delete_kafkasource
view_kafkasource
add_kinesissource
change_kinesissource
delete_kinesissource
view_kinesissource
activity_project
add_project
change_project
delete_project
stats_project
view_project
add_siemsource
change_siemsource
delete_siemsource
view_siemsource
alter_table_sql
dictGet_sql
insert_sql
select_sql
select_catalog_sql
select_metadata_sql
show_columns_sql
show_databases_sql
show_dictionaries_sql
show_tables_sql
add_summarysource
change_summarysource
delete_summarysource
view_summarysource
activity_table
add_table
change_table
delete_table
generate_table
populate_catalog_table
stats_table
truncate_table
view_table
add_transform
change_transform
delete_transform
view_transform
add_view
change_view
delete_view
view_view
ALL

Organization Permissions

Hydrolix supports the following permissions at the organization scope:

add_alterjob
cancel_alterjob
commit_alterjob
delete_alterjob
retry_alterjob
status_alterjob
verify_alterjob
view_alterjob
add_batchjob
cancel_batchjob
delete_batchjob
retry_batchjob
status_batchjob
view_batchjob
add_catalog
delete_catalog
view_catalog
add_dictionary
change_dictionary
delete_dictionary
view_dictionary
add_dictionaryfile
change_dictionaryfile
delete_dictionaryfile
view_dictionaryfile
add_function
change_function
delete_function
view_function
add_hdxstorage
change_hdxstorage
delete_hdxstorage
view_hdxstorage
add_kafkasource
change_kafkasource
delete_kafkasource
view_kafkasource
add_kinesissource
change_kinesissource
delete_kinesissource
view_kinesissource
activity_org
fetch_merge_pools_org
fetch_query_options_org
purgejobs_org
update_merge_pools_org
update_query_options_org
view_org
activity_project
add_project
change_project
delete_project
stats_project
view_project
add_siemsource
change_siemsource
delete_siemsource
view_siemsource
all_sql
alter_table_sql
create_function_sql
create_temporary_table_sql
dictGet_sql
drop_function_sql
insert_sql
select_sql
select_catalog_sql
select_metadata_sql
select_usage_sql
show_columns_sql
show_databases_sql
show_dictionaries_sql
show_tables_sql
add_summarysource
change_summarysource
delete_summarysource
view_summarysource
activity_table
add_table
change_table
delete_table
generate_table
populate_catalog_table
stats_table
truncate_table
view_table
add_transform
change_transform
delete_transform
view_transform
add_view
change_view
delete_view
view_view
ALL

Notes

ALL Permission

Administrator Permissions

Administrators can use the ALL permission with a scope including all organizations to have access to all resources and actions at all times.

Single Table Write Table Permissions

Similarly, users who require write access to a single table can use the ALL permission in the context of the one table.