Skip to content

Org Transition Planning

This page contains guidance for Hydrolix cluster administrators wanting to understand the transition to multiple organization support.

Administrators of single-tenant or dedicated clusters don't need to do anything. See single-tenant Org transition planning.

Administrators of multi-tenant or shared clusters must set the tunable turbine_api_feature_migrate_projects_to_orgs before upgrading to Hydrolix v6.1. Additional tools in Hydrolix v6.1 and subsequent releases provide the ability to modify the Customer objects which become Organizations when the transition completes. See multi-tenant Org transition planning.

For a depiction of the transitional, secondary Customer hierarchy, see transition to new hierarchy.

What's changing?⚓︎

Hydrolix clusters will support multiple organizations. Each organization contains projects, users, credentials, storage definitions, and more.

After several releases, when the transition is complete, the Customers become Organizations. The original hierarchy is replaced by the hierarchy defined by the newly-promoted Organization objects.

Two projects in every cluster are always assigned their own organizations, regardless of the tunable setting.

  • the administrative hydro project, which contains Hydrologs and Audit Logging tables
  • the demonstration sample_project

Why?⚓︎

In Hydrolix v6.0 and earlier, the Config API only supports a single organization. For multi-tenant clusters, this incurs a user management burden on the administrator.

Benefits⚓︎

Support for multiple organizations includes the following benefits:

  • An Org naturally models a single tenant and their linked resources in a single administrative domain.
  • Users have a primary or default Org and can belong to multiple Orgs.
  • Storage and credential definitions belong to an Org and can optionally be shared with other Orgs.
  • Org and project UUIDs are no longer required for many Config API endpoints, allowing shorter URLs.
  • Cluster services switch from a single monolithic configuration to a manifest and many files, decreasing time to activate changes.
  • Administration responsibilities can be delegated from cluster administrator to Org administrators.
  • Permissions management for clusters with many projects and tables becomes simpler.

What's not changing?⚓︎

  • The super_admin role permissions remain global to the cluster.
  • Primary storage definitions and their related credentials must remain global to the cluster. This storage is used for logs, config blobs, and backups.
  • The conceptual model remains the same before and after the transition. See also transition to new hierarchy.

Transition stages⚓︎

The installation and switch to the new organizations hierarchy will occur over several software releases.

Detailed upgrade instructions will accompany release notes for each phase of the transition.

Parallel hierarchy installation⚓︎

In release v6.1, a parallel Customer hierarchy is created. Each Customer is a future Org.

  • Preparation: Select upgrade path using a tunable. See Upgrade to v6.1.
  • Upgrade process: The upgrade automatically creates Customer objects, according to the tunable setting.
  • After upgrade: Multi-tenant administrators can change project to customer associations.

See detailed description of the parallel hierarchy in v6.1.

Establish relationships⚓︎

This change occurs in a future release.

Customers are automatically assigned users, credentials, storage definitions, and more.

  • Preparation:
    • All Projects must be associated with a Customer. No project may be orphaned.
    • Audit user roles and permissions, prior to automatic assignment to Customer objects.
  • Upgrade process:
    • Users are automatically assigned to Customers based on their exisiting roles.
    • Invites, Credentials, Storages, and other objects are associated to their respective customer objects.
  • After upgrade: Administrators can audit and correct ownership and usage relationships.

Adoption of new hierarchy⚓︎

This change occurs in a future release.

The Config API publishes v2 endpoints with shorter URLs and cluster services begin switching to the newer format of internal configuration files.

  • Preparation:
    • All Customer, Storage, and Credential associations must be correct.
    • All Users must have a default Customer, except for users with super_admin permissions.
  • Upgrade process: No automated changes.
  • After upgrade:
    • Ingestion and query systems begin using configs generated from the Customer hierarchy.
    • The background job for automatic creation of Customer objects stops.
    • Many new relationship validations are enforced, for example, project creation now requires an existing Customer.

Retirement of old hierarchy⚓︎

This change occurs in a future release.

The new Customer hierarchy replaces the original Org hierarchy. Every Customer is now an Org. The transition is complete.

  • Preparation:
    • Clients must stop using the v1 endpoints which include the retired, original Org ID. Using exclusively v2 endpoints is better.
  • Upgrade process: Customers automatically become Orgs.
  • After upgrade: The transition is complete.