Skip to content

Org Transition Planning

This page contains guidance for Hydrolix cluster administrators wanting to understand the transition to multiple organization support.

Administrators of single-tenant or dedicated clusters don't need to do anything. See single-tenant Org transition planning.

Administrators of multi-tenant or shared clusters must set the tunable turbine_api_feature_migrate_projects_to_orgs before upgrading to Hydrolix v6.1. Additional tools in subsequent releases provide the ability to modify relationships between objects in the cluster and especially the Customer objects which become Organizations to complete the transition. See multi-tenant Org transition planning.

The installation and switch to the new organizations hierarchy will occur over several software releases.

Detailed transition instructions will accompany release notes for each stage.

For a depiction of the secondary, Customer hierarchy, see transition to new hierarchy.

What's changing?⚓︎

Hydrolix clusters will support multiple organizations. Each organization contains projects, users, credentials, storage definitions, and more.

After several releases, when the transition is complete, the Customers become Organizations. The original hierarchy is replaced by the hierarchy defined by the newly-promoted Organization objects.

Two projects in every cluster are always assigned their own organizations, regardless of the tunable setting.

  • the demonstration sample_project
  • the administrative hydro project, which contains Hydrologs and Audit Logging tables

Why?⚓︎

In Hydrolix v6.0 and earlier, the Config API only supports a single organization. For multi-tenant clusters, this incurs a management burden on the administrator.

Benefits⚓︎

Support for multiple organizations includes the following benefits:

  • An Org naturally models a single tenant and their linked resources in a single administrative domain.
  • Users have a primary or default Org and can belong to multiple Orgs.
  • Storage and credential definitions belong to an Org and can optionally be shared with other Orgs.
  • Org and project UUIDs are no longer required for many Config API endpoints, allowing shorter URLs.
  • Cluster services switch from a single monolithic configuration to a manifest and many files, decreasing time to activate changes.
  • Administration responsibilities can be delegated from cluster administrator to Org administrators.
  • Permissions management for clusters with many projects and tables becomes simpler.

What's not changing?⚓︎

  • The super_admin role permissions remain global to the cluster.
  • Primary storage definitions and their related credentials must remain global to the cluster. This storage is used for logs, config blobs, and backups.
  • Project names must be unique across a cluster.
  • The conceptual model remains the same before and after the transition.

Transition stages⚓︎

Parallel hierarchy installation⚓︎

In release v6.1, a parallel Customer hierarchy is created. Each Customer is a future Org.

  • Preparation: Select upgrade path using a tunable. See v6.1 Release Notes, Upgrade instructions.
  • Upgrade process: The upgrade automatically creates Customer objects, according to the tunable setting.
  • After upgrade: Multi-tenant administrators can change project to customer associations.

See detailed description and instructions in parallel hierarchy.

Establish relationships⚓︎

In releases v6.2 and v6.3, users, storage definitions, credentials, and other objects are automatically assigned to Customers.

The Config API begins introducing shorter URLs in the v2 endpoints.

Relationships are derived from existing permissions.

  • Preparation:
    • Audit user roles and permissions for accuracy.
  • Background jobs:
    • User accounts are assigned to Customers automatically.
    • Credential, Storage, and Invite objects are associated to Customers.
  • After upgrade: Administrators can audit and correct ownership and relationships.

See detailed description and instructions in relationship establishment.

Adoption of new hierarchy⚓︎

This change occurs in a future release.

Cluster services begin using new, multi-org configuration files and Config API v2 endpoints.

  • Preparation:
    • Perform a relationship audit.
    • Customer, Storage, and Credential assignments must be correct.
    • Users must have a default Customer, except for users with super_admin permissions.
  • Background jobs:
    • Background relationship establishment jobs stop.
  • After upgrade:
    • All services, like ingestion and query systems, begin using configs generated from the Customer hierarchy.
    • Project to Customer assignment is immutable.

Retirement of original hierarchy⚓︎

This change occurs in a future release.

The new Customer hierarchy replaces the original Org hierarchy. Every Customer is now an Org. The transition is complete.

  • Preparation:
  • Upgrade process: Customers automatically become Orgs.
  • After upgrade: The transition is complete.

Transition to new hierarchy⚓︎

erDiagram
  BGlobal["Global"]
  BOrg[Org]
  BProject[Project]
  BTable[Table]
  BGlobal ||--|| BOrg : contains-one
  BOrg ||--o{ BProject : can-contain-multiple
  BProject ||--o{ BTable : can-contain-multiple

  DGlobal[Global]
  DOrg[Org]
  DCustomer[Customer]
  DProject[Project]
  DTable[Table]
  DGlobal ||--o{ DOrg : contains-one
  DGlobal ||--o{ DCustomer : can-contain-multiple
  DOrg ||--o{ DProject : can-contain-multiple
  DCustomer ||--o{ DProject : can-contain-multiple
  DProject ||--o{ DTable : can-contain-multiple

  AGlobal[Global]
  AOrg["Org (was Customer)"]
  AProject[Project]
  ATable[Table]
  AGlobal ||--o{ AOrg : can-contain-multiple
  AOrg ||--o{ AProject : can-contain-multiple
  AProject ||--o{ ATable : can-contain-multiple