Container Security Contexts

Version availability

Hydrolix hardened the default container security context in Hydrolix version 6.4. See Settings changed in version 6.4.

Hydrolix applies a Kubernetes security context to every workload it deploys. The security context constrains what a container can do on the node:

• which user identity the process assumes
• whether it can write to its own root filesystem
• whether it can gain new privileges
• which Linux capabilities it holds.

This page describes the defaults Hydrolix applies, the workloads that require relaxed settings, and the upgrade path for clusters moving to version 6.4.

Default security context

Hydrolix applies a restricted security context by default to everything it deploys. Every workload runs under these settings unless it overrides the default security context. Defaults are applied to pods, including all their replicas, and containers.

The pod-level settings establish the user identity the pod's processes assume. The container-level settings restrict what a container can do once it's running.

Pod security contextContainer security context

securityContext:
  runAsNonRoot: true
  runAsUser: 1000

securityContext:
  runAsNonRoot: true
  readOnlyRootFilesystem: true
  allowPrivilegeEscalation: false
  runAsUser: 1000
  capabilities:
    drop:
    - ALL

Some containers receive a /tmp directory

Because containers run with a read-only root filesystem, workloads that need scratch space receive an emptyDir volume mounted at the path they write to, most often /tmp. A container that writes to an unmounted path on its root filesystem fails at runtime rather than silently writing to ephemeral storage.

Sidecars and init containers

A workload's own security context reaches the pod and the workload's main container. It doesn't extend to the other containers in the pod; sidecars and init containers each carry a security context of their own, so a single pod can run containers under different security contexts.

For example, there are three different containers with varying security contexts running in an intake-head pod:

Container	Type of container	Security context
intake-head	Main container	Set by the workload (user and group 1000)
turbine	Sidecar	Restricted default
wait-for-init-turbine-api	Init container	Restricted default

Where a pod-level setting and a container-level setting cover the same field, Kubernetes applies the container-level value. Of the settings Hydrolix sets, only runAsUser, runAsGroup, and runAsNonRoot overlap this way.

Setting	Level
fsGroup, fsGroupChangePolicy	Pod only
runAsUser, runAsGroup, runAsNonRoot	Pod and container
readOnlyRootFilesystem, allowPrivilegeEscalation, capabilities	Container only

To inspect a container's security context, use the following command.

kubectl get pod <pod-name> -n <namespace> \
  -o jsonpath='{range .spec.initContainers[*]}{.name}{"\t"}{.securityContext}{"\n"}{end}{range .spec.containers[*]}{.name}{"\t"}{.securityContext}{"\n"}{end}'

Settings changed in version 6.4

The defaults for these container-level settings changed in 6.4.

Setting	Version 6.3 and earlier	Version 6.4 and later
readOnlyRootFilesystem	Not set, so containers could write anywhere on the root filesystem	true
allowPrivilegeEscalation	Not set, so a process could gain privileges through a setuid binary	false
capabilities.drop	Not set, so containers kept the default capability set	["ALL"]

All other settings remain the same. The full list of settings can be found in Sidecars and init containers.

Workloads with relaxed or non-default settings

Some workloads can't run under the full restricted context. Hydrolix relaxes specific settings for these workloads and leaves the rest of the context intact.

Workload	Relaxed settings	Reason
vector	Runs as the default user for the image with a writable root filesystem	Vector mounts host paths to hold state files that persist across pod restarts.
tooling	Runs as root, writable root filesystem, privilege escalation allowed, capabilities retained	The tooling pod supports debugging tasks that need root, such as installing packages.
hdx-node	Runs as root, writable root filesystem, with the NET_ADMIN and NET_RAW capabilities added	The workload runs operating-system-level networking commands.
hdx-gate	Runs as root, writable root filesystem, privilege escalation allowed	The workload runs operating-system-level commands. Applies only when the cluster uses the hdx-gate proxy mode.

While RabbitMQ and Traefik don't hold extra privileges, they do use a non-default user identity.

Workload	Non-default user settings	Reason
rabbitmq	Runs as user and group 999	RabbitMQ images expect that identity. The restricted container-level settings still apply.
traefik	Runs as user and group 65532	The Traefik image expects that identity. The restricted container-level settings still apply.

Upgrade path for RabbitMQ

Versions v5.9.5 through v6.3 deploy RabbitMQ with an init container that runs as root and corrects file ownership on the RabbitMQ data volume. v6.4 removes that init container.

Upgrade through a version that corrects RabbitMQ file ownership

When upgrading a Hydrolix cluster, make sure to upgrade sequentially through each major.minor release. See the v6.4 upgrade instructions.

To upgrade a cluster from a version earlier than v5.9:

1. Upgrade to a version in the v5.9 through v6.3 range
2. Let RabbitMQ start so the init container corrects ownership on the data volume
3. Upgrade to v6.4

Upgrading straight to v6.4 leaves root-owned files on the volume that the unprivileged RabbitMQ process can't read.

Root user override

The force_container_user_root tunable sets the initial user for all containers to root. It defaults to false.

Tunable sets initial user but doesn't alter filesystem restrictions

This tunable overrides only the user identity. It doesn't enable writes to the root filesystem, allow privilege escalation, or return dropped capabilities, so a container running as root under this tunable still may not be able to write to its root filesystem. Use it for diagnostics only.

spec:
  force_container_user_root: true