Preserve Client IP Addresses

This feature was introduced in Hydrolix v5.8.5.

Configure Traefik to preserve the original client IP addresses to enable IP-based access control, audit logging, and troubleshooting.

Prerequisites

• Hydrolix v5.8.5 or higher
• kubectl access to the cluster
• Permission to modify the HydrolixCluster specification
• For AWS (EKS) only: Helm and eksctl

Use cases

• Audit logging - Track which client IPs access the cluster and when
• IP-based access control - Apply IP allowlists at the application level
• Troubleshooting - Identify the source of problematic requests
• Security analysis - Analyze access patterns and detect anomalies

How it works

The implementation varies by cloud provider based on the type of load balancer.

• Linode (LKE) and AWS (EKS): Use proxy protocol, which passes client IP information through Layer 4 (TCP) load balancers
• Google Cloud (GKE) and Azure (AKS): Use externalTrafficPolicy=Local, which routes traffic only to nodes with Traefik pods

Enable client IP preservation

Setup steps vary by cloud provider.

Choose your cloud platform:

Linode (LKE)AWS (EKS)Google Cloud (GKE)Azure (AKS)

Linode supports proxy protocol through service annotations. Add this to your HydrolixCluster configuration:

apiVersion: hydrolix.io/v1
kind: HydrolixCluster
spec:
  kubernetes_profile: lke
  traefik_service_annotations:
    service.beta.kubernetes.io/linode-loadbalancer-port-443: '{"proxy-protocol": "v2"}'
    service.beta.kubernetes.io/linode-loadbalancer-port-80: '{"proxy-protocol": "v2"}'

The Linode controller updates the load balancer dynamically. Changes take effect within a few minutes.

EKS requires the AWS Load Balancer Controller to enable proxy protocol. The built-in controller doesn't support this feature reliably.

Install the AWS Load Balancer Controller (if not already installed):

export POLICY_NAME=AWSLoadBalancerControllerIAMPolicy
export ACCOUNT_ID=<your-account-id>
export CLUSTER_NAME=<your-cluster-name>
export REGION=<your-region>

# Create IAM policy
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.14.1/docs/install/iam_policy.json
aws iam create-policy --policy-name $POLICY_NAME --policy-document file://iam_policy.json

# Create service account
eksctl create iamserviceaccount \
  --cluster=$CLUSTER_NAME \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --attach-policy-arn=arn:aws:iam::$ACCOUNT_ID:policy/$POLICY_NAME \
  --override-existing-serviceaccounts \
  --region $REGION \
  --approve

# Install controller with Helm
helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=$CLUSTER_NAME \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

Configure the HydrolixCluster:

apiVersion: hydrolix.io/v1
kind: HydrolixCluster
spec:
  kubernetes_profile: eks
  traefik_load_balancer_class: service.k8s.aws/nlb
  traefik_service_annotations:
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol-per-target-group: "443, 80"
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-type: external

If proxy protocol doesn't activate on the existing load balancer, the load balancer may need to be replaced. Contact Hydrolix support for help with load balancer replacement.

GKE uses externalTrafficPolicy=Local to preserve client IPs. This requires a backend service-based load balancer with health checks that track per-node pod readiness, and extending Traefik's pre-stop delay so the pod remains available long enough for those health checks to detect it as unavailable before it terminates.

Configure pre-stop delay:

GKE health checks have an 8-second interval with a 3-failure threshold (24 seconds total). Set the Traefik pre-stop delay to 30 seconds so the pod remains available long enough for health checks to fail and reroute traffic before the pod terminates:

apiVersion: hydrolix.io/v1
kind: HydrolixCluster
spec:
  kubernetes_profile: gke
  traefik_pre_stop_seconds: 30
  traefik_service_annotations:
    cloud.google.com/l4-rbs: "enabled"

The l4-rbs annotation switches the load balancer to a backend service-based architecture. This enables health checks that track pod readiness on each node, which is required for externalTrafficPolicy: Local to route traffic only to nodes running a ready Traefik pod. Adding this annotation requires replacing the load balancer. Contact Hydrolix support for a load balancer replacement.

Enable local traffic policy:

After the load balancer is configured with pod-readiness health checks, enable local traffic policy:

apiVersion: hydrolix.io/v1
kind: HydrolixCluster
spec:
  kubernetes_profile: gke
  traefik_pre_stop_seconds: 30
  traefik_service_annotations:
    cloud.google.com/l4-rbs: "enabled"
  traefik_use_local_policy: true

AKS health check defaults (5-second interval, 1-failure threshold) are aggressive enough that the Traefik pre-stop hook does not need to be configured. The pre-stop hook is a configurable sleep that delays pod termination, giving load balancer health checks time to detect unavailability. Enable local traffic policy:

apiVersion: hydrolix.io/v1
kind: HydrolixCluster
spec:
  kubernetes_profile: aks
  traefik_use_local_policy: true

Verify configuration

After applying the configuration, verify that Traefik is preserving client IPs.

Query the audit log

Make an authenticated API call to the Hydrolix cluster, then query the audit log to check the recorded IP address:

SELECT ip_address, username, type, timestamp
FROM hydro.audit_logs
WHERE timestamp >= now() - INTERVAL 5 MINUTE
ORDER BY timestamp DESC
LIMIT 10

If ip_address shows your public IP, the configuration is working correctly. If it shows a private IP (10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16), the configuration isn't working. Wait five minutes for settings to propagate, then check again.

Client IPs show as private IPs

If the logs show private IP addresses after waiting five minutes:

• Verify the configuration matches the cloud provider's example
• Check that annotations are applied correctly to the Traefik service: kubectl -n <namespace> get svc traefik -o yaml
• For EKS, confirm the AWS Load Balancer Controller is installed and managing the service

Rollback procedure

To revert the configuration, remove the annotations and tunables:

• LKE/EKS: Remove the traefik_service_annotations
• GKE/AKS: Set traefik_use_local_policy: false or remove the setting
• GKE only: You can leave traefik_pre_stop_seconds configured without affecting functionality