Skip to content

Pod - Zookeeper

Overview⚓︎

Apache Zookeeper™ provides coordination services to the query subsystem.

Responsibilities⚓︎

Zookeeper retains state of the query system.

It is not used for any other purposes and is not reachable from outside the cluster.

In production clusters, three instances run, elect a leader and ensure high availability.

Deployment and scale⚓︎

comment
Type StatefulSet
Replicas default 3, minimum 1
Storage default 5 GiB
CPU 500 m
memory 512 MiB

Containers⚓︎

  • zookeeper only

Startup dependencies⚓︎

None.

Runtime dependencies⚓︎

None.

Runtime provides⚓︎

Zookeeper provides services

  • to query-head - a listing of available query-peers

  • to query-peer - communication services with query-head for assigned jobs

  • zookeeper holds the state of query-peer membership for each pool

Network listeners⚓︎

External⚓︎

None.

Internal⚓︎

  • tcp/2181 - query-head and query-peer clients connect here
  • tcp/2888 - zookeeper followers connect to leader's port 2888
  • tcp/3888 - used for leader election, if more than one zookeeper process

HTTP APIs⚓︎

None.

Configuration⚓︎

Secrets⚓︎

Hydrolix tunables⚓︎

Hydrolix Config API⚓︎

Tunables⚓︎

None.

Metrics⚓︎

Key health metrics⚓︎

Other application metrics⚓︎

Logs⚓︎

  • Logs appear in primary storage bucket under logs/zookeeper/zookeeper/YYYY-mm-dd/.
  • In hydro.logs, they can be found

Effects of breakage⚓︎

  • The query subsystem always relies on Zookeeper. The query subsystem is utterly non-functional until Zookeeper is present.
  • In the absence of Zookeeper, the query head applications will not be able to find query peer applications.

Troubleshoot⚓︎