Scale Profiles

Dev

Suggested profile for a minimally-resourced development environment

service              replicas    cpu  memory    storage    data_storage
-------------------  ----------  -----  --------  ---------  --------------
alter-head                    0    500m  1Gi       1Gi        -
alter-peer                    0    500m  1Gi       1Gi        -
autoingest                    1    500m  512Mi     1Gi        -
batch-head                    1    500m  1Gi       1Gi        -
batch-peer                    1    500m  1Gi       1Gi        -
decay                         1    500m  512Mi     1Gi        -
indexer                       0    500m  1Gi       1Gi        -
init-cluster                  1    500m  1Gi       1Gi        -
init-turbine-api              1    500m  1Gi       1Gi        -
intake-api                    1    500m  1Gi       1Gi        -
kafka-peer                    0    500m  1Gi       1Gi        -
keycloak                      1    1     1Gi       1Gi        -
kinesis-peer                  0    500m  1Gi       1Gi        -
load-sample-project           1    500m  1Gi       1Gi        -
merge-head                    1    500m  1Gi       1Gi        -
merge-peer                    1    500m  1Gi       1Gi        -
operator                      1    500m  1Gi       1Gi        -
otel                          0    500m  1Gi       1Gi        -
postgres                      1    1     1Gi       1Gi        5Gi
prometheus                    1    1     1Gi       1Gi        5Gi
promwaltz                     0    500m  512Mi     1Gi        -
query-head                    1    1     1Gi       1Gi        -
query-peer                    1    1     1Gi       1Gi        -
rabbitmq                      1    500m  1Gi       1Gi        5Gi
reaper                        1    500m  512Mi     1Gi        -
redpanda                      1    1     1Gi       1Gi        5Gi
stream-head                   1    500m  1Gi       1Gi        -
stream-peer                   1    500m  1Gi       1Gi        -
traefik                       1    500m  512Mi     1Gi        -
traefik-cfg                   0    500m  512Mi     1Gi        -
turbine-api                   1    500m  1Gi       1Gi        -
ui                            1    500m  1Gi       1Gi        -
validator                     1    500m  1Gi       1Gi        -
vector                        0    100m  512Mi     1Gi        -
version                       1    500m  1Gi       1Gi        -
zookeeper                     1    500m  1Gi       1Gi        1Gi

Minimal

Suggested profile for a small-scale trial environment

service                replicas    cpu  memory    storage    data_storage
-------------------  ----------  -----  --------  ---------  --------------
alter-head                    0   1      1Gi       5Gi        -
alter-peer                    0   1      1Gi       5Gi        -
autoingest                    1   500m   512Mi     1Gi        -
batch-head                    1   500m   1Gi       5Gi        -
batch-peer                    1   2      4Gi       8Gi        -
decay                         1   500m   1Gi       5Gi        -
indexer                       0   1      4Gi       8Gi        -
init-cluster                  1   500m   1Gi       5Gi        -
init-turbine-api              1   500m   1Gi       5Gi        -
intake-api                    1   500m   1Gi       5Gi        -
kafka-peer                    0   2      4Gi       8Gi        -
keycloak                      1   1      2Gi       5Gi        -
kinesis-peer                  0   2      4Gi       8Gi        -
load-sample-project           1   500m   1Gi       1Gi        -
merge-head                    1   500m   1Gi       5Gi        -
merge-peer                    2   2      8Gi       8Gi        -
operator                      1   500m   1Gi       5Gi        -
otel                          0   500m   1Gi       1Gi        -
postgres                      1   4      8Gi       5Gi        100Gi
prometheus                    1   1      1Gi       5Gi        50Gi
promwaltz                     0   500m   512Mi     1Gi        -
query-head                    1   3      12Gi      20Gi       -
query-peer                    1   6      24Gi      20Gi       -
rabbitmq                      1   1      4Gi       5Gi        10Gi
reaper                        1   2      1Gi       5Gi        -
redpanda                      2   2      4Gi       5Gi        1Ti
stream-head                   1   2      2Gi       8Gi        -
stream-peer                   1   2      4Gi       8Gi        -
traefik                       1   1      512Mi     5Gi        -
traefik-cfg                   0   500m   512Mi     1Gi        -
turbine-api                   1   500m   1Gi       5Gi        -
ui                            1   500m   1Gi       5Gi        -
validator                     1   1      1Gi       1Gi        -
vector                        0   500m   1Gi       1Gi        -
version                       1   500m   1Gi       1Gi        -
zookeeper                     1   1      1Gi       5Gi        5Gi

Prod

Suggested profile for a reliable production environment (1-4 TB/day)

service                replicas    cpu  memory    storage    data_storage
-------------------  ----------  -----  --------  ---------  --------------
alter-head                    0   1     1Gi       5Gi        -
alter-peer                    0   2     1Gi       5Gi        -
autoingest                    1   500m  512Mi     1Gi        -
batch-head                    1   500m  1Gi       5Gi        -
batch-peer                    2   2     8Gi      10Gi        -
decay                         1   500m  1Gi       5Gi        -
indexer                       0   2     8Gi      10Gi        -
init-cluster                  1   500m  1Gi       5Gi        -
init-turbine-api              1   500m  1Gi       5Gi        -
intake-api                    2   500m  1Gi       5Gi        -
kafka-peer                    0   2     8Gi      10Gi        -
keycloak                      1   1     2Gi       5Gi        -
kinesis-peer                  0   2     8Gi      10Gi        -
load-sample-project           1   500m  1Gi       1Gi        -
merge-head                    1   500m  1Gi       5Gi        -
merge-peer                    4   2     8Gi      10Gi        -
operator                      1   500m  1Gi       5Gi        -
otel                          0   500m  1Gi       1Gi        -
postgres                      1   6     16Gi      5Gi        100Gi
prometheus                    1   2     2Gi       1Gi        50Gi
promwaltz                     0   500m  512Mi     1Gi        -
query-head                    2   6     24Gi      50Gi       -
query-peer                    3   12    48Gi      50Gi       -
rabbitmq                      3   1     4Gi       5Gi        10Gi
reaper                        1   2     1Gi       5Gi        -
redpanda                      2   2     8Gi       5Gi        1Ti
stream-head                   2   4     4Gi      10Gi        -
stream-peer                   2   2     8Gi      10Gi        -
traefik                       2   2     1Gi       5Gi        -
traefik-cfg                   0   500m  512Mi     1Gi        -
turbine-api                   2   500m  1Gi       5Gi        -
ui                            1   500m  1Gi       5Gi        -
validator                     1   1     1Gi       1Gi        -
vector                        0   500m  1Gi       1Gi        -
version                       1   500m  1Gi       1Gi        -
zookeeper                     3   1     1Gi       5Gi        5Gi

Mega

Suggested profile for a large-scale production environment (10-50 TB/day)

service                replicas    cpu  memory    storage    data_storage
-------------------  ----------  -----  --------  ---------  --------------
alter-head                    0   1     1Gi       5Gi        -
alter-peer                    0   2     1Gi       5Gi        -
autoingest                    1   500m  512Mi     1Gi        -
batch-head                    1   500m  1Gi       5Gi        -
batch-peer                    3   2     8Gi      10Gi       -
decay                         1   500m  1Gi       5Gi        -
indexer                       0   2     8Gi      10Gi        -
init-cluster                  1   500m  1Gi       5Gi        -
init-turbine-api              1   500m  1Gi       5Gi        -
intake-api                    2   500m  1Gi       5Gi        -
kafka-peer                    0   2     8Gi      10Gi        -
keycloak                      1   1     2Gi       5Gi        -
kinesis-peer                  0   2     8Gi      10Gi        -
load-sample-project           1   500m  1Gi       1Gi        -
merge-head                    1   500m  1Gi       5Gi        -
merge-peer                16-64   1.8   8Gi      10Gi        -
operator                      1   500m  1Gi       5Gi        -
otel                          0   500m  1Gi       1Gi        -
postgres                      1   8     16Gi      5Gi        100Gi
prometheus                    1   3     3Gi       1Gi        50Gi
promwaltz                     0   500m  512Mi     1Gi        -
query-head                    2   6     24Gi      50Gi       -
query-peer                    8   12    48Gi      50Gi       -
rabbitmq                      3   1     4Gi       5Gi        10Gi
reaper                        2   2     1Gi       5Gi        -
redpanda                      5   2     8Gi       5Gi        1Ti
stream-head                4-16   4     4Gi      10Gi        -
stream-peer                8-32   2     8Gi      10Gi        -
traefik                       3   4     1Gi       5Gi        -
traefik-cfg                   0   500m  512Mi     1Gi        -
turbine-api                   2   500m  1Gi       5Gi        -
ui                            1   500m  1Gi       5Gi        -
validator                     1   1     1Gi       1Gi        -
vector                        0   500m  1Gi       1Gi        -
version                       1   500m  1Gi       1Gi        -
zookeeper                     3   1     1Gi       5Gi        5Gi

Did this page help you?