Prepare your AWS Account

To ensure a successful deployment of a Hydrolix Data Platform we suggest preparing your AWS account for the install.

Hydrolix is a Data Platform that contains a number of components that are created and configured automatically at time of deployment. In order to be successful, it is important to set-up the AWS account to start.

We've broken this down to:

  1. AWS Account Preparation
  2. Users and Roles
  3. Marketplace Subscription
  4. Component Scaling



👍

Account Preparation

We would suggest installing Hydrolix into a Sub account.

The installer creates everything, including the VPC, networking components, service components, roles and policies needed to run the service, see what is deployed take a look here.

This means the installer requires Administrative access or a policylike the below to be used.

Make sure to grab the Account ID, you'll need it for your licence.




👍

Users and Roles

In order to successfully create your cluster you will need a user or policy that has rights to create and manage AWS components, roles and policies.

This can either be achieved through the use of a Administrative rights or by using the Policy below.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateStack",
                "cloudformation:DeleteStack",
                "cloudformation:DescribeStackEvents",
                "cloudformation:DescribeStacks",
                "cloudformation:GetStackPolicy",
                "cloudformation:ListStacks",
                "iam:AttachRolePolicy",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:DeleteRolePolicy",
                "iam:DetachRolePolicy",
                "iam:GetRole",
                "iam:GetRolePolicy",
                "iam:PassRole",
                "iam:PutRolePolicy",
                "lambda:CreateFunction",
                "lambda:DeleteFunction",
                "lambda:GetFunction",
                "lambda:GetFunctionCodeSigningConfig",
                "lambda:InvokeFunction",
                "lambda:UpdateFunctionCode",
                "s3:CreateBucket",
                "s3:DeleteBucket",
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:ListAllMyBuckets",
                "s3:ListBucket",
                "s3:PutBucketPolicy",
                "s3:PutEncryptionConfiguration",
                "s3:PutObject"
            ],
            "Resource": "*"
        }
    ]
}

More information on AWS Policies can be found here - AWS Permissions and Polices




👍

Marketplace Subscription

Subscribe to our AMI HDXReader from the Amazon Marketplace.

Just subscribe to the AMI, no need to deploy.

For more information on permission and market place usage you can read the AWS guide




📘

Account Scaling

Some users may have scale limits on their account. If so you should ensure you have the ability to scale the following instance sizes. Note this scale is the initial basic scale that is deployed when the architecture is created.

More information on what is deployed can be found here - AWS Hydrolix Architecture

Service/Pool

Count

Server Type

Disk

Bastion

1

t3.micro

10

Batch-peer

1

r5.2xlarge

30

Grafana

1

t3.small

10

Head (Query)

1

c5n.xlarge

10

Query-peer

1

c5n.4xlarge

10

Intake-misc

1

t3.small

10

Merge-peer

1

r5.xlarge

30

Prometheus

1

t2.medium

50

Stream-head

1

r5.xlarge

10

Stream-peer

1

r5.xlarge

10

Superset

1

t3.medium

10

Web

1

t3.small

10

Zookeeper

3

t2.small

10

Catalog

2

RDS (db.r5.large)

S3

S3 Bucket Created with your client_id


Did this page help you?