AWS Account Preparation

To ensure a successful deployment of a Hydrolix Data Platform we suggest preparing your AWS account for the install.

Hydrolix is a Data Platform that contains a number of components that are created and configured automatically at time of deployment. In order to be successful, it is important to set-up the AWS account to start.

We've broken this down to:

  1. AWS Account Preparation
  2. Users and Roles
  3. Marketplace Subscription
  4. Component Scaling



👍

Account Preparation

We would suggest installing Hydrolix into a Sub account.

The installer creates everything, including the VPC, networking components, service components, roles and policies needed to run the service, see what is deployed take a look here.

This means the installer requires Administrative access or a policylike the below to be used.

Make sure to grab the Account ID, you'll need it for your licence.




👍

Users and Roles

In order to successfully create your cluster you will need a user or policy that has rights to create and manage AWS components, roles and policies.

This can either be achieved through the use of a Administrative rights or by using the Policy below.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudformation:CreateStack",
        "cloudformation:DeleteStack",
        "cloudformation:DescribeStackEvents",
        "cloudformation:DescribeStacks",
        "cloudformation:GetStackPolicy",
        "cloudformation:ListStacks",
        "iam:AttachRolePolicy",
        "iam:CreateRole",
        "iam:DeleteRole",
        "iam:DeleteRolePolicy",
        "iam:DetachRolePolicy",
        "iam:GetRole",
        "iam:GetRolePolicy",
        "iam:PassRole",
        "iam:PutRolePolicy",
        "lambda:CreateFunction",
        "lambda:DeleteFunction",
        "lambda:GetFunction",
        "lambda:GetFunctionCodeSigningConfig",
        "lambda:InvokeFunction",
        "lambda:UpdateFunctionCode",
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListAllMyBuckets",
        "s3:ListBucket",
        "s3:PutBucketPolicy",
        "s3:PutEncryptionConfiguration",
        "s3:PutObject",
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "*"
    }
  ]
}

More information on AWS Policies can be found here - AWS Permissions and Polices




👍

Marketplace Subscription

Subscribe to our AMI HDXReader from the Amazon Marketplace.

Just subscribe to the AMI, no need to deploy.

For more information on permission and market place usage you can read the AWS guide




📘

Account Scaling

Some users may have scale limits on their account. If so you should ensure you have the ability to scale the following instance sizes. Note this scale is the initial basic scale that is deployed when the architecture is created.

More information on what is deployed can be found here - AWS Hydrolix Architecture

Service/PoolCountServer TypeDisk
Bastion1t3.micro10
Batch-peer1r5.2xlarge30
Grafana1t3.small10
Head (Query)1c5n.xlarge10
Query-peer1c5n.4xlarge10
Intake-misc1t3.small10
Merge-peer1r5.xlarge30
Prometheus1t2.medium50
Stream-head1r5.xlarge10
Stream-peer1r5.xlarge10
Superset1t3.medium10
Web1t3.small10
Zookeeper3t2.small10
Catalog2RDS (db.r5.large)-
S3-S3 Bucket Created with your client_id-