Trading system 002 --- production environment on AWS

on

 

Introduction

After you prepare the development environment locally, it is time to push the k8s stack to AWS. There at least 2 ways to do that:

  • EKS
  • Kops

In this article we will talk about how to deploy your k8s stack to AWS through EKS.

Workloads of the k8s cluster in this example:

  • elastic agent
  • kube state metrics
  • logstash 

Reference

learn from

Steps

Create IAM user group and user on AWS

We will use eksctl to create the k8s cluster. eksctl will leverage AWS CLI to finish the underlying implementation. We need to login to AWS CLI with a user with enough permission. In this section, we will create the needed user group and user.

User group


json file

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "eks:*",
            "Resource": "*"
        },
        {
            "Action": [
                "ssm:GetParameter",
                "ssm:GetParameters"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

User

Create a new user under the eksGroup


k8s cluster creation

You can build a gitlab CI/CD pipeline for this, but the easiest way is use this image and then run the following commands:

aws configure set aws_access_key_id "$AWS_EKS_USER_ID" # you can get this ID from AWS IAM
aws configure set aws_secret_access_key "$AWS_EKS_USER_PW" # you can get his from AWS IAM at the first time after you created this resource
aws configure set region "us-east-1" # pick the region that is suitable for you

eksctl create cluster --name YourCluster --nodes-min=2 --node-type t2.medium

This may take 20 minutes

Install workload to k8s cluster

The scripts are the same as we used before in the minikube. We used the minikube as a development environment and EKS as the production one. The only different here is how we create the k8s cluster. After that, all the steps are very similar — except volume preparation on AWS for example.

# install kube-state-metrics
helm upgrade --wait --timeout=1200s --install kube-state-metrics ./kube-state-metrics --namespace kube-system

# install elastic agent
kubectl apply -f ./elastic_agent/elastic-agent-managed-kubernetes.yaml

# install logstash
helm upgrade --wait --timeout=1200s --install --values ./logstash/custom_config/prod/values.customer.yaml logstash-elastic ./logstash

Please refer the file ./logstash/custom_config/prod/values.customer.yaml from here.

The end

This is done for the set up. The last thing you may want to know to how can you delete the cluster. Here is the eksctl command.

eksctl delete cluster --name YourCluster

Comments

Post a Comment