Setting up a Kubernetes cluster on EC2 is easy and I followed the article in [1]. When following that, I faced some problems. So in this article I will go through those with troubleshooting guide.
Following are the steps you need to follow.
1. Install and configure Kubectlin your local machine.
Kubectl is the client program we use for communication with kubernetes master node.
For Darwin
wget https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/darwin/amd64/kubectl
For Linux
wget https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kubectl
chmod +x kubectlmv kubectl /usr/local/bin/
2. Install and configure the AWS Command Line Interface in your local machine.
wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py
sudo pip install awscli
This program is used to communicate with EC2. For an example you can spawn instances using this program.
3. Run aws configure command in the terminal to provide all the required configurations.
You have to provide following details.
AWS Access Key ID [None]: ***************
AWS Secret Access Key [None]: ****************
Default region name [None]: ap-southeast-1
Default output format [None]: json
4. Create the Kubernetes Security Group.
aws ec2 create-security-group –group-name kubernetes –description “Kubernetes Security Group”
aws ec2 authorize-security-group-ingress –group-name kubernetes –protocol tcp –port 22 –cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress –group-name kubernetes –protocol tcp –port 80 –cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress –group-name kubernetes –protocol tcp –port 4500 –cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress –group-name kubernetes –source-security-group-name kubernetes
5. Download the master.yaml file from [2]. Better to get the file from a released tag rather than taking from the master branch.
Note: In [2] look for master.yaml file.
6. Download the node.yaml file from [3]. Better to get the file from a released tag rather than taking from the master branch. You will need to change the node.yaml file in a next step.
Note: In [3] look for node.yaml file.
7. Launch the master.
– Get the AMI ID for Core Os from [4]. Alpha channel is preferred. (eg: ami-c6fcc494)
– Run the following command in the terminal. Make sure you are running that command from where you have master.yaml file.
aws ec2 run-instances –image-id ami-c6fcc494 –key-name <Key_Pair> –region ap-southeast-1 –security-groups kubernetes –instance-type m3.medium –user-data file://master.yaml
This will spawn a new instance in EC2 which is the kubernetes master node. You can verify you have added the master.yaml file properly with the spawned instance by following below steps.
7.1 – Select the spawned instance and go to View/Change User Data.
7.2 – Output should be as follows.
8. Update the node.yaml cloud-config file.
For this we need the kubernetes master node private ip. You can find that from the EC2 console as well. If you click on the spawned instance, under the details you can find the private ip. Also you can use the method mentioned in [1].
Now replace all instances of the <master-private-ip> in the node.yaml file with the private IP address of the master node.
9. Launch 3 worker nodes which is called as minions.
aws ec2 run-instances –count 3 –image-id ami-c6fcc494 –key-name <Key_Pair> –region ap-southeast-1 –security-groups kubernetes –instance-type m3.medium –user-data file://node.yaml
Make sure node.yaml file is in the location where you execute the above command. This will spawn another 3 ec2 instances.
10. Configure the Kubectl SSH tunnel
ssh -i key-pair-file -f -nNT -L 8080:127.0.0.1:8080 core@<master-public-ip>
Use kubernetes master public ip to create the ssh tunnel.
11. Set KUBERNETES_MASTER environment variable. You can add this to .bashrc file as well as you can export it.
export KUBERNETES_MASTER=http://127.0.0.1:8080
12. You can use following commands to verify your kubernetes cluster.
12.1. Use kubectl commands
kubectl get nodes – list down minions
kubectl get services – list down services
kubectl api-versions – list down api versions
Read more on [5]
12.2. Use the dashboard – http://localhost:8080/static/app/#/dashboard/
12.3. Run Apache Stratos Kubernetes API Client live test [6] against the Kubernetes cluster on EC2.
TROUBLESHOOT GUIDE
1. You can ssh to Kubernetes nodes as below.
ssh -i <kye_pair_file> core@<public_ip>
Note : In these machines username is core. It is not root.
2. Use command sudo netstat -taupen to see all the services running in the core os instance.
3. Use following commands to read logs in the core os machine.
journalctl -f – This is the main log.
systemctl -f – This will list down services with their status
You can check the log for any particular service as below.
journalctl -b -u service
systemctl status -l service
eg : systemctl status -l kube-apiserver
kube-apiserver is important as that service is used for communication between kubernetes master node and kubectl client.
More service names can be found on [2].
More details on troubleshooting on Core OS can be found in [7].
[1] – https://cwiki.apache.org/confluence/display/STRATOS/4.1.0+Install+Stratos+with+Kubernetes+on+EC2
[4] – https://coreos.com/docs/running-coreos/cloud-providers/ec2/
[5] – https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/kubectl.md
[6] – https://github.com/apache/stratos/tree/master/components/org.apache.stratos.kubernetes.client
Pubci World 