I am already running a single master kubernetes cluster now and I am doing research about setting up Highly available Kubernetes clusters. I was thinking of Multi master cluster setup then realized self-hosted cluster might be a better option to go future ready.
Additional challenge is I am doing it in Bare Metal (Meaning, I am going to use cloud vms from these cloud provider, Hetzner, Linode, DigitialOcean and they have CSI driver, cloud controller manager etc., )
In this case, I see 2 options.
Setup with bootkube (https://github.com/kubernetes-sigs/bootkube)
Setup with kubeadm self-hosting. (https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/self-hosting/)
I assume this is still an early topic hence I am not able to find guidance to choose the right approach and then correct documentation. I need this for a scalable production environment where I will start small with at least 8 nodes and can grow faster.
Is bootkube considerable for future readiness?
or kubeadm self-hosting is still in alpha stage, am I getting into a risk running a production environment?
Any good, documentation, blog, article to go in this direction?
I use Keepalived + Haproxy and Ansible to deploy HA kubernetes cluster. Now kubeadm supports join control plane command, so it easy to integrate with ansible.
You can also refer: https://github.com/kubernetes-sigs/kubespray.
Related
I am now running two kubernetes clusters.
First Cluster is running on bare metal, and Second Cluster is running on EKS.
but since maintaining EKS costs a lot, so I am finding ways to change this service as Single Cluster that autoscales on AWS.
I did tried to consider several solutions such as RHACM, Rancher and Anthos.
But those solutions are for controlling multi cluster.
I just want to change this cluster as "onpremise based cluster that autoscales (on AWS) when lack of resources"
I could find "EKS anywhere" solution but since price is too high, I want to build similar architecture.
need advice for any use cases for ingress controller, or (physical) loadbalancer, or other architecture that could satisfies those conditions
Cluster API is probably what you need. It is a concept of creating Clusters with Machine objects. These Machine objects are then provisioned using a Provider. This provider can be Bare Metal Operator provider for your bare metal nodes and Cluster API Provider AWS for your AWS nodes. All resting in a single cluster (see the docs below for many other provider types).
You will run a local Kubernetes cluster which will have the Cluster API running in it. This will include components that will allow you to be able to create different Machine objects and tell Kubernetes also how to provision those machines.
Here is some more reading:
Cluster API Book: Excellent reading on the topic.
Documentation for CAPI Provider - AWS.
Documentation for the Bare Metal Operator I worked on this project for a couple of years and the community is pretty amazing. This GitHub repository hosts the CAPI Provider for bare metal nodes.
This should definitely get you going. You can start by running different providers individually to get a taste of how they work and then work with Cluster API and see it in function.
I'm developing a Kubernetes scheduler and I want to test its performance when nodes join and leave a cluster, as well as how it handles node failures.
What is the best way to test this locally on Windows 10?
Thanks in advance!
Unfortunately, you can't add nodes to Docker Desktop with Kubernetes enabled. Docker Desktop is single-node only.
I can think of two possible solutions, off the top of my head:
You could use any of the cloud providers. Major (AWS, GCP, Azure) ones have some kind of free tier (under certain usage, or timed). Adding nodes in those environments is trivial.
Create local VM for each node. This is less than perfect solution - very resource intesive. To make adding nodes easier, you could use kubeadm to provision your cluster.
I have tried with
minikube tool, It's a single node.
kubeadm tool, It's a multinode but single master.
I am looking for the tool which can be configure multi master kubernetes cluster in
local.
There's no tool to install a multi-master Kubernetes cluster locally as of this writing. Generally, a multi-master setup is meant for production environments and a local setup is generally far from what someone would describe as a production environment.
You can probably piece together a local installation from this and Kubernetes the Hard Way.
Kubeadm can be used to create a multi-master highly available setup. Documentation regarding this can be found # https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/.
If you only have access to one physical machine, but want to create a multi master setup you can use manually provision several VMs and create the cluster, or you can automate everything by using tools such as Vagrant and Ansible Playbooks. Tutorials regarding this is available # https://github.com/justmeandopensource/kubernetes/tree/master/kubeadm-ha-multi-master. You can also have a look at justmeandopensource channel on youtube (https://www.youtube.com/user/wenkatn) for detailed tutorials (I used them and was of great help).
if have a limited amount of the physical machine and you want to run the setup of multiple masters you can use the LXD container to first create the VMs and use those VM containers to setup the K8s clusters.
Some of resource link : https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/
with kubeadm : https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/
also as mentioned by #rico kubernetes the hard way is the ultimate thing to use : https://github.com/kelseyhightower/kubernetes-the-hard-way
here one nice tutorial link of youtube using kubeadm: https://www.youtube.com/watch?v=q92MYG-EW-w
you can also follow this github opensource repo guide : https://github.com/hub-kubernetes/kubernetes-multi-master
I am trying to install kubernetes on Self-hosted production environment running on Ubuntu 16.04. I am not able to find any helpful guide to setup production grade kubernetes master and connect worked nodes to it.
any help is much appreciated.
you can use the kubespray to self Host production environment.
https://github.com/kubernetes-incubator/kubespray
Depends on what you understand by saying "self-host". The most people think it's about deploying kubernetes in the own environment.
If you want to compare different approaches to deploy k8s in a custom environment, refer to this article which covers a bunch of options suitable for that.
If you are interested in how to set up an HA Kubernetes cluster using kubeadm, refer to this article.
However, in kubernetes, there is a different definition of "self-hosted". It means running kubernetes itself as a workload in kubernetes. If you are interested in a real self-hosted approach (on a custom environment), refer to this article
Hope this helps
You can use typhoon which can be used to provision an HA kubernetes cluster.
Here is a sample configuration which I used to bring up my own home cluster.
A few advantages of typhoon are that you have the option of choosing your choice of a cloud provider for provisioning your infrastructure, which is done using terraform and the fact that it gives you upstream k8s is a big plus too.
Internally, it uses bootkube to bring up the temporary control plane, which would consist of
api-server
controller-manager
scheduler
and then when we have the temporary control plane object, we inject the objects to the API server to have our k8s cluster.
Have a look at this kubecon talk given by CoreOS which explains how this is working.
Kubernetes 1.2:
How do you bootstrap a second master for an HA configuration?
Can you use kube-up?
The HA doc doesn't really get into that:
https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/admin/high-availability.md
Thanks
There aren't any automated scripts (like kube-up.sh) checked into the Kubernetes github repository that will create an HA cluster; you will need to understand the intricacies of building a cluster (many of which are described in the Creating a Custom Cluster from Scratch guide) and build an HA cluster from scratch or modify a "normal" cluster to make it into an HA configuration.
If you are interested in helping contribute to developing better tools for HA masters, you can join the Kubernetes High Availability special interest group.
kops (https://github.com/kubernetes/kops) is able to provision HA Kubernetes with multiple masters: https://github.com/kubernetes/kops/blob/master/docs/commands.md#other-interesting-modes
AFAIK some more upcoming work is being done in k8s 1.5 and 1.6
Some other platforms based on kubernetes supports multiple masters.
For example, Openshift origin support setup cluster with multiple masters and on load balance.
However, Openshift built some customized solutions into kubernetes. Currently k8s couldn't be decoupled from Openshift gracefully.
Personally speaking, I also prefer multiple masters configuration for HA.