I have 2 namespaces in my kubernetes cluster: development and production. I'm currently adding a third namespace: staging.
I'm NOT using terraform with which this task would have been supposedly simpler.
I'm looking for solutions within the GCP ecosystem to provision a workload in the staging namespace with all the environment variables and configurations of the development namespace.
Please check my answer to similar question here. Unfortunately there is no ready solution for that within GCP, especially if you want to migrate workload from the existing cluster between different namespaces. However you can use for that purpose Heptio Velero. It's nicely described in this article.
Related
I have recently been reading more about infrastructure as a service (IaaS) and platform as a service (PaaS) and had some questions. I see when we opt for a PaaS solution, it is generally very easy to create the infrastructure as the cloud providers handle that for us and we can even automate the deployment using an infrastructure as code solution like Terraform.
But if we use an IaaS solution or even a local on premise cluster, we lose a lot of the automation it seems that PaaS allows. So I was curious, are there any good tools out there for automating infrastructure deployment on a local cluster that is not in the cloud?
The best thing I could think of was to run a local Kubernetes cluster and then Dockerize each of the infrastructure components, but this seems difficult as each node in the cluster will need its own specific configuration files.
From my basic Googling, it seems like there is not a good solution to this.
Edit:
I was not clear enough with my original intentions. I have two problems I am trying to solve.
How do I automate infrastructure deployment locally? For example, suppose I wanted to create a Hadoop HDFS cluster. I would need to configure one node to be the namenode with an accessible IP, and the other nodes to be datanodes that are aware of the namenode's IP. At the moment, I have to do this manually by logging into each node, checking it's IP, and then configuring each one. How would I automate this? If I were to use a Kubernetes approach, how do I specify that one of the running pods needs to be the namenode and the others are datanodes? How do I find the pods' IPs and have them be aware of the namenode IP?
The next problem I have is very similar to the first, but a slight modification. How would I deploy specific configuration files to each node. For instance in Kafka, the configuration file for one node, requires the IPs of the Zookeeper nodes, as well as the IP it should listen on. This may be different for every node in the cluster. Is there a good way to make these config files pod specific, so that I do not have to do bash text processing to insert the correct contents into each pod's config files?
You can use Terraform for all of your on-premise Infra. Automation, and Ansible for configuration management.
Let's say you have three HPE servers, Install K8s or VMware on them using Ansible, then you can treat them as three Avvaliabilty zones in one region, same as AWS. from this you can start deploying dockerize apps, or helm charts using Terraform.
Summary:
Ansbile for installing and configuration K8s.
Terraform for provisioning K8s.
Helm for installing apps on K8s.
After this you gonna have a base automated on-premise Infra.
I'm trying to figure out ways to automate k8s deployments in an EKS cluster. I'm trying to set up namespaces for each specific environment. One for dev, one for staging, and one for production. My production namespace is in a separate region and also in a separate cluster (dev & staging are in one cluster). I'm a little new to this concept, but does it make sense to have each respective application load balancer in it's respective namespace? Is that practice common or best practice? Any ideas on automating deployments would be appreciated.
Hi Dave Michaels,
I assume there are two questions in your post above:
If we use a dedicated namespace in the same cluster (dev & staging setup), can we use a dedicated load balancer for each of these namespaces? Is this good practice.
Answer: Yes. As you are using the namespace concept for each environment in the same cluster, it is Ok to create a dedicated load balancer (promise me you will use ingress :)) in each of these namespaces as we need an easier way to access those environments. To be frank, I am not a fan of using namespaces for environments, because as your cluster grows and lots of microservices getting added to it, you might want to use namespace for another reason eg., namespace per team or domain to have granular access rights. But I have seen teams using it for different environments successfully as well.
Suggest automated Kubernetes deployments possibilities?
This is a large topic by itself.
As your microservices grow, you will have multiple Kubernetes manifests to handle, first thing I will suggest is to either use a configuration manager like Kustomize or a package manager like Helm to segregate variables from actual manifests, this will help to easily automate deployment across environments (same cluster or different clusters). Coming to actual deployment automation, if there is no existing CD in place I would suggest exploring tools that support natively Kubernetes that supports GitOps, like FluxCD or ArgoCD etc
I cannot find any articles answering question: Is it safe/right to deploy Spinnaker to same Kubernetes cluster which Spinnaker will manage? Mainly I mean for production, HA deployments.
I think the architectures of Spinnaker and Kubernetes compliment each other very well, and running Spinnaker in the same K8s cluster it is managing is definitely safe.
As per your comment in #mdirkse's answer, there is a codelab, which is official Spinnaker documentation, that explains how to create a set of basic pipelines for deploying code from a Github repo to a production Kubernetes cluster in the form of a Docker container.
In this documentation, it specifically states the following:
We will be deploying Spinnaker to the same Kubernetes cluster it will be managing. ...
Not sure if this is exactly what you are looking for though.
I'm not sure about "right", but I'd definitely say that it is safe to run Spinnaker on the same Kubernetes cluster that it manages, if you set it up right. Kubernetes (and Docker) gives you all the tools you need to properly separate Spinnaker from the other things running on the cluster (namespaces, quotas, node affinities etc). Indeed the whole point of Kubernetes is to be able to easily run software in an HA/fault tolerant way, and since Spinnaker consists of a collection of stateless microservices it really plays to the strenghts of k8s.
I am trying to install kubernetes on Self-hosted production environment running on Ubuntu 16.04. I am not able to find any helpful guide to setup production grade kubernetes master and connect worked nodes to it.
any help is much appreciated.
you can use the kubespray to self Host production environment.
https://github.com/kubernetes-incubator/kubespray
Depends on what you understand by saying "self-host". The most people think it's about deploying kubernetes in the own environment.
If you want to compare different approaches to deploy k8s in a custom environment, refer to this article which covers a bunch of options suitable for that.
If you are interested in how to set up an HA Kubernetes cluster using kubeadm, refer to this article.
However, in kubernetes, there is a different definition of "self-hosted". It means running kubernetes itself as a workload in kubernetes. If you are interested in a real self-hosted approach (on a custom environment), refer to this article
Hope this helps
You can use typhoon which can be used to provision an HA kubernetes cluster.
Here is a sample configuration which I used to bring up my own home cluster.
A few advantages of typhoon are that you have the option of choosing your choice of a cloud provider for provisioning your infrastructure, which is done using terraform and the fact that it gives you upstream k8s is a big plus too.
Internally, it uses bootkube to bring up the temporary control plane, which would consist of
api-server
controller-manager
scheduler
and then when we have the temporary control plane object, we inject the objects to the API server to have our k8s cluster.
Have a look at this kubecon talk given by CoreOS which explains how this is working.
I'm trying to dig into Rancher, and was wondering if having Rancher plugged in with Kubernetes has any additional benefits over Cattle which is Rancher's in home orchestration framework. So far, I haven't been able to figure out why someone would opt for Rancher with Kubernetes. Does it only help ease out the initial setup of Kubernetes? How do these options differ from a stand alone setup of Kubernetes ?
There is now a very good answer to this. Rancher just moved 100% into Kubernetes by announcing Rancher 2.0: http://rancher.com/announcing-rancher-2-0/. It does not use Cattle anymore.
After 1+ year of using Rancher + Cattle and Kubernetes in production simultaneously, I can share some personal observations, but first let me share some thoughts on Kubernetes vs Rancher + Cattle:
Kubernetes is astonishingly flexible and designed to be extendable. There are two pillars on which Kubernetes cluster stands - kube-apiserver as a main unit and etcd as a storage. Other basic components are vital for a cluster, but they were designed to use kube-apiserver's API as any Kubernetes extension should. You can design your cluster the way you need, use different network providers, authentication/authorization services, volume providers and substitute almost every component in cluster - and if configured correctly your extended Kubernetes cluster will be more stable than basic Cattle cluster in almost any case, not speaking of Cattle extensions.
Kubernetes has a lot of useful out-of-the-box features for persisting your data, keeping your data secret and control access to it. This is important part, rancher-nfs is quite fragile, moreover, NFS-storage is the only more or less stable option you have in Cattle to keep volumes, not mentioning local storage.
Kubernetes is an overkill for non-frequent deployments and development in a small team. No doubts, Cattle has been created as a simple clone of Kubernetes and has inherited its basic concepts of namespaces, pods, scheduling, etc., yet making it very easy to deploy and maintain a cluster, making it useful for developers with no understanding of containerisation and networking. Due to its simplicity it became pretty unusable for heavy and dynamically changing production projects.
Kubernetes community is growing fast, being supported by cloud providers and by uncountable number of open-source project and startups, so it's wiser to learn Kubernetes than Cattle, which is one foot in grave now.
And finally - Cattle is unstable, very unstable. Especially if you plan to use it with bare metal. Believe me, you don't want to use Cattle in production.
And speaking about Rancher + Kubernetes tied together - it's a very nice combination, Rancher web UI makes Kubernetes concepts simpler in a good way, cluster maintenance and provisioning become much easier, but there's always some "buts": Rancher 2.0 is very unstable, too, bugs make it unusable for production now. Can't say anything about Rancher 1.x + Kubernetes.
In our experience,
Cattle is just a toy. It's just suit for development or some small environment. If you used Cattle in PROD, you will know what I say.
But Rancher is far more concise and simple than k8s.
So I'm looking forward to the Rancher 2.0,
Rancher 2.0 is a complete container management platform built on Kubernetes
rancher v2 is more complicated as ks8 is complicated but its contain great features to manage containers ,for pros and cons between cattle rancher v1 and ks8 rancher v2 here some of most important differences :
1- rancher v1 cattle is more like docker-compose than modern orchestration tools and lack of support new storage drivers or real high available loadbalancer approach like ingress in ks8.
2- rancher v1 have great community catalogs that make create of high available services lots more easy but its kind of tricky to resolve problems that can happen.
3 - rancher v2 support community catalogs and helm but need more contribution to become stable and great as v1 was but k8s instead is completely stable and using kubectl to manage pods and resources is easy.
my opinion is use rancher v2 if you want to start docker orchestration , ks8 have great cli tools but when you need upgrade container or change configuration rancher web gui can help effectively.