We have multiple(20+) services running inside docker containers which are being managed using Kubernetes. These services include databases, streaming pipelines and custom applications. We want to make this product available as an on-premises solution so that it can be easily installed, like a one-click installation sort of thing, hiding all the complexity of the infrastructure.
What would be the best way of doing this? Currently we have scripts managing this but as we move into production there will be frequent upgrades and it will become more and more complex to manage all the dependencies.
I am currently looking into helm and am wondering if I am exploring in the right direction. Any guidance will be really helpful to me. Thanks.
Helm seems like the way to go, but what you need to think about in my opinion is more about how will you deliver updates to your software. For example, will you provide a single 'version' of your whole stack, that translates into particular composition of infra setup and microservice versions, or will you allow your customers to upgrade single microservices as they are released. You can have one huge helm chart for everything, or you can use, like I do in most cases, an "umbrella" chart. It contains subcharts for all microservices etc.
My usual setup contains a subchart for every service, then services names are correctly namespaced, so they can be referenced within as .Release.Name-subchart[-optional]. Also, when I need to upgrade, I just upgraed whole chart with something like --reuse-values --set subchart.image.tag=v1.x.x which gives granular control over each service version. I also gate each subcharts resources with if .Values.enabled so I can individualy enabe/diable each subcharts resources.
The ugly side of this, is that if you do want to release single service upgrade, you still need to run the whole umbrella chart, leaving more surface for some kind of error, but on the other hand it gives this capability to deploy whole solution in one command (the default tags are :latest so clean install will always install latest versions published, and then get updated with tagged releases)
Related
There are web applications - WebAppA and WebAppB. Each web application depend on a Postgres database. We want to ship these applications to a customer who will deploy the applications on its own k8s cluster.
We want to create three packages - "WebappA", "WebAppB" and "Datastore". The webapp itself made of multiple services, not mentioning for the sake of simplicity.
We want to provide apt-get/brew/yum kind of experience, so that customer can deploy one or both the applications like al-carte. Most importantly while deploying, it should identify if the common package "DataStore" is running and not spin off yet another Postgres instance.
Is there any to package applications as packages for Kubernetes which can make installation easy with dependency handling?
Of course! One way to start would be using Helm charts. You can read more about them here.
Helm defines dependency relationships declaratively using charts, allows you to manipulate/maintain dependencies simply by managing some YAML manifests. It also allows you to have personalised repositories where you can put your images to be fetched from. It's really nice.
I am interested in generating docker-compose.yaml files from Helm charts. Is there a good way or tool to do this?
I realize that this is in the opposite direction from what most people are doing. Why I want to do this:
Our production systems run Kubernetes via Helm charts. We've got a full blown k8s and Helm setup already; no need to use a tool like Kompose to get us there. The question is how to convert Helm to docker-compose, not the other way around.
We want our Helm charts to be the single authoritative source of container configuration. They are able to describe a superset of what docker-compose can.
Running a set of services using Helm on a development machine is more time and resource consuming than running the same set of services via docker-compose. We do not want to slow development down by having engineers run using Helm/k8s.
We do not want to maintain two sets of configurations.
Can anybody recommend how to do this, or suggest a different solution to the time/resources issue encountered on development machines?
I have a big and fairly complex system for install into the k8s cluster.
60 microservices and 10 helm charts installed to 5 namespaces.
Currently, we run 5 helm install/upgrade commands with a pause of 30 seconds between commands. However, this strategy incurs a serious load on nodes due to the fact that we pull docker images and start applications. We have a long and not clear execution time(timeline) that often results in timeouts of components such as consul, Elasticsearch, and applications that depend on the aforementioned components.
I would like to hear opinions about ways to turn this situation around. First, here is our approach so far:
Write the script that controls installation by helm charts.
Write an ansible-playbook that runs Helm charts and controls the installation status of components.
Write an ansible-playbook install components (either using Jinja2 templates or Golang templates)
Write the k8s operator that installs components and controls the system status.
To answer my own question, I created an installation that can be used as a quick solution to fairly complex installations.
The solution relies on Ansible as an installation orchestrator and Helm as a package manager.
You can browse my github repo contains the code.
There's a lot of ways of doing this. But you can use the kubernetes api directly. You can create any tech server such as Spring Boot, NodeJS, etc that controls the creation of the Kubernetes objects that you want.
This way, basically, you'll be doing a customized Helm API, but the main difference is that you'll customize in your way with your needs.
I’m currently using kubernetes and I came across of helm.
Let’s say I don’t like the idea of “infecting” my kubernetes cluster with a process that is not related to my applications but I would gladly accept it if it could be beneficial.
So I made some researches but I still can’t find anything I can’t easily do by using my yaml descriptor and kubectl so for now I can’t find an use except,maybe, for the environizing.
For example (taking it from guides I read:
you can easily install application, eg. helm install nginx —> I add an nginx image to my deployment descriptor, done
repositories -> I have docker ones (where I pull my images from)
you can easily helm rollback in case of release failure-> I just change the image version to the previous one in my kubernetes descriptor, easy
What bothers me is that, at level of commands, I do pretty much the same effort (helm update->kubectl apply).
In exchange for that I have a lot of boilerplate because of keeping the directory structure helm wants and I feel like missing the control I have with plain deployment descriptors ...what am I missing?
It is totally understandable your question. For small and simple deploys the benefits is not actually that great. But when the deploy of something is very complex Helm helps a lot.
Think that you have a couple squads that develop microservice for some company. If you can make a Chart that works for most of them, the deploy of each microservices would differ only by the image and the resources required. This way you get an standardized deployment and easier to all developers.
Another use case is deploying applications which requires a lot of moving parts. For example, if you want to deploy a Grafana server on Kubernetes you're probably going to need at least a Deployment and a Configmap, then you would need a service that matches this deployment. And if you want to expose it to the internet you need an ingress too.
One relatively simple application, would require 4 different YAMLs that you would to manually configure and make sure everything is correct instead you could do a simple helm install and reuse the configuration that someone has already made, sometimes even the company who created the Application.
There are a lot of other use cases, but these two are the ones that I would say are the most common.
Here's three suggestions of ways Helm can be useful:
Your continuous deployment system somewhat routinely produces new builds and wants to send them to the Kubernetes cluster. You can use templating to specify the image name and tag in a deployment, and so helm upgrade ... --set tag=201907211931 to request a specific tag.
You might have various service-specific controls like the log level or external database hostnames. The Helm values mechanism gives a uniform way to specify them, without having to know the details of the Kubernetes YAML files.
There is a repository of pre-packaged application charts, so if you want replicated PostgreSQL with in-cluster persistent storage, that's already built for you and you can just depend on it, rather than figuring out the right combination of StatefulSets and PersistentVolumeClaims yourself.
You can combine these in interesting (and potentially complex) ways: use an in-cluster database for developer testing but use a cloud-hosted and backed-up database for production, for example, and compute the database host name based on what combination of settings are provided.
There are, of course, alternative ways to do all of these things. Kustomize in particular can change the image value fairly straightforwardly, and is notable for having been included in the kubectl tool since Kubernetes 1.14 (see also Declarative Management of Kubernetes Objects Using Kustomize in the Kubernetes documentation). The "operator" pattern gives an alternate path to install software in your cluster, but even more so than Helm you're trusting an arbitrary program with API access.
If the installation of OpenEBS can be completed with a single command, why would a developer use helm install ? (It is probably more a helm benefits question). I'd like to understand the additional benefits OpenEBS charts can present to a helm user, if any.
I guess you're looking at the two current supported options for OpenEBS installation and noting that the helm install section is much larger with more steps than the operator-based install option. If so, note that the helm section has two sub-sections - you only need one or the other and the one that uses the stable helm charts repo is just a single command. But one might still wonder why install helm in the first place.
One of the main advantages of helm is the availability of standard, reusable charts for a wide range of applications. This is including but not limited to the official charts repo. Relative to pure kubernetes descriptors, helm charts are easier to pass parameters into since they work as templates from which kubernetes descriptor files are generated.
Often the level of parameterisation that you get from templating is needed to ensure that an app can be installed to lots of different clusters and provide the full range of installation options that the app needs. Things like turning on or off certain permissions or pointing at storage. Different apps need different levels of configurability.
If you look at the OpenEBS non-helm deployment descriptor at https://openebs.github.io/charts/openebs-operator-0.7.0.yaml, you'll see it defines a list of resources. The same resources defined in https://github.com/helm/charts/tree/master/stable/openebs/templates Within the non-helm version the number of replicas for maya-apiserver is set at 1. To change this, you'd need to download the file and edit it or change it in your running kubernetes. With the helm version it's one of a range of parameters that you can set at install time (https://github.com/helm/charts/blob/master/stable/openebs/values.yaml#L19) as options on the helm install command