Imagin one product chart contains two sub charts, app and config, app depends on config, for example, a deployment depend on configmap mounting volume. The real senario could be multiple app charts sharing same config, but only one here for simplicity.
Goal
product chart design need support coexistance of multiple releases in same namespace, isolated with each other.
Need app ci success alone, here success only means the app chart can be successfully deployed to k8s, no need function tests.
Problem
To accomplish goal 1, I think the natual way is template prefixing config name with {{ .Release.Name }}.
But when in ci, the dependency {{ .Release.Name }}-config is missing, so apparently ci would fail. So violates goal 2.
Possible solutions
Deploy config before(or during) app ci, but this causes extra effort and resource usage to ci.
Add if else logic in app chart to determine whether currently is in ci, if true, not render those config parts. but introduce a chart logic only for ci seems foolish.
So is this common problem in k8s helm charts development, and what's the best practice? Is there any existing k8s mechanism to easily mock a object during ci deployment?
Related
I am trying to set up a helmfile deployment for my local kubernetes cluster which is running using 'kind' (a lightweight alternative to minikube). I have charts set up for my app which are all deploying correctly, however I require an nginx-ingress controller. Luckily 'kind' provides one, which I am currently applying with the command:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
It seems perverse that I should have everything else set up to deploy at the touch of a button, but still have to 'remember' (and also train my colleagues to remember...) to run this additional command.
I realise I could copy and paste and create my own version, but I would like to keep up to date with any changes made at source. Is it possible to create a chart that makes a reference to an external template?
I am looking at solutions using either helm or helmfile.
Your linked YAML file seems to have been generated from the ingress-nginx chart.
Subchart
You can include ingress-nginx as a subchart in Helm 3 by adding it as a dependency to your own chart. In Helm 3, this is done with the dependencies field in Chart.yaml, e.g.:
apiVersion: v2
name: my-chart
version: 0.1.0
dependencies:
- name: ingress-nginx
version: ~4.0.6
repository: https://kubernetes.github.io/ingress-nginx
condition: ingress-nginx.enabled
This may be problematic, however, if you need to install multiple versions of your own chart in the same cluster. To handle this, you'd need to consider the implications of multiple Ingress controllers.
Chart
Ingress controllers are capable of handling ingresses from various releases across multiple namespaces. Therefore, I would recommend maintaining ingress-nginx separately from your own releases that depend on it. This would mean installing ingress-nginx like you already are or as a separate chart (guide).
If you go this route, there are tools that help make it easier for devs to take a hands-off approach for setting up their K8s environments. Some popular ones include Skaffold, DevSpace, Tilt, and Helmfile.
I have several microservices that have practically the same settings in YAML, some values change (e.g. image, version, a specific environment variable ...), and they are in different repositories, with a different pipeline each. How do I use the same template between them without getting repeated code?
This is how we do it in the place I currently work.
We have our own generic Helm chart that is version controlled and hosted in our Artifactory, every parameter in that chart that may need changing is exposed in values.yaml.
The Artifactory gets added to helm as repository, then you only need separate a values.yaml for each microservice you want deployed as the chart gets sourced centrally.
helm install -f values.yaml microservice01 artifcatory/global-helm-chart
On top of that we use helmfile, but this is not necessary in order to achieve your goal.
The key points are:
make the chart generic
host it centrally
add the repository to helm.
You can also update the values.yaml from pipeline and then package the chart and deploy it..In that way you can still have the same yaml file but the values will differ from which pipeline they are deployed.
Alternatively and easy way will be to maintain different values.yaml for different environment in the helm chart itself and pass them during helm install/upgrade from pipeline.
We do it for about 90 microservices. We have common Chart and we run values file through kindof sed script which changes what we need. Then whole package gets deployed
Currently, I have 2 Helm Charts - Chart A, and Chart B. Both Chart A and Chart B have the same dependency on a Redis instance, as defined in the Chart.yaml file:
dependencies:
- name: redis
version: 1.1.21
repository: https://kubernetes-charts.storage.googleapis.com/
I have also overwritten Redis's name since applying the 2 Charts consecutively results in 2 Redis instances, as such:
redis:
fullnameOverride: "redis"
When I try to install Chart A and then Chart B I get the following error:
Error: rendered manifests contain a resource that already exists. Unable to continue with install: existing resource conflict: kind: PersistentVolumeClaim, namespace: default, name: redis
I was left with the impression that 2 charts with identical dependencies would use the same instance if it's already present?
When you install a chart using Helm, it generally expects each release to have its own self-contained set of Kubernetes objects. In the basic example you show, I'd expect to see Kubernetes Service objects named something like
release-a-application-a
release-a-redis
release-b-application-b
release-b-redis
There is a general convention that objects are named starting with {{ .Release.Name }}, so the two Redises are separate.
This is actually an expected setup. A typical rule of building microservices is that each service contains its own isolated storage, and that services never share storage with each other. This Helm pattern supports that, and there's not really a disadvantage to having this setup.
If you really want the two charts to share a single Redis installation, you can write an "umbrella" chart that doesn't do anything on its own but depends on the two application charts. The chart would have a Chart.yaml file and (in Helm 2) a requirements.yaml file that references the two other charts, but not a templates directory of its own. That would cause Helm to conclude that a single Redis could support both applications, and you'd wind up with something like
umbrella-application-a
umbrella-application-b
umbrella-redis
(In my experience you usually don't want this – you do want a separate Redis per application – and so trying to manage multiple installations using an umbrella chart doesn't work especially well.)
Unfortunately, Helm can't handle multiple resources with the same name or in other word there isn't any share resource capability.
You can follow This issue
I think you can use kustomize template to use share resources. There is a really good kustomize vs helm article.
Currently I am working with a project based on a micro service architecture. For making this project, I have 20 Spring Boot micro service projects are there. I for for every root folder I placed my Dockerfile for image building. And I am using Kubernetes cluster for deployment through Helm chart.
My confusion here that, when I created Helm chart, it giving the service.yaml and deployment.yaml inside template directory.
If I am deploying these 20 microservices, do I need to create 20 separate helm chart ? Or Can I create service for every 20 within 1 chart?
I am new to Kubernetes and Helm chart. So I am confused about the standard way of using yaml files with chart. Do I need to create 20 separate chart or can I include in 1 chart?
How can I follow the standard way of chart creation for my micro service projects please?
What I ended up doing (working with a similar stack), is create one microservice Chart, which is stored in an internal Chart repository. Inside of the Helm Chart, I gave enough configuration options, so teams have the flexibility to control their own deployments, but I made sure to set sensible defaults (e.g. make sure the Deployment utilises a RollingUpdateStrategy and readiness probes are configured with sensible defaults).
These configuration options can be passed by the values.yaml file. Teams deploy their microservice via a CICD pipeline, passing the values.yaml file to the helm command (with the -f flag).
I would certainly recommend you read the Helm Template Developer guide, before making the decision. It really depends on how similar your microservices are, but I recommend going for 1 Helm Chart if you have a homogenous environment (which also was the case for me).
i would want to know how do you manage your service with Helm ?
I already know that we are going to have more than 200 microservices. How to manage them easily ?
Each microservice with is own yaml files (deployment,service,ingress ,values etc..)
or one several large (deployment,ingress, etc.. )yaml files for all microservices and i push the values yaml file with the specific params for the application.
I'd suggest aiming for an umbrella chart that includes lots of subcharts for the individual services. You can deploy each chart individually but using a single umbrella makes it easier to deploy the whole setup consistently to different environments.
Perhaps some microservices will be similar enough that for them you could use the same chart with different parameters (maybe including docker image parameter) but you'll have to work through them to see whether you can do that. You can include the same chart as a dependency multiple times within an umbrella chart to represent different services.
Ideally you also want a chart for a service to be individually-deployable so you can deploy and check that service in isolation. To do this you would give each chart its own resources including its own Ingress. But you might decide that for the umbrella chart you prefer to disable the Ingresses in the subcharts and put in a single fan-out Ingress for everything - that comes down to what works best for you.