I have a containerized daemon that I need to run one instance of for every thing. Each thing has a unique set of configs associated with it, but the container image is the same. The configs can be set simply as environment variables. I have a list of the configs, and I need to define the desired state as having exactly 1 pod running for each thing. What is the appropriate way to construct this in Kubernetes with or without Helm?
My understanding is that ReplicaSets and Deployments work on identical containers, in other words they would all be spun up with the same environment variables? I understand that StatefulSet may be able to represent this, but the deamons do not need to hold state really, they do not need persistent storage, they can be killed at will, so long as another with the same configs comes up soon afterwards.
One clue I was given by somebody was to use Helmfile or Helm partials. That is the extent of what they told me. I have not yet investigated whether those are appropriate or not.
You are correct saying that Deployment and ReplicaSets are running on identical containers, so the way I see it you have 2 options:
Deploy multiple deployments with different configs defined in the values file:
You can see an example here, where multiple configs are set in the values file and using {{ range }} to iterate and create multiple deployments
Iterate over you configurations names/files using scripting language of your choice and create separate release for each of your configuration via the command line for example: --set configName=
Personally, I would go with the 2nd option since multiple helm releases can harness the helm cli to better understand what is running and it's state. also, any CRUD action you would like to do would be less dangerous since the deployments are decoupled
Related
I have written a number of related OKD object definitions, each in its own YAML file. These together essentially make up an application deployment. I am doing something like the following to install my application on an OKD cluster, which works to my satisfaction when none of the objects already exist [on the cluster]:
oc create -f deploymentconfig.yaml,service.yaml,route.yaml,configmap.yaml,secret.yaml
However, if some of the objects oc create is asked to create, already exist on the cluster, then oc create refuses to re-create them (naturally) but it will have created all the other ones that did not exist.
This isn't ideal when the objects I am creating on the cluster were made to behave "in tandem", and are parts of an application where they depend on one another -- the configuration map, for instance, is pretty much a hard requirement as without it the container will fail to start properly (lacking configuration data through a mounted volume).
I'd like to know, can oc create be made to behave like either all of the objects specified on the command line, are installed, or none if some of them already exist or if there were errors?
I am aware OKD has template faculties and other features that may greatly help with application deployment, so if I am putting too much (misplaced) faith on oc create here, I'll take an alternative solution if oc create by design does not do "transactions". This is just me trying what seems simple from where I currently stand -- not being much of an OKD expert.
Unfortunately, there is no such thing.
In Kubernetes (and so in Openshift), manifests are declarative, but they are declarative by resource.
You can oc apply or oc replace to create or modify some resource in a atomic way, but the same cannot be done with a lot of resources because Kubernetes don't see them as a unity.
Even if you have a Template or a List, some resources may have problems and you will end with a part of the whole.
For this kind of thing helm is much more versatile and works as you want with --atomic flag.
I have deployed on prod 10+ java/js microservices in GKE, all is good, none use external volumes, its a simple process in pipeline of generating new image, pushing to container registry and when upgrading the app to new version, just deploy new deployment with the new image and pods using rolling update are upgraded.
My question is how would it look like with Common Lisp application ? The main benefit of the language is that the code can be changed in runtime. Should the config .lisp files be attached as ConfigMap? (update to ConfigMap still requires recreation of pods for the new ConfigMap changes to be applied) Or maybe as some volume? (but what about there being 10x pods of the same deployment? all read from the same volume? what if there are 50 pods or more (wont there be some problems?)) And should the deploy of new version of the application look like v1 and v2 (new pods) or do we use somehow the benefits of runtime changes (with solutions I mentioned above), and the pods version stays the same, while the new code is added via some external solution
I would probably generate an image with the compiled code, and possibly a post-dump image, then rely on Kubernetes to restart pods in your Deployment or StatefulSet in a sensible way. If necessary (and web-based), use Readiness checks to gate what pods will be receiving requests.
As an aside, the projected contents of a ConfigMap should show up in side the container, unless you have specified the filename(s) of the projected keys from the ConfigMap, so it should be possible to keep the source that way, then have either the code itself check for updates or have another mechanism to signal "time for a reload". But, unless you pair that with compilation, you would probably end up with interpreted code.
I have a container type that I will need to intermittently add and delete from my cluster, but each different container instance will need a unique configuration in the form of environment variables.
What is the best way to structure this with Kubernetes? Should I have a separate workload for each container? Should I have one common workload and update the pod with new containers as needed?
The containers are isolated applications that don't have anything to do with their siblings.
If you want to do this you can use the jobs in kubernetes, if there is no use case of PVC or PV
For each different type create a new job with the different type of environment variables.
I have a container type that I will need to intermittently add and
delete from my cluster, but each different container instance will
need a unique configuration in the form of environment variables.
configure multiple jobs or deployment in which you can give different options of environment variables.
I do it by
Having a common deployment yaml. You should mount configmaps as environment variables
Packaging with Helm
Deploying with different values.yaml for each instance.
As Harsh stated, you can also use Jobs pattern but I prefer having single file & many values files instead of different jobs definitions.. Because if a key changes, you will need to update all job definitions..
I have a requirement where my client applications are having almost same properties and even the URL is same, as they are running behind a load balancer, the only change they have is a particular set of environment properties that differ.
Is it possible to register them uniquely based on that property.
I would say there are a few approaches.
One would be loading Environment Variables from a Kubernetes Secret.
Second using helm(https://helm.sh/)
Helm helps you manage Kubernetes applications — Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.
Charts are easy to create, version, share, and publish — so start using Helm and stop the copy-and-paste.
Explanation:
If you would use a secret option, you would probably create two separate secrets with env variables that you need and load those based on the app name, or if you have them setup in different namespaces then copy the secret over to each as those resources will not work between different namespaces.
If you would use helm, you will have to write your chart and put the env variables into values.yaml or mix it together and load secret from inside Kubernetes.
This will work on Kubernetes, I do not know (based on your tags) if it's the same on OpenShift.
Please provide some samples of what you have already done and I'll provide more details.
Okay you can easy install application but where is the benefit compared to normal .yaml files from Kubernetes?
Can someone give me a example where it is useful to use helm and why normal Kubernetes is not sufficient?
Also a confrontation for helm and Kubernetes would be nice.
With Helm, a set of resources (read as Kubernetes manifests) logically define a release - and you need to treat this group of resources as a single unit.
A simple example on why this is necessary: Imagine an application bundle that has, let's say, 10 kubernetes objects in total. On the next release, due to the changes in the app, now 1 of the resources is not needed anymore - there are 9 objects in total. How would I roll out this new release? If I simply do kubectl apply -f new_release/, that wouldn't take care of the deletion of that 1 resource that is not needed anymore. This means, I cannot roll upgrades that doesn't need manual intervention. Helm takes care of this.
Helm also keeps a history of releases with their exact set of resources, so you can rollback to a previous release with a single command, in case things go wrong.
Also, one of the things you need often is templating your resources - imagine you want to deploy multiple instances of the same exact application. What would you do?
Kubernetes doesn't offer many options to tackle this problem - one solution is to use different namespaces: Don't specify namespace in the manifests, but give it in the command, such as kubectl apply -n my_namespace -f resources/, but what if you want to deploy two of this instances on the same namespace? Then you need some kind of name/label/selector templating, and Helm takes care of that.
These are some examples for the use cases that Helm addresses.