I have my application deployed on Kubernetes cluster, managed by operator (A). I am mounting secrets with ssl key materials to the deployment, so application could access to the content.
I have separate operator (B) deployment, which is responsible to create those secrets with the ssl key materials. Now I have a use case where my secrets are recreated by operator (B), and it deletes/restarts the pods, which managed by operator (A).
I am trying understand - is it common practice to allow separately deployed operator delete pods?
My perception was that operator should work only with resources it manages, nothing more.
Community wiki to summarise the topic.
If it is as you say:
both operators are proprietary,
it is impossible to give a definite yes or no answer. Everything will depend on what is really going on there, and we are not able to check and evaluate it.
Look at the well provided comments by David Maze:
That sort of seems like a bug...but also, Pods are pretty disposable and the usual expectation is that a ReplicaSet or another controller will recreate them...?
Note that the essence of the Kubernetes controller model is that the controller looks at the current state of the Kubernetes configuration store (not changes or events, just which objects exist and which don't) and tries to make the things it manages match that, so if the controller believes it should manage some external resource and there's not a matching Kubernetes object, it could delete it.
Related
I have created a custom resource definition (CRD) and a custom resource (CR) in my Kubernetes cluster, but in my CR controller, how do I create Kubernetes events which are attached to my custom resource? I hope with events, users will be able to see important messages associated with the CR when they run kubectl describe <cr>.
You're looking for an operator. There are already a few pre-written ones out there, check OperatorHub. They are essentially controllers for custom resources, watching for events and acting on them.
You can also write your own, e.g. with the Operator SDK. When you write your own, you can update the status of the CR, depending on what your operator should do, which then shows up when calling kubectl describe.
A good start might be this post on Operators, and this here for listening to events from your custom resources.
I have an app I'm building on Kubernetes which needs to dynamically add and remove worker pods (which can't be known at initial deployment time). These pods are not interchangeable (so increasing the replica count wouldn't make sense). My question is: what is the right way to do this?
One possible solution would be to call the Kubernetes API to dynamically start and stop these worker pods as needed. However, I've heard that this might be a bad way to go since, if those dynamically-created pods are not in a replica set or deployment, then if they die, nothing is around to restart them (I have not yet verified for certain if this is true or not).
Alternatively, I could use the Kubernetes API to dynamically spin up a higher-level abstraction (like a replica set or deployment). Is this a better solution? Or is there some other more preferable alternative?
If I understand you correctly you need ConfigMaps.
From the official documentation:
The ConfigMap API resource stores configuration data as key-value
pairs. The data can be consumed in pods or provide the configurations
for system components such as controllers. ConfigMap is similar to
Secrets, but provides a means of working with strings that don’t
contain sensitive information. Users and system components alike can
store configuration data in ConfigMap.
Here you can find some examples of how to setup it.
Please try it and let me know if that helped.
I have a application consisting of frontend, backend and a database.
At the moment the application is running on a kubernetes cluster.
Front-, backend and database is inside its own Pod communicating via services.
My consideration is to put all these application parts (Front-, Backend and DB) in one Pod, so i can make a Helm chart of it and for every new customer i only have to change the values.
The Question is, if this is a good solution or not to be recommended.
No, it is a bad idea, this is why:
First, the DB is a stateful container, when you update any of the components, you have to put down all containers in the POD, let's say this is a small front end update, it will put down everything and the application will be unavailable.
Let's say you have multiple replicas of this pod to avoid the issue mentioned above, this will make extremely hard to scale the application, because you will need a copy of every container scaled, when you might likely need only FE or BE to scale, also creating multiple replicas of a database, depending how it replicates the data, will make it slower. You also have to consider backup and restore of the data in case of failures.
In the same example above, multiple replicas will make the PODs consume too much resources, even though you don't need it.
If you just want to deploy the resources without much customization, you could just deploy them into separate namespaces and add policies to prevent one namespace talking to each other and deploy the raw yaml there, only taking care to use config maps to load the different configurations for each.
If you want just a simple templating and deployment solution, you can use kustomize.
If you want to have the complex setup and management provided by Helm, you could defined all pods in the chart, an example is the Prometheus chart.
You can create a helm chart consisting of multiple pods or deployments, so you do not need to put them in one pod just for that purpose. I would also not recommend that, as for example the Database would most likely fit better in a StatefulSet.
One of the documented best practices for Kubernetes is to store the configuration in version control. It is mentioned in the official best practices and also summed up in this Stack Overflow question. The reason is that this is supposed to speed-up rollbacks if necessary.
My question is, why do we need to store this configuration if this is already stored by Kubernetes and there are ways with which we can easily go back to a previous version of the configuration using for example kubectl? An example is a command like:
kubectl rollout history deployment/nginx-deployment
Isn't storing the configuration an unnecessary duplication of a piece of information that we will then have to keep synchronized?
The reason I am asking this is that we are building a configuration service on top of Kubernetes. The user will interact with it to configure multiple deployments, I was wondering if we should keep a history of the Kubernetes configuration and the content of configMaps in a database for possible roll backs or if we should just rely on kubernetes to retrieve the current configuration and rolling back to previous versions of the configuration.
You can use Kubernetes as your store of configuration, to your point, it's just that you probably shouldn't want to. By storing configuration as code, you get several benefits:
Configuration changes get regular code reviews.
They get versioned, are diffable, etc.
They can be tested, linted, and whatever else you desired.
They can be refactored, share code, and be documented.
And all this happens before actually being pushed to Kubernetes.
That may seem bad ("but then my configuration is out of date!"), but keep in mind that configuration is actually never in date - just because you told Kubernetes you want 3 replicas running doesn't mean there are, or if there were that 1 isn't temporarily down right now, and so on.
Configuration expresses intent. It takes a different process to actually notice when your intent changes or doesn't match reality, and make it so. For Kubernetes, that storage is etcd and it's up to the master to, in a loop forever, ensure the stored intent matches reality. For you, the storage is source control and whatever process you want, automated or not, can, in a loop forever, ensure your code eventually becomes reflected in Kubernetes.
The rollback command, then, is just a very fast shortcut to "please do this right now!". It's for when your configuration intent was wrong and you don't have time to fix it. As soon as you roll back, you should chase your configuration and update it there as well. In a sense, this is indeed duplication, but it's a rare event compared to the normal flow, and the overall benefits outweigh this downside.
Kubernetes cluster doesn't store your configuration it runs it, as you server runs your application code.
I'm working on a mysql users operator and I'm somewhat stuck on what's the proper way to report any issues.
The plan is to watch on CRD for MysqlUser and create Secrets and mysql users in the specified DB. Obviously, either of that can go wrong, at which point I need to report an error.
Some k8s object track events in the status.conditions. There's also the Event object, but I've only seen that used by kubelet / controllermanager insofar.
If say, I have a problem creating mysql user because my operator cannot talk to mysql, but otherwise the CRD is valid, should it go to event or to CRD's status?
CRDs do not have a status part yet (1.7). Notifying via events is perfectly fine, that's the reason for having them in the first place.
This sounds similar to events reported from volume plugin (kubelet) where, for example, kubelet is unable to mount a volume from NFS server because server address is invalid, thus can not take to it.
Tracking events in status.conditions is less useful in this scenario since users typically have no control over how kubelet (or operator in your case) interacts with underline resources. In general, status.conditions only signals the status of the object, not why it is in this condition.
This is just my understanding of how to make the choice. I don't know if there is any rules around it.