I would like to implement functionality (or even better reuse existing libraries/APIs!) that would intercept a kubectl command to create an object and perform some pre-creation validation tasks on it before allowing kubectl command to proceed.
e.g.
check various values in the yaml against external DB for example
check a label conforms to the internal naming convention
and so on..
Is there an accepted pattern or existing tools etc?
Any guidance appreciated
The way to do this is by creating a ValidatingAdmissionWebhook. It's not for the faint of heart and even a brief example would be an overkill as a SO answer. A few pointers to start:
https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook
https://banzaicloud.com/blog/k8s-admission-webhooks/
https://container-solutions.com/a-gentle-intro-to-validation-admission-webhooks-in-kubernetes/
I hope this helps :-)
I usually append - - dry-run to kubectl command to check and validate the YAML config
Related
Lets say I need to create environment variables or ConfigMap entries like this:
- name: JDBC_URL
value: "jdbc:db2://alice-service-a:50000/db1"
- name: KEYCLOAK_BASE_URL
value: "http://alice-keycloak:8080/auth"
Where alice- is the namePrefix. How do I do this using Kustomize?
The containers I use actually do need references to other containers that are string concatenations of "variables" like above.
It doesn't look like Kustomize's vars can do this. The documentation entry Unstructured Edits seems to describe this and is under a heading called "Eschewed Features", so I guess that isn't going to happen. A similar feature request, #775 Support envsubst style variable expansion was closed.
Coming from Helm, that was easy.
What are my options if I want to move from Helm to Kustomize, but need to create an env or ConfigMap entry like e.g. jdbc:db2://${namePrefix}-service-b:${dbPort}/${dbName} (admittedly a contrived example)?
I'm guessing I'll have to resort to functionality external to Kustomize, like envsubst. Are there any best practices for cobbling this together, or am I writing my own custom-deploy-script.sh?
I'm afraid I've come up against one of the limitations of Kustomize.
The State of Kubernetes Configuration Management: An Unsolved Problem | by Jesse Suen | Argo Project has this to say under "Kustomize: The Bad":
No parameters & templates. The same property that makes kustomize applications so readable, can also make it very limiting. For example, I was recently trying to get the kustomize CLI to set an image tag for a custom resource instead of a Deployment, but was unable to. Kustomize does have a concept of “vars,” which look a lot like parameters, but somehow aren’t, and can only be used in Kustomize’s sanctioned whitelist of field paths. I feel like this is one of those times when the solution, despite making the hard things easy, ends up making the easy things hard.
Instead, I've started using gomplate: A flexible commandline tool for template rendering in addition to Kustomize to solve the challenge above, but having to use two tools that weren't designed to work together is not ideal.
EDIT: We ended up using ytt for this instead of gomplate.
I can heavily recommend the article: The State of Kubernetes Configuration Management: An Unsolved Problem. Nice to know I'm not the only one hitting this road block.
In the kubectl Cheat Sheet (https://kubernetes.io/docs/reference/kubectl/cheatsheet/), there are 3 ways to modify resources. You can either update, patch or edit.
What are the actual differences between them and when should I use each of them?
I would like to add a few things to night-gold's answer. I would say that there are no better and worse ways of modifying your resources. Everything depends on particular situation and your needs.
It's worth to emphasize the main difference between editing and patching namely the first one is an interactive method and the second one we can call batch method which unlike the first one may be easily used in scripts. Just imagine that you need to make change in dozens or even a few hundreds of different kubernetes resources/objects and it is much easier to write a simple script in which you can patch all those resources in an automated way. Opening each of them for editing wouldn't be very convenient and effective. Just a short example:
kubectl patch resource-type resource-name --type json -p '[{"op": "remove", "path": "/spec/someSection/someKey"}]'
Although at first it may look unnecessary complicated and not very convenient to use in comparison with interactive editing and manually removing specific line from specific section, in fact it is a very quick and effective method which may be easily implemented in scripts and can save you a lot of work and time when you work with many objects.
As to apply command, you can read in the documentation:
apply manages applications through files defining Kubernetes
resources. It creates and updates resources in a cluster through
running kubectl apply. This is the recommended way of managing
Kubernetes applications on production.
It also gives you possibility of modifying your running configuration by re-applying it from updated yaml manifest e.g. pulled from git repository.
If by update you mean rollout ( formerly known as rolling-update ), as you can see in documentation it has quite different function. It is mostly used for updating deployments. You don't use it for making changes in arbitrary type of resource.
I don't think I have the answer to this but I hope this will help.
All three methods do the same thing, they modify some resources configuration but the command and way to it is not the same.
As describe in the documentation:
Editing is when you open the yaml configuration file that is in the kubernetes cluster and edit it (with vim or other) to get direct modification on your cluster. I would not recommand this outside of testing purpose, reapplying conf from orignal yaml file will delete modificaitons.
Patching seems the same to me, but without opening the file and targetting specific parts of the resources.
Updating in the documentation it seems that's it's all other method to update a resource without using patch or edit. Some of those can be used for debug/testing, for example forcing a resource replace, or update an image version. Others are used to update them with new configurations.
From experience, I only used editing and some command of update for testing, most of time I reapply the configurations.
I was wondering what is the best name pattern for a service object in k8s environment.
Should it be %service-name%-service or just %service-name%?
workflow-service or just workflow?
What are the arguments for both sides?
In kubernetes the service dns follow the below pattern
<service-name>.<namespace-name>.svc.cluster.local
i have seen people using svc or service appended to the service name with '-' as the delimiter like below, say, redis
redis-service
redis-svc
redis
all the three are perfectly fine but the first one makes more sense interms of readability and standard way of representing the service object.
In fact, while creating a service it is not needed to append "-service" in the name. The general way of doing it is to name the service same as the name of the pods it is pointing to. Hope this helps.
Thank you!
this is simply a matter of taste. if you want verbosity, add -service. but since resources are separate anyway, why be verbose.
I have a gke cluster with a number of different namespaces. I would like to be able to in effect namespace my images the same way my other resources are namespaced. That is, I would like pods in different namespaces to be able to reference an image using the same name but for them to get different images depending on which namespace they are in. One way to achieve this (if it were supported) might be to substitute the name of the namespace into the image name in the yml, eg:
containers:
- image: eu.gcr.io/myproject/$(NAMESPACE)-myimage
name: myimage
Then I could push eu.gcr.io/myproject/mynamespace-myimage to make my image available to namespace mynamespace.
Is there any tidy way to achieve this kind of thing. If not, and since I've been unable to find anybody else asking similar questions, is there some way in which this is a bad thing to want to do?
I don't think this is possible. Kubernetes supports expansion on fields like command and args. This lets you use substitutions with a variable set in container's env field, which can come from configmap/secret. Example: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#use-environment-variables-to-define-arguments
However I don't think variable expansion works for the image field. :(
What you're trying to do does not seem like a great idea: Having different images across multiple environments defeats the purpose of having test/staging environments.
Instead you should probably should use the same image to test on all platforms, by changing env vars, configMaps etc.
Would anyone have suggestions, pointers, examples of updating a current ASG's tagging?
I have a need to update tags on ASGs rather frequently. So far, playing with ec2_asg, I'm unable to figure out if it's possible. My next best so far has been having ansible run the CLI commands themselves. Before I start going the boto path and writing my own script, I thought I'd ask the community.
Any suggestion, pointer or example is always appreciated. Thank you.
You can use the ec2_tag module. Just apply the tags you want on the ASG ARN.
Creates, removes and lists tags from any EC2 resource. The resource is referenced by its resource id (e.g. an instance being i-XXXXXXX). It is designed to be used with complex args (tags), see the examples. This module has a dependency on python-boto.