So a typical k8s deployment file that I'm woking on looks like this:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
...
name: ${service-name}
spec:
replicas: 1
strategy:
...
template:
metadata:
...
spec:
serviceAccountName: test
...
the goal is to create multiple services who have access to the same serviceAccount.
This structure works fine when test exists in
kubectl get serviceaccount
The question is how can I set serviceAccountName to default serviceAccount if test does not exist in the namespace (for any reason)?
I don't wanna fail the deployment
I essentially need to have something like
serviceAccountName: {test:-default}
P.S. clearly I can assign a variable to serviceAccountName and parse the yaml file from outside, but wanted to see if there's a better option
As long as you want run this validation inside the cluster, the only way would be to use MutatingAdmissionWebhook.
This will intercepts requests matching the rules defined in MutatingWebhookConfiguration before presisting into etcd. MutatingAdmissionWebhook executes the mutation by sending admission requests to webhook server. Webhook server is just plain http server that adhere to the API.
Thus, you can validate if the service account exists and set default sa if it's not.
Here is an example of the weebhook, which validates and sets custom labels.
More info about Admission Controller Webhooks
Related
Is it possible to provide environment variables which will be set in all pods instead of configuring in each pods spec?
If not natively possible in Kubernetes, what would be an efficient method to accomplish it? We have Helm, but that still requires a lot of duplication.
This old answer suggested "PodPreset" which is no longer part of Kubernetes: Kubernetes - Shared environment variables for all Pods
You could do this using a mutating admission webhook to inject the environment variable into the pod manifest.
There are more details on implementing webhooks here.
I am not sure if you can do that for EVERY single pod in the cluster (if that is what you meant), but you CAN do it for every single pod within an application or service.
For example, via a Deployment, you can set a variable within the pod template, and all replicas will carry that value.
apiVersion: apps/v1
kind: Deployment
metadata:
...
spec:
replicas: 5
template:
metadata:
...
spec:
containers:
- image: nginx
name: nginx
...
env:
- name: VAR_NAME # <---
value: "var_value" # <---
...
In this (edited) example, all 5 replicas of the nginx will have the environment variable VAR_NAME set to the value var_value.
You could also use a configmap (https://kubernetes.io/docs/concepts/configuration/configmap/) or secrets (https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables) to set environments variables from a shared location, depending on your requirements.
i'm playing around with knative currently and bootstrapped a simple installation using gloo and glooctl. Everything worked fine out of the box. However, i just asked myself if there is a possibility to change the generated url, where the service is made available at.
I already changed the domain, but i want to know if i could select a domain name without containing the namespace, so helloworld-go.namespace.mydomain.com would become helloworld-go.mydomain.com.
The current YAML-definition looks like this:
apiVersion: serving.knative.dev/v1alpha1
kind: Service
metadata:
labels:
name: helloworld-go
namespace: default
spec:
template:
spec:
containers:
- image: gcr.io/knative-samples/helloworld-go
env:
- name: TARGET
value: Go Sample v1
Thank you for your help!
This is configurable via the ConfigMap named config-network in the namespace knative-serving. See the ConfigMap in the deployment resources:
apiVersion: v1
data:
_example: |
...
# domainTemplate specifies the golang text template string to use
# when constructing the Knative service's DNS name. The default
# value is "{{.Name}}.{{.Namespace}}.{{.Domain}}". And those three
# values (Name, Namespace, Domain) are the only variables defined.
#
# Changing this value might be necessary when the extra levels in
# the domain name generated is problematic for wildcard certificates
# that only support a single level of domain name added to the
# certificate's domain. In those cases you might consider using a value
# of "{{.Name}}-{{.Namespace}}.{{.Domain}}", or removing the Namespace
# entirely from the template. When choosing a new value be thoughtful
# of the potential for conflicts - for example, when users choose to use
# characters such as `-` in their service, or namespace, names.
# {{.Annotations}} can be used for any customization in the go template if needed.
# We strongly recommend keeping namespace part of the template to avoid domain name clashes
# Example '{{.Name}}-{{.Namespace}}.{{ index .Annotations "sub"}}.{{.Domain}}'
# and you have an annotation {"sub":"foo"}, then the generated template would be {Name}-{Namespace}.foo.{Domain}
domainTemplate: "{{.Name}}.{{.Namespace}}.{{.Domain}}"
...
kind: ConfigMap
metadata:
labels:
serving.knative.dev/release: "v0.8.0"
name: config-network
namespace: knative-serving
Therefore, your config-network should look like this:
apiVersion: v1
data:
domainTemplate: {{ '"{{.Name}}.{{.Domain}}"' }}
kind: ConfigMap
metadata:
name: config-network
namespace: knative-serving
You can also have a look and customize the config-domain to configure the domain name that is appended to your services.
Assuming you're running knative over an istio service mesh, there's an example of how to use an Istio Virtual Service to accomplish this at the service level in the knative docs.
https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-armor-backendconfig
I have only seen example assigning one securityPolicy but I want to assign multiple ones.
I created the following backend config with 2 policies and applied to my service with beta.cloud.google.com/backend-config: my-backend-config
apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
namespace: cloud-armor-how-to
name: my-backend-config
spec:
securityPolicy:
name: "policy-one"
name: "policy-two"
When I deploy only "policy-two" is applied. Can I assign two policies somehow? I see no docs for this
There's nothing in the docs that says that you can specify more than one policy. Even the spec says securityPolicy the singular and the YAML structure is not an array.
Furthermore, if you look at your spec:
spec:
securityPolicy:
name: "policy-one"
name: "policy-two"
The YAML standard completely ignores the first name: "policy-one" which explains why only name: "policy-two" is used. You can check it on YAMLlint. To have one more value on your YAML you would have to convert securityPolicy to an array. Something like this:
apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
namespace: cloud-armor-how-to
name: my-backend-config
spec:
securityPolicy:
- name: "policy-one"
- name: "policy-two"
The issue with this is that it's probably not supported by GCP.
This same behavior happens to the regular HTTP(S) Load Balancers. It looks like it's only possible to add only a single Security Policy per target and the same behavior affects the HTTP(S) load Balancers created by the GKE ingress.
It's possible to add more rules for that only security policy. The new rules can be added in the same way as the first rule was added; however, the priorities of these rules must be different like in the example below:
~$ gcloud beta compute security-policies rules create 1000 \
--security-policy ca-how-to-security-policy \
--src-ip-ranges "192.0.2.0/24" \
--action "deny-404"
~$ gcloud beta compute security-policies rules create 1001 \
--security-policy ca-how-to-security-policy \
--src-ip-ranges "11.16.0.0/24" \
--action "deny-404"
I have 4 k8s pods by setting the replicas of Deployment to 4 now.
apiVersion: v1
kind: Deployment
metadata:
...
spec:
...
replicas: 4
...
The POD will get items in a database and consume it, the items in database has a column class_name.
now I want one pod only get one class_name's item.
for example pod1 only get item which class_name equals class_name_1, and pod2 only get item which class_name equals class_name_2...
So I want to pass different class_name as environment variables to different Deployment PODs. Can I define it in the yaml file of Deployment?
Or is there any other way to achieve my goal?(like something other than Deployment in k8s)
For distributed job processing Deployments are not very good, because they don't have any type of ordering or consistent pod hostnames. You'd better use StatefulSet for it, because they have consistent naming, like pod-0, pod-1, pod-2. You can rely on that hostname index.
For example, if your class_name_idx - is the index of class name in class names list, num_replicas - is the number of replicas in StatefulSet and pod_idx - is the index of pod in StatefulSet, then pod should run the job only if: class_name_idx % num_replicas == pod_idx.
Unfortunately number of StatefulSet replicas cannot be obtained within the pod dynamically using Downward API, so you can either hardcode it or use Kubernetes API to obtain it from cluster.
Neither Deployment nor anything else won't help to achieve your goal. Your goal is some kind of logic and it should be implemented via code in your application.
Since the Deployment is some instances of the same application the only thing that might be useful for you is: using multiple deployments, each for its own task. The first could get class_name_1 item, while other class_name_2, class_name_3 etc. But it is not a good idea
I would not recommend this approach, but the closest thing to do what you want is using the stateful-set and use the pod name as the index.
When you deploy a stateful set, the pods will be named after their statefulset name, in the following sample:
apiVersion: v1
kind: Service
metadata:
name: kuard
labels:
app: kuard
spec:
type: NodePort
ports:
- port: 8080
name: web
selector:
app: kuard
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: kuard
spec:
serviceName: "kuard"
replicas: 3
selector:
matchLabels:
app: kuard
template:
metadata:
labels:
app: kuard
spec:
containers:
- name: kuard
image: gcr.io/kuar-demo/kuard-amd64:1
ports:
- containerPort: 8080
name: web
The pods created by the statefulset will be named as:
kuard-0
kuard-1
kuard-2
This way you could either, name the stateful-set according to the classes, i.e: class-name and the pod created will be class-name-0 and you can replace the _ by -. Or just strip the name out to get the index at the end.
To get the name just read the environment variable HOSTNAME
This naming is consistent, so you can make sure you always have 0, 1, 2, 3 after the name. And if the 2 goes down, it will be recreated.
Like I said, I would not recommend this approach because you tie the infrastructure to your code, and also can't scale(if needed) because each service are unique and adding new instances would get new ids.
A better approach would be using one deployment for each class and pass the proper values as environment variables.
I'm trying to understand why kubernetes docs recommend to specify service before deployment in one configuration file:
The resources will be created in the order they appear in the file. Therefore, it’s best to specify the service first, since that will ensure the scheduler can spread the pods associated with the service as they are created by the controller(s), such as Deployment.
Does it mean spread pods between kubernetes cluster nodes?
I tested with the following configuration where a deployment is located before a service and pods are distributed between nods without any issues.
apiVersion: apps/v1
kind: Deployment
metadata:
name: incorrect-order
namespace: test
spec:
selector:
matchLabels:
app: incorrect-order
replicas: 2
template:
metadata:
labels:
app: incorrect-order
spec:
containers:
- name: incorrect-order
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: incorrect-order
namespace: test
labels:
app: incorrect-order
spec:
type: NodePort
ports:
- port: 80
selector:
app: incorrect-order
Another explanation is that some environment variables with service URL will not be set for pods in this case. However it also works ok in case a configuration is inside one file like the example above.
Could you please explain why it is better to specify service before the deployment in case of one configuration file? Or may be it is some outdated recommendation.
If you use DNS as service discovery, the order of creation doesn't matter.
In case of Environment Vars (the second way K8S offers service discovery) the order matters, because once that vars are passed to the starting pod, they cannot be modified later if the service definition changes.
So if your service is deployed before you start your pod, the service envvars are injected inside the linked pod.
If you create a Pod/Deployment resource with labels, this resource will be exposed through a service once this last is created (with proper selector to indicate what resource to expose).
You are correct in that it effects the spread among the worker nodes.
Deployments without a Service will simply be scheduled onto the nodes with the least cpu/memory allocation. For instance, a brand new and empty node will get all new pods from a new deployment.
With a Deployment that also has a service the Scheduler tries to spread the pods between nodes, disregarding the cpu/memory load (within limits), to help the Service survive better.
It puzzles me that a Deployment on it's own doesn't cause a optimal spread but it doesn't, not yet at least.
This is the answer from the official documentation:
The resources will be created in the order they appear in the file.
Therefore, it's best to specify the service first, since that will
ensure the scheduler can spread the pods associated with the service
as they are created by the controller(s), such as Deployment.
Kubernetes Documentation/Concepts/Cluster/Administration/Managing Resources