After trying all possible configurations I ask here if anyone knows how to enable ttlAfterFinished=true?
I use the K8S version 1.17.1.
You need to enable it via the feature gate in kube controller manager and kube API Server. If Kube controller manager and kube API Server is deployed as static pod then you can find the deployment yaml at
/etc/kubernetes/manifests/kube-controller-manager.yaml
and
/etc/kubernetes/manifests/kube-apiserver.yaml
in the master node.
Edit both manifests files and add this line at the bottom of the command section:
- --feature-gates=TTLAfterFinished=true
After the yaml is edited and saved, the kube controller manager and the kube API Server pod will be automatically recreated with this feature enabled.
You can verify by checking logs of kube controller manager pod and you should see below
I0308 06:04:43.886097 1 ttlafterfinished_controller.go:105] Starting TTL after finished controller
Tip: you can specify multiple feature gates using comma, for example:
--feature-gates=TTLAfterFinished=true,OtherFeature=true
Related
I setup a private K8S cluster with RKE 1.2.2 and so my K8S version is 1.19. We have some internal services, and it is necessary to access each other using custom FQDN instead of simple service names. As I searched the web, the only solution I found is adding rewrite records for CoreDNS ConfigMap described in this REF. However, this solution results in manual configuration, and I want to define a record automatically during service setup. Is there any solution for this automation? Does CoreDNS have such an API to add or delete rewrite records?
Note1: I also tried to mount the CoreDNS's ConfigMap and update it via another pod, but the content is mounted read-only.
Note2: Someone proposed calling kubectl get cm -n kube-system coredns -o yaml | sed ... | kubectl apply .... However, I want to automate it during service setup or in a pod or in an initcontainer.
Note3: I wish there were something like hostAliases for services, something called serviceAliases for internal services (ClusterIP).
Currently, there is no ready solution for this.
Only thing comes to my mind is to use MutatingAdmissionWebhook. It would need catch moment, when new Kubernetes service was created and then modify ConfigMap for CoreDNS as it's described in CoreDNS documentation.
After that, you would need to reload CoreDNS configuration to apply new configuration from ConfigMap. To achieve that, you can use reload plugin for CoreDNS. More details about this plugin can be found here.
Instead of above you can consider using sidecarContainer for CoreDNS, which will send SIGUSR1 signal to CoreDNS conatiner.
Example of this method can be found in this Github thread.
We have an on-premise kubernetes deployment in our data center. I just finished deploying the pods for Dex, configured hooked up with our LDAP server to allow LDAP based authentication via Dex, ran tests and was able to retrieve the OpenID connect token for authentication.
Now I would like to change our on-premise k8s API server startup parameters to enable OIDC and point it to the Dex container.
How do I enable OIDC to the API server startup command without downtime to our k8s cluster? Was reading this doc https://kubernetes.io/docs/reference/access-authn-authz/authentication/ but the site just says "Enable the required flags" without the steps
Thanks!
I installed Dex + Active Directory Integration few months ago on a cluster installed by kubeadmn .
Let's assume that Dex is now running and it can be accessible thru
https://dex.example.com .
In this case,..
Enabling ODIC at the level of API server has 3 steps :
These steps have to be done on each of your Kubernetes master nodes.
1- SSH to your master node.
$ ssh root#master-ip
2- Edit the Kubernetes API configuration.
Add the OIDC parameters and modify the issuer URL accordingly.
$ sudo vim /etc/kubernetes/manifests/kube-apiserver.yaml
...
command:
- /hyperkube
- apiserver
- --advertise-address=x.x.x.x
...
- --oidc-issuer-url=https://dex.example.com # <-- 🔴 Please focus here
- --oidc-client-id=oidc-auth-client # <-- 🔴 Please focus here
- --oidc-username-claim=email # <-- 🔴 Please focus here
- --oidc-groups-claim=groups # <-- 🔴 Please focus here
...
3- The Kubernetes API will restart by itself.
I recommend also to check a full guide like this tuto.
The OIDC flags are for Kubernetes API Server. You have not mentioned how you have installed Kubernetes on prem. Ideally you should have multiple master nodes fronted by a LoadBalancer.
So you would disable traffic to one master node from the loadbalancer and login to that master node and edit the manifest of api server in /etc/kubernetes/manifests and add the OIDC flags. Once you change the manifest api server pod will be restarted automatically.
You repeat the same process for all master nodes and since at any given point in time you have at least one master node available there should not be any downtime.
I'm new to K8s. In process to config Openstack Cinder as K8s StorageClass, i have to add some flags to my kube controller manager, and I found that it's my big problem.
I'm using K8s 1.11 in VMs, and my K8s cluster has a kube-controller-manager pod, but I don't know how to add these flags to my kube-controller-manager.
After hours search, i found that there's a lot of task require add flag to kube-controller-manager, but no exactly document guide me how to do that. Please share me the way to go over it.
Thank you.
You can check /etc/kubernetes/manifests dir on your master nodes.
This dir would contain yaml files for master components.
These are also known as static pods.
More Info : https://kubernetes.io/docs/tasks/administer-cluster/static-pod/
Update these files and you would be able to see your changes as kubelet should restart the pod on file change.
As a more long term solution, you will need to incorporate the flags to the tooling that you use to generate your k8s cluster.
I have a container cluster in Google Container Engine with Stackdriver logging agent enabled. It is correctly pulling stdout logs from my containers. Now I would like to change the fluentd config to specify a log parser so that the logs shown in the GCP Logging view will have the correct severity and component.
Following this Stackdriver logging guide from kubernetes.io, I have attempted to:
Get the fluentd ConfigMap as a yml file
Added a new <filter> according to my log4js log format
Created a new ConfigMap named fluentd-cm-2 in kube-system namespace
Edited the DaemonSet for fluentd and set its ConfigMap to fluentd-cm-2. I did this using kubectl edit ds instead of kubectl replace -f because the latter failed with an error message: "the object has been modified", even after getting a fresh copy of the DaemonSet yaml.
Unexpected result: The DaemonSet is restarted, but its configuration is reverted back to the original ConfigMap, so my changes did not take effect.
I have also tried editing the ConfigMap directly (kubectl edit cm fluentd-gcp-config-v1.1 --namespace kube-system) and saved it, but it was also reverted.
I noticed that the DaemonSet and ConfigMap for fluentd are tagged with addonmanager.kubernetes.io/mode: Reconcile. I would conclude that GKE has overwritten my settings because of this "reconcile" mode.
So, my question is: how can I change the fluentd configuration in a Google Container Engine cluster, when the logging agent was installed by GKE on cluster provisioning?
Please take a look at the Prerequisites section on the documentation page you mentioned. It's mentioned there, that on GKE you cannot change the default Stackdriver Logging integration. The reason is that GKE maintains this configuration: updates the agent, watches its health and so on. It's not possible to provide the same level of support for all possible configurations.
However, you can always disable the default integration and deploy your own, patched version of DaemonSet. You can find out how to disable the default integration in the GKE documentation:
gcloud beta container clusters update [CLUSTER-NAME] \
--logging-service=none
Note, that after you disabled the default integration, you have to maintain the new deployment yourself: update the agent, set the resources, watch its health.
Here is a solution for using your own fluentd daemonset that is very much like the one included with GKE.
https://cloud.google.com/solutions/customizing-stackdriver-logs-fluentd
Docker: 1.12.6
rancher/server: 1.5.10
rancher/agent: 1.2.2
Tried two ways to install Kubernetes cluster on rancher/server.
Method 1: Use Kubernetes environment
Infrastructure/Hosts
Agent hosts disconnected sometimes.
Stacks
All green except kubernetes-ingress-lbs. It has 0 containers.
Method 2: Use Default environment
Infrastructure/Hosts
Set some labels to rancher server and agent hosts.
Stacks
All green except kubernetes-ingress-lbs. It has 0 containers.
Both of them have this issue: kubernetes-ingress-lbs 0 services 0 containers. Then can't access Kubernetes dashboard.
Why didn't been installed by rancher?
And, is it necessary to add those labels for Kubernetes cluster?
Here is RIGHT Kubernetes Cluster deployed on Rancher server:
Turning on the Show System, you can find the service of kubernetes-dashboard under the namespace of kube-system.
Well, by using the version of kubernetes is v1.5.4, you should prepare in advance to pull the below Docker Images:
By reading rancher/catalog and rancher/kuberetes-package, you can know and even modify the config files(like docker-compose.yml, rancher-compose.yml and so on) by yourself.
When you enable to "Show System" containers in the UI, you should be able to see the dashboard container running under Namespace: kube-system. If this container is not running then the dashboard will not be able to load.
You might have to enable kubernetes add-on service within rancher environment template.
manage environments >> edit kubernetes default template >> enable add-on service and save the new template with the preferred name.
Now launch the cluster using customized templates.