This page: https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/iam-policy.md
uses:
kubectl set env daemonset aws-node -n kube-system CLUSTER_NAME=${YourClusterName}
and according to this page: https://kubernetes.io/docs/reference/kubectl/
everything after set seems to be a subcommand.
However, I don't understand what subcommand this is: daemonset aws-node -n kube-system CLUSTER_NAME=${YourClusterName}
Can someone explain (hopefully with more docs)?
...don't understand what subcommand this is: daemonset aws-node -n kube-system CLUSTER_NAME=${YourClusterName}
Every EKS node runs an instance of aws-node pod which provide CNI functionality to the node. The command set the environment variable named CLUSTER_NAME value to the name of your cluster for every pod managed by the daemonset. So that you do not need to go node by node to change each running pod environment variable, the command set and automatically restart all the pod(s) for you.
Related
I have a cluster in Azure (AKS), in this cluster, I have 2 pools: a System one and a User one to run apps.
When executing kubectl get pod command inside a pod, I get the error:
The connection to the server localhost:8080 was refused - did you specify the right host or port?
the pod is running on the system node(not working on the user node either) in one my own namespace(let's call it cron)
But when running the same command in a pod belonging to the namespace kube-system on the system node, it's working fine.
it looks like to be link to the configuration of kubectl (kubeconfig) but I don't get how it works in the kube-system namespace and not the cron one
what did I miss in AKS to not be able to run kubectl command in pod not belonging to kube-system namespace?
Edit1:
Environment variables are different especially those linked to Kubernetes:
On the pod running under kube-system got the URL:
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_PORT=443
KUBE_DNS_PORT=udp://10.0.0.10:53
KUBE_DNS_PORT_53_TCP_ADDR=10.0.0.10
KUBERNETES_DASHBOARD_PORT_443_TCP_PORT=443
KUBERNETES_DASHBOARD_SERVICE_PORT=443
KUBE_DNS_SERVICE_HOST=10.0.0.10
KUBERNETES_PORT_443_TCP=tcp://*****.hcp.japaneast.azmk8s.io:443
KUBE_DNS_PORT_53_TCP_PORT=53
KUBE_DNS_PORT_53_UDP=udp://10.0.0.10:53
KUBE_DNS_PORT_53_UDP_PROTO=udp
KUBE_DNS_SERVICE_PORT_DNS=53
KUBE_DNS_PORT_53_TCP_PROTO=tcp
KUBE_DNS_PORT_53_UDP_ADDR=10.0.0.10
KUBERNETES_DASHBOARD_PORT_443_TCP_ADDR=10.0.207.97
KUBE_DNS_SERVICE_PORT_DNS_TCP=53
KUBERNETES_DASHBOARD_PORT_443_TCP=tcp://10.0.207.97:443
KUBERNETES_DASHBOARD_PORT=tcp://10.0.207.97:443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBE_DNS_PORT_53_UDP_PORT=53
KUBERNETES_PORT_443_TCP_ADDR=****.hcp.japaneast.azmk8s.io
KUBERNETES_SERVICE_HOST=*****.hcp.japaneast.azmk8s.io
KUBERNETES_PORT=tcp://*****.hcp.japaneast.azmk8s.io:443
KUBERNETES_PORT_443_TCP_PORT=443
KUBE_DNS_PORT_53_TCP=tcp://10.0.0.10:53
KUBERNETES_DASHBOARD_PORT_443_TCP_PROTO=tcp
KUBE_DNS_SERVICE_PORT=53
KUBERNETES_DASHBOARD_SERVICE_HOST=10.0.207.97
In my own env cron
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT_443_TCP=tcp://10.0.0.1:443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_ADDR=10.0.0.1
KUBERNETES_SERVICE_HOST=10.0.0.1
KUBERNETES_PORT=tcp://10.0.0.1:443
KUBERNETES_PORT_443_TCP_PORT=443
About namespaces. I labeled my own namespace with the same labels as kube-system but no luck.
about the config: both kubectl config view are coming back empty (requested inside the pod):
apiVersion: v1
clusters: null
contexts: null
current-context: ""
kind: Config
preferences: {}
users: null
I'd need to force moving a Pod from one Openshift Node to another, in order to do some performance tests. From the documentation it seems that setting the nodeSelector in the Deployment config is the way to go, but it doesn't work, according to my tests.
Here is what I have tried as a test:
Create nginx Pod
oc new-app -f https://raw.githubusercontent.com/sclorg/nginx-ex/master/openshift/templates/nginx.json
The Pod is running on "ip-10-0-121-229.us-east-2.compute.internal" Node.
Now I patch the node selector, setting a target Node:
oc patch dc nginx-example -p '{"spec":{"template":{"spec":{"nodeSelector":{"kubernetes.io/hostname": "ip-10-0-169-74.us-east-2.compute.internal"}}}}}'
However the Pod is still running on the same Node. Even by killing the Pod it re-attaches there.
Any suggestion?
Thanjs
you can try this -
kubectl get pod -o wide
this will give you the VM on which your pod is running
then execute
kubectl cordon {name_of_that_node_in_which_POD_is_running}
then delete the pods, those you want to get aligned to other node
then run
kubectl uncordon {the_node_that_was_cordoned_above}
You can use nodeName field in your pod spec to schedule the pod yourself, rather than scheduler do it.
oc explain pod.spec.nodeName
FIELD: nodeName <string>
DESCRIPTION:
NodeName is a request to schedule this pod onto a specific node. If it is
non-empty, the scheduler simply schedules this pod onto that node, assuming
that it fits resource requirements.
you can patch it in similar way. Don't forget to delete nodeSelctor fields
oc patch dc nginx-example -p '{"spec":{"template":{"spec":{"nodeName": "ip-10-0-169-74.us-east-2.compute.internal"}}}}'
How to flush CoreDNS Cache on kubernetes cluster?
I know it can be done by deleting the CoreDNS pods, but is there a proper way to to the cache flush ?
#coollinuxoid's answer is not suitable for production environment, it will have temporary downtime because the commands will terminate all pods at the same time. Instead, you should use kubernetes deployment's rolling update mechanism by setting an environment variable to avoid the downtime with command:
kubectl -n kube-system set env deployment.apps/coredns FOO="BAR"
The best way, as you said, would be restarting coredns pods. This can be done easily, by scaling the coredns deployment to "0" and then, scale it back to the desired number.
Like in the sample command below:
kubectl scale deployment.apps/coredns -n kube-system --replicas=0
kubectl scale deployment.apps/coredns -n kube-system --replicas=2
without timout:
kubectl rollout restart deployment coredns -n kube-system
Thanks #Nick for comment
execute this in each rabbitmq pod to remove mnesia, then restart pods
rm -rf /bitnami/rabbitmq/mnesia
I keep reading documentation that gives parameters for kube-proxy, but does not explain how where these parameters are supposed to be used. I create my cluster using az aks create with the azure-cli program, then I get credentials and use kubectl. So far everything I've done has involved yaml for services and deployments and such, but I can't figure out where all this kube-proxy stuff fits into all of this.
I've googled for days. I've opened question issues on github with AKS. I've asked on the kubernetes slack channel, but nobody has responded.
The kube-proxy on all your Kubernetes nodes runs as a Kubernetes DaemonSet and its configuration is stored on a Kubernetes ConfigMap. To make any changes or add/remove options you will have to edit the kube-proxy DaemonSet or ConfigMap on the kube-system namespace.
$ kubectl -n kube-system edit daemonset kube-proxy
or
$ kubectl -n kube-system edit configmap kube-proxy
For a reference on the kube-proxy command line options you can refer to here.
I have 3 nodes in kubernetes cluster. I create a daemonset and deployed it in all the 3 devices. This daemonset created 3 pods and they were successfully running. But for some reasons, one of the pod failed.
I need to know how can we restart this pod without affecting other pods in the daemon set, also without creating any other daemon set deployment?
Thanks
kubectl delete pod <podname> it will delete this one pod and Deployment/StatefulSet/ReplicaSet/DaemonSet will reschedule a new one in its place
There are other possibilities to acheive what you want:
Just use rollout command
kubectl rollout restart deployment mydeploy
You can set some environment variable which will force your deployment pods to restart:
kubectl set env deployment mydeploy DEPLOY_DATE="$(date)"
You can scale your deployment to zero, and then back to some positive value
kubectl scale deployment mydeploy --replicas=0
kubectl scale deployment mydeploy --replicas=1
Just for others reading this...
A better solution (IMHO) is to implement a liveness probe that will force the pod to restart the container if it fails the probe test.
This is a great feature K8s offers out of the box. This is auto healing.
Also look into the pod lifecycle docs.
kubectl -n <namespace> delete pods --field-selector=status.phase=Failed
I think the above command is quite useful when you want to restart 1 or more failed pods :D
And we don't need to care about name of the failed pod.