when and where a service ip created for kubernetes static pods - kubernetes

Question

I'm using kubernetes 1.6.7 cluster, 3 nodes as master (node1, node2 and node3).
The apiserver cluster ip as bellow:
root#node1:/etc/kubernetes/manifests# kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.233.0.1 <none> 443/TCP 14d
root#node1:/etc/kubernetes/manifests# kubectl get endpoints
kubernetes 172.16.10.11:6443,172.16.10.12:6443,172.16.10.13:6443 14d
The problem is when node2 was down, tiller pod can't access 10.233.0.1:443, logs as bellow:
[storage] 2018/08/28 06:46:34 listing all releases with filter
[storage/driver] 2018/08/28 06:46:34 list: failed to list: Get https://10.233.0.1:443/api/v1/namespaces/kube-system/configmaps?labelSelector=OWNER%3DTILLER: dial tcp 10.233.0.1:443: connect: no route to host
and the service kubernetes still has 3 endpoints, but i think the 172.16.10.12 should be delete from the endpoint list. iptables as bellow:
-A KUBE-SVC-LRLUS54FOYJDJ5GT -m comment --comment "default/wangxj35-nginx:nginx" -j KUBE-SEP-GGIOR777CPKVSIB7
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 10800 --reap --name KUBE-SEP-CS4Z4M5BFAAWBFXQ --mask 255.255.255.255 --rsource -j KUBE-SEP-CS4Z4M5BFAAWBFXQ
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 10800 --reap --name KUBE-SEP-MQZA2ZXHX6GRRY22 --mask 255.255.255.255 --rsource -j KUBE-SEP-MQZA2ZXHX6GRRY22
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 10800 --reap --name KUBE-SEP-ZX77NMWHNJWLXYWF --mask 255.255.255.255 --rsource -j KUBE-SEP-ZX77NMWHNJWLXYWF
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-CS4Z4M5BFAAWBFXQ
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-MQZA2ZXHX6GRRY22
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -j KUBE-SEP-ZX77NMWHNJWLXYWF
-A KUBE-SEP-CS4Z4M5BFAAWBFXQ -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-CS4Z4M5BFAAWBFXQ --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 172.16.10.11:6443
-A KUBE-SEP-MQZA2ZXHX6GRRY22 -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-MQZA2ZXHX6GRRY22 --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 172.16.10.12:6443
-A KUBE-SEP-ZX77NMWHNJWLXYWF -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-ZX77NMWHNJWLXYWF --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 172.16.10.13:6443
Could anyone help me where the static pods service created by kubernetes source code?
Thanks.

Answer 1

Controller.go creates the kubernetes service:
// UpdateKubernetesService attempts to update the default Kube service.
func (c *Controller) UpdateKubernetesService(reconcile bool)> error {
// Update service & endpoint records.
Also, to work around this problem, kubernetes1.9.0 has added --endpoint-reconciler-type=lease to allow apiserver to track the alive apiserver, see issue51698, while for release below 1.9.0, you can unset the --apiserver-count flag to let alive apiserver to preempt the only one endpoint position.