KUBERNETES PODS:unable to mount nfs volume - kubernetes

Question

I have some trouble mounting an nfs volume inside a docker container.
I have one minion(labcr3) and one pod(httpd-php).
Below you can find all relevant details.
Can you help me ?
Thanks and Regards
Phisical host(minion) is able to mount the nfs volume.
[root#LABCR3 ~]# df -h /NFS
Filesystem Size Used Avail Use% Mounted on
192.168.240.1:/mnt/VOL1/nfs-ds 500G 0 500G 0% /NFS
Here the priviliged mode enable on kubernetes on minion
root#LABCR3 ~]# cat /etc/kubernetes/config | grep PRIV
KUBE_ALLOW_PRIV="--allow_privileged=true"
Describe of pod
[root#labcr1 pods]# kubectl describe pod httpd-php
Name: httpd-php
Namespace: default
Image(s): centos7-httpd-php-alive-new
Node: labcr3/10.99.1.203
Labels: cont=httpd-php-mxs,name=httpd-php
Status: Pending
Reason:
Message:
IP:
Replication Controllers:
Containers:
httpd-php-mxs:
Image: centos7-httpd-php-alive-new
State: Waiting
Reason: Image: centos7-httpd-php-alive-new is ready, container is creating
Ready: False
Restart Count: 0
Conditions:
Type Status
Ready False
Events:
FirstSeen LastSeen CountFrom SubobjectPath Reason Message
Fri, 04 Dec 2015 15:29:48 +0100 Fri, 04 Dec 2015 15:29:48 +0100 1 {scheduler } scheduled Successfully assigned httpd-php to labcr3
Fri, 04 Dec 2015 15:31:53 +0100 Fri, 04 Dec 2015 15:38:09 +0100 4 {kubelet labcr3} failedMount Unable to mount volumes for pod "httpd-php_default": exit status 32
Fri, 04 Dec 2015 15:31:53 +0100 Fri, 04 Dec 2015 15:38:09 +0100 4 {kubelet labcr3} failedSync Error syncing pod, skipping: exit status 32
Kubelet logs on minons
-- Logs begin at ven 2015-12-04 14:25:39 CET, end at ven 2015-12-04 15:34:39 CET. --
dic 04 15:33:58 LABCR3 kubelet[1423]: E1204 15:33:58.986220 1423 pod_workers.go:111] Error syncing pod 7915461d-9a93-11e5-b8eb-d4bed9b48f94, skipping:
dic 04 15:33:58 LABCR3 kubelet[1423]: E1204 15:33:58.973581 1423 kubelet.go:1190] Unable to mount volumes for pod "httpd-php_default": exit status 32;
dic 04 15:33:58 LABCR3 kubelet[1423]: Output: mount.nfs: Connection timed out
dic 04 15:33:58 LABCR3 kubelet[1423]: Mounting arguments: labsn1:/mnt/VOL1/nfs-ds /var/lib/kubelet/pods/7915461d-9a93-11e5-b8eb-d4bed9b48f94/volumes/kube
dic 04 15:33:58 LABCR3 kubelet[1423]: E1204 15:33:58.973484 1423 mount_linux.go:103] Mount failed: exit status 32
dic 04 15:31:53 LABCR3 kubelet[1423]: E1204 15:31:53.939521 1423 pod_workers.go:111] Error syncing pod 7915461d-9a93-11e5-b8eb-d4bed9b48f94, skipping:
dic 04 15:31:53 LABCR3 kubelet[1423]: E1204 15:31:53.927865 1423 kubelet.go:1190] Unable to mount volumes for pod "httpd-php_default": exit status 32;
dic 04 15:31:53 LABCR3 kubelet[1423]: Output: mount.nfs: Connection timed out
dic 04 15:31:53 LABCR3 kubelet[1423]: Mounting arguments: labsn1:/mnt/VOL1/nfs-ds /var/lib/kubelet/pods/7915461d-9a93-11e5-b8eb-d4bed9b48f94/volumes/kube
dic 04 15:31:53 LABCR3 kubelet[1423]: E1204 15:31:53.927760 1423 mount_linux.go:103] Mount failed: exit status 32
dic 04 15:21:31 LABCR3 kubelet[1423]: E1204 15:21:31.119117 1423 reflector.go:183] watch of *api.Service ended with: 401: The event in requested in
UPDATE 12/18/2015
Yes, is the first pod with nfs mount.
nfs is mounted on the minion, so is working fine and there isn't ACL applyed to the nfs
this morning I've changed the config like this:
apiVersion: v1
kind: Pod
metadata:
name: httpd-php
labels:
name: httpd-php
cont: httpd-php-mxs
spec:
containers:
- name: httpd-php-mxs
image: "centos7-httpd-php-alive-new"
command: ["/usr/sbin/httpd","-DFOREGROUND"]
imagePullPolicy: "Never"
ports:
- containerPort: 80
volumeMounts:
- mountPath: "/NFS/MAXISPORT/DOCROOT"
name: mynfs
volumes:
- name: mynfs
persistentVolumeClaim:
claimName: nfs-claim
[root#labcr1 KUBE]# cat pv/nfspv.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
name: nfspv
spec:
capacity:
storage: 500Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /mnt/VOL1/nfs-ds
server: 10.99.1.202
[root#labcr1 KUBE]# cat pv/nfspvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: nfs-claim
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 500Gi
[root#labcr1 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
httpd-php 1/1 Running 1 29m
[root#labcr1 ~]# kubectl get pv
NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON
nfspv <none> 536870912000 RWX Bound default/nfs-claim
[root#labcr1 ~]# kubectl get pvc
NAME LABELS STATUS VOLUME
nfs-claim map[] Bound nfspv
[root#labcr1 ~]# kubectl describe pod httpd-php
Name: httpd-php
Namespace: default
Image(s): centos7-httpd-php-alive-new
Node: labcr3/10.99.1.203
Labels: cont=httpd-php-mxs,name=httpd-php
Status: Running
Reason:
Message:
IP: 172.17.76.2
Replication Controllers: <none>
Containers:
httpd-php-mxs:
Image: centos7-httpd-php-alive-new
State: Running
Started: Fri, 18 Dec 2015 15:12:39 +0100
Ready: True
Restart Count: 1
Conditions:
Type Status
Ready True
Events:
FirstSeen LastSeen Count From SubobjectPath Reason Message
Fri, 18 Dec 2015 14:44:20 +0100 Fri, 18 Dec 2015 14:44:20 +0100 1 {kubelet labcr3} implicitly required container POD pulled Pod container image "gcr.io/google_containers/pause:0.8.0" already present on machine
Fri, 18 Dec 2015 14:44:20 +0100 Fri, 18 Dec 2015 14:44:20 +0100 1 {scheduler } scheduled Successfully assigned httpd-php to labcr3
Fri, 18 Dec 2015 14:44:20 +0100 Fri, 18 Dec 2015 14:44:20 +0100 1 {kubelet labcr3} implicitly required container POD created Created with docker id 3b6d520e66b0
Fri, 18 Dec 2015 14:44:20 +0100 Fri, 18 Dec 2015 14:44:20 +0100 1 {kubelet labcr3} implicitly required container POD started Started with docker id 3b6d520e66b0
Fri, 18 Dec 2015 14:44:21 +0100 Fri, 18 Dec 2015 14:44:21 +0100 1 {kubelet labcr3} spec.containers{httpd-php-mxs} started Started with docker id 859ea73a6cdd
Fri, 18 Dec 2015 14:44:21 +0100 Fri, 18 Dec 2015 14:44:21 +0100 1 {kubelet labcr3} spec.containers{httpd-php-mxs} created Created with docker id 859ea73a6cdd
Fri, 18 Dec 2015 15:12:38 +0100 Fri, 18 Dec 2015 15:12:38 +0100 1 {kubelet labcr3} implicitly required container POD pulled Pod container image "gcr.io/google_containers/pause:0.8.0" already present on machine
Fri, 18 Dec 2015 15:12:38 +0100 Fri, 18 Dec 2015 15:12:38 +0100 1 {kubelet labcr3} implicitly required container POD created Created with docker id bdfed6fd4c97
Fri, 18 Dec 2015 15:12:38 +0100 Fri, 18 Dec 2015 15:12:38 +0100 1 {kubelet labcr3} implicitly required container POD started Started with docker id bdfed6fd4c97
Fri, 18 Dec 2015 15:12:39 +0100 Fri, 18 Dec 2015 15:12:39 +0100 1 {kubelet labcr3} spec.containers{httpd-php-mxs} created Created with docker id ab3a39784b4e
Fri, 18 Dec 2015 15:12:39 +0100 Fri, 18 Dec 2015 15:12:39 +0100 1 {kubelet labcr3} spec.containers{httpd-php-mxs} started Started with docker id ab3a39784b4e
Now the minion is up and running but inside the pods I can't see the nfs mounted....
[root#httpd-php /]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/docker-253:2-3413- f90c89a604c59b6aacbf95af649ef48eb4df829fdfa4b51eee4bfe75a6a156c3 99G 444M 93G 1% /
tmpfs 5.9G 0 5.9G 0% /dev
shm 64M 0 64M 0% /dev/shm
tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup
tmpfs 5.9G 0 5.9G 0% /run/secrets
/dev/mapper/centos_labcr2-var 248G 3.8G 244G 2% /etc/hosts