I am make a nfs file share and using it in kubernetes pods, but when I start pods, it give me tips :
2020-05-31 03:00:06+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.30-1debian10 started.
chown: changing ownership of '/var/lib/mysql/': Operation not permitted
I searching from internet and understand the nfs default map other root login to nfsnobody account, if the privillege not correct, this error should happen, but I follow the steps and still not solve it. This is the ways I having tried:
1 addd unsecure config no_root_squash in /etc/exports:
/mnt/data/apollodb/apollopv *(rw,sync,no_subtree_check,no_root_squash)
2 remove the PVC and PV and directly using nfs in pod like this:
volumes:
- name: apollo-mysql-persistent-storage
nfs:
server: 192.168.64.237
path: /mnt/data/apollodb/apollopv
containers:
- name: mysql
image: 'mysql:5.7'
ports:
- name: mysql
containerPort: 3306
protocol: TCP
env:
- name: MYSQL_ROOT_PASSWORD
value: gfwge4LucnXwfefewegLwAd29QqJn4
resources: {}
volumeMounts:
- name: apollo-mysql-persistent-storage
mountPath: /var/lib/mysql
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext: {}
schedulerName: default-scheduler
this tell me the problem not in pod define but in the nfs config itself.
3 give every privillege using this command
chmod 777 /mnt/data/apollodb/apollopv
4 chown to nfsnobody like this
sudo chown nfsnobody:nfsnobody -R apollodb/
sudo chown 999:999 -R apollodb
but the problem still not solved,so what should I try to make it works?
You wouldn't set this via chown, you would use fsGroup security setting instead.
Related
Can I create multiple empty dir volume and copy the content in it. Deployment has started failing after adding securityContext as:
securityContext:
readOnlyRootFilesystem: true
Due to read only volume, deployment fails with the error:
OSError: [Errno 30] Read-only file system: '
As per your issue Nginx must be provided access to the paths which were shown in the error.
In order to prevent these errors we need to pass tmpfs arguments.
Tmpfs can be used to rectify the error: OSError: [Errno 30] Read-only file system: ' permission issue. Tmpfs theoretically works as the regular volume, which allows us to mount storage from outside the container however it’s not persistent in nature. It mounts an area of the host memory to the specified location in the container. Deploy the container by passing tmpfs arguments
Example:
<docker run -d -p 8080:80 --read-only --tmpfs:/var/cache/nginx --tmpfs:/var/run nginx:alphine>
Now if you check the logs you will be able to find the deployment is successful.
If you are trying to deploy the containers using Kubernetes we use ephemeral storage for this purpose. When we use emptyDir as volume, Kubernetes will attach a local folder from the underlying worker-node, which lives as long as the pod. Please follow this official documentation for more information on ephemeral storages in Kubernetes. Please follow the below yaml file example for configuring emptyDir in Kubernetes.
apiVersion: v1
kind: Pod
metadata:
name: webserver
labels:
name: webserver
spec:
containers:
- name: webserver
image: nginx:alpine
securityContext:
readOnlyRootFilesystem: true
ports:
- containerPort: 80
volumeMounts:
- mountPath: /var/run
name: tmpfs-1
- mountPath: /var/cache/nginx
name: tmpfs-2
volumes:
- name: tmpfs-1
emptyDir: {}
# - name: tmpfs-ram
# emptyDir:
# medium: "Memory"
- name: tmpfs-2
emptyDir: {}
On my Jetson NX, I like to set a yaml file that can mount 2 cameras to pod,
the yaml:
containers:
- name: my-pod
image: my_image:v1.0.0
imagePullPolicy: Always
volumeMounts:
- mountPath: /dev/video0
name: dev-video0
- mountPath: /dev/video1
name: dev-video1
resources:
limits:
nvidia.com/gpu: 1
ports:
- containerPort: 9000
command: [ "/bin/bash"]
args: ["-c", "while true; do echo hello; sleep 10;done"]
securityContext:
privileged: true
volumes:
- hostPath:
path: /dev/video0
type: ""
name: dev-video0
- hostPath:
path: /dev/video1
type: ""
name: dev-video1
but when I deploy it as pod, get the error:
MountVolume.SetUp failed for volume "default-token-c8hm5" : failed to sync secret cache: timed out waiting for the condition
I had tried to remove volumes in yaml, and the pod can be successfully deployed. Any comments on this issue?
Another issue is that when there is a pod got some issues, it will consume the rest of my storage of my Jetson NX, I guess maybe k8s will make lots of temporary files or logs...? when something wrong happening, any solution to this issue, otherwise all od my pods will be evicted...
I've created the manifest file, that looks as follows:
apiVersion: v1
kind: Pod
metadata:
name: kuard
spec:
volumes:
- name: "kuard-data"
hostPath:
path: "/home/developer/kubernetes/exercises"
containers:
- image: gcr.io/kuar-demo/kuard-amd64:1
name: kuard
volumeMounts:
- mountPath: "/data"
name: "kuard-data"
ports:
- containerPort: 8080
name: http
protocol: TCP
As you can see, the hostpath is:
path: "/home/developer/kubernetes/exercises"
and the mountPath is:
mountPath: "/data"
I've created a hello.txt file in the folder /home/developer/kubernetes/exercises and when I enter into the pod via kubectl exec -it kuard ash I can not find the file hello.txt.
Where is the file?
kind is using Docker containers to simulate Kubernetes nodes. So when you are creating files on your host (your ubuntu machine) the containers will not automatically have access to them.
(This gets even more complicated when using macos or windows and docker is running in a separate virtual machine...)
I assume that there are some shared folders visible inside the kind-docker-nodes, but I could not find it documented.
You can verify the filesystem content of the docker node from inside the container using docker exec -it kind-control-plane /bin/sh and then work with the usual tools.
If you need to make content from your development machine available you might want to have a look at ksync: https://github.com/vapor-ware/ksync
I try to mount a folder that is non-root user(xxxuser) at kubernetes and I use hostPath for mounting. But whenever container is started, it is mounted with user (1001) not xxxuser. It is always started with user (1001). How can I mount this folder with xxxuser ?
There is many types of volumes but I use hostPath. Before started; I change folder user and group with chown and chgrp commands. And then; mounted this folder as volume. Container started and I checked user of folder but it always user (1001). Such as;
drwxr-x---. 2 1001 1001 70 May 3 14:15 configutil/
volumeMounts:
- name: configs
mountPath: /opt/KOBIL/SSMS/home/configutil
volumes:
- name: configs
hostPath:
path: /home/ssmsuser/configutil
type: Directory
drwxr-x---. 2 xxxuser xxxuser 70 May 3 14:15 configutil/
You may specify the desired owner of mounted volumes using following syntax:
spec:
securityContext:
fsGroup: 2000
I try what you have recomend but my problem is still continue. I add below line to my yaml file:
spec:
securityContext:
runAsUser: 999
runAsGroup: 999
fsGroup: 999
I use 999 because I use 999 inside my Dockerfile. Such as;
RUN groupadd -g 999 ssmsuser && \
useradd -r -u 999 -g ssmsuser ssmsuser
USER ssmsuser
I am trying to run DPDK L2FWD application on a container managed by Kubernetes.
To achieve this I have done the below steps -
I have created single node K8s setup where both master and client are running on host machine. As network plug-in, I have used Calico Network.
To create customized DPDK docker image, I have used the below Dockerfile
FROM ubuntu:16.04 RUN apt-get update
RUN apt-get install -y net-tools
RUN apt-get install -y python
RUN apt-get install -y kmod
RUN apt-get install -y iproute2
RUN apt-get install -y net-tools
ADD ./dpdk/ /home/sdn/dpdk/
WORKDIR /home/sdn/dpdk/
To run DPDK application inside POD, below host's directories are mounted to POD with privileged access:
/mnt/huge
/usr
/lib
/etc
Below is k8s deployment yaml used to create the POD
apiVersion: v1
kind: Pod
metadata:
name: dpdk-pod126
spec:
containers:
- name: dpdk126
image: dpdk-test126
imagePullPolicy: IfNotPresent
command: ["/bin/sh"]
args: ["-c", "while true; do echo hello; sleep 10;done"]
resources:
requests:
memory: "2Gi"
cpu: "100m"
volumeMounts:
- name: hostvol1
mountPath: /mnt/huge
- name: hostvol2
mountPath: /usr
- name: hostvol3
mountPath: /lib
- name: hostvol4
mountPath: /etc
securityContext:
privileged: true
volumes:
- name: hostvol1
hostPath:
path: /mnt/huge
- name: hostvol2
hostPath:
path: /usr
- name: hostvol3
hostPath:
path: /home/sdn/kubernetes-test/libtest
- name: hostvol4
hostPath:
path: /etc
Below configurations are already done in host -
Huge page mounting.
Interface binding in user space.
After successful creation of POD, when trying to run a DPDK L2FWD application inside POD, I am getting the below error -
root#dpdk-pod126:/home/sdn/dpdk# ./examples/l2fwd/build/l2fwd -c 0x0f -- -p 0x03 -q 1
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: 1007 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
EAL: FATAL: Cannot get hugepage information.
EAL: Cannot get hugepage information.
EAL: Error - exiting with code: 1
Cause: Invalid EAL arguments
According to this, you might be missing
medium: HugePages from your hugepage volume.
Also, hugepages can be a bit finnicky. Can you provide the output of:
cat /proc/meminfo | grep -i huge
and check if there's any files in /mnt/huge?
Also maybe this can be helpful. Can you somehow check if the hugepages are being mounted as mount -t hugetlbfs nodev /mnt/huge?
First of all, you have to verify, that you have enough hugepages in your system. Check it with kubectl command:
kubectl describe nodes
where you could see something like this:
Capacity:
cpu: 12
ephemeral-storage: 129719908Ki
hugepages-1Gi: 0
hugepages-2Mi: 8Gi
memory: 65863024Ki
pods: 110
If your hugepages-2Mi is empty, then your k8s don't see mounted hugepages
After mounting hugepages into your host, you can prepare your pod to work with hugepages. You don't need to mount hugepages folder as you shown. You can simply add emptyDir volume like this:
volumes:
- name: hugepage-2mi
emptyDir:
medium: HugePages-2Mi
HugePages-2Mi is a specific resource name that corresponds with hugepages of 2Mb size. If you want to use 1Gb size hugepages then there is another resource for it - hugepages-1Gi
After defining the volume, you can use it in volumeMounts like this:
volumeMounts:
- mountPath: /hugepages-2Mi
name: hugepage-2mi
And there is one additional step. You have to define resource limitations for hugepages usage:
resources:
limits:
hugepages-2Mi: 128Mi
memory: 128Mi
requests:
memory: 128Mi
After all this steps, you can run your container with hugepages inside container
As #AdamTL mentioned, you can find additional info here