Mounting Windows local folder into pod - kubernetes

I'm running a Ubuntu container with SQL Server in my local Kubernetes environment with Docker Desktop on a Windows laptop.
Now I'm trying to mount a local folder (C:\data\sql) that contains database files into the pod.
For this, I configured a persistent volume and persistent volume claim in Kubernetes, but it doesn't seem to mount correctly. I don't see errors or anything, but when I go into the container using docker exec -it and inspect the data folder, it's empty. I expect the files from the local folder to appear in the mounted folder 'data', but that's not the case.
Is something wrongly configured in the PV, PVC or pod?
Here are my yaml files:
apiVersion: v1
kind: PersistentVolume
metadata:
name: dev-customer-db-pv
labels:
type: local
app: customer-db
chart: customer-db-0.1.0
release: dev
heritage: Helm
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
hostPath:
path: /C/data/sql
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: dev-customer-db-pvc
labels:
app: customer-db
chart: customer-db-0.1.0
release: dev
heritage: Helm
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Mi
apiVersion: apps/v1
kind: Deployment
metadata:
name: dev-customer-db
labels:
ufo: dev-customer-db-config
app: customer-db
chart: customer-db-0.1.0
release: dev
heritage: Helm
spec:
selector:
matchLabels:
app: customer-db
release: dev
replicas: 1
template:
metadata:
labels:
app: customer-db
release: dev
spec:
volumes:
- name: dev-customer-db-pv
persistentVolumeClaim:
claimName: dev-customer-db-pvc
containers:
- name: customer-db
image: "mcr.microsoft.com/mssql/server:2019-latest"
imagePullPolicy: IfNotPresent
volumeMounts:
- name: dev-customer-db-pv
mountPath: /data
envFrom:
- configMapRef:
name: dev-customer-db-config
- secretRef:
name: dev-customer-db-secrets
At first, I was trying to define a volume in the pod without PV and PVC, but then I got access denied errors when I tried to read files from the mounted data folder.
spec:
volumes:
- name: dev-customer-db-data
hostPath:
path: C/data/sql
containers:
...
volumeMounts:
- name: dev-customer-db-data
mountPath: data
I've also tried to Helm install with --set volumePermissions.enabled=true but this didn't solve the access denied errors.

Based on this info from GitHub for Docker there is no support hostpath volumes in WSL 2.
Thus, next workaround can be used.
We need just to append /run/desktop/mnt/host to the initial path on the host /c/data/sql. No need for PersistentVolume and PersistentVolumeClaim in this case - just remove them.
I changed spec.volumes for Deployment according to information about hostPath configuration on Kubernetes site:
volumes:
- name: dev-customer-db-pv
hostPath:
path: /run/desktop/mnt/host/c/data/sql
type: Directory
After applying these changes, the files can be found in data folder in the pod, since mountPath: /data

Related

Kubernetes mount volume keeps timeing out even though volume can be mounted from sudo mount

I have a read only persistent volume that I'm trying to mount onto the statefulset, but after making some changes to the program and re-creating the pods, the pod can now no longer mount to the volume.
PV yaml file:
apiVersion: v1
kind: PersistentVolume
metadata:
name: foo-pv
spec:
capacity:
storage: 2Gi
accessModes:
- ReadOnlyMany
nfs:
server: <ip>
path: "/var/foo"
claimRef:
name: foo-pvc
namespace: foo
PVC yaml file:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: foo-pvc
namespace: foo
spec:
accessModes:
- ReadOnlyMany
storageClassName: ""
volumeName: foo-pv
resources:
requests:
storage: 2Gi
Statefulset yaml:
apiVersion: v1
kind: Service
metadata:
name: foo-service
spec:
type: ClusterIP
ports:
- name: http
port: 80
selector:
app: foo-app
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: foo-statefulset
namespace: foo
spec:
selector:
matchLabels:
app: foo-app
serviceName: foo-app
replicas: 1
template:
metadata:
labels:
app: foo-app
spec:
serviceAccountName: foo-service-account
containers:
- name: fooContainer
image: <image>
imagePullPolicy: Always
volumeMounts:
- name: writer-data
mountPath: <path>
- name: nfs-objectd
mountPath: <path>
volumes:
- name: nfs-foo
persistentVolumeClaim:
claimName: foo-pvc
volumeClaimTemplates:
- metadata:
name: writer-data
spec:
accessModes: [ "ReadWriteMany" ]
storageClassName: "foo-sc"
resources:
requests:
storage: 2Gi
k describe pod reports "Unable to attach or mount volumes: unmounted volumes=[nfs-foo]: timed out waiting for the condition". There is a firewall between the machine running kubernetes and the NFS, however the port has been unblocked, and the folder has been exported for mounting on the NFS side. Running sudo mount -t nfs :/var/foo /var/foo is able to successfully mount the NFS, so I don't understand why kuebernetes isn't about to mount it anymore. Its been stuck failing mount for several days now. Is there any other way to debug this?
Thanks!
Based on the error “unable to attach or mount volumes …….timed out waiting for condition”, there were some similar issues reported to the Product Team and it is a known issue. But, this error is more observed on the preemptible/spot nodes when the node is preempted. In similar occurrences of this issue for other users, upgrading the control plane version resolved this issue temporarily in preemptible/spot nodes.
Also, if you are not using any preemptible/spot nodes in your cluster, this issue might have happened when the old node is replaced by a new node. If you are still facing this issue, try upgrading the control plane to the same version i.e. you can execute the following command:
$ gcloud container clusters upgrade CLUSTER_NAME --master --zone ZONE --cluster-version VERSION
Another workaround to fix this issue would be remove the stale VolumeAttachment with the following command:
$ kubectl delete volumeattachment [volumeattachment_name]
After running the command and thus removing the VolumeAttachment, the pod should eventually pick up and retry. You can read more about this issue and its cause here.

How to create a PV and PVC which is accessible by multiple pods and cronjobs in Kubernetes on a local setup

I am trying to create a PV and PVC which is accessible by a pod and a cronjob too on the same node.
Currently I am using the following yaml file
apiVersion: v1
kind: PersistentVolume
metadata:
name: wwwroot
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "wwwroot"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
app: wwwroot
name: wwwroot
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
status: {}
in the statefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
labels:
app: web
name: web
spec:
selector:
matchLabels:
app: web
serviceName: web
replicas: 1
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: xy:latest
ports:
- containerPort: 80
- containerPort: 443
volumeMounts:
- mountPath: /app/wwwroot
name: wwwroot
subPath: tempwwwroot
imagePullSecrets:
- name: xy
restartPolicy: Always
serviceAccountName: ""
volumeClaimTemplates:
- metadata:
name: wwwroot
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 100Gi
This works fine with the pods, but I have a cronjob that needs to access the same wwwroot PV.
apiVersion: batch/v1beta1
kind: CronJob
metadata:
labels:
app: import
name: import
spec:
schedule: "09 16 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: import
image: xy:latest
imagePullPolicy: Always
volumeMounts:
- mountPath: /app/wwwroot
name: wwwroot
subPath: tempwwwroot
imagePullSecrets:
- name: xy
restartPolicy: OnFailure
volumes:
- name: wwwroot
persistentVolumeClaim:
claimName: wwwroot
To the best of my knowledge, you cannot bind these PVs to a folder on the host machine like in Docker (which would be nice, but I can live with that). When I run a kubectl get pvc with the following setup, it seem to use the same PVC name wwwroot, but clearly it is not. I have /bin/bash-d into the pod created by the cronjob and the wwwroot folder does not contain the files from the web pod wwwroot folder.
Any idea what am I doing wrong? Also what is the best option the create PV/PVCs on a local 1 node Kubernetes setup? I was also thinking about to create a NFS share on the host for easier access to the files.
Also when first created the PV/PVC and deployed the web statefulSet, the web pod should populate the wwwroot PV with CSS and JS files, but seems like when it bounds to the PV, it deletes everything from that folder and I have to manually copy those files again. Any workaround on that?
You are mounting pv on two pod created and using pv by setting accessModes: ReadWriteOnce
You can check them below.
https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes
Also instant of local you can use storage class NFS volume which can give you read-write-many. I am not sure Storage class local provide it as it is not in official documents. Chk below
https://kubernetes.io/docs/concepts/storage/persistent-volumes/
P.S. Never use storage class in actual deployment as node in k8s come and go.

How to have data persist in GKE kubernetes StatefulSet with postgres?

So I'm just trying to get a web app running on GKE experimentally to familiarize myself with Kubernetes and GKE.
I have a statefulSet (Postgres) with a persistent volume/ persistent volume claim which is mounted to the Postgres pod as expected. The problem I'm having is having the Postgres data endure. If I mount the PV at var/lib/postgres the data gets overridden with each pod update. If I mount at var/lib/postgres/data I get the warning:
initdb: directory "/var/lib/postgresql/data" exists but is not empty
It contains a lost+found directory, perhaps due to it being a mount point.
Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.
Using Docker alone having the volume mount point at var/lib/postgresql/data works as expected and data endures, but I don't know what to do now in GKE. How does one set this up properly?
Setup file:
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: sm-pd-volume-claim
spec:
storageClassName: "standard"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1G
---
apiVersion: "apps/v1"
kind: "StatefulSet"
metadata:
name: "postgis-db"
namespace: "default"
labels:
app: "postgis-db"
spec:
serviceName: "postgis-db"
replicas: 1
selector:
matchLabels:
app: "postgis-db"
template:
metadata:
labels:
app: "postgis-db"
spec:
terminationGracePeriodSeconds: 25
containers:
- name: "postgis"
image: "mdillon/postgis"
ports:
- containerPort: 5432
name: postgis-port
volumeMounts:
- name: sm-pd-volume
mountPath: /var/lib/postgresql/data
volumes:
- name: sm-pd-volume
persistentVolumeClaim:
claimName: sm-pd-volume-claim
You are getting this error because the postgres pod has tried to mount the data directory on / folder. It is not recommended to do so.
You have to create subdirectory to resolve this issues on the statefulset manifest yaml files.
volumeMounts:
- name: sm-pd-volume
mountPath: /var/lib/postgresql/data
subPath: data

How to deploy MariaDB on kubernetes with some default schema and data?

For some context, I'm trying to build a staging / testing system on kubernetes which starts with deploying a mariadb on the cluster with some schema and data. I have a trunkated / clensed db dump from prod to help me with that. Let's call that file : dbdump.sql which is present in my local box in the path /home/rjosh/database/script/ . After much reasearch here is what my yaml file looks like:
apiVersion: v1
kind: PersistentVolume
metadata:
name: m3ma-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 30Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: m3ma-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 30Gi
---
apiVersion: v1
kind: Service
metadata:
name: m3ma
spec:
ports:
- port: 3306
selector:
app: m3ma
clusterIP: None
---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: m3ma
spec:
selector:
matchLabels:
app: m3ma
strategy:
type: Recreate
template:
metadata:
labels:
app: m3ma
spec:
containers:
- image: mariadb:10.2
name: m3ma
env:
# Use secret in real usage
- name: MYSQL_ROOT_PASSWORD
value: password
ports:
- containerPort: 3306
name: m3ma
volumeMounts:
- name: m3ma-persistent-storage
mountPath: /var/lib/mysql/
- name: m3ma-host-path
mountPath: /docker-entrypoint-initdb.d/
volumes:
- name: m3ma-persistent-storage
persistentVolumeClaim:
claimName: m3ma-pv-claim
- name: m3ma-host-path
hostPath:
path: /home/smaikap/database/script/
type: Directory
The MariaDB instance is coming up but not with the schema and data that is present in /home/rjosh/database/script/dbdump.sql.
Basically, the mount is not working. If I connect to the pod and check /docker-entrypoint-initdb.d/ there is nothing. How do I go about this?
A bit more details. Currently, I'm testing it on minikube. But, soon it will have to work on GKE cluster. Looking at the documentation, hostPath is not the choice for GKE. So, what the correct way of doing this?
Are you sure your home directory is visible to Kubernetes? Minikube generally creates a little VM to run things in, which wouldn't have your home dir in it. The more usual way to handle this would be to make a very small new Docker image yourself like:
FROM mariadb:10.2
COPY dbdump.sql /docker-entrypoint-initdb.d/
And then push it to a registry somewhere, and then use that image instead.

Minio data does not persist through reboot

I deployed Minio on Kubernetes on an Ubuntu desktop. It works fine, except that whenever I reboot the machine, everything that was stored in Minio mysteriously disappears (if I create several buckets with files in them, I come back to a completely blanks slate after the reboot - the buckets, and all their files, are completely gone).
When I set up Minio, I created a persistent volume in Kubernetes which mounts to a folder (/mnt/minio/minio - I have a 4 TB HDD mounted at /mnt/minio with a folder named minio inside that). I noticed that this folder seems to be empty even when I store stuff in Minio, so perhaps Minio is ignoring the persistent volume and using the container storage? However, I don't know why this would be happening; I have both a PV and a PV claim, and kubectl shows that they are bound to each other.
Below are the yaml files I applied to deploy my minio installation:
kind: PersistentVolume
apiVersion: v1
metadata:
name: minio-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/minio/minio"
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: minio-pv-claim
labels:
app: minio-storage-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 99Gi
apiVersion: apps/v1 # for k8s versions before 1.9.0 use apps/v1beta2 and before 1.8.0 use extensions/v1beta1
kind: Deployment
metadata:
# This name uniquely identifies the Deployment
name: minio-deployment
spec:
selector:
matchLabels:
app: minio
strategy:
type: Recreate
template:
metadata:
labels:
# Label is used as selector in the service.
app: minio
spec:
# Refer to the PVC created earlier
volumes:
- name: storage
persistentVolumeClaim:
# Name of the PVC created earlier
claimName: minio-pv-claim
containers:
- name: minio
# Pulls the default Minio image from Docker Hub
image: minio/minio:latest
args:
- server
- /storage
env:
# Minio access key and secret key
- name: MINIO_ACCESS_KEY
value: "minio"
- name: MINIO_SECRET_KEY
value: "minio123"
ports:
- containerPort: 9000
hostPort: 9000
# Mount the volume into the pod
volumeMounts:
- name: storage # must match the volume name, above
mountPath: "/mnt/minio/minio"
apiVersion: v1
kind: Service
metadata:
name: minio-service
spec:
type: LoadBalancer
ports:
- port: 9000
targetPort: 9000
protocol: TCP
selector:
app: minio
you need to mount container's /storage directory in the directory you are mounting on the container /mnt/minio/minio/;
args:
- server
- /mnt/minio/minio/storage
But consider deploying using StatefulSet, so when your pod restarts it will retain everything of the previous pod.