Deployment.apps is invalid: spec.template.spec.containers[0].volumeMounts[1].name: Not found: "data" - kubernetes

I am deployment an app named soa-illidan-hub-service with a Persistent Volume in kubernetes version v1.16.0. When I apply the yaml, it gives me this error:
Deployment.apps "soa-illidan-hub-service" is invalid: spec.template.spec.containers[0].volumeMounts[1].name: Not found: "data"
This is my yaml file:
kind: Deployment
apiVersion: apps/v1
metadata:
name: soa-illidan-hub-service
namespace: dabai-pro
selfLink: /apis/apps/v1/namespaces/dabai-pro/deployments/soa-illidan-hub-service
uid: 01a06200-f8d4-4d60-bd79-a7acf76d0a30
resourceVersion: '6232127'
generation: 62
creationTimestamp: '2020-06-08T01:42:11Z'
labels:
k8s-app: soa-illidan-hub-service
annotations:
deployment.kubernetes.io/revision: '52'
spec:
replicas: 1
selector:
matchLabels:
k8s-app: soa-illidan-hub-service
template:
metadata:
name: soa-illidan-hub-service
creationTimestamp: null
labels:
k8s-app: soa-illidan-hub-service
annotations:
kubectl.kubernetes.io/restartedAt: '2020-07-09T17:41:29+08:00'
spec:
volumes:
- name: agent
emptyDir: {}
initContainers:
- name: init-agent
image: 'harbor.google.net/miaoyou/dabai-pro/skywalking-agent:6.5.0'
command:
- sh
- '-c'
- >-
set -ex;mkdir -p /skywalking/agent;cp -r /opt/skywalking/agent/*
/skywalking/agent;
resources: {}
volumeMounts:
- name: agent
mountPath: /skywalking/agent
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
containers:
- name: soa-illidan-hub-service
image: >-
harbor.google.net/miaoyou/dabai-pro/soa-illidan-hub-service#sha256:4ac4c6ddceac3fde05e95219b20414fb39ad81a4f789df0fbf97196b72c9e6f0
env:
- name: SKYWALKING_ADDR
value: 'dabai-skywalking-skywalking-oap.apm.svc.cluster.local:11800'
- name: APOLLO_META
valueFrom:
configMapKeyRef:
name: pro-config
key: apollo.meta
- name: ENV
valueFrom:
configMapKeyRef:
name: pro-config
key: env
resources: {}
volumeMounts:
- name: agent
mountPath: /opt/skywalking/agent
- name: data
mountPath: /var/export/data
livenessProbe:
httpGet:
path: /actuator/liveness
port: 11024
scheme: HTTP
initialDelaySeconds: 120
timeoutSeconds: 60
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /actuator/health
port: 11024
scheme: HTTP
initialDelaySeconds: 120
timeoutSeconds: 60
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
securityContext:
privileged: false
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext: {}
imagePullSecrets:
- name: harbor-regcred
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 25%
revisionHistoryLimit: 10
volumeClaimTemplates:
- metadata:
name: data
creationTimestamp: null
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
volumeMode: Filesystem
progressDeadlineSeconds: 600
to add PV, I add volumeClaimTemplates config:
volumeClaimTemplates:
- metadata:
name: data
creationTimestamp: null
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
volumeMode: Filesystem
and I am using this volume in my pod like this:
volumeMounts:
- name: data
mountPath: /var/export/data
Am I missing something?What should I do fix this problem?

I believe that your deployment definition is the issue.
Checking the k8s docs, I found this example:
kind: Pod
metadata:
name: test-pd
spec:
containers:
- image: k8s.gcr.io/test-webserver
name: test-container
volumeMounts:
- mountPath: /cache
name: cache-volume
volumes:
- name: cache-volume
emptyDir: {}
Basically you need to define the volumeMounts under your container, and also refer that volumeMount to a valid volume under the volumes section.
Just to highlight, the names should match, otherwise, it will also fail.

volumeClaimTemplates is only available for statefulset.
kubeclt explain statefulset.spec.volumeClaimTemplates
KIND: StatefulSet
VERSION: apps/v1
RESOURCE: volumeClaimTemplates <[]Object>
DESCRIPTION:
volumeClaimTemplates is a list of claims that pods are allowed to
reference. The StatefulSet controller is responsible for mapping network
identities to claims in a way that maintains the identity of a pod. Every
claim in this list must have at least one matching (by name) volumeMount in
one container in the template. A claim in this list takes precedence over
any volumes in the template, with the same name.
PersistentVolumeClaim is a user's request for and claim to a persistent
volume
FIELDS:
apiVersion <string>
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
kind <string>
Kind is a string value representing the REST resource this object
represents. Servers may infer this from the endpoint the client submits
requests to. Cannot be updated. In CamelCase. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
metadata <Object>
Standard object's metadata. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
spec <Object>
Spec defines the desired characteristics of a volume requested by a pod
author. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
status <Object>
Status represents the current information/status of a persistent volume
claim. Read-only. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
volumeClaimTemplates is not available for deployment
kubectl explain deployment.spec.volumeClaimTemplates
error: field "volumeClaimTemplates" does not exist
So you can not use volumeClaimTemplates for deployment

Related

Nexus on k3s on restart does not persist Users and data

I have installed on K3S raspberry pi cluster nexus with the following setups for kubernetes learning purposes. First I created a StatefulSet:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nexus
namespace: dev-ops
spec:
serviceName: "nexus"
replicas: 1
selector:
matchLabels:
app: nexus-server
template:
metadata:
labels:
app: nexus-server
spec:
containers:
- name: nexus
image: klo2k/nexus3:latest
env:
- name: MAX_HEAP
value: "800m"
- name: MIN_HEAP
value: "300m"
resources:
limits:
memory: "4Gi"
cpu: "1000m"
requests:
memory: "2Gi"
cpu: "500m"
ports:
- containerPort: 8081
volumeMounts:
- name: nexusstorage
mountPath: /sonatype-work
volumes:
- name: nexusstorage
persistentVolumeClaim:
claimName: nexusstorage
Storage class
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nexusstorage
provisioner: driver.longhorn.io
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
numberOfReplicas: "3"
staleReplicaTimeout: "30"
fsType: "ext4"
diskSelector: "ssd"
nodeSelector: "ssd"
pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nexusstorage
namespace: dev-ops
spec:
accessModes:
- ReadWriteOnce
storageClassName: nexusstorage
resources:
requests:
storage: 50Gi
Service
apiVersion: v1
kind: Service
metadata:
name: nexus-server
namespace: dev-ops
annotations:
prometheus.io/scrape: 'true'
prometheus.io/path: /
prometheus.io/port: '8081'
spec:
selector:
app: nexus-server
type: LoadBalancer
ports:
- port: 8081
targetPort: 8081
nodePort: 32000
this setup will spin up nexus, but if I restart the pod the data will not persist and I have to create all the setups and users from scratch.
What I'm missing in this case?
UPDATE
I got it working, nexus needs on mount permissions on directory. The working StatefulSet looks as it follow
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nexus
namespace: dev-ops
spec:
serviceName: "nexus"
replicas: 1
selector:
matchLabels:
app: nexus-server
template:
metadata:
labels:
app: nexus-server
spec:
securityContext:
runAsUser: 200
runAsGroup: 200
fsGroup: 200
containers:
- name: nexus
image: klo2k/nexus3:latest
env:
- name: MAX_HEAP
value: "800m"
- name: MIN_HEAP
value: "300m"
resources:
limits:
memory: "4Gi"
cpu: "1000m"
requests:
memory: "2Gi"
cpu: "500m"
ports:
- containerPort: 8081
volumeMounts:
- name: nexus-storage
mountPath: /nexus-data
volumes:
- name: nexus-storage
persistentVolumeClaim:
claimName: nexus-storage
important snippet to get it working
securityContext:
runAsUser: 200
runAsGroup: 200
fsGroup: 200
I'm not familiar with that image, although checking dockerhub, they mention using a Dockerfile similar to that of Sonatype. Then, I would change the mountpoint for your volume, to /nexus-data
This is the default path storing data (they set this env var, then declare a VOLUME). Which we can confirm, looking at the repository that most likely produced your arm-capable image
And following up on your last comment, let's try to also mount it in /opt/sonatype/sonatype-work/nexus3...
In your statefulset, change volumeMounts, to this:
volumeMounts:
- name: nexusstorage
mountPath: /nexus-data
- name: nexusstorage
mountPath: /opt/sonatype/sonatype-work/nexus3
volumes:
- name: nexusstorage
persistentVolumeClaim:
claimName: nexusstorage
Although the second volumeMount entry should not be necessary, as far as I understand. Maybe something's wrong with your storage provider?
Are you sure your PVC is write-able? Reverting back to your initial configuration, enter your pod (kubectl exec -it) and try to write a file at the root of your PVC.

Postgresql storage on K3s

I am trying to setup a Postgresql Stateful-Set on a k3s single node (Raspberry Pi 4 8Gb) cluster which will be common between any services that end up needing postgresql. Currently, since this is just a single node, I am using the Rancher local-path storage provisioner (this will probably change as I add nodes) pointing to an external hdd set to mount, though this will probably change when I get around to adding nodes.
My pod spins up (after figuring out a small headache with mounting the data directory which was as per this issue), and I can access the postgresql instance with kubectl port-forward -n common pod/postgresql-stateful-set-0 5432:5432, and making whatever changes I need.
At this point, my problem manifests, whereby I notice that no data is persisted in my data directory. I have found this issue which is the exact issue I've encountered, but following all the examples of "fixes", from playing with the directory locations (which gave me chmod issues like encountered with the mounting data directory issues), to changing to a pv I defined.
Find below my yml file which is use kubectl apply -f postgres.yml and is based on the bitnami helm template:
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: postgresql-storage
provisioner: rancher.io/local-path
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
---
kind: PersistentVolume
apiVersion: v1
metadata:
name: postgresql-pv
namespace: common
spec:
storageClassName: postgresql-storage
persistentVolumeReclaimPolicy: Retain
capacity:
storage: 20Gi
accessModes:
- ReadWriteMany
hostPath:
path: "/mnt/storage/k3s/common/postgresql"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
namespace: common
name: pvc-postgresql
spec:
storageClassName: postgresql-storage
accessModes:
- ReadWriteMany
resources:
requests:
storage: 20Gi
---
apiVersion: v1
kind: Secret
metadata:
name: postgresql-password
namespace: common
data:
POSTGRES_PASSWORD: <PWD>
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgresql-configmap
namespace: common
data:
POSTGRESQL_PORT_NUMBER: "5432"
PGDATA: /var/lib/postgresql/data/pgdata
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgresql-stateful-set
namespace: common
labels:
name: postgres-stateful-set
spec:
replicas: 1
serviceName: postgresql-stateful-set
updateStrategy:
rollingUpdate: {}
type: RollingUpdate
selector:
matchLabels:
app: postgresql
template:
metadata:
labels:
app: postgresql
spec:
containers:
- name: postgresql
image: docker.io/postgres:14.2-alpine
envFrom:
- configMapRef:
name: postgresql-configmap
- secretRef:
name: postgresql-password
ports:
- name: tcp-postgresql
containerPort: 5432
livenessProbe:
failureThreshold: 6
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
exec:
command:
- /bin/sh
- -c
- exec pg_isready -U "rootAdmin" -h 127.0.0.1 -p 5432 -d rootDefault
readinessProbe:
failureThreshold: 6
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
exec:
command:
- /bin/sh
- -c
- -e
- exec pg_isready -U "rootAdmin" -h 127.0.0.1 -p 5432 -d rootDefault
resources:
limits:
memory: "300Mi"
cpu: "300m"
requests:
cpu: 250m
memory: 256Mi
volumeMounts:
- name: dshm
mountPath: /dev/shm
- name: postgresql-data
mountPath: /var/lib/postgresql
volumes:
- name: dshm
emptyDir:
medium: Memory
volumeClaimTemplates:
- metadata:
name: postgresql-data
spec:
storageClassName: postgresql-storage
accessModes:
- "ReadWriteOnce"
resources:
requests:
storage: "20Gi"
---
kind: Service
apiVersion: v1
metadata:
name: postgresql-svc
namespace: common
spec:
selector:
app: postgresql
type: ClusterIP
clusterIP: None
ports:
- name: tcp-postgresql
port: 5432
protocol: TCP
Couple of things I should probably mention:
I added my own Storage class for the purposes of retaining the record rather then delete on the record being removed.
using the alpine version just for the sake of smaller image size.
The base OS that k3s is hosted on is Ubuntu 20.04 LTS
K3s was setup using this ansible playbook here
I have exec-ed into the container to see what is in the directory defined by PGDATA and confirmed that data was written.

Redis pod failing

I have redis DB setup running on my minikube cluster. I have shutdown my minikube and started after 3 days and I can see my redis pod is failing to come up with below error from pod log
Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>.
Below is my Stateful Set yaml file for redis master deployed via a helm chart
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
meta.helm.sh/release-name: test-redis
meta.helm.sh/release-namespace: test
generation: 1
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: redis
helm.sh/chart: redis-14.8.11
name: test-redis-master
namespace: test
resourceVersion: "191902"
uid: 3a4e541f-154f-4c54-a379-63974d90089e
spec:
podManagementPolicy: OrderedReady
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/name: redis
serviceName: test-redis-headless
template:
metadata:
annotations:
checksum/configmap: dd1f90e0231e5f9ebd1f3f687d534d9ec53df571cba9c23274b749c01e5bc2bb
checksum/health: xxxxx
creationTimestamp: null
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: redis
helm.sh/chart: redis-14.8.11
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/name: redis
namespaces:
- tyk
topologyKey: kubernetes.io/hostname
weight: 1
containers:
- args:
- -c
- /opt/bitnami/scripts/start-scripts/start-master.sh
command:
- /bin/bash
env:
- name: BITNAMI_DEBUG
value: "false"
- name: REDIS_REPLICATION_MODE
value: master
- name: ALLOW_EMPTY_PASSWORD
value: "no"
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
key: redis-password
name: test-redis
- name: REDIS_TLS_ENABLED
value: "no"
- name: REDIS_PORT
value: "6379"
image: docker.io/bitnami/redis:6.2.5-debian-10-r11
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- sh
- -c
- /health/ping_liveness_local.sh 5
failureThreshold: 5
initialDelaySeconds: 20
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 6
name: redis
ports:
- containerPort: 6379
name: redis
protocol: TCP
readinessProbe:
exec:
command:
- sh
- -c
- /health/ping_readiness_local.sh 1
failureThreshold: 5
initialDelaySeconds: 20
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 2
resources: {}
securityContext:
runAsUser: 1001
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /opt/bitnami/scripts/start-scripts
name: start-scripts
- mountPath: /health
name: health
- mountPath: /data
name: redis-data
- mountPath: /opt/bitnami/redis/mounted-etc
name: config
- mountPath: /opt/bitnami/redis/etc/
name: redis-tmp-conf
- mountPath: /tmp
name: tmp
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1001
serviceAccount: test-redis
serviceAccountName: test-redis
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 493
name: test-redis-scripts
name: start-scripts
- configMap:
defaultMode: 493
name: test-redis-health
name: health
- configMap:
defaultMode: 420
name: test-redis-configuration
name: config
- emptyDir: {}
name: redis-tmp-conf
- emptyDir: {}
name: tmp
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/name: redis
name: redis-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 8Gi
volumeMode: Filesystem
status:
phase: Pending
Please let me know your suggestions on how can I fix this.
I am not an Redis expert but from what I can see:
kubectl describe pod red3-redis-master-0
...
Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>
...
Means that your appendonly.aof file was corrupted with invalid byte sequences in the middle.
How we can proceed if redis-master is not working?:
Verify pvc attached to the redis-master-pod:
kubectl get pvc
NAME STATUS VOLUME
redis-data-red3-redis-master-0 Bound pvc-cf59a0b2-a3ee-4f7f-9f07-8f4922518359
Create new redis-client pod wit the same pvc redis-data-red3-redis-master-0:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: redis-client
spec:
volumes:
- name: data
persistentVolumeClaim:
claimName: redis-data-red3-redis-master-0
containers:
- name: redis
image: docker.io/bitnami/redis:6.2.3-debian-10-r0
command: ["/bin/bash"]
args: ["-c", "sleep infinity"]
volumeMounts:
- mountPath: "/tmp"
name: data
EOF
Backup your files:
kubectl cp redis-client:/tmp .
Repair appendonly.aof file:
kubectl exec -it redis-client -- /bin/bash
cd /tmp
# make copy of appendonly.aof file:
cp appendonly.aof appendonly.aofbackup
# verify appendonly.aof file:
redis-check-aof appendonly.aof
...
0x 38: Expected prefix '*', got: '"'
AOF analyzed: size=62, ok_up_to=56, ok_up_to_line=13, diff=6
AOF is not valid. Use the --fix option to try fixing it.
...
# repair appendonly.aof file:
redis-check-aof --fix appendonly.aof
# compare files using diff:
diff appendonly.aof appendonly.aofbackup
Note:
As per docs:
The best thing to do is to run the redis-check-aof utility, initially without the --fix option, then understand the problem, jump at the given offset in the file, and see if it is possible to manually repair the file: the AOF uses the same format of the Redis protocol and is quite simple to fix manually. Otherwise it is possible to let the utility fix the file for us, but in that case all the AOF portion from the invalid part to the end of the file may be discarded, leading to a massive amount of data loss if the corruption happened to be in the initial part of the file.
In addition as described in the comments by #Miffa Young you can verify where your data is stored using k8s.io/minikube-hostpath provisioner:
kubectl get pv
...
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM
pvc-cf59a0b2-a3ee-4f7f-9f07-8f4922518359 8Gi RWO Delete Bound default/redis-data-red3-redis-master-0
...
kubectl describe pv pvc-cf59a0b2-a3ee-4f7f-9f07-8f4922518359
...
Source:
Type: HostPath (bare host directory volume)
Path: /tmp/hostpath-provisioner/default/redis-data-red3-redis-master-0
...
Your redis instance is failing down because your appendonly.aof is malformed and stored permanently under this location.
You can ssh into your vm:
minikube -p redis ssh
cd /tmp/hostpath-provisioner/default/redis-data-red3-redis-master-0
# from there you can backup/repair/remove your files:
Another solution is to install this chart using new name in this case new set of pv,pvc for redis StatefulSets will be created.
I think your redis is not quit Gracefully , so the AOF file is in a bad format What is AOF
you should repair aof file using a initcontainer by command (./redis-check-aof --fix .)
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
meta.helm.sh/release-name: test-redis
meta.helm.sh/release-namespace: test
generation: 1
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: redis
helm.sh/chart: redis-14.8.11
name: test-redis-master
namespace: test
resourceVersion: "191902"
uid: 3a4e541f-154f-4c54-a379-63974d90089e
spec:
podManagementPolicy: OrderedReady
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/name: redis
serviceName: test-redis-headless
template:
metadata:
annotations:
checksum/configmap: dd1f90e0231e5f9ebd1f3f687d534d9ec53df571cba9c23274b749c01e5bc2bb
checksum/health: xxxxx
creationTimestamp: null
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: redis
helm.sh/chart: redis-14.8.11
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/name: redis
namespaces:
- tyk
topologyKey: kubernetes.io/hostname
weight: 1
initContainers:
- name: repair-redis
image: docker.io/bitnami/redis:6.2.5-debian-10-r11
command: ['sh', '-c', "redis-check-aof --fix /data/appendonly.aof"]
containers:
- args:
- -c
- /opt/bitnami/scripts/start-scripts/start-master.sh
command:
- /bin/bash
env:
- name: BITNAMI_DEBUG
value: "false"
- name: REDIS_REPLICATION_MODE
value: master
- name: ALLOW_EMPTY_PASSWORD
value: "no"
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
key: redis-password
name: test-redis
- name: REDIS_TLS_ENABLED
value: "no"
- name: REDIS_PORT
value: "6379"
image: docker.io/bitnami/redis:6.2.5-debian-10-r11
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- sh
- -c
- /health/ping_liveness_local.sh 5
failureThreshold: 5
initialDelaySeconds: 20
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 6
name: redis
ports:
- containerPort: 6379
name: redis
protocol: TCP
readinessProbe:
exec:
command:
- sh
- -c
- /health/ping_readiness_local.sh 1
failureThreshold: 5
initialDelaySeconds: 20
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 2
resources: {}
securityContext:
runAsUser: 1001
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /opt/bitnami/scripts/start-scripts
name: start-scripts
- mountPath: /health
name: health
- mountPath: /data
name: redis-data
- mountPath: /opt/bitnami/redis/mounted-etc
name: config
- mountPath: /opt/bitnami/redis/etc/
name: redis-tmp-conf
- mountPath: /tmp
name: tmp
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1001
serviceAccount: test-redis
serviceAccountName: test-redis
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 493
name: test-redis-scripts
name: start-scripts
- configMap:
defaultMode: 493
name: test-redis-health
name: health
- configMap:
defaultMode: 420
name: test-redis-configuration
name: config
- emptyDir: {}
name: redis-tmp-conf
- emptyDir: {}
name: tmp
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: test-redis
app.kubernetes.io/name: redis
name: redis-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 8Gi
volumeMode: Filesystem

kubernetes StorageClass does not retain existing data

My Kubernetes StorageClass volume doesn't retain existing data when the pod is deleted and deployed back with my postgresql database. When I delete the pod, the new pod is created but the database is empty.
I have followed variations of the different versions of the tutorials (https://kubernetes.io/docs/concepts/storage/persistent-volumes/) but nothing seems to work.
I paste all the YAML files cause the problem might be in the combination.
storage-google.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: spingular-pvc
spec:
storageClassName: standard
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 7Gi
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: standard
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-standard
zone: us-east4-a
jhipsterpress-postgresql.yml
apiVersion: v1
kind: Secret
metadata:
name: jhipsterpress-postgresql
namespace: default
labels:
app: jhipsterpress-postgresql
type: Opaque
data:
postgres-password: NjY0NXJxd24=
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: jhipsterpress-postgresql
namespace: default
spec:
replicas: 1
template:
metadata:
labels:
app: jhipsterpress-postgresql
spec:
volumes:
- name: data
persistentVolumeClaim:
claimName: spingular-pvc
containers:
- name: postgres
image: postgres:10.4
env:
- name: POSTGRES_USER
value: jhipsterpress
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: jhipsterpress-postgresql
key: postgres-password
ports:
- containerPort: 5432
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/
---
apiVersion: v1
kind: Service
metadata:
name: jhipsterpress-postgresql
namespace: default
spec:
selector:
app: jhipsterpress-postgresql
ports:
- name: postgresqlport
port: 5432
type: LoadBalancer
jhipsterpress-deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jhipsterpress
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: jhipsterpress
version: "v1"
template:
metadata:
labels:
app: jhipsterpress
version: "v1"
spec:
initContainers:
- name: init-ds
image: busybox:latest
command:
- '/bin/sh'
- '-c'
- |
while true
do
rt=$(nc -z -w 1 jhipsterpress-postgresql 5432)
if [ $? -eq 0 ]; then
echo "DB is UP"
break
fi
echo "DB is not yet reachable;sleep for 10s before retry"
sleep 10
done
containers:
- name: jhipsterpress-app
image: galore/jhipsterpress
env:
- name: SPRING_PROFILES_ACTIVE
value: prod
- name: SPRING_DATASOURCE_URL
value: jdbc:postgresql://jhipsterpress-postgresql.default.svc.cluster.local:5432/jhipsterpress
- name: SPRING_DATASOURCE_USERNAME
value: jhipsterpress
- name: SPRING_DATASOURCE_PASSWORD
valueFrom:
secretKeyRef:
name: jhipsterpress-postgresql
key: postgres-password
- name: JAVA_OPTS
value: " -Xmx256m -Xms256m"
resources:
requests:
memory: "256Mi"
cpu: "500m"
limits:
memory: "512Mi"
cpu: "1"
ports:
- name: http
containerPort: 8080
readinessProbe:
httpGet:
path: /management/health
port: http
initialDelaySeconds: 20
periodSeconds: 15
failureThreshold: 6
livenessProbe:
httpGet:
path: /management/health
port: http
initialDelaySeconds: 120
jhipsterpress-service.yml
apiVersion: v1
kind: Service
metadata:
name: jhipsterpress
namespace: default
labels:
app: jhipsterpress
spec:
selector:
app: jhipsterpress
type: LoadBalancer
ports:
- name: http
port: 8080
When I included a Retain Policy I was getting this error:
#cloudshell:~ (academic-veld-230622)$ kubectl apply -f storage-google.yaml
error: error validating "storage-google.yaml": error validating data:
ValidationError(PersistentVolumeClaim.spec): unknown field "persistentVolumeReclaimPolicy" in io.k8s.api.core.v1.PersistentVolumeClaimSpec; if you choose to ignore these errors, turn validation off with --validate=false
Please, if you know of a complete example on a public image that works (in postgresql, I can make it work with Mongo), I will really appreciate it.
Thanks to all.
Note that for this to work you need to have your PVC dynamically provision a PV to satisfy its requirements, then there will be a permanent binding between the PVC and PV and every time your workload uses the PVC then it will use the same PV. Specifically indicated by this excerpt:
If a PV was dynamically provisioned for a new PVC, the loop will always bind that PV to the PVC
If in your case the Google Persistent Disk is being provisioned by the PVC, and you can verify that on GCP it's the same PV used every time, then it's probably an issue with the pod startup process where it's removing all the data. (Is there any reason why you are using /var/lib/postgresql/ vs /var/lib/postgresql?)
Also, persistentVolumeReclaimPolicy: Retain applies to a PV, not a PVC. For dynamically provisioned PVs the value is Delete. In your case, it wouldn't apply because your dynamically provisioned volume should be bound to your PVC. In other words, you are not reclaiming the volume.
Having said all that the recommended way to deploy a DB is using StatefulSets similar to this mysql example using a volumeClaimTemplate.

Kubernetes: Error kubectl edit deployment

I'm trying to edit deployment in kubernetes by:
kubectl -n <namespace> edit deployment <depolyment_name>.
after entering the command, vi windows for editing appears, then I make some changes for example in the command section or in volumeMounts section.
but I get the following error:
A copy of your changes has been stored to "/tmp/kubectl-edit-hv5dh.yaml"
error: map: map[] does not contain declared merge key: name
someone can help with it?
attached the edit deployment file of apiserver:
kubectl -n federation-system edit deployment apiserver
(codes between ** ** are the lines i added)
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
federation.alpha.kubernetes.io/federation-name: fed
creationTimestamp: 2018-04-01T13:26:40Z
generation: 1
labels:
app: federated-cluster
name: apiserver
namespace: federation-system
resourceVersion: "393140"
selfLink: /apis/extensions/v1beta1/namespaces/federation-system/deployments/apiserver
uid: <uid>
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: federated-cluster
module: federation-apiserver
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
federation.alpha.kubernetes.io/federation-name: fed
creationTimestamp: null
labels:
app: federated-cluster
module: federation-apiserver
name: apiserver
spec:
containers:
- command:
- /fcp
- federation-apiserver
- --admission-control=NamespaceLifecycle
- --advertise-address=<master-ip>
- --bind-address=0.0.0.0
- --client-ca-file=/etc/federation/apiserver/ca.crt
- --etcd-servers=http://localhost:2379
- --secure-port=8443
- --tls-cert-file=/etc/federation/apiserver/server.crt
- --tls-private-key-file=/etc/federation/apiserver/server.key
**- --enable-admission-plugins=SchedulingPolicy
- --admission-control-config-file=/etc/kubernetes/admission/config.yml**
image: gcr.io/k8s-jkns-e2e-gce-federation/fcp-amd64:v1.9.0-alpha.3
imagePullPolicy: IfNotPresent
name: apiserver
ports:
- containerPort: 8443
name: https
protocol: TCP
- containerPort: 8080
name: local
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/federation/apiserver
name: apiserver-credentials
readOnly: true
**volumeMounts:
- mountPath: /etc/kubernetes/admission
name: admission-config**
- command:
- /usr/local/bin/etcd
- --data-dir
- /var/etcd/data
image: gcr.io/google_containers/etcd:3.1.10
imagePullPolicy: IfNotPresent
name: etcd
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
imagePullSecrets:
- {}
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- name: apiserver-credentials
secret:
defaultMode: 420
secretName: apiserver-credentials
**- name: admission-config
configMap:
name: admission**
status:
availableReplicas: 1
conditions:
- lastTransitionTime: 2018-04-01T13:26:40Z
lastUpdateTime: 2018-04-01T13:26:40Z
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: 2018-04-01T13:26:40Z
lastUpdateTime: 2018-04-01T13:27:20Z
message: ReplicaSet "apiserver-8484fd45f8" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
it's happened after I created configMap file:
kubectl create -f scheduling-policy-admission.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: admission
namespace: federation-system
data:
config.yml: |
apiVersion: apiserver.k8s.io/v1alpha1
kind: AdmissionConfiguration
plugins:
- name: SchedulingPolicy
path: /etc/kubernetes/admission/scheduling-policy-config.yml
scheduling-policy-config.yml: |
kubeconfig: /etc/kubernetes/admission/opa-kubeconfig
opa-kubeconfig: |
clusters:
- name: opa-api
cluster:
server: http://opa.federation-system.svc.cluster.local:8181/v0/data/kubernetes/placement
users:
- name: scheduling-policy
user:
token: deadbeefsecret
contexts:
- name: default
context:
cluster: opa-api
user: scheduling-policy
current-context: default
I'm trying to configure Admission Controller in the Federation API.
Thanks,
dnsPolicy: ClusterFirst
# DELETE imagePullSecrets:
# DELETE - {}
restartPolicy: Always
I would strongly recommend removing that imagePullSecrets block. Since those objects have a mergeKey of name, but that object has no name, it would very easily cause the error you are experiencing. If the YAML was given to your editor in that condition, then I am almost certain that is a kubernetes bug: it should always(?) allow round-tripping YAML via kubectl edit, if for no other reason than this situation right here.