Issue mounting Cloud SQL credentials to a container in Kubernetes - postgresql

Question

I'm attempting to migrate over to Cloud SQL (Postgres). I have the following deployment in Kubernetes, having followed these instructions https://cloud.google.com/sql/docs/mysql/connect-container-engine :
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: menu-service
spec:
replicas: 1
template:
metadata:
labels:
app: menu-service
spec:
volumes:
- name: cloudsql-instance-credentials
secret:
secretName: cloudsql-instance-credentials
- name: cloudsql
emptyDir:
- name: ssl-certs
hostPath:
path: /etc/ssl/certs
containers:
- image: gcr.io/cloudsql-docker/gce-proxy:1.11
name: cloudsql-proxy
command: ["/cloud_sql_proxy", "--dir=/cloudsql",
"-instances=tabb-168314:europe-west2:production=tcp:5432",
"-credential_file=/secrets/cloudsql/credentials.json"]
volumeMounts:
- name: cloudsql-instance-credentials
mountPath: /secrets/cloudsql
readOnly: true
- name: ssl-certs
mountPath: /etc/ssl/certs
- name: cloudsql
mountPath: /cloudsql
- name: menu-service
image: eu.gcr.io/tabb-168314/menu-service:develop
imagePullPolicy: Always
env:
- name: MICRO_BROKER
value: "nats"
- name: MICRO_BROKER_ADDRESS
value: "nats.staging:4222"
- name: MICRO_REGISTRY
value: "kubernetes"
- name: ENV
value: "staging"
- name: PORT
value: "8080"
- name: POSTGRES_HOST
value: "127.0.0.1:5432"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: cloudsql-db-credentials
key: username
- name: POSTGRES_PASS
valueFrom:
secretKeyRef:
name: cloudsql-db-credentials
key: password
- name: POSTGRES_DB
value: "menus"
ports:
- containerPort: 8080
But unfortunately I'm getting this error when trying to update the deployment:
MountVolume.SetUp failed for volume "kubernetes.io/secret/69b0ec99-baaf-11e7-82b8-42010a84010c-cloudsql-instance-credentials" (spec.Name: "cloudsql-instance-credentials") pod "69b0ec99-baaf-11e7-82b8-42010a84010c" (UID: "69b0ec99-baaf-11e7-82b8-42010a84010c") with: secrets "cloudsql-instance-credentials" not found
Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "staging"/"menu-service-1982520680-qzwzn". list of unattached/unmounted volumes=[cloudsql-instance-credentials]
Have I missed something here?

Answer 1

You are missing (at least) one of the secrets needed to start up this Pod, namely cloudsql-instance-credentials.
From https://cloud.google.com/sql/docs/mysql/connect-container-engine:
You need two secrets to enable your Container Engine application to access the data in your Cloud SQL instance:
The cloudsql-instance-credentials secret contains the service account.
The cloudsql-db-credentials secret provides the proxy user account and password. (I think you have this created, I can't see an error message about this one)
To create your secrets:
Create the secret containing the Service Account which enables authentication to Cloud SQL:
kubectl create secret generic cloudsql-instance-credentials \
--from-file=credentials.json=[PROXY_KEY_FILE_PATH]
[...]
The link above also describes how to create a GCP service account for this purpose, if you don't have one created already.