IBM File Storage on Kubernetes stuck "Pending" - kubernetes

Question

I am trying to use the following https://cloud.ibm.com/docs/containers?topic=containers-file_storage#add_file:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ibmc-file
labels:
billingType: 'monthly'
region: us-south
zone: dal10
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 12Gi
storageClassName: ibmc-file-silver
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: postgres
spec:
replicas: 1
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:11
imagePullPolicy: Always
ports:
- containerPort: 5432
envFrom:
- configMapRef:
name: postgres-config
volumeMounts:
- name: postgres-storage
mountPath: /var/lib/postgresql/data
volumes:
- name: postgres-storage
persistentVolumeClaim:
claimName: ibmc-file
But the PVC is never "Bound" and gets stuck as "Pending".
➜ postgres-kubernetes kubectl describe pvc ibmc-file
Name: ibmc-file
Namespace: default
StorageClass: ibmc-file-silver
Status: Pending
Volume:
Labels: billingType=monthly
region=us-south
zone=dal10
Annotations: ibm.io/provisioning-status=failed: Storage creation failed with error: {Code:E0013, Description:User doesn't have permissions to create or manage Storage [Backend Error:Validation failed due to missin...
kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"labels":{"billingType":"monthly","region":"us-south","zone":"dal10"},"n...
volume.beta.kubernetes.io/storage-provisioner=ibm.io/ibmc-file
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Provisioning 10m (x3 over 10m) ibm.io/ibmc-file_ibm-file-plugin-5d7684d8c5-xlvks_db50c480-500f-11e9-ba08-cae91657b92d External provisioner is provisioning volume for claim "default/ibmc-file"
Warning ProvisioningFailed 10m (x3 over 10m) ibm.io/ibmc-file_ibm-file-plugin-5d7684d8c5-xlvks_db50c480-500f-11e9-ba08-cae91657b92d failed to provision volume with StorageClass "ibmc-file-silver": Storage creation failed with error: {Code:E0013, Description:User doesn't have permissions to create or manage Storage [Backend Error:Validation failed due to missing permissions[NAS_MANAGE] for User[id:xxx, name:xxxm_2018-11-20-07.35.49, email:xxx, account:xxx]], Type:MissingStoragePermissions, RC:401, Recommended Action(s):Run `ibmcloud ks api-key-info` to see the owner of the API key that is used to order storage. Then, contact the account administrator to add the missing storage permissions. If infrastructure credentials were manually set via `ibmcloud ks credentials-set`, check the permissions of that user. Delete the PVC and re-create it. If the problem persists, open an IBM Cloud support case.}
Normal ExternalProvisioning 7m (x22 over 10m) persistentvolume-controller waiting for a volume to be created, either by external provisioner "ibm.io/ibmc-file" or manually created by system administrator
Normal ExternalProvisioning 11s (x26 over 6m) persistentvolume-controller waiting for a volume to be created, either by external provisioner "ibm.io/ibmc-file" or manually created by system administrator

Answer 1

#atkayla Could you try running kubectl get secret storage-secret-store -n kube-system -o yaml | grep slclient.toml: | awk '{print $2}' | base64 --decode to see what API key is used in the storage secret store? If this also shows your name and email address, then the file storage plug-in uses the permissions that are assigned to you.
You might have the permissions to create the cluster, but you might lack some storage permissions that do not let you create the storage. Are you the owner of the account and have the possibility to check the permissions? You should have Add/Upgrade Storage (StorageLayer), and Storage Manage.
If you do not have these permissions, add these and then run ibmcloud ks api-key-set to update the API key. The storage secret store is automatically refreshed after 5-15 minutes. Then, you can try again.