How to use private registry provider, Service Account - from Kubernertes deployments - kubernetes

Question

Update I suspect this to be a google issue, I have created a new more clean question here.
Update: yes this is different than the suggested "This question may already have an answer here:", as this is about a "Service Account" - not a "User accounts".
Do you now how to use a private registry like Google Container Registry from DigitalOcean or any other Kubernetes not running on the same provider?
I tried following this, but unfortunately it did not work for me.
Update: I suspect it to be a Google SA issue, I will go and try using Docker Hub and get back if that succeeds. I am still curious to see the solution for this, so please let me know - thanks!
Update: Also tried this
Update: tried to activate Google Service Account
Update: tried to download Google Service Account key
Update: in the linked description is says:
kubectl create secret docker-registry $SECRETNAME \
--docker-server=https://gcr.io \
--docker-username=_json_key \
--docker-email=user#example.com \
--docker-password="$(cat k8s-gcr-auth-ro.json)"
Is the --docker-password="$(cat k8s-gcr-auth-ro.json)" really the password?
If I do cat k8s-gcr-auth-ro.json the format is:
{
"type": "service_account",
"project_id": "<xxx>",
"private_key_id": "<xxx>",
"private_key": "-----BEGIN PRIVATE KEY-----\<xxx>\n-----END PRIVATE KEY-----\n",
"client_email": "k8s-gcr-auth-ro#<xxx>.iam.gserviceaccount.com",
"client_id": "<xxx>",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/k8s-gcr-auth-ro%<xxx>.iam.gserviceaccount.com"
}
kubectl get pods
I get: ...is waiting to start: image can't be pulled
from a deployment with:
image: gcr.io/<project name>/<image name>:v1
deployment.yaml
# K8s - Deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: <image-name>-deployment-v1
spec:
replicas: 1
template:
metadata:
labels:
app: <image-name>-deployment
version: v1
spec:
containers:
- name: <image-name>
image: gcr.io/<project-name>/<image-name>:v1
imagePullPolicy: Always
ports:
- containerPort: 80
imagePullSecrets:
- name: <name-of-secret>
I can see from the following that it logs: repository does not exist or may require 'docker login'
kubectl describe pod :
k describe pod <image-name>-deployment-v1-844568c768-5b2rt
Name: <image-name>-deployment-v1-844568c768-5b2rt
Namespace: default
Priority: 0
PriorityClassName: <none>
Node: my-cluster-digitalocean-1-7781/10.135.153.236
Start Time: Mon, 25 Mar 2019 15:51:37 +0100
Labels: app=<image-name>-deployment
pod-template-hash=844568c768
version=v1
Annotations: <none>
Status: Pending
IP: <ip address>
Controlled By: ReplicaSet/<image-name>-deployment-v1-844568c768
Containers:
chat-server:
Container ID:
Image: gcr.io/<project-name/<image-name>:v1
Image ID:
Port: 80/TCP
Host Port: 0/TCP
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-dh8dh (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-dh8dh:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-dh8dh
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 50s default-scheduler Successfully assigned default/<image-name>-deployment-v1-844568c768-5b2rt to my-cluster-digitalocean-1-7781
Normal Pulling 37s (x2 over 48s) kubelet, my-cluster-digitalocean-1-7781 pulling image "gcr.io/<project-name><image-name>:v1"
Warning Failed 37s (x2 over 48s) kubelet, my-cluster-digitalocean-1-7781 Failed to pull image "gcr.io/<project-name>/<image-name>:v1": rpc error: code = Unknown desc = Error response from daemon: pull access denied for gcr.io/<project-name>/<image-name>, repository does not exist or may require 'docker login'
Warning Failed 37s (x2 over 48s) kubelet, my-cluster-digitalocean-1-7781 Error: ErrImagePull
Normal SandboxChanged 31s (x7 over 47s) kubelet, my-cluster-digitalocean-1-7781 Pod sandbox changed, it will be killed and re-created.
Normal BackOff 29s (x6 over 45s) kubelet, my-cluster-digitalocean-1-7781 Back-off pulling image "gcr.io/<project-name>/<image-name>:v1"
Warning Failed 29s (x6 over 45s) kubelet, my-cluster-digitalocean-1-7781 Error: ImagePullBackOff
Just a note: docker pull on local machine pulls the image alright