CloudRun: Debug authentication error from curl - kubernetes

Question

I am spinning up a container (pod/Job) from a GKE.
I have set up the appropriate Service Account on the cluster's VMs.
Therefore, when I manually perform a curl to a specific CloudRun service endpoint, I can perform the request (and get authorized and have 200 in my response)
However, when I try to automate this by setting an image to run in a Job as follows, I get 401
- name: pre-upgrade-job
image: "google/cloud-sdk"
args:
- curl
- -s
- -X
- GET
- -H
- "Authorization: Bearer $(gcloud auth print-identity-token)"
- https://my-cloud-run-endpoint
Here are the logs on Stackdriver
{
httpRequest: {
latency: "0s"
protocol: "HTTP/1.1"
remoteIp: "gdt3:r787:ff3:13:99:1234:avb:1f6b"
requestMethod: "GET"
requestSize: "313"
requestUrl: "https://my-cloud-run-endpoint"
serverIp: "212.45.313.83"
status: 401
userAgent: "curl/7.59.0"
}
insertId: "29jdnc39dhfbfb"
logName: "projects/my-gcp-project/logs/run.googleapis.com%2Frequests"
receiveTimestamp: "2019-09-26T16:27:30.681513204Z"
resource: {
labels: {
configuration_name: "my-cloud-run-service"
location: "us-east1"
project_id: "my-gcp-project"
revision_name: "my-cloudrun-service-d5dbd806-62e8-4b9c-8ab7-7d6f77fb73fb"
service_name: "my-cloud-run-service"
}
type: "cloud_run_revision"
}
severity: "WARNING"
textPayload: "The request was not authorized to invoke this service. Read more at https://cloud.google.com/run/docs/securing/authenticating"
timestamp: "2019-09-26T16:27:30.673565Z"
}
My question is how can I see if an "Authentication" header does reach the endpoint (the logs do not enlighten me much) and if it does, whether it is appropriately rendered upon image command/args invocation.

Answer 1

In your job you use this container google/cloud-sdk which is a from scratch installation of gcloud tooling. It's generic, without any customization.
When you call this $(gcloud auth print-identity-token) you ask for the identity token of the service account configured in the gcloud tool.
If we put together this 2 paragraphs, you want to generate an identity token from a generic/blank installation of gcloud tool. By the way, you don't have defined service account in your gcloud and your token is empty (like #johnhanley said).
For solving this issue, add an environment variable like this
env:
- GOOGLE_APPLICATION_CREDENTIAL=<path to your credential.json>
I don't know where is your current credential.json of your running environment. Try to perform an echo of this env var to find it and forward it correctly to your gcloud job.
If you are on compute engine or similar system compliant with metadata server, you can get a correct token with this command:
curl -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=<URL of your service>"
UPDATE
Try to run your command outside of the gcloud container. Here the update of your job
- name: pre-upgrade-job
image: "google/cloud-sdk"
entrypoint: "bash"
args:
- -c
- "curl -s -X GET -H \"Authorization: Bearer $(gcloud auth print-identity-token)\" https://my-cloud-run-endpoint"
Not sure that works. Let me know

Answer 2

In your Job, gcloud auth print-identity-token likely does not return any tocken.
The reason is that locally, gcloud uses your identity to mint a token, but in a Job, you are not logged into gcloud.