I have created Service accounts in GCP Console IAM & Admin > Service Accounts >> Create
Assigned Storage Admin, Storage Object Admin, Storage Object Creator
Created Key and downloaded json file
To test this service account I ran this command on GCP VM
$ gcloud auth activate-service-account --key-file service-account.json
Activated service account credentials for: [ingester#xxx-123.iam.gserviceaccount.com]
Then I was able to cp file to storage bucket
$ gsutil cp image-src.txt gs://bucket-images
Copying file://image-src.txt [Content-Type=text/plain]...
/ [1 files][ 19.5 KiB/ 19.5 KiB]
Operation completed over 1 objects/19.5 KiB.
But when I try to use service-account.json with node.js app I get
(node:1) UnhandledPromiseRejectionWarning: Error: ingester#xxx-123.iam.gserviceaccount.com does not have storage.objects.create access to the Google Cloud Storage object.
at new ApiError (/ingester/node_modules/#google-cloud/common/build/src/util.js:59:15)
at Util.parseHttpRespBody (/ingester/node_modules/#google-cloud/common/build/src/util.js:194:38)
at Util.handleResp (/ingester/node_modules/#google-cloud/common/build/src/util.js:135:117)
at retryRequest (/ingester/node_modules/#google-cloud/common/build/src/util.js:434:22)
at onResponse (/ingester/node_modules/retry-request/index.js:206:7)
at /ingester/node_modules/teeny-request/build/src/index.js:170:17
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 5)
Am i creating service account correctly ?
Update : After v1gnus answer
I tried to get
cloudshell:~ $ gcloud iam service-accounts get-iam-policy ingester#xxx-123.iam.gserviceaccount.com
etag: ACAB
Then I tried to set policy
cloudshell:~ $ gcloud projects add-iam-policy-binding xxx-123 --member serviceAccount:ingester#xxx-123.ia
m.gserviceaccount.com --role roles/storage.admin
Updated IAM policy for project [xxx-123].
bindings:
...
- members:
- serviceAccount:ingester#xxx-123.iam.gserviceaccount.com
role: roles/storage.admin
- members:
- serviceAccount:ingester#xxx-123.iam.gserviceaccount.com
role: roles/storage.objectAdmin
- members:
- serviceAccount:ingester#xxx-123.iam.gserviceaccount.com
role: roles/storage.objectCreator
etag: rfvtgyh-4Gw=
version: 1
After this I still get this
cloudshell:~ $ gcloud iam service-accounts get-iam-policy ingester#xxx-123.iam.gserviceaccount.com
etag: ACAB
It seems you have an issue with the permissions on your service account
“Error: ingester#xxx-123.iam.gserviceaccount.com does not have storage.objects.create access to the Google Cloud Storage object.”
You can view which role has been assigned to your Service Account by typing this command:
gcloud iam service-accounts get-iam-policy ingester#xxx-123.iam.gserviceaccount.com
For more information on how to use Google Cloud Storage Node.js Client API check this link
Related
Is there any difference between a key that I create using the gcloud iam command below vs. going thru the console to create a json key? Each results in a .json file that other than obvious differences in the private_key_id and private_key values are identical. Here are the gcloud commands I am using:
gcloud iam service-accounts create my-sa-name
gcloud projects add-iam-policy-binding my-project-id --member="serviceAccount:my-sa-name#my-project-id.iam.gserviceaccount.com" --role="roles/owner"
gcloud iam service-accounts keys create key.json --iam-account=serviceAccount:my-sa-name#my-project-id.iam.gserviceaccount.com
However, when I try and leverage the one pulled down thru the command line, I get:
google.auth.exceptions.RefreshError: ('invalid_grant: Invalid JWT Signature.', '{"error":"invalid_grant","error_description":"Invalid JWT Signature."}')
Oddly if I go to the console, create a key for the same service account, and put the file that downloads to my computer in place of the one from the CLI, all works fine.
How am I using the key you ask? I'm using the function-frameworks to locally run and debug a cloud function that will access a cloud storage bucket, so my code is using the google.cloud.storage client library (python 3.8). I run the local process with the environment variable GOOGLE_APPLICATION_CREDENTIALS set to the location/filename of the json key file. I know this all works fine b/c of the console-downloaded key working fine.
I have also tried using gcloud auth service-account --key-file=key.json and this also gives me the Invalid JWT Signature error.
Fortunately I'm not blocked b/c I can use the manually-created key, but I would REALLY like to automate every possible step here...
So... can anyone explain this? Seen it, figured it out and know how to fix it?
gcloud iam service-accounts keys create key.json \
--iam-account=my-sa-name#my-project-id.iam.gserviceaccount.com
One way to clarify this is:
ACCOUNT="[[YOUR-ACCOUNT]]"
PROJECT="[[YOUR-PROJECT]]"
EMAIL="${ACCOUNT}#${PROJECT}.iam.gserviceaccount.com"
gcloud iam service-accounts create ${ACCOUNT} \
--project=${PROJECT}
gcloud projects add-iam-policy-binding ${PROJECT} \
--member="serviceAccount:${EMAIL}" \
--role="roles/owner"
gcloud iam service-accounts keys create ${ACCOUNT}.json \
--iam-account=${EMAIL} \
--project=${PROJECT}
I'm working through the example at this URL, but I'm running into the error below. What is the source of that error?
ERROR: (gcloud.run.services.add-iam-policy-binding) NOT_FOUND: Requested entity was not found.
I'm running the script below, but obviously with a different parameter set. I verified in the GUI/Console that the service account was created, the $serviceAccount variable has the correct value, the service exists, and the service name is correct. Also, I'm able to manually add the service account to the service in the invoker role in the GUI/Console.
param(
$cloudRunInvokerServiceAccount = 'some-service-account',
$projectId = 'some-project-id',
$nameOfServiceBeingInvoked = 'some-service'
)
gcloud iam service-accounts create $cloudRunInvokerServiceAccount --display-name $cloudRunInvokerServiceAccount
$serviceAccount = "serviceAccount:$cloudRunInvokerServiceAccount#$projectId.iam.gserviceaccount.com"
gcloud run services add-iam-policy-binding $nameOfServiceBeingInvoked `
--member=$serviceAccount `
--role=roles/run.invoker `
--platform=managed
I also reviewed the docs for good measure.
Also, I ran gcloud run services list and gcloud iam service-accounts list to verify the existence of the services/accounts and their status (enabled).
I have seen this error with old versions of the Google Cloud CLI gcloud.
The solution is to update the Google Cloud SDK and components.
Windows:
Open an elevated Windows Commmand Prompt.
Execute this command and follow the prompts: gcloud components update
Linux:
Execute this command and follow the prompts: sudo gcloud components update
During the startup of my server I am getting the following error:
StorageException: server#<project>.iam.gserviceaccount.com does not have storage.buckets.create access to project <project-id>.
From here the docs I understand that adding Storage Object Creator the the roles of my service account should be sufficient to get the storage.buckets.create permissions.
However, the exception above tells my I am missing something.
The Object Creator role is for creating Storage objects not buckets. To administrate buckets you need the Storage Admin role.
To create that Service Account from the Cloud Shell:
gcloud iam service-accounts create buckets-sa \
--description "Storage admin Service account" \
--display-name "buckets-sa"
# Main role to create Buckets in Google Storage
gcloud projects add-iam-policy-binding [PROJECT] \
--member serviceAccount:buckets-sa#[PROJECT].iam.gserviceaccount.com \
--role roles/storage.admin
# Role for testing: Service Account Token Creator
gcloud projects add-iam-policy-binding [PROJECT] \
--member serviceAccount:buckets-sa#[PROJECT].iam.gserviceaccount.com \
--role roles/iam.serviceAccountTokenCreator
# Create Key
gcloud iam service-accounts keys create key-file.json \
--iam-account buckets-sa#[PROJECT].iam.gserviceaccount.com
# Test: Impersonate service account
gcloud auth activate-service-account buckets-sa#[PROJECT].iam.gserviceaccount.com --key-file=key-file.json
gsutil -i "buckets-sa#[PROJECT].iam.gserviceaccount.com" mb gs://new-bucket
# To restore your account in Cloud Shell uncomment execute the following line.
# gcloud auth login
How do I connect to the database on Cloud SQL without having to add my credentials file inside the container?
UPDATE: to connect to Cloud SQL from Cloud Run see the official documentation
Cloud SQL is now supported by the fully managed version of Cloud Run (Cloud Run on GKE users were already able to use Cloud SQL using a private IP)
To get started:
if you do not have one already, create a Cloud SQL instance.
make sure that the Cloud SQL admin API is enabled
deploy a new revision of your Cloud Run service with gcloud alpha and the following flag:
$ gcloud run services update --add-cloudsql-instances [INSTANCE_CONNECTION_NAME]
Where is INSTANCE_CONNECTION_NAME is of the type project:region:instancename.
When you do this, Cloud Run will activate and configure the Cloud SQL proxy for you. You should then connect to it via the /cloudsql/[INSTANCE_CONNECTION_NAME] Unix socket.
CONNECTING FROM CLOUD RUN (fully managed) TO CLOUD SQL USING UNIX DOMAIN SOCKETS (Java)
At this time Cloud Run (fully managed) does not support connecting to
the Cloud SQL instance using TCP. Your code should not try to access the instance
using an IP address such as 127.0.0.1 or 172.17.0.1.
link
1.Install and initialize the Cloud SDK
2.Update components:
gcloud components update
3.Create a new project
gcloud projects create run-to-sql
gcloud config set project run-to-sql
gcloud projects describe run-to-sql
4.Enable billing
gcloud alpha billing projects link run-to-sql --billing-account XXXXXX-XXXXXX-XXXX
5.Set the compute project-info metadata:
gcloud compute project-info describe --project run-to-sql
gcloud compute project-info add-metadata --metadata google-compute-default-region=europe-west2,google-compute-default-zone=europe-west2-b
6.Enable the Cloud SQL Admin API:
gcloud services enable sqladmin.googleapis.com
7.Create a Cloud SQL instance with public Ip
#Create the sql instance in the same region as App Engine Application
gcloud --project=run-to-sql beta sql instances create database-external --region=europe-west2
#Set the password for the "root#%" MySQL user:
gcloud sql users set-password root --host=% --instance database-external --password root
#Create a user
gcloud sql users create user_name --host=% --instance=database-external --password=user_password
#Create a database
gcloud sql databases create user_database --instance=database-external
gcloud sql databases list --instance=database-external
gcloud sql instances list
Cloud Run (fully managed) uses a service account to authorize your
connections to Cloud SQL. This service account must have the correct
IAM permissions to successfully connect. Unless otherwise configured,
the default service account is in the format
PROJECT_NUMBER-compute#developer.gserviceaccount.com.
8.Ensure that the service account for your service has one of the following IAM roles:Cloud SQL Client (preferred)
gcloud iam service-accounts list
gcloud projects add-iam-policy-binding run-to-sql --member serviceAccount:PROJECT_NUMBER-compute#developer.gserviceaccount.com. --role roles/cloudsql.client
9.Clone the java-docs-repository
git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git
cd java-docs-samples/cloud-sql/mysql/servlet/
ls
#Dockerfile pom.xml README.md src
10.Inspect the file that handle the connection to Cloud SQL
cat src/main/java/com/example/cloudsql/ConnectionPoolContextListener.java
11.Containerizing the app and uploading it to Container Registry
gcloud builds submit --tag gcr.io/run-to-sql/run-mysql
12.Deploy the service to Cloud Run
gcloud run deploy run-mysql --image gcr.io/run-to-sql/run-mysql
13.Configure the service for use with Cloud Run
gcloud run services update run-mysql --add-cloudsql-instances run-to-sql:europe-west2:database-external --set-env-vars CLOUD_SQL_CONNECTION_NAME=run-to-sql:europe-west2:database-external DB_USER=user_name,DB_PASS=user_password,DB_NAME=user_database
14.Test it
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://run-mysql-xxxxxxxx-xx.x.run.app
SUCCESS!
I was facing an issue with connecting from a dockerized FastApi application to CloudSQL via private ip. I took the following 3 steps to resolve my issue:
Ensure your application is utilizing the proper database-connection-string.
Sanity check, always do this first. You don't want to spend hours researching a solution without first ruling out a wrong connection string.
When testing (and only when testing): consider logging the db connection string on app init so you can explicitly confirm your connection string is correct.
Provide the Cloud SQL Client role to my cloudrun default service account.
Contains the following permissions:
cloudsql.instances.connect
cloudsql.instances.get
Create a VPC connector within the network of the database (documentation). And assign the VPC connector to the Cloud Run service.
I'm able to login successfully with : ibmcloud cr login
but when i try to create a repository in the registry, i have the following error :
docker push registry.eu-gb.bluemix.net/fdutreg/ksrepo
The push refers to repository [registry.eu-gb.bluemix.net/fdutreg/ksrepo]
428c97da766c: Preparing
unauthorized: The login credentials are not valid, or your IBM Cloud account is not active.
Any idea ?
Replace registry.eu-gb.bluemix.net by registry.eu-de.bluemix.net and now this is ok.
2 years later but probably someone could be experimenting the same issue. The thing is that you are not authenticate to the registry. You can authenticate with an API key using:
docker login -u iamapikey -p apikey registry_url
For the apikey field you can create an apikey in Manage > IAM > APIkeys > Create an IBM Cloud API key in ibm.cloud.com
It is important to know that Using --password via the CLI is insecure. Use --password-stdin. You can find alternatives in https://cloud.ibm.com/docs/Registry?topic=Registry-registry_access
Log your local Docker daemon into the IBM Cloud Container Registry.run the following command:
ibmcloud cr login