I am trying to submit a build using this command in the gcloud CLI:
gcloud builds submit --config cloudbuild.yaml .
but it returns this error:
(gcloud.builds.submit) The user is forbidden from accessing the bucket
[fastapi-api_cloudbuild]. Please check your organization's policy or
if the user has the "serviceusage.services.use" permission
I checked in the IAM admin panel and it says that my account has the owner role and that i have most permissions, and i authenticated in the gcloud cli, but i still get said error, i havent tried much since i have no idea what i could try since i already have the highest role. thank you.
Have you set your project in the cli with gcloud config set project <project>?
Otherwise the answer from this thread might help (billing).
or this one with Storage Admin
I had a similar issues (caused by the change of the billing account), and I fixed it by disabling Google Cloud Build API and re-enabling.
I used this commands:
gcloud services disable cloudbuild.googleapis.com --project "my_project"
gcloud services enable cloudbuild.googleapis.com --project "my_project"
Related
I'm trying to use terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster module of version "~> 7.3" to create a cluster in region europe-west2. But I keep getting error:
Error: googleapi: Error 403: Google Compute Engine: Required 'compute.regions.get'
permission for 'projects/***/regions/europe-west2'., forbidden
The weird thing is that I'm trying to do that using user with Editor role. So, it should have permission to read region. I tried to add more roles for the user (made it all kind of admins), but the result is still the same.
Could you please advice, where can be my mistake?
Please check if your [id]#cloudservices.gserviceaccount.com service account has the editor role.
List all service accounts with gcloud projects get-iam-policy [project-id] command and look for the account. it should look similar to this:
- members:
- serviceAccount:67993345594-compute#developer.gserviceaccount.com
- serviceAccount:679934532594#cloudservices.gserviceaccount.com
- serviceAccount:service-674567382594#containerregistry.iam.gserviceaccount.com
- serviceAccount:test2-468#asdf.iam.gserviceaccount.com
- serviceAccount:asdf#appspot.gserviceaccount.com
role: roles/editor
Second from the top is the account you're looking for and the bottom line says "roles/editor" which is the correct situation.
If this account doesn't have this role you can grant it using command:
gcloud projects add-iam-policy-binding [project] /
--member serviceAccount:[id]#cloudservices.gserviceaccount.com --role roles/editor
It's all described in the documentation.
Very similar issues were discussed on StackOverflow here and here.
I managed to solve that problem. It appeared that my provisioner dropped some roles, which are required for GKE to work properly. In particular,
serviceAccount:service-${project-number}#compute-system.iam.gserviceaccount.com must be roles/compute.serviceAgent.
serviceAccount:service-${project-number}#container-engine-robot.iam.gserviceaccount.com must be roles/compute.serviceAgent.
To find that I disabled Kubernetes engine service and enabled it back and google cloud automatically recovered required roles for that service accounts.
I had the same problem after switching account in gcloud.
I solve it by running
gcloud auth application-default login
It sets application default credentials for Packer/Terraform to call Google APIs.
source: https://cloud.google.com/sdk/gcloud/reference/auth/application-default
I am trying to integrate CircleCi with gcloud Kubernetes engine.
I created a service account with Kubernetes Engine Developer and Storage Admin roles.
Created CircleCi yaml file and configured CI.
Part of my yaml file includes:
docker:
- image: google/cloud-sdk
environment:
- PROJECT_NAME: 'my-project'
- GOOGLE_PROJECT_ID: 'my-project-112233'
- GOOGLE_COMPUTE_ZONE: 'us-central1-a'
- GOOGLE_CLUSTER_NAME: 'my-project-bed'
steps:
- checkout
- run:
name: Setup Google Cloud SDK
command: |
apt-get install -qq -y gettext
echo $GCLOUD_SERVICE_KEY > ${HOME}/gcloud-service-key.json
gcloud auth activate-service-account --key-file=${HOME}/gcloud-service-key.json
gcloud --quiet config set project ${GOOGLE_PROJECT_ID}
gcloud --quiet config set compute/zone ${GOOGLE_COMPUTE_ZONE}
gcloud --quiet container clusters get-credentials ${GOOGLE_CLUSTER_NAME}
Everything runs perfectly except that the last command:
gcloud --quiet container clusters get-credentials ${GOOGLE_CLUSTER_NAME}
It keeps failing with the error:
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/my-project-112233/zones/us-central1-a/clusters/my-project-bed". See https://cloud.google.com/kubernetes-engine/docs/troubleshooting#gke_service_account_deleted for more info.
I tried to give the ci account the role of project owner but I still got that error.
I tried to disable and re-enable the Kubernetes Service but it didn't help.
Any idea how to solve this? I am trying to solve it for 4 days...
This is an old thread, this is how this issue handled today in case using cloud build :
Granting Cloud Build access to GKE
To deploy the application in your Kubernetes cluster, Cloud Build needs the Kubernetes Engine Developer Identity and Access Management Role.
Get Project Number:
PROJECT_NUMBER="$(gcloud projects describe ${PROJECT_ID} --format='get(projectNumber)')"
Add IAM Policy bindings:
gcloud projects add-iam-policy-binding ${PROJECT_NUMBER} \
--member=serviceAccount:${PROJECT_NUMBER}#cloudbuild.gserviceaccount.com \
--role=roles/container.developer
More info can be found here.
I believe it's not the CI Service account but the k8s service account used to manage your GKE cluster, where its email should look like this (Somebody must have deleted it):
k8s-service-account#<project-id>.iam.gserviceaccount.com
You can re-create it an give it project owner permissions.
Step 1 : gcloud init
Step 2 : Select [2] Create a new configuration
Step 3 : Enter configuration name. Names start with a lower case letter and
contain only lower case letters a-z, digits 0-9, and hyphens '-': kubernetes-service-account
Step 4 : Choose the account you would like to use to perform operations for
this configuration:[2] Log in with a new account
Step 5 : Do you want to continue (Y/n)? y
Step 6 : Copy paste the link to brwoser and login with the ID which is used to create your google Cloud Account
Step 7 : Copy the verification code provided by google after login and paste it in to the console.
Step 8 : Pick cloud project to use:
Step 9: Do you want to configure a default Compute Region and Zone? (Y/n)? y
Step 10 : Please enter numeric choice or text value (must exactly match list item): 8
Your Google Cloud SDK is configured and ready to use!
The details of the above mentioned errors are explained in this help center article.
To add the Kubernetes Engine Service account (if you don't have it), please run the following command, in order to properly recreate the Kubernetes Service Account with the "Kubernetes Engine Service Agent" role,
gcloud services enable container.googleapis.com
In my case, these 2 steps solved my issue:
In the command,
gcloud container clusters get-credentials my-cluster-1 --zone=asia-south1-a --
project=thelab-240901
the --project should have the projectID value, not the project name
In the your travis account, go to your project repository -> more options -> settings -> Environment Variables. Now make sure you have only one set of encrypted_iv and encrypted_key environment variables as follows:
If you have encrypted different service accounts (json key files), this could add more than one set of encrypted_iv and encrypted_key environment variables. So, delete all those variables and freshly create the encrypted keys, may be by using travis encrypt-file --pro yourServiceAccountJSONFile.json --add
I had this problem using gcloud with my main owner account (!)
What fixed it was including --zone and --project params in the command to get the kubectl credentials.
I faced this issue with different scenarios, Listing it down below hope it will help someone.
1. If you did a fresh installation of google-cloud-sdk then you must login
with gcloud using the below command.
gcloud auth login
The above command will open ur browser and ask for login with your GCP account.
2. Sometimes provisioning is not reflected. Hence i revoke my provision and granted access (In this case my role is Owner) again. Then it worked.
I was getting the same error when trying to connect to my newly created cluster:
gcloud container clusters get-credentials <foo-cluster> --zone europe-central2-a --project <foo-project>
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/foo-project/zones/europe-central2-a/clusters/foo-cluster".
I tried a few things:
I enabled Kuberentes API - no success
I added a key to service account and loged in using downloaded key:
gcloud auth activate-service-account --key-file=<path-to-the-downloaded-json-file>/foo-project-xxxx.json
Activated service account credentials for: [xxxx-compute#developer.gserviceaccount.com]
I run:
gcloud components update
However, I had a problem retriving data, all kubectl command were giving TLS handshake timeout, for example: kubectl get namespace was giving an error:
Unable to connect to the server: net/http: TLS handshake timeout
This is when I tried again:
gcloud container clusters get-credentials <foo-cluster> --zone europe-central2-a --project <foo-project>
and it worked
Fetching cluster endpoint and auth data.
kubeconfig entry generated for foo-project.
I implemented a few tasks with BashOperator. Ones with "gsutil rm" and "gsutil cp" worked fine. But one with "gcloud alpha firestore export" generates this error:
{bash_operator.py:101} INFO - ERROR: (gcloud.alpha.firestore.export) PERMISSION_DENIED: The caller does not have permission
This command itself works fine in gcloud shell. I tried to give some Firestore related permissions to the service account used by the Composer but it still doesn't work. Any idea
It might be that you don't have permissions for a particular project.
The error I was getting was: PERMISSION_DENIED: Caller does not have required permission to use project project:random-id-11111.
The way I resolved it was by running gcloud config set project 'the-right-project-id' and then the actual gcloud command.
I think you need Cloud Datastore Import Export access. Following are the steps as per current Current Google Cloud platform layout.
https://console.cloud.google.com > Left drawer > IAM & admin > Against user - Edit Icon > Add another role > Data Store > Cloud Datastore Import & Export > Save
Try creating a new service account with the Firestore-related permissions needed and using that on a freshly created environment. https://cloud.google.com/composer/docs/how-to/access-control
Other debugging ideas: * Try ssh-ing into the Kubernetes workers on your Composer environment and running the command. *Is the Firestore API enabled on your project?
Open https://console.cloud.google.com/iam-admin/iam
Find the service account you're using for the backups
Add the Owner role to the service account
It's not really intuitive or logic because there are not permissions or roles for Firestore.
Unfortunately it took me way to long to figure it out. I hope it helps others!
Similar to Roy's answer, the issue for me was that gcloud was set to a different project.
check which project it is set to
gcloud config list
list which projects you have access to
gcloud projects list
set the correct project
gcloud config set project 'foo-project'
Grant firebase admin role to the default service account that your service is using.
Adding Owner role to the service account seems too much privilege for just taking backup.
In IAM & Admin make sure your #appspot.gserviceaccount.com must have access for 3 things:
Cloud Functions Admin
Cloud Datastore Import Export Admin
Storage Admin
you need to set your project first where you are owner
gcloud config set project project-id
You can find your project id by clicking on gcloud console it will be there in popup in project-name-somerandomnumbers
I got caught out on this today. The issue was that I had set up my service account correctly in the IAM settings, but hadn't realised an invitation had been sent to that email address which I needed to accept. Worked immediately once I accepted the invite.
I am trying to create an internal app to upload files to google cloud. I don't want each individual user or this app to log in so I'm using a service account. I login into the service account and everything is ok, but when I try to upload it gives me this error:
ServiceException: 401 Anonymous caller does not have storage.objects.list access to bucket
As you can see I am logged in with a service account and my account and(neither service or personal) works
I had similar problem, and as always, it took me 2 hours but the solution was trivial, if only it was written somewhere... I needed to login (or authorize, what suits you) to the gsutil in addition to being authorized to the gcloud. I thought they are linked or whatever, but nah. After I ran gsutil config and authorized via the provided link (and code that I pasted back to the console), it started working for me.
Note that I was also logged in to gcloud via a service account linked to my project and having the service account .json key saved locally (see gcloud auth activate-service-account --help.
gcloud auth login solved my issue. You need both steps:
gcloud auth login
gcloud auth application-default login
It happened to me because I had an incomplete initialisation while running gcloud init.
I reinitialised the configuration using gcloud init command and it worked fine.
I can only think of a few things that might cause you to see this error:
Maybe you have an alias set up to a standalone installation of gsutil (which doesn't share credentials with gcloud)?Edit: it's also possible you're invoking the wrong gsutil entry point - make sure you're using <path-to-cloud-sdk>/google-cloud-sdk/bin/gsutil, and not <path-to-cloud-sdk>/google-cloud-sdk/platform/gsutil/gsutil. The platform path will not automatically know about your configured gcloud auth options.
Maybe your service account credentials have moved/are invalid now? If your boto file is referring to a keyfile path and the keyfile was moved, this might happen.
Maybe the gcloud boto file (that gcloud created to use with gsutil when you ran gcloud auth login) is gone. You can run gsutil version -l to see if it's shown in your config path. If gcloud's boto file is present, you should see a line similar to this:
config path(s):
/Users/Daniel/.config/gcloud/legacy_credentials/email#domain.tld/.boto
You can run gsutil version -l to get a bit more info and look into the possibilities above. In particular, these attributes from the output will probably be the most helpful: using cloud sdk, pass cloud sdk credentials to gsutil, config path(s), and gsutil path.
Use this command to resolve some issues
gsutil config
Follow the browser to get a code, then set it in your terminal.
I had the same issue, tried to do gsutil config then it recommended me gcloud auth login which opened google in the browser. After i logged in, i could download with gsutil cp -r gs://my_bucket/Directory local_save_path the entire bucket and save it locally.
I faced the same problem. It took me two days to get this thing working.
I am writing about the whole setup. please refer to step 2 for the answer to the question. FYI my OS is windows 10
Step 1:
Firstly, I faced problems installing gcloud and this is what i did.
The script(.\google-cloud-sdk\install.bat) which is supposed to add gcloud to the path was not working due to permission issues.
I had to add the path manually in two places
1) In the system variables, to the "PATH" variable i added the path to the gcloud bin which should look like - C:\Users\774610\google-cloud-sdk\bin - in my case
2) Additionally gcloud needs python so to the "PATHEXT" variable i appended ".PY" at the end.
After Performing these tasks gcloud started working.
Step 2:
Even though gcloud is working, maven is not able to connect to cloud storage and the error was "401 Anonymous caller does not have storage.objects.list access to bucket"
I was pretty sure i did login to my account and selected the correct project. I also tried adding environment variable as shown in this documentation "https://cloud.google.com/docs/authentication/getting-started"
Nothing seemed to be working even though all the credentials were perfectly setup.
while going through the gcloud documentation I came across this command - "gcloud auth application-default login" which was exactly what i needed.
Refer here for difference between gcloud auth login and gcloud auth application default login
In short what this command does is it obtains your credentials via a web flow and stores them in 'the well-known location for Application Default Credentials' and any code/SDK you run will be able to find the credentials automatically
After this, maven was successfully able to connect to google storage and do its stuff.
Hope this helps, thanks
Does your service account actually have the required permission? The role(s) that will give you this permission are roles/storage.objectViewer / roles/storage.objectAdmin / roles/storage.admin.
Please ensure the service account actually have the permissions in your Cloud Console and then it should work.
--- UPDATE ---
Since you have the correct permission in the account, there it's likely the correct account wasn't used in the gsutil command. This can happen if you have multiple installations of your gsutil tool, please ensure your gsutil has the correct path point to a .BOTO file. There's a similar issue reported on the github repo. You can see if the solution there works.
Ultimately, you can use a new machine / vm with a fresh install to test it out to see if it works. You can this easily by going to the Cloud Console and using the Cloud Shell. No real installation needed, should be very simple to test.
This should work and it will basically isolate your issue (to that of multiple installation) on your original machine. After that, you basically just have to do a clean install to fix it.
If you installed gsutil using python (without gcloud SDK), it may help to run gsutil config and complete steps of initialisation.
Thank you for all the replies.
I would like to share my own experience.
I had to login under the user which is defined when installing Gitlab Runner.
By default, the user indicated in the installation doc is : "gitlab-runner".
So, first, I added a password on this user:
passwd gitlab-runner
then :
su - gitlab-runner
gcloud auth login
gcloud auth application-default login
The issue is solved.
Maybe there is a better way, by directly putting the Google auth files under /home/gitlab-runner
I faced same issue. I used
gcloud auth login
and follow the link
If you are using a service account you need first to authorize it, otherwise gsutil won't have the permission to read/write
gcloud auth activate-service-account --key-file=service_account_file.json
Personally, I had an account with proper permissions registered but I got that error as well despite verifying that my account was running using "sudo gcloud init"
What solved it for me was navigating to the ~/.gutil directory and writing the following
sudo chown jovyan:jovyan *
which let my JupyterLab terminal run, not from root, but from default jovyan.
After that it used my account, not Anonymous caller
Here is another way to edit roles:
gsutil iam ch allUsers:objectViewer gs://tf-learn-objectdetection
Fore more documentation:
gsutil iam help
Use gcloud auth login
Goto mention link
Copy Verification code
Paste Verification code
In my case, even after using gsutils solutions discussed in other answers, I got the error. After checking other google search results, I found out that the reason was that I was authenticating with "my user" while running the gsutils as the root.
Thanks to the answer in the gsutils page in github: https://github.com/GoogleCloudPlatform/gsutil/issues/457
Let me expain what helped me step by step:
First my requirement is to enable CORS, but faced the asked issue, So I followed the below steps:
On Browser side:
Open google cloud console on your browser.
Open Cloud shell editor.
Type gcloud auth login.
Now it will show an command with an url.
Copy that command Don't close browser.
On PC GCloud software side:
Download GCloud Sdk Installer.exe
Open GCLoud in your pc It will ask you to sign In via browser
Signin with correct email id
Select your project from the shown list
Paste the previously copied command
Again it will ask you to signIn
Select the proper account to sign in
Now the GCloud cmd will show you another command with url as output
Copy the output Open your browser, then paste it.
Done! It will show like You are now logged in as xyz#gmail.com
Now I'm able to set CORS without any exception.
Hope these steps will be helpfull for someone who is new to the issue.
Looks like account information is not stored with gsutil
Step 1:
gsutil config
Step 2:
copy url in browser
Step3:
select account and grant permission
Step 4:
Copy key and share it in gsutil promt "step1 will be asking for this key to proceed"
Step 5:
Run command whose access was denied
Thank you Petr Krýže!!! you saved my day...
I launched an instance with service level accounts enabled. For example it has storage-rw set. I verfied that the instance has those. Now whenever i run gsutil ls gs://my_bucket from within the instance I get the error: Failure: unauthorized_client.
gcloud auth list returns
Credentialed accounts:
- xxxx#developer.gserviceaccount.com (active)
I need to use gcloud sdk from an instance because i need more components other than the gcutil and gsutil.
So my question is how can I authorize gcloud to use the xxxx#developer.gserviceaccount.com account and thus the permissions only specified on the instance and not my personal user account which has full permissions to everything?
The gcloud CLI definitely handles Google Compute Engine service accounts. If you see it as "(active)" when you do $ gcloud auth list, that should be sufficient.
Two things can be going wrong here:
You are using the wrong gsutil.
When you install the Google Cloud SDK, it will create google-cloud-sdk/bin/gsutil, and THAT is the one you want to run. Do $ which gsutil to double check. If you're running google-cloud-sdk/platform/gsutil/gsutil, that's the wrong one, and it won't know about anything that gcloud can tell it.
The account doesn't have permissions to access the bucket you're trying to inspect. You'll have to ask the owner of the bucket to add it to the project that owns that bucket.
Source: Engineer for the Google Cloud SDK
See "Authenticating to Google Compute Engine" section in this doc: https://developers.google.com/compute/docs/gcutil/