Unable to delete Google Cloud Storage object change notification webhook - google-cloud-storage

I created a webhook to monitor changes on a bucket in 2015 using the command gsutil notification watchbucket https://my-project.appspot.com/myWebHook gs://my-bucket/
Today I would like to delete it but I'm unable to do it.
gsutil notification list -o gs://my-bucket/ is showing me the current webhooks
It is returning something like that:
Notification channel 1:
Channel identifier: someid-905e-905e-905e-1fdca170d689
Resource identifier: myIdentifierXXXfncnI0SuU-U
Application URL: url=https://my-project.appspot.com/myWebHook&appEngine=true
Created by:
Creation time: 2015-09-08 11:44:26.775000
when I try to delete it with the command gsutil notification stopchannel someid-905e-905e-905e-1fdca170d689 myIdentifierXXXfncnI0SuU-U
I get this:
NotFoundException: 404 Channel 'someid-905e-905e-905e-1fdca170d689' not found for project '32555940559'
All this is done with my account which is the owner of the project. By the way, the id of the project in the error message is not the one of my project 🤔
If I switch to a service account, I get a different error: AccessDeniedException: 403 Caller not owner of subscription
Do you have any idea how I could solve this?

According to the official documentation Object change notification:
There are three steps to configuring authorization in an object change
notification scenario:
1.Creating a Service Account
2.Configuring gsutil to Use the Service Account
3.Identifying a Domain to Receive Notifications
To be able to remove the notification channel you have to identify the service account used for creating the channel (owner service account of this channel). Then, you have to configure gsutil to use that specific service account and remove the notification channel. Therefore, trying to perform this action by authorizing gsutil using your personal account will not work.

I was getting this same problem and solved it by authenticating gcloud with my main default service account (this may have been the account used when creating the watcher):
gcloud auth activate-service-account my-project#appspot.gserviceaccount.com --key-file ~/Downloads/my-project-1234abcdef.json
I could then list all my watchers to get the Channel ID and Resource ID with
gsutil notification list -o gs://my-project.appspot.com/
Note that in this next command, I couldn't use the bucket path like gs://my-project.appspot.com/ but rather had to use the goofy "Resource identifier" value from the list response above which looks like
Bucket my-project.appspot.com has the following active Object Change Notifications:
Notification channel 1:
Channel identifier: prod
Resource identifier: asdf1234
Application URL: url=https://my-project.appspot.com/api/gcs&appEngine=true
Created by:
Creation time: 2019-02-11 16:16:07.955000
Notification channel 2:
Channel identifier: dev
Resource identifier: 1234asdf
Application URL: url=https://my-project.appspot.com/api/gcs&appEngine=true
Created by:
Creation time: 2019-02-11 16:21:40.864000
This command then removed the prod channel:
gsutil notification stopchannel prod asdf1234
Removing channel prod with resource identifier asdf1234 ...
Succesfully removed channel.

Related

Unable to set notifications for bucket

I am using the golang storage API to add notifications to my bucket.
https://godoc.org/cloud.google.com/go/storage#BucketHandle.AddNotification
I am getting the following error: Unable to set notifications for bucket: googleapi: Error 403: The service account 'dev-lm#gs-project-accounts.iam.gservice
account.com' does not have permission to publish messages to to the Cloud Pub/Sub topic '//pubsub.googleapis.com/projects/dev-lm/topics/local_sanket_ms-transcript-incoming', or that topic does not exist., forbidden
Clearly, its a permission issue. Although -- I am not sure why the service account looks like that. I am not sure where this account is being construed from or how do I override it with an account I know is valid. Any hints where to look?

How can I view currently configured Object Change Notifications (OCNs) for a given GCS Bucket?

How can I view currently configured Object Change Notifications (OCNs) for a given GCS Bucket?
I thought it would be as follows:
gsutil notification list gs://my-bucket-name
But then I was surprised by this in gsutil help notification:
LIST The list sub-command provides a list of notification configs
belonging to a given bucket. The listed name of each notification
config can be used with the delete sub-command to delete that
specific notification config.
No object change notifications will be listed. Only Cloud Pub/Sub
notification subscription configs will be listed.
Notice the last paragraph.
You should add -o like following example:
gsutil notification list -o gs://${BucketName}
And this will list Object Change Notifications:
Bucket ${BucketName} has the following active Object Change Notifications:
Notification channel 1:
Channel identifier: xxxxxxxx
Resource identifier: xxxxxx
Application URL: url=https://xxxxxx.appspot.com/notify
Created by:
Creation time: xxxx
Reference: notification command

Creating a bucket using Google Cloud Platform Deployment Manager Template

I'm trying to create a bucket using GCP Deployment Manager. I already went through the QuickStart guide and was able to create a compute.v1.instance. But I'm trying to create a bucket in Google Cloud Storage, but am unable to get anything other than 403 Forbidden.
This is what my template file looks like.
resources:
- type: storage.v1.bucket
name: test-bucket
properties:
project: my-project
name: test-bucket-name
This is what I'm calling
gcloud deployment-manager deployments create deploy-test --config deploy.yml
And this is what I'm receiving back
Waiting for create operation-1474738357403-53d4447edfd79-eed73ce7-cabd72fd...failed.
ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation operation-1474738357403-53d4447edfd79-eed73ce7-cabd72fd: <ErrorValue
errors: [<ErrorsValueListEntry
code: u'RESOURCE_ERROR'
location: u'deploy-test/test-bucket'
message: u'Unexpected response from resource of type storage.v1.bucket: 403 {"code":403,"errors":[{"domain":"global","message":"Forbidden","reason":"forbidden"}],"message":"Forbidden","statusMessage":"Forbidden","requestPath":"https://www.googleapis.com/storage/v1/b/test-bucket"}'>]>
I have credentials setup, and I even created an account owner set of credentials (which can access everything) and I'm still getting this response.
Any ideas or good places to look? Is it my config or do I need to pass additional credentials in my request?
I'm coming from an AWS background, still finding my way around GCP.
Thanks
Buckets on Google Cloud Platform need to be unique.
If you try to create a bucket with a name that is already used by somebody else (on another project), you will receive an ERROR MESSAGE. I would test by creating a new bucket with another name.

Gsutil stopchannel can not stop with AccessDeniedException

I can not stopchannel with throwed exception: AccessDeniedException: 403 Caller not owner of subscriptio
I set permission in google developers console already with owner role, but still return exception.
gsutil version: 4.6
You need to use the same service account that was used to create the channel, otherwise you get this error.
You can use the commands gcloud auth list to view the account list that you have in the instance. Use gcloud config set account ACCOUNT to set the service account as active.

The gsutil tool is not working to register a channel in object change notification

When executin the follow command:
gsutil notifyconfig watchbucket -i myapp-channel -t myapp-token https://myapp.appspot.com/gcsnotify gs://mybucket
I receive the follow answer, but I used the same command before in another buckets and it worked:
Watching bucket gs://mybucket/ with application URL https://myapp.appspot.com/gcsnotify...
Failure: <HttpError 401 when requesting https://www.googleapis.com/storage/v1beta2/b/mybucket/o/watch?alt=json returned "Unauthorized WebHook callback channel: https://myapp.appspot.com/gcsnotify">.
I used gsutil config to set permissions and tried with gsutil config -e also.
I already tried to set the permissions, made myself owner of the project, but is not working, any help?
I was getting the same error. You must configure gsutil to use a service account before you can watch a bucket.
An additional security requirement was recently added for Object Change Notification. You must add your endpoint domain as a trusted domain on your cloud project. To do that, the domain first has to be whitelisted with the Google Webmaster Tools.
See instructions here:
https://developers.google.com/storage/docs/object-change-notification#_Authorization
I also determined that I needed to:
Whitelist my appspot domain
Create a service account before I can watch a bucket.
At first I was using the google cloud shell and I figured it should just be authenticated. gsutil ls listed the objects in my bucket so I assumed I was authenticated. However that is not the case.
You need to instal gsutil or google cloud sdk, log in, get the .p12 file from the service account, and auth it as Wind Up Toy described. After that it will work.