I'm using Google Cloud Storage to serve my static website to the public and wondering is there a way to enable Apache/NGINX like access logs for the bucket via the GCP web interface?
GCS does offer access logs, although they arrive as CSV files with a bunch of information and not as Apache logs.
Enabling them is fairly simple. Say you want access logs for bucket "mybucket".
First, create another bucket to hold the access logs. Let's call it "mylogsbucket".
Second, give GCS permission to write logs to that bucket with this gsutil command:
gsutil acl ch -g cloud-storage-analytics#google.com:W gs://mylogsbucket
Third, activate logging:
gsutil logging set on -b gs://mylogsbucket gs://mybucket
Usage logs for mybucket will now show up about once per hour in mylogsbucket, and storage logs recording how much data is being stored will show up once per day.
More documentation on this feature is here: https://cloud.google.com/storage/docs/access-logs
Related
Using Google Cloud Launcher we've deployed a Mongodb replicaset.
We are know configuring backups being uploaded to buckets.
Under the VM Cloud API access scopes, with the machine stoped, we've given Full access to Storage
When we try to upload using gsutil cp, we get the following error:
Copying file://whateverfilewe try [Content-Type=application/octet-stream]...
AccessDeniedException: 403 Insufficient OAuth2 scope to perform this operation.
Acceptable scopes: https://www.googleapis.com/auth/cloud-platform
Reading documentation, that scope seems way too much.
How should we proceed in order to give access to to a bash script (using the machines assigned service account) within a Compute engine instance upload access to a bucket inside the same project? Is full access really necessary?
Seems gsutil was caching credentials. This comment gave me the solution:
gsutil copy returning "AccessDeniedException: 403 Insufficient Permission" from GCE
I have developed a Software in which is configured directories to save files. I run it on Linux. These directories are informed by config file.
I would like to use compute engine nodes because I need to increase its performance. Therefore, I would like to use Google Storage to save these files into a save repository.
In [1] is showed mounting a bucket as file system. I tried it, but no success. I receive authentication error.
Can anyone help me to get success in order to access my bucket by compute engine nodes ?
[1] https://cloud.google.com/compute/docs/disks/gcs-buckets
Best regards,
It sounds like you did not start your GCE instance with a service account.
According to the docs you linked, you need to configure a service account or run gcloud auth login to configure your credentials for accessing cloud storage.
If you are trying to set up gcsfuse without running on GCE you will need to use the gcloud auth login approach.
I want to know the right/best way of having one machine copying data to Google Storage.
I need one machine to be able to write to a bucket, but not be able to create or delete other buckets.
While researching, I found out that you should create a account service so this account can log in to GC and then use the storage.
But the problem is, when the machine is from GCE, there are scopes. When setting up the scope "Default" it can Read from Google Storage, but can not write to it. Even after authenticated with a service account.
When the scope is Devstorage.read_write now the machine can create and remove buckets from that storage without login. I find that to risk.
Does anyone have any recommendations?
Thanks
The core problem here is that the "write" scope covers both write and delete, and that the GCE service account is likely a member of project-editors, which can create and delete buckets. It sounds like what you want to do is restrict a service account to only being able to affect a single bucket. You should be able to do this with these steps:
Create a service account in your project (and save the private key file).
In the permissions page for the project, make sure that service account is not a project editor for your project.
Using an account that does have full permissions to your project, create the bucket, then grant the service account write access to the bucket. Example gsutil commands to do this:
gsutil mb gs://yourbucket
gsutil acl ch -u your-service-account-name#gserviceaccount.com:W gs://yourbucket
Create a VM that does not have a GCE service account enabled.
Push the service account's private key file to that VM.
On the VM, gcloud auth activate-service-account --key-file=your-key-file.json
Now gsutil commands run on the VM should be able to write to (and delete) objects in that bucket, but not any other buckets in your project.
I've got a bucket in Google Cloud Storage, and a website. People can currently upload to the bucket through the website (using Google authentication).
However, I need to set it so that anyone can view the files that are uploaded (and can't modify them).
This can't be something that Google needs to authenticate, as some of our clients' IT departments have blocked Google (for whatever reason) and refuse to budge. It could be something where the request is made from my website, it could allow it (as I'll record the URL on the website's database).
Preferably, if this could be done without using gsutil that would be great.
You can set a default object ACL on the bucket that makes all objects uploaded to that bucket publicly readable. For example you could do it using gsutil:
gsutil defacl ch -u AllUsers:R gs://your-bucket
Note that the above command only affects newly written objects. If you already have objects in your bucket that need to be made public you could accomplish that with gsutil as well:
gsutil acl ch -u AllUsers:R gs://your-bucket/**
Regarding your point about making sure anyone can view the files but not modify them: You can accomplish this by making sure the bucket ACL only allows you (or your service account) to write objects, not all users.
A project, a Google Group have been set up for controlling data access following the DCM guide: https://support.google.com/dcm/partner/answer/3370481?hl=en-GB&ref_topic=6107456
The project does not contain the bucket I want to access(under Storage->Cloud Storage), since it's Google owned bucket, for which I only have read only access. I can see the bucket in my browser since I am allowed to with my Google account(since I am a member of the ACL).
I used the gsutil tool to configure the service account of the project that was linked with the private bucket using
gsutil config -e
but when I try to access that private bucket with
gsutil ls gs://<bucket_name>
I always get 403 errors, and I don't know why is that. Did anyone tried that before or any ideas are welcome.
Since the bucket is private and in project A, service accounts in your project (project B) will not have access. The service account for your project (project B) would need to be added to the ACL for that bucket.
Note that since you can access this bucket with read access as a user, you can run gsutil config to grant your user credentials to gsutil and use that to read the bucket.