This seems like a simple question, but I have not been able to find the answer online. What specific actions can the OWNER of a bucket do that a WRITER of that bucket (or any object) cannot do? The reason I ask is that I noticed that one of my "logs" buckets has cloud-logs#google.com as an OWNER and I'm wondering if I can (and possibly should?) change this to WRITER -- is it the OWNER because it actually created the bucket for me? if an Google app creates a bucket for me, would it be a good idea for it to them remove itself as an OWNER after making "project-owners" the OWNER? Also, wouldn't it make more sense for the "role" of project-editors to be WRITER rather than OWNER so that there is a difference between a project "owner" and a project "editor"?
For complete details on the different permissions, there's a complete chart of the differences between Reader, writer, and owner for buckets and objects in the GCS documentation.
A reader of a bucket can list its contents. A writer of a bucket can also delete objects and insert new ones. An owner of a bucket can also edit bucket metadata and ACLs.
According to the Cloud Logging Docs, OWNER is the correct permission for cloud-logs#google.com.
Related
I'm trying to figure out if anyone can offer advice around bucket creation for an app that will have users with an album of photos. I was initially thinking of creating a single bucket and then prefixing the filename by user id, since google cloud storage doesn't recognize subdirectories, like so: /bucket-name/user-id1/file.png
Alternatively, I was considering creating a bucket and naming it by user id like so: /user-id1-which-is-also-bucket-name/file.png
I was wondering what I should consider in terms of cost and organization when setting up my google cloud storage. Thank you!
There is no difference in term of cost. In term of organization, it's different:
For the deletion, it's simpler to delete a bucket and not a folder in the unique bucket.
For performances, sharding is better is you have separate bucket (you have less chance to create an hotspot)
At billing perspective, you can add labels on the buckets, and get them in the billing exported to BigQuery. You can know the cost of the bucket per user, and maybe do a rebill to them
The biggest advantage of 1 bucket per user model is the security. You can grant a user (if the users have direct access to the bucket and don't use a backend service to access it) on a bucket, without the use of legacy (and almost deprecated) ACL on object. In addition, if you use ACL, you can't set ACL per folder, ACL are per object. So, everytime that you add an object in the unique bucket, you have to set the ACL on it. It's harder to achieve.
IMO, 1 bucket per user is the best model.
I am working on a project for a client and a couple of weeks ago most of the content "disappeared".
Images and videos are routed through FileStack (a file processing service) but actually stored on Google Cloud Storage in one bucket.
On the day in question everything was working, and then everything stopped working. When we investigated it turned out that the bucket FileStack was pointing to was non-existent, so we created a new bucket with the same name and everything magically worked itself out.
Now my question is, where did all the files from the disappeared bucket go? Is it possible to get them back? Is it possible to figure out what happened?
I have extensively reviewed the audit log in the Activity tab and it shows zero activity for the bucket in question. Is there anywhere else we can investigate?
Can you please send email to gs-team#google.com, noting the bucket name and an example object name from that bucket, along with the last time you were successfully able to access that bucket/object? Doing it that way will avoid exposing these names on the public forum. Please mention my name in the message, so I will get it and can investigate.
Thanks,
Mike Schwartz
GCS Team
When an object is deleted, it's deleted from the system and there isn't any option to recover it [1]. You can prevent this behavior by using object versioning [2]. And to get a better overview of the activity in Cloud Storage you can enable the "Data Access logs" [3].
About the reason why the objects has disappeared, as a first workaround you can review if there's an Object Lifecycle enabled [4].
https://cloud.google.com/storage/docs/deleting-objects
https://cloud.google.com/storage/docs/object-versioning
https://cloud.google.com/storage/docs/audit-logs
https://cloud.google.com/storage/docs/lifecycle
When objects are uploaded into a GCS bucket and shared publicly, the owner of the bucket is responsible for all of the costs of users downloading these objects. How do I change that so that the downloaders are billed instead of me?
This feature is called "Requester Pays." Its documentation is here: https://cloud.google.com/storage/docs/requester-pays
The idea is that you mark a bucket as being a "requester pays" bucket. Once you've done that, your project is only responsible for the price of storing the objects in the bucket (and, if it's a nearline or coldline bucket, any early deletion fees). Anyone that wants to download an object from this bucket (or upload a new object, copy an object, etc) must specify which of their projects GCS should bill for the operation.
This is a very useful configuration for situations where you wish to make objects publicly available but don't want to be responsible for the cost of distributing it to many end users.
To enable Requester Pays on a bucket, open the Cloud Storage browser, find your bucket, and click the "off" button in the "Requester Pays" column, and follow the prompts. You can also set this flag in other ways, see the docs: https://cloud.google.com/storage/docs/using-requester-pays#enable
Downloading objects from requester pays buckets requires a Google Cloud project with billing enabled. Once you have that, you can download the object from the cloud console or using gsutil:
$> gsutil -u [PROJECT_ID] cp gs://[BUCKET_NAME]/[OBJECT_NAME] [OBJECT_DESTINATION]
The trick to this command is the -u [PROJECT_ID] bit, which specifies which project should be billed for the download.
You can also download the object using our other APIs or with the cloud console. More in the docs: https://cloud.google.com/storage/docs/using-requester-pays#using
I need to use google cloud storage to store some files that can contain sensitive information. File names will be generated using crypto function, and thus unguessable. Files will be made public.
Is it safe to assume that the file list will not be available to public ? I.e. file can only be accessed by someone who knows the file name.
I have ofc tried accessing the parent dir and bucket, and I do get rejected with unauthenticated error. I am wondering if there is or will ever be any other way to list the files.
Yes, that is a valid approach to security through obscurity. As long as the ACL to list the objects in a bucket is locked down, your object names should be unguessable.
However, you might consider using Signed URLs instead. They can have an expiration time set so it provides extra security in case your URLs are leaked.
Yes, but keep in mind that the ability to list objects in a bucket is allowed for anyone with read permission or better on the bucket itself. If your object names are secret, make sure to keep the bucket's read permissions locked down as much as possible.
jterrace's suggestion about preferring signed URLs is a good one. The major downside to obscure object names is that it's very difficult to remove access to a particular entity later without deleting the resource entirely.
I am currently structuring a web application to serve out segments of our database represented as html iframes. I need to host my Django app's static files (such as bootstrap) in a static file store on Google Cloud Storage in order to correctly represent the HTML elements. However, when I try to create a bucket called 'static', GCS replies with the following error:
Sorry, that name is not available. Please try a different one.
Not only that, it is not allowing me to access or modify the URI, displaying a "Forbidden" message when I attempt to.
Does anyone know how to change this default setting by Google? There is no documentation regarding this..
It seems that the bucket with the given name has been already created by someone else. You have to choose a globally unique name.
Bucket names reside in a single Google Cloud Storage namespace. As a consequence, every bucket name must be unique across the entire Google Cloud Storage namespace. If you try to create a bucket with a bucket name that is already taken, Google Cloud Storage responds with an error message.
Use another name or use the default bucket. If your app was created after the App Engine 1.9.0 release, it should have a default GCS bucket named [your-app-id].appspot.com available. You can create your static files in that bucket and mimic directory structure as follows.
[your-app-id].appspot.com/static/my-file-1