How does one associate a bucket with an actual GCS folder/file? I see that one creates buckets, then registers clients to access it, but not seeing how that bucket creation is associated to the actual filesystem resource...
If my understanding of your question is correct, this would be an answer
gs://bucket/folder/subfolder/file"
Related
I'm trying to figure out if anyone can offer advice around bucket creation for an app that will have users with an album of photos. I was initially thinking of creating a single bucket and then prefixing the filename by user id, since google cloud storage doesn't recognize subdirectories, like so: /bucket-name/user-id1/file.png
Alternatively, I was considering creating a bucket and naming it by user id like so: /user-id1-which-is-also-bucket-name/file.png
I was wondering what I should consider in terms of cost and organization when setting up my google cloud storage. Thank you!
There is no difference in term of cost. In term of organization, it's different:
For the deletion, it's simpler to delete a bucket and not a folder in the unique bucket.
For performances, sharding is better is you have separate bucket (you have less chance to create an hotspot)
At billing perspective, you can add labels on the buckets, and get them in the billing exported to BigQuery. You can know the cost of the bucket per user, and maybe do a rebill to them
The biggest advantage of 1 bucket per user model is the security. You can grant a user (if the users have direct access to the bucket and don't use a backend service to access it) on a bucket, without the use of legacy (and almost deprecated) ACL on object. In addition, if you use ACL, you can't set ACL per folder, ACL are per object. So, everytime that you add an object in the unique bucket, you have to set the ACL on it. It's harder to achieve.
IMO, 1 bucket per user is the best model.
I want to link multiple domains to one bucket with gcs
However, in an official document, the bucket name will be the domain as it is, so it seems that you can not associate multiple domains.
Do not you know someone?
GCS does not support this directly. Instead, to accomplish this, you'd likely need to make use of Google Cloud Load Balancing with your GCS bucket as a backing store. With it, you can obtain a dedicated, static IP address which you can map several domains to, and it also allows you to map static and dynamic content under the same domain, and it allows you to swap out which bucket is being served at the same path. The main downside to it is added complexity and cost.
When objects are uploaded into a GCS bucket and shared publicly, the owner of the bucket is responsible for all of the costs of users downloading these objects. How do I change that so that the downloaders are billed instead of me?
This feature is called "Requester Pays." Its documentation is here: https://cloud.google.com/storage/docs/requester-pays
The idea is that you mark a bucket as being a "requester pays" bucket. Once you've done that, your project is only responsible for the price of storing the objects in the bucket (and, if it's a nearline or coldline bucket, any early deletion fees). Anyone that wants to download an object from this bucket (or upload a new object, copy an object, etc) must specify which of their projects GCS should bill for the operation.
This is a very useful configuration for situations where you wish to make objects publicly available but don't want to be responsible for the cost of distributing it to many end users.
To enable Requester Pays on a bucket, open the Cloud Storage browser, find your bucket, and click the "off" button in the "Requester Pays" column, and follow the prompts. You can also set this flag in other ways, see the docs: https://cloud.google.com/storage/docs/using-requester-pays#enable
Downloading objects from requester pays buckets requires a Google Cloud project with billing enabled. Once you have that, you can download the object from the cloud console or using gsutil:
$> gsutil -u [PROJECT_ID] cp gs://[BUCKET_NAME]/[OBJECT_NAME] [OBJECT_DESTINATION]
The trick to this command is the -u [PROJECT_ID] bit, which specifies which project should be billed for the download.
You can also download the object using our other APIs or with the cloud console. More in the docs: https://cloud.google.com/storage/docs/using-requester-pays#using
I need to use google cloud storage to store some files that can contain sensitive information. File names will be generated using crypto function, and thus unguessable. Files will be made public.
Is it safe to assume that the file list will not be available to public ? I.e. file can only be accessed by someone who knows the file name.
I have ofc tried accessing the parent dir and bucket, and I do get rejected with unauthenticated error. I am wondering if there is or will ever be any other way to list the files.
Yes, that is a valid approach to security through obscurity. As long as the ACL to list the objects in a bucket is locked down, your object names should be unguessable.
However, you might consider using Signed URLs instead. They can have an expiration time set so it provides extra security in case your URLs are leaked.
Yes, but keep in mind that the ability to list objects in a bucket is allowed for anyone with read permission or better on the bucket itself. If your object names are secret, make sure to keep the bucket's read permissions locked down as much as possible.
jterrace's suggestion about preferring signed URLs is a good one. The major downside to obscure object names is that it's very difficult to remove access to a particular entity later without deleting the resource entirely.
I am currently structuring a web application to serve out segments of our database represented as html iframes. I need to host my Django app's static files (such as bootstrap) in a static file store on Google Cloud Storage in order to correctly represent the HTML elements. However, when I try to create a bucket called 'static', GCS replies with the following error:
Sorry, that name is not available. Please try a different one.
Not only that, it is not allowing me to access or modify the URI, displaying a "Forbidden" message when I attempt to.
Does anyone know how to change this default setting by Google? There is no documentation regarding this..
It seems that the bucket with the given name has been already created by someone else. You have to choose a globally unique name.
Bucket names reside in a single Google Cloud Storage namespace. As a consequence, every bucket name must be unique across the entire Google Cloud Storage namespace. If you try to create a bucket with a bucket name that is already taken, Google Cloud Storage responds with an error message.
Use another name or use the default bucket. If your app was created after the App Engine 1.9.0 release, it should have a default GCS bucket named [your-app-id].appspot.com available. You can create your static files in that bucket and mimic directory structure as follows.
[your-app-id].appspot.com/static/my-file-1