I am evaluating google cloud storage for following use case. I need restrict my users (they do not have gmail accounts) so they can access only their files.
I know that can be done using gsutil signurl. But its gonna be lots of small files and generating signed url for every file is crazy. So wondering if there is trick to provide access to some subfolder using signed url?
Mentioned documentation says that wildcards can be used. Does it mean that it will generate many urls or one url that will apply to all files within wildcard?
You should use per-object ACL for this, absolutely. Signed URLs might be more difficult to implement, and if you're already thinking of managing user accounts, you'll want to do this through OAuth2.0 for Login anyways, so sending the user's Bearer token with any requests you make to the API should come as a magical bonus of doing your user accounts in this way. Read more about Auth with Cloud Storage here.
Unlike the gsutil ls command, the signurl command does not support operations on sub-directories. For example, unless you have an object named some-directory/ stored inside the bucket some-bucket, the following command returns an error: gsutil signurl gs://some-bucket/some-directory/
Related
I have a bucket with a sub-folder structure to add media
e.g.
bucket/Org1/ ...
bucket/Org2/ ...
and I want to generate a signed url for all the media inside each subfolder, so users that belongs to organization 1 only can view they files.
Of course I don't want to generate a signed url for each file (can be a lot) and also ACL doesn't work, because my users are logged with a non-google account (and can haven't)
so there is any way to allow like bucket/Org1/* ?
Unfortunately, no. For retrieving objects, signed URLs need to be for exact objects. You'd need to generate one per object.
One way to accomplish this would be to write a small App Engine app that they attempt to download from instead of directly from GCS which would check authentication according to whatever mechanism you're using and then, if they pass, generate a signed URL for that resource and redirect the user.
I followed up this tutorial to allow upload of files from GWT frontend directly to Google Cloud Storage using signed URLs. I've extended the Java example by specifying content type which worked just fine. Then, I saw that files uploaded this way weren't publicly readable. To get this working I've tried:
I've set up default ACL for newly uploaded objects gsutil defacl set public-read gs://<bucket>. Uploaded file again - no luck, stil not visible.
Then tried to set ACL on that object directly gsutil acl set public-read gs://<bucket>/<file> but it gave me AccessDeniedException: 403 Forbidden. It makes sense since gsutil is authenticated with my Google account and signed URL is being created with service account and it's P12 key.
I've tried to set up ACL at upload phase therefore added "x-goog-acl:public-read\n" canonicalized extension headers and appropriate query string param to pass signature check. Damn, stil no luck!
My assumption is that maybe this extension header I'm using is wrong? Then according to documentation all authenticated requests to GCS will apply private ACL by default.
Anyway - why I can't make these files publicly readable from Google Console when I'm logged in as project owner? I can make so for all files uploaded through console (I know that in that case the owner is project owner and not the service account).
What I'm doing wrong? How can I make them publicly readable by anyone?
Thanks in advice!
I think if you gone through the given docs. It clearly mention that, if you need the user to download the object without using the google account then this method provides an assigned URL for specific time to the User to download the object. I am assuming that might be its not possible to make those objects publicly available as they are signed. If still you need that functionality I would recommend you to go through the resumable upload or simple upload of the object.
Also try to put the service account of your project as the owner in the "Edit default permission of Object" in the developer console on the right side of your bucket name.
I have a newbie question regarding google cloud storage. I'm looking to create a website where people can upload files (similar to Flickr in concept) and view them. I was looking at google's cloud storage option and it seemed interesting. I got a little hung up on the authentication process. Do all users need a Google account or did I just misread it? I guess my question is can I create a site where everyone doesn't have to sign up for a google account?
Thank you for your patience and help
Nope, there's no need for customers to have their own Google accounts. Google Cloud Storage supports that, in case you want to say "these accounts have access to this data", but you can also let your app grant access on a per-request basis using whatever logic or authentication scheme you like.
This is usually accomplished with signed URLs. Basically, you'd use your credentials to sign a very specific request (download object X, upload an object with name Y) and pass that URL to the user for them to use. Signed URLs are only valid for as long as you like (one of the parameters is how long the URL is good for).
Documentation on signed URLs are here: https://developers.google.com/storage/docs/accesscontrol#Signed-URLs
It's not strictly necessary to require a user to be logged in in order to write to your bucket (e.g., we have a public-read-write canned ACL for buckets), but it's generally a bad idea. Any data stored in your bucket will ultimately be charged to you, so allowing anyone to write without authentication opens you up to a great deal of abuse.
Likewise, you could make all of your objects public-read, but then you get charged for the bandwidth costs and have no control over it (though this is a much more reasonable thing to do than public-write).
A safer option would be to proxy bytes for your users - i.e., only you/your app can write to your bucket, and your users hand their bytes to you.
Generally speaking, though, the only types of authentication we support are Google accounts, signed URLs, and anonymous users.
I'm storing objects in buckets on google cloud storage. I would like to provide a http url to the object for download. Is there a standard convention or way to expose files stored in cloud storage as http urls?
Yes. Assuming that the objects are publicly accessible:
http://BUCKET_NAME.storage.googleapis.com/OBJECT_NAME
You can also use:
http://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME
Both HTTP and HTTPS work fine. Note that the object must be readable by anonymous users, or else the download will fail. More documentation is available at https://developers.google.com/storage/docs/reference-uris
If it is the case that the objects are NOT publicly accessible and you only want the one user to be able to access them, you can generate a signed URL that will allow only the holder of the URL to download the object, and even then only for a limited period of time. I recommend using one of the GCS client libraries for this, as it's easy to get the signing code slightly wrong: https://developers.google.com/storage/docs/accesscontrol#Signed-URLs
One way is to use https://storage.cloud.google.com// see more documentation at
https://developers.google.com/storage/docs/collaboration#browser
If the file is not public, you can use this link to the file and it will authenticate with your signed in Google account:
https://storage.cloud.google.com/{bucket-name}/{folder/filename}
Otherwise generate a signed URL:
gsutil signurl -d 10m Desktop/private-key.json gs://example-bucket/cat.jpeg
It seems like this would be really, really easy - but I can't get it to work. All I need to do is to be able to serve files from Google cloud storage while restricting access to my google apps domain. I easily did this before using Google App engine simply by choosing that I wanted to limit access to my domain and setting the app.yaml appropriately. I can't find anything that tells me what I might be missing - I've tried using gsutil to set the ACL to restrict to my domain, which processes successfully through the command line, but then when I try to look at the bucket or object permissions through the cloud web console, I get "unexpected ACL entity type: domain".
I'm trying to access using storage.googleapis.com/bucket/object (of course with my bucket and object name) and I always get a 403 error even though I'm definitely logged in to gmail, and as the administrator of the domain, it seems like it should work because even if the ACL's were otherwise wrong (and I've tried it both with and without the domain restriction), and that it would work for me at least. The only way I can serve content using the above url is if I make it public - which obviously is NOT what I want to do.
I'm sure I'm missing something completely stupid, or some fundamental principles about how this should work - can anyone give me any ideas?
I'm not 100% sure what your use case is, but I'm guessing that your users are attempting to access the objects directly from a web browser. storage.cloud.google.com accepts Google authorization cookies, which means that if a user is logged in to an appropriate Google account, they can access resources restricted to certain users, groups, or domains. However, the other endpoints do not accept cookies as authorization, and so this use case won't work.
These users have permission to access objects using storage.googleapis.com, but doing so requires explicitly authorizing requests.
In othe words, a simple <img src="http://storage.cloud.google.com/bucket/object" /> link will work fine for signed-in users, but using storage.googleapis.com requires explicitly authorizing requests with via OAuth 2.