I am currently trying to read a pdf file that I have stored in my Google Cloud Storage bucket. I have finished writing my code using the Google Cloud Vision API to read pdf files, but for some reason when I try to access the file that I have stored, I receive an error in my code saying that permission was denied to the file. How can I fix this, and if needed what code should I include to fix this?
Related
I am trying to create an environment in Google Cloud Composer. Link here
When creating the environment from scratch and selecting all the default fields, the following error appears:
CREATE operation on this environment failed 22 hours ago with the following error message:
CREATE operation failed. Composer Agent failed with: Cloud Storage Assertions Failed: Unable to write to GCS bucket.
GCS bucket write check failed.
I then created a google cloud storage bucket within the same project to see if that would help and the same error still appears.
Has anyone been able successfully create a Google Cloud Composer environment and if so please provide guidance on why this error message continues to appear?
Update: Need to update permissions to allow access it seems like. Here is a screenshot of my permissions page but not editable.
It seems like you haven't given the required IAM policies to the service account. I would advise you to read more about the IAM policies on Google Cloud here
When it comes to the permissions of the bucket, there are permissions like the Storage Object Admin that might fit your needs.
I just configured my application to use Google Cloud Storage for storing user uploads. So far I was able to retrieve the file path of an uploaded by doing so:
ActiveStorage::Blob.service.send(:path_for, file.key)
However, on Google Cloud Storage this gives me the following error:
NoMethodError (undefined method `path_for' for #<ActiveStorage::Service::GCSService:...>)
How can I retrieve the filepath here?
You can use url_for_direct_upload(key, expires_in:, checksum:, **) or url(key, expires_in:, filename:, content_type:, disposition:) to get the GCS urls.
Reference
I have some data in Google Cloud SQL, which I am trying to transfer to Google Cloud Storage using Cloud SQL API(beta) (export) function. For this, I have installed jupyter notebook on google compute engine and I am running the python code (to connect to cloud sql API and export the date to cloud storage in a csv file) on that.
The python code does not throw any error and also there is no error body in API response. However, the API's response says "u'status': u'PENDING'" and I see the following error (in logs of Cloud SQL Dashboard) :
error uploading CSV file to GCS: gs://[BUCKET_NAME]/[FILE_NAME].csv: Access denied for account [SERVICE_ACCOUNT_NAME]#speckle-umbrella-11.iam.gserviceaccount.com (permission issue?)
How do I ensure I have all the relevant access for the account ([SERVICE_ACCOUNT_NAME]#speckle-umbrella-11.iam.gserviceaccount.com) ? I am unable to locate this account or give myself access to this .
By default Google Compute Engine instances have read only scope for Google Cloud Storage.
You should confirm that the Google Compute Engine instance scope is set to read/write.
You need to grant the Cloud SQL instance service account access to your bucket. Visit the export documentation and click the GCLOUD (2ND GEN) tab which will show you the commands you need to use.
The command to grant the service account access to the bucket is
gsutil acl ch -u [SERVICE_ACCOUNT_ADDRESS]:W gs://[BUCKET_NAME]
I have migrated files from Parse.com to my hosted parse server using "https://github.com/parse-server-modules/parse-files-utils" tool by applying "Option-2".
Now My problem is when I click on the image in my hosted parse server dashboard, it will show me message "File not found." and my url is like,
http://ip of my server:1337/parse/files/OE9gP1wrd2OT9avp3RBmt8zysmM25wRTMtDOxsfe/tfss-6ca44378-72fb-4ddf-aef2-11af0485b11b-profile-pic
If I upload new image from mobile aap, its working fine.
I have installed mongodb and migrated parse.com data to newly created database in mongodb.
I am not using any FileAdapter in my new created parse server.
Thanks in advance, kindly please look into this issue and help me that how can I display migrated images in our hosted parse server.
What is the http error you are seeing (404?) Are you sure the error is "file not found"? Maybe your server folder permission is set not set to public, so you can't publicly access the files (should be a 403 error).
You usually store the image files to a container (storage) that can be accessed via APIs like Amazon (AWS) or Microsoft Azure. It's usually more efficient to keep you local server file storage small and have fast access speeds to your images.
You can find out how to setup an Amazon S3 bucket or Google Cloud Storage here.
You can find out how to setup an Azure Storage here and connect it to your parse-server using this adapter.
I'm not sure about AWS, but Google and Azure gives you free credits if you sign up, and (at least for Azure) the storage aren't too expensive, so the free credits can last you a while...
I want to upload 120 files, each around 1.2GB so about 150GB in total, from an HTTPS website onto my Google Cloud Storage.
I really, really don't want to have to download them all locally, and then upload them individually.
Is there any way around this? Surely I can just give Google Cloud Storage a URL to pull from? I don't control the HTTPS server.
It seems to be possible to upload from S3 to Google Cloud Storage, but S3 seems to suffer from the same problem.
If your website allows public access you can use the GCS Transfer Service to do it: https://cloud.google.com/storage/transfer/