I have files in a virtual machine hosted and created in Google Cloud and I want to be able to access them in google colab to run selenium.
Should I send the files to Google storage? It seems that I would then be able to ve found a tutorial there, it shows me how to access Google Cloud Storage files in Colab Notebooks.
You can use many methods to upload the files from your Compute Engine machine, for example uploading the files to Google Drive, or Google Cloud Storage. This document explains it well. Note that your CE machine wouldn't be stricty "local" to you but it would act the same way in that context.
With Cloud Storage, you can use gsutil to upload the files, with this command:
gsutil cp $file_to_upload gs://$bucket_name
Related
Is there a way to directly load / edit / save files to a given bucket in Google Cloud Storage without having to download the file, edit it, and then upload it again?
We have a GCS bucket with about 20 config files that we edit for various reasons. We would really just like to load the bucket into VS Code and then browse between updating the files and saving edits.
I have tried the vscode-bucket-explorer extension for VS Code but this just seems to provide viewing capability with no editing/saving capability. Unless I am missing something?
Is there a way to mount a bucket as a drive on a Mac? With read/write ability?
Is there a way to directly load / edit / save files to a given bucket
in Google Cloud Storage without having to download the file edit it and then upload it again
No, blobs objects in Google Cloud Storage can not be edited in place.
As with buckets, existing objects cannot be directly renamed. Instead,
you can copy an object, give the copied version the desired name, and
delete the original version of the object. See Renaming an object for
a step-by-step guide, including instructions for tools like gsutil and
the Google Cloud Console, which handle the renaming process
automatically.
Is there a way to mount a bucket as a drive on a Mac? With read/write
ability?
You can use Cloud Storage FUSE where the mounted bucket will behave similarly to a persistent disk.
Cloud Storage FUSE is an open source FUSE adapter that allows you to
mount Cloud Storage buckets as file systems on Linux or macOS systems.
It also provides a way for applications to upload and download Cloud
Storage objects using standard file system semantics. Cloud Storage
FUSE can be run anywhere with connectivity to Cloud Storage, including
Google Compute Engine VMs or on-premises systems
after successfully uploading files on google cloud storage using slingshot package, i didn't find documentation about how to delete files from google cloud throw my application. is there any idea to work around ? Thanks.
I download a lot of csv files via ftp from different sources on a daily basis. I then upload these files into Google Cloud Storage.
Are there any programs/api/tools to automate this?
Looking for a best way, if possible, to load these files directly into Google Cloud Storage without having to locally download them. Something that I can deploy on Google Compute, so I don't need to run a local programs like Filezilla/CrossFTP. The program/tool will keep checking the remote location on a regular basis and load new files into Google Cloud Storage; ensuring a checksum match.
I apologize in advance if this is too vague/generic question.
Sorry, no. Automatically importing objects from a remote FTP server is not currently a feature of GCS.
I generated Access & Secret Key put it into Bucket Explorer just as I saw on their screenshots but I get an error that Keys are not recognized on AWS, so it keeps trying to connect to AWS. I have latest version for OS X.
Looking at the Bucket Explorer website, it sounds like you need a special version of their software -- however, the download link does not actually seem to have that version available. Have you contact the developers of Bucket Explorer about it?
Alternatively, you can use the Google Cloud Console to upload and access your data in Google Cloud Storage without depending on a particular client application.
I installed the CKAN from source and trying to activate the Cloud Filestore option without success.
I double-checked my Google API console and activated interoperable access keys (GOOG...) to no avail. I keep getting "Unable to Upload File" when I try to upload.
Couldn't get it to work so just switched to S3 which was more straightforward. The recent S3 price drops helped too :)