I'm trying to follow some of Tensorflow's tutorials. Specifically this one:
https://www.tensorflow.org/lite/tutorials/model_maker_object_detection
It has a step that uses files in Google cloud storage. That step fails. It seems I should be able to manually download that file(s). But I can't click on the URL. I've poked around in Google Cloud storage, but it appears it costs money and I can't seem to figure out how to "copy" that file(s) from someone else's cloud storage to mine.
Am I missing something simple?
You can use gsutil command line tool
For example
gsutil cp gs://cloud-ml-data/img/openimage/csv/salads_ml_use.csv /tmp/
It may ask you to login first - just follow the commands that shows on screen.
For more details see https://cloud.google.com/storage/docs/downloading-objects#gsutil
For that example it does need login details. You can open the notebook on Google colabs and download as in the above:
! gsutil cp gs://cloud-ml-data/img/openimage/csv/salads_ml_use.csv /content/
Related
I'm trying to download the networks models of a Neural Network project used to detect NSFW images. The packed models are available in the assets section of a release at this URL: https://github.com/notAI-tech/NudeNet/releases
I would like to create a DOCKERFILE which download these assets when built. I started using the ADD command, but the files didn't get downloaded entirely (only few kB over the 120 MB of some files). So, I tried in my Linux CLI using wget and curl... But nothing worked as expected. For example the command :
curl -OJL https://github.com/notAI-tech/NudeNet/releases/download/v0/classifier_model.onnx
Starts the download but only download an HTML file instead of the actual ONNX file... It seems Github is doing some kind of redirection and I don't know how I can handle it with curl/wget and finally with the ADD command of a DOCKERFILE ?
I did visit
https://github.com/notAI-tech/NudeNet/releases/download/v0/classifier_model.onnx
in my browser and I did get login page, so apparently it is not publicly available. That would explain why you did get small HTML file (file with login form).
Github is doing some kind of redirection and I don't know how I can
handle it with(...)wget
You need to provide authentication data, I do not know how it is exactly done in this case, but I suspect they might use one of popular methods: basic authentication (see wget options --http-user=user and --http-password=pass) or cookies based solution (see wget options --load-cookies file and --save-cookies file and --keep-session-cookies).
Mentonied options are described in wget man page, which you might access by click link or doing man wget in terminal.
I'm trying to read files from my Github repository in Google Colab. I found this answer that instructed me to use:
!npx degit Jainu-s/urldata/al -f
Which copies the files from my repository as local Colab files. It works perfectly, but I have no ideia what this code does exactly and I could not find any Colab documentation on it.
Can anyone explain to me how it works? What is its syntax, what does it do? And what are the options I could use with it?
You can read its documentation.
https://github.com/Rich-Harris/degit
Basically,
the program is degit
npx is to execute without install it
/al means you download that directory content
f is force download, because the directory is not empty
To read more
!npx degit --help
This appears to be fairly recent, but previously if I were to click a folder in the Google Cloud Storage web browser, it would take me to the directory even if there were spaces in the directory. The name of the folders in the breadcrumb navigation are showing up as:
Unknown%20Company
When previously it would have been:
Unknown Company
It seems like this has recently started happening, is there some kind of changelog for the frontend of this application? All the files are still accessible through gsutils if I do something like:
gsutil ls -a gs://bucket-name/Folder\ Name
I've confirmed this is an issue in the GCS UI and is being worked on. I'll post an update here when it has been resolved.
Update: As of 2018-05-08 03:12 PDT, the issue has been resolved.
I've been trying to figure out how to use the google closure library for d-pad navigation for my web app. I cannot for the life of me get closure to work. I'm following google's guide here https://developers.google.com/tv/web/lib/closure/gettingstarted
The guide asks to checkout from their SVN repo, but I'm just using the downloaded zip of the library from github. The guide asks to perform the following command
gtv-ui-lib/closure-library/closure/bin/build/depswriter.py \
--root_with_prefix="gtv-ui-lib/source ../../../../gtv-ui-lib/source" \
--output_file=tv-deps.js
to create a deps.js file. Can somebody explain to me how to execute this command? Is it supposed to be in a python command prompt or something else? When I copy and paste their exact html and javascript page code and use the premade deps.js file for the guide, my page doesn't function with keyboard navigation, so I'm assuming the deps.js file is my problem because everything else is exactly from the guide.
You will need to have python installed on your development machine and make sure that it is added to your path environment. The closure depswriter can be executed by using python from the command line (if you are using *nix then you can just invoke the file from the cli). Try:
python WHERE-YOU-UNZIPPED-THE-ZIP-PATH/gtv-ui-lib/closure-library/closure/bin/build/depswriter.py \
--root_with_prefix="gtv-ui-lib/source ../../../../gtv-ui-lib/source" \
--output_file=tv-deps.js
I would also recommend checking out the source from the SVN repo and not using the github zip file. The SVN repo is the most upto date and the github is a mirror that can sometimes fall behind. To ensure you have the latest version of the code do install the SVN tools and checkout from the specified repo.
I require a small space online (free) where I can
upload/download few files automatically using a script.
Space requirement is around 50 MB.
This should be such that it could be automated so I can set
it to run without manual interaction i.e. No GUI
I have a dynamic ip & have no tech on setting up a server.
Any help would be appreciated. Thanks.
A number of online storage services provide 1-2 GB space for free. Several of those have command-line clients. E.g. SpiderOak that I use has a client that can run in a headless (non-GUI) mode to upload files, and there's even a way to download files from it by wget or curl.
You just set up things in GUI mode, then put files into the configured directory and run SpiderOak with right options; files get uploaded. Then you either download ('restore') all or some of the files via another SpiderOak call or get them via HTTP.
About the same applies to Dropbox, but I have no experience with that.
www.bshellz.net gives you a free shell running Linux. I think everyone gets 50mb so you're in luck!