I am a windows users with a scala kernel set up on Jupyter Notebook. I have a ML model saved as .pmml file saved in a jar file and that jar file I need to put in the snowflake stage. However in the snowflake documentation the following command is used to do that
The command for windows is :
put file://c:\data\data.csv #~/staged;
My question is where should I execute the command with my user directory details,should it be in the scala kernel in Jupyter notebook, in cmd or in the snowflake itself?
My analogous command:
put file:C:\Users\psengar\myJar.jar #~/staged
You can install and run the put command on SnowSQL
Related
I've got a notebook that has got a bit unwieldy and I'm doing some refactoring which isn't fun.
I was wondering if it would be possible to execute code in this notebook from the command line for debugging.
Ideally, I would run something like:
run-in-jupyter $notebook file.py
and see the output from the command line. There is an interpreter in jupyterlab that can do this, so this make me think that it is possible.
I have a brief search but couldn't find much
How to run an .ipynb Jupyter Notebook from terminal? I explicitly don't want to do this (I want to run commands in an existing instace)
There is this library but this seems quite involved and some of the results I found on the internet where people not being able to use the library
jupyter console (pip install jupyter-console) connects to a running jupyter kernel from the kernel. Details on running kernels can be found amongst jupyter's run time files, on my box these live in ~/.local/share/jupyter/runtime. You can find the path to the kernel data file corresponding to an open workbook with %config IPKernelApp.connection_file which will look something like ~/.local/share/jupyter/runtime/kernel-55da8a07-b67d-4584-9ec6-f24e4a26cbbd.json.
You can then connect from the command line with
jupyter console --existing ~/.local/share/jupyter/runtime/kernel-55da8a07-b67d-4584-9ec6-f24e4a26cbbd.json
You can pipe commands into it as shown
echo h=87 | jupyter console --existing 55da8a07-b67d-4584-9ec6-f24e4a26cbbd 'h=57' --simple-prompt -y
I want to use commands such as mongodump/mongorestore/mongoexport on my local windows computer and have therefor downloaded the .msi extension version of mongodb database tool and also executed the .msi so it could be installed.
But what do i do now to start using the commands. I cant find any documentaions of this. Is there something i have to configue?
When i execute a command like mongodump on my system command line i get the response:
'mongodump' is not recognized as an internal or external command, operable program or batch file.
Do i maybe have to execute the command from a certain dir location?
Look for the highlighted folder on your system. When you install the database tools this folder is created. Inside this folder there is a bin folder which contains all the commands you are looking for.
Once you find this folder then navigate to bin folder. And open a command prompt at that location. And try running the commands.
Alternatively:-
You can set PATH to bin folder and then you will be able to run the commands from anywhere.
Using jupyter-lab
%run otherNotebook.ipynb
gives the following error message
Error: file not found otherNotebook.ipynb.py
How can I use the magic method and prevent it from adding .py to the file
As described here %run is for running a named file inside IPython as a program. Jupyter notebooks are not Python programs.
Notebooks can be converted to Python programs/scripts using Jupytext. Following that conversion you could then use %run.
Alternatively, you can use nbconvert to execute a notebook or use Papermill to execute a notebook. Papermill allows you to easily pass in parameters at the time of run. I have an example of both commented out in code under 'Step #5' here and 'Step#2' here.
If you are actually trying to bring the code into your present notebook, then you may want to explore importing Jupyter notebooks as modules. importnb is recommended here for making importing notebooks more convenient. Or, I just came across the subnotebook project that let's you run a notebook as you would call a Python function, pass parameters and get results back, including output contents.
I have a lot of .ipynb file stored in one folder. I have create a .sh file which contain only code jupyter notebook. Then I have change the preference > behavior in nautilus file manager to ask each time. Then when I double click the .sh file the box appear and I select run from terminal. But It is not executing and not opening then jupyter notebook. but if I do with terminal code it works as follows:
chmod +x /path/to/ipynbStart.sh
./ipynbStart.sh
I do the first approach with .bat file in windows and it works fine. I tried a lot But I could not find a solution. Thanks in advance.
It was not that difficult. I just create a new sh file (jupyter-start.sh) with the following command in the project directory I wanted:
#!/bin/bash
clear
sh -c /home/datapsycho/miniconda3/bin/jupyter-notebook
In the third line it's the path to the jupyter-notebook in my system.
Then run the chmod +x jupyter-start.sh to make it executable. Now I can place that file to all my project folders where I need to start jupyter notebook and it will always start in the specific project directory I want.
Now every time I just can execute that program to start jupyter notebook with out writing any thing in the shell.
I'm trying to launch PySpark notebook in my browser by typing in pyspark from the console, but I get the following error:
c:\Spark\bin>pyspark
python: can't open file 'notebook': [Errno 2] No such file or directory
What am I doing wrong here?
Please help?
Sounds like the jupyter notebook is either not installed or not in your path.
I prefer to use Anaconda for my python distribution and Jupyter comes standard and will install all necessary path information as well.
After that as long as you have set PYSPARK_PYTHON_DRIVER=jupyter and PYSPARK_PYTHON_DRIVER_OPTS='notebook' correctly you are good to go.
You want to launch the jupyter notebook when you invoke the command pyspark. Therefore you need to add the following to the bash_profile or zshrc_profile.
export PYSPARK_SUBMIT_ARGS="pyspark-shell"
export PYSPARK_DRIVER_PYTHON=ipython
export PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark