When I run code
import open datasets as pd
dataset = 'link of data set'
od.download(dataset)
I get asked to input my username which after inputting and running jupyter keeps on running and I don't get the output to input my api
Any reason
Related
Assuming I executed pytest on a remote host, export results to JSON (using pytest-json-report) and I transfer resulting JSON file to another PC. How can I "import" the results so the caller get the exact same output like he/she executed tests locally (with all pretty printing stuff)?
I am running a databricks notebook from Azure devops pipeline using Execute Databricks notebook task. I am passing vertical name and branch name to my notebooks. Using dbutils am able to get the required values. I have Scala code and shell script code in the same notebook in different cells. I have to use the notebook parameters in the shell script which is in same notebook different cell.
Can someone please suggest me how i can use notebook parameters in the shell script
%scala
val verticalName = dbutils.widgets.get("vertical")
val branchName = dbutils.widgets.get("branch")
println(verticalName)
println(branchName)
%sh
echo $verticalName
echo $branchName
I know about nbconvert and use it to generate static html or ipynb files with the results output. However, I want to be able to generate a notebook that stays attached to a kernel that I already have running, so that I can do further data exploration after all of the template cells have been run. Is there a way to do that?
Apparently, you can do this through the Python API. I didn't try it myself, but for someone who will be looking for a solution, this PR has an example in the comments:
from nbconvert.preprocessors.execute import executenb, ExecutePreprocessor
from nbformat import read as nbread
from jupyter_client.manager import start_new_kernel
nb = nbread('parsee.ipynb', as_version=4)
kernel_name = nb.metadata.get('kernelspec', {}).get('name', 'python')
km, kc = start_new_kernel(kernel_name=kernel_name)
executenb(nb, kernel=(km, kc))
kc.execute_interactive('a') # a is a variable defined in parsee.ipynb with 'a = 1'
Not quite sure about your purpose. But my general solutions are,
to execute the notebook in command line and see the execution at the same time,
jupyter nbconvert --debug --allow-errors --stdout --execute test.ipynb
this will show the execute through all cells in debug mode even exception happens. but I can't see the result until the end of the execution.
to output the result to a html file, and then open the html file to see the results. I found this is more convenient.
jupyter nbconvert --execute --allow-errors --stdout test.ipynb >> result.html 2>&1
if you open result.html, it will be,
and all the errors and results will be shown on the page.
I would like to learn other answers/solutions from you all. thank you.
If I understood correctly you wish to open a Python console, and connect Jupyter notebook to that kernel instance?
Perhaps your solution would be to edit jupyter scripts itself and run the server in separate thread/background task implementing some sort of connection between threads and work in the jupyter console? Currently it's impossible because main thread is running the server.
This would require some work and I don't have any solution as-is, but I will look into that and maybe edit this answer if I can make it work.
But it seems that the easiest solution is to simply add another field in the notebook and do whatever you wish to do there. Is there a reason for not doing that?
I'm relatively new to NumPy/SciPy and IPython.
To execute a python script in the python interactive mode, we may use the following commands.
>>> import os
>>> os.system('executable.py')
Then the print outputs can be seen from the python prompt.
But the same idea doesn't work with IPython notebook.
In [64]:
import os
os.system('executable.py')
Out[64]:
0
In this case, I cannot see any print outputs. The notebook only tells weather execution was successful or not. Are there any ways to see the outputs when I use IPython notebook?
Use the magic function %run:
%run executable.py
This properly redirects stdout to the browser and you will see the output from the program in the notebook.
It gives you both, the typical features of running from command line plus Python tracebacks if there is exception.
Parameters after the filename are passed as command-line arguments to
the program (put in sys.argv). Then, control returns to IPython's
prompt.
This is similar to running at a system prompt python file args,
but with the advantage of giving you IPython's tracebacks, and of
loading all variables into your interactive namespace for further use
(unless -p is used, see below).
The option -t times your script. With -d it runs in the debugger pdb. More nice options to explore.
I'm interested in implementing a behavior in IPython that would be like a combination of ! and !!. I'm trying to use an IPython terminal as an adjunct to my (Windows) shell. For a long running command (e.g., a build script) I would like to be able to watch the output as it streams by as ! does. I would also like to capture the output of the command into the output history as !! does, but this defers printing anything until all output is available.
Does anyone have any suggestions as to how to implement something like this? I'm guessing that a IPython.utils.io.Tee() object would be useful here, but I don't know enough about IPython to hook this up properly.
Here is a snippet of code I just tried in iPython notebook v2.3, which seems to do what was requested:
import sys
import IPython.utils.io
outputstream = IPython.utils.io.Tee("outputfile.log", "w", channel="stdout")
outputstream.write("Hello worlds!\n")
outputstream.close()
logstream=open("outputfile.log", "r")
sys.stdout.write("Read back from log file:\n")
sys.stdout.write(logstream.read())
The log file is created in the same directory as the iPython notebook file, and the output from running this cell is displayed thus:
Hello worlds!
Read back from log file:
Hello worlds!
I haven't tried this in the iPython terminal, but see no reason it wouldn't work as well there.
(Researched and answered as part of the Oxford participation in http://aaronswartzhackathon.org)