How to disable line breaks in vscode when printing a spark-dataframe in a jupyter-notebook - visual-studio-code

I have my vscode set up to connect to a remote JupyterLab that connects to a spark cluster.
When printing a wide dataframe using df.show(false) the output wraps around and becomes unreadable.
What I have tried:
The same thing that works in JypterLab:
%%html
<style>
div.jp-OutputArea pre {
white-space: pre;
}
</style>
Setting Word Wrap to off in vscode (alt+z)
Google. I have found how to turn off wrapping the code, but nothing about the output.

Related

vscode python lint problem at the first column of first line

I have python modules that I run in VSCode for development.
And everything works fine until today VSCode suddenly said that one of the module had some binding problem and made the file name tab red.
VSCode the problem occurs in the first character of the first line which is the following standard line at every module of my program.
from __future__ import print_function # (at top of module)
But strangely VSCode did not raise the same problem for other module that starts with the same line.
The problem message is like this.
An internal error occurred while binding file "aModule.py": Error: Debug Failure. createKeyForReference received unexpected expression type
at createKeyForReference (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\codeFlowTypes.ts:240:9)
at createKeyForReference (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\codeFlowTypes.ts:218:25)
at C.visitAssignment (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:849:50)
at C.visitNode (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:132:29)
at C.walk (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:98:37)
at forEach (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:107:22)
at Array.forEach (<anonymous>)
at C.walkMultiple (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:105:15)
at C.walk (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:100:18)
at C._walkStatementsAndReportUnreachable (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:2583:22)
at C.visitSuite (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:345:14)
at C.visitNode (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:330:29)
at C.walk (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:98:37)
at callback (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:1290:22)
at C._trackCodeFlowExpressions (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:3236:9)
at C.visitIf (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:1265:48)
at C.visitNode (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:184:29)
at C.walk (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:98:37)
at C._walkStatementsAndReportUnreachable (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:2583:22)
at C.visitSuite (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:345:14)
at C.visitNode (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:330:29)
at C.walk (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\parseTreeWalker.ts:98:37)
at Object.callback (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:545:22)
at C._bindDeferred (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:4140:22)
at C.bindModule (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\binder.ts:298:14)
at callback (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\sourceFile.ts:1197:28)
at s.timeOperation (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\common\timing.ts:40:20)
at callback (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\sourceFile.ts:1184:38)
at t.LogTracker.log (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\common\logTracker.ts:43:20)
at t.SourceFile.bind (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\sourceFile.ts:1181:33)
at O._bindFile (c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\program.ts:931:34)
at c:\Users\Farn\.vscode\extensions\ms-python.vscode-pylance-2022.7.40\dist\pyright-internal\src\analyzer\program.ts:1051:18
Can someone help?
Thanks
Farn

how to fix/work-around cr/lf / LF no longer working in dev tools console

[honestly, this is a WTF for chromium project once again fixing something that wasn't broken -- until a recent update to chromium-based browsers (it shows up in Chrome 91 and Edge 91, but does not manifest itself in Opera 60, nor the latest Firefox)]
To wit: previously, the console would honor/interpret LF (ie \n ) in a character string written to the console log. for example, if you entered
xyz="\nabc\ndef\ng"
the console would display ->
"
abc
def
g"
Now it displays:
"\nabc\ndef\ng"
This totally bollixes all of the embedded functions previously written to be invoked from the console that are intended to write to the console log.
Using \n in output to console log has been promoted on this forum for nearly a decade.
So, is this a new 'feature' or a bug?

parcel watch only detects first file change

I have the following in ./js/parcel/build-js.js (it is more or less a simplification of exactly what the API docs example does, except that it takes an optional --watch argument):
#!/usr/bin/env node
const Bundler = require('parcel-bundler');
const path = require('path');
const watch = process.argv.indexOf('--watch') > 0;
if (watch) console.log('Watching files...');
(async function bundleJs() {
const jsBundler = new Bundler(path.join(__dirname, '../src/common.js'), {
watch,
hmr: false,
});
jsBundler.on('bundled', () => {
console.log('bundled!');
});
const bundle = await jsBundler.bundle();
console.log('done');
})();
When I run node js/parcel/build-js.js --watch, it detects the first change to src/common.js and prints:
Watching files...
✨ Built in 585ms.
bundled!
done
This is as I'd expect. When I edit and save src/common.js, it sees that and then the total output becomes (done gets deleted):
Watching files...
✨ Built in 585ms.
bundled!
✨ Built in 86ms.
bundled!
But after that, no file changes are detected. I make changes and save but it just sits there, producing no more output or updating the build. Why only once?
Note: If I do strace node js/parcel/build-js.js --watch, it seems to just sit on an unfinished epoll_wait(3,, which I guess means it's waiting for something, but maybe watching the wrong file...
Edit: Versions!
parcel-bundler: 1.12.3
node: 10.15.1
Ubuntu 18.04
Edit: using parcel watch
This appears to be a system-wide thing for me. I did yarn globals add parcel (which also installed 1.12.3), and now watching any JS file with parcel watch path/to/file.js does the same thing.
It turned out to be a conflict between Parcel's change detection and the default Vim setup. From the Hot Module Replacement docs:
Some text editors and IDE's have a feature called safe write that basically prevents data loss, by taking a copy of the file and renaming it when saved.
When using Hot Module Reload (HMR) this feature blocks the automatic detection of file updates, to disable safe write use the options provided below:
I added set backupcopy=yes to my .vimrc and it started working.
The solution for other editors is documented there as well.
It is a Parcel issue! I dropped it (until they fix it)
IMHO: I do not have to change my editor's behavior just to make bundler work correctly. (webpack works fine in the situation)

zeppelin-0.7.3 Interpreter pyspark not found

I get the below error when I use pyspark via Zeppelin.
The python & spark interpreters work and all environment variables are set correctly.
print os.environ['PYTHONPATH']
/x01/spark_u/spark/python:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python/lib/pyspark.zip:/x01/spark_u/spark/python:/x01/spark_u/spark/python/pyspark:/x01/spark_u/zeppelin/interpreter/python/py4j-0.9.2/src:/x01/spark_u/zeppelin/interpreter/lib/python
zepplin-env.sh is set with the below vars
export PYSPARK_PYTHON=/usr/local/bin/python2
export PYTHONPATH=${SPARK_HOME}/python:${SPARK_HOME}/python/lib/py4j-0.10.4-src.zip:${PYTHONPATH}
export SPARK_YARN_USER_ENV="PYTHONPATH=${PYTHONPATH}"
See the below log file
INFO [2017-11-01 12:30:42,972] ({pool-2-thread-4}
RemoteInterpreter.java[init]:221) - Create remote interpreter
org.apache.zeppelin.spark.PySparkInterpreter
org.apache.zeppelin.interpreter.InterpreterException:
paragraph_1509038605940_-1717438251's Interpreter pyspark not
found
Thank you in advance
I found a workaround for the above issue.The interpreter not found issue does not happen when I create note inside a directory.The issue only happens when I use notes at toplevel.Addionally I foud out that this issue does not happen in 0.7.2 version
Ex :
enter image description here

ipython notebook is not updating when I change my code

So, I ran into a weird issue using an ipython notebook and not sure what to do. Normally, when I run a part of the code, if there is an error, I would trace it back, fix it, and then re-run the code. I was doing a similar thing but even after making changes to the code, it looks like nothing is changing!
Here is the example... I am using Python 3.5 so xrange is gone. This then caused an error to be thrown:
XXXX
24 XXXX
25 XXXX
---> 26 for t in xrange(0,len(data),1):
27
28 XXXX
NameError: name 'xrange' is not defined
but after changing my code (which you can see below the difference in line 26), the same error pops up!
XXXX
24 XXXX
25 XXXX
---> 26 for t in range(0,len(data),1):
27
28 XXX
NameError: name 'xrange' is not defined
Any ideas on why this would be happening?
As Thomas K said, you're probably making a change in an external file that was not imported. There is a very useful command in ipython notebook for such cases, called autoreaload. With autoreaload, whenever you modify an external file you do not have to import it again because the extension takes care of it for you. For more information check: ipython autoreload.
Whenever using external files along with Ipython use autoreload. It will reload the external files every time before executing any code in IPython.
Add this at first cell of the IPython.
%load_ext autoreload
%autoreload 2
For me this was due to one of the following:
Cause 1: imported module not updated
Solution:
import importlib
importlib.reload(your_module)
Cause 2: other
Solution: restart the kernel, for jupyter notebook this is how
I have the same problem. I tried jupyter magic autoreload but it didn't work. Finally, I solved it in this way:
in the first cell, add
import My_Functions as my
import importlib
importlib.reload(my)
But notice if the module is imported in this way:
from My_Functions import *
I couldn't reload it properly.
I have the same issue sometimes. I think it has to do with memory - if I have a bunch of dataframes hanging around it seems to cause issues. If I restart the kernel using the Kernel > Restart option the problem goes away.
I have the same problem sometimes. I restarted the kernels but it didn't work.I try to run the cell (ctr+ enter) two or three times. then the result will be displayed according to the updated codes. I hope it helps.
insert new empty cell with + option, go to Kernel, choose Restart & Run All.
Then, fill in the new inserted cell and run again in Kernel, choose Restart & Run All.
It works with me.