I am trying to use snowpark(0.6.0) via Jupiter notebooks(after installing Scala almond kernel). I am using Windows laptop. Had to change the examples here a bit to work around windows. Following documentation here
https://docs.snowflake.com/en/developer-guide/snowpark/quickstart-jupyter.html
Ran into this error
java.lang.NoClassDefFoundError: Could not initialize class com.snowflake.snowpark.Session$
ammonite.$sess.cmd5$Helper.<init>(cmd5.sc:6)
ammonite.$sess.cmd5$.<init>(cmd5.sc:7)
ammonite.$sess.cmd5$.<clinit>(cmd5.sc:-1)
Also tried earlier with IntelliJ IDE,got bunch of errors with missing dependencies for log4j etc.
Can I get help.
Have not set it up in Widows but only with Linux.
You have to do the setup steps for each notebook that is going to use Snowpark (part from installing the kernel).
It's important to make sure you are using a unique folder for each notebook, as in step 2 in the guide.
What was the output of the import $ivy.com.snowflake:snowpark:0.6.0?
Related
I am trying to get the MNIST dataset in Julia using the MLDatasets package via the following commands in a .jl file:
using MLDatasets
dataset_train = MNIST(:train)
dataset_test = MNIST(:test)
This is what's indicated in the MLDatasets documentation. However, I am getting the following error:
ERROR: MethodError: objects of type Module are not callable
Stacktrace:
[1] top-level scope
# c:\my\julia\file\...
For some reason Julia is interpreting MNIST to be a module. It is worth noticing that this exact same code works without problem in a friend's computer, who has the same operating system, and the same Julia version (OS: Windows, x86_64-w64-mingw32; Julia version: 1.8.2).
The MLDatasets package is installed and does import correctly with the using command. What may be the problem?
The most likely reason is that you have an older version of MLDatasets installed on your computer. The current version is 0.7.5, while you probably have a 0.5.x version installed. This can happen if there are package conflicts that prevent the latest version of MLDatasets from being installed.
If you're using the default v1.8 environment to install all packages, conflicts like this are more likely to happen. It's recommended to instead create a separate environment for your project, and install only the packages you need for that particular project in that environment.
Older versions of MLDatasets had MNIST as a sub-module rather than as a type, which is the reason for this particular error message.
You can compare your environment's status (] status) and the one in your friend's setup. ] status --outdated can also help you see which particular package is causing the issue here. But I would suggest just keeping your base Julia environment (v1.8) minimal, uninstalling packages except the ones that you need to be available everywhere (eg. IJulia). See also the "Creating your own projects" section of the Pkg documentation.
pyodbc has a .pyi file but when running pytest-mypy, I have this error:
__________________________________________________________________________________________________ connexion.py __________________________________________________________________________________________________
3: error: Cannot find implementation or library stub for module named "pyodbc"
3: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
_
This should only happen when the library doesn't have stub files but it appears there are stub files. What should I do?
I'm using Python 3.10.2 and i've updated pyodbc to the latest version (pyodbc==4.0.34)
Let's assume your python is installed in /usr. In that case, your python executable will be in /usr/bin, and any libraries you install with pip will be installed in /usr/lib/python3.10/site-packages. In this case, all the sources for pyodbc can be found in /usr/lib/python3.10/site-packages/pyodbc.
Following this pattern we would expect to find type stubs in /usr/lib/python3.10/site-packages/pyodbc.pyi, but there is an issue in pyodbc, so that the stubs are actually installed in /usr/pyodbc.pyi.
In order to pick up this path, you will need to modify settings in your development environment. In Linux, try setting PYTHONPATH=/usr in your environment. The link mentioned in rogdham's comment includes others' comments on how to make this work in VS Code. Other development environments should support similar workarounds.
Based on web search and as highly recommended, I am trying to run Jupyter on my local for Scala (using spylon-kernel).
I was able to create a notebook but while trying to run/play a Scala code snippet, I see this message initializing scala interpreter and in the console, I see this error:
ValueError: Couldn't find Spark, make sure SPARK_HOME env is set or Spark is in an expected location (e.g. from homebrew installation).
I am not planning to install Spark. Is there a way I can still use Jupyter for Scala without installing Spark?
I am new to Jupyter and the ecosystem. Pardon me for the amateur question.
Thanks
I actually work on zeppelin with spark and scala. I want to import the library which contain : import com.databricks.spark.xml.
I tried but I have still the same mistake in zeppelin mistake : <console>:25: error: object databricks is not a member of package com.
What I've done actually ? I create a note in Zeppelin with this code : %dep
z.load("com.databricks:spark-xml_2.11:jar:0.5.0"). Even with that, the interpreter don't work. It's like it don't succeed to load the library.
Have you an idea why it don't work ?
Thanks for your help and have a nice day !
Your problem is very common and not intuitive to solve. I resolved an issue similar to this (I wanted to load the postgres jdbc connector in AWS EMR and I was using a linux terminal). Your issue can be resolved by checking if you can:
load the jar file manually to the environment that is hosting Zeppelin.
add the path of the jar file to your CLASSPATH environment variable. I don't know where you're hosting your files that manage your CLASSPATH env, but in EMR, my file, viewed from the Zeppelin root directory, was here: /usr/lib/zeppelin/conf/zeppelin-env.sh
download the zeppelin interpreter with
$ sudo ./bin/install-interpreter.sh --name "" --artifact
add the interpreter in Zeppelin wby going to the Zeppelin Interpreter GUI and add in the interpreter group.
Reboot Zeppelin with:
$ sudo stop zeppelin
$ sudo start zeppelin
It's very likely that your configurations may vary slightly, but I hope this helps provide some structure and relevance.
I want to install Go on CentOS 5.x, but Go website here http://golang.org/doc/install mentions that it is not supported.
Is there still some way to install Go Language on CentOS since CentOS is just a different flavour of Linux?
When I installed the same and ran the sample program hello.go mentioned at above website
I got the error
hello.go:3:8: import "fmt": cannot find package
package runtime: import "runtime": cannot find package
% export GOROOT=~/
% go run test.go
test.go:3:8: import "fmt": cannot find package
package runtime: import "runtime": cannot find package
Not finding the "runtime" package tells you that the $GOROOT enviroment variable isn't set to golang root directory.
You need to set $GOROOT. Also you should set the $GOPATH variable if you use external packages (go help gopath for more information).
A typical setup (not specific to CentOS) would be:
export GOROOT="/usr/local/go"
export PATH="$GOROOT/bin:$PATH"
export GOPATH=/Users/rodowi/gocode
At least for some programs it works fine. I have a production program that I compile on RHEL 6.x and deploy on 5.x and 6.x and it is working without any problems.
Edit: I used to use it under 1.0.3, but a few months ago I upgraded to "tip go" (the soon to be 1.1 version). Under 1.x it'd crash on 5.9 when accepting a tcp connection but that was fixed a few days after I reported it on the mailing list.
http://dave.cheney.net/2013/06/18/how-to-install-go-1-1-on-centos-5
This page explains rather well what kernel primitives are missing from the CentOS-5 kernel, their impact, and possible fixes.
I'm affraid you're out of luck because of the bit too much aged kernel version of CentOS 5.x. If the go runtime uses features simply not present in that kernel, then I see no easy way how it could work.
you could download the rpm package and begin installing locally, check below and download according to your OS:
http://pkgs.org/download/golang
or you could alternatively compile and build from source:
http://dave.cheney.net/2013/06/18/how-to-install-go-1-1-on-centos-5