I am trying to run ibm_db in a jupyter notebook. When I run ibm_db I get the below error.
ImportError Traceback (most recent call last)
in ()
----> 1 import ibm_db
ImportError: dlopen(/Users/myName/anaconda/envs/householding/lib/python3.6/site-packages/ibm_db.cpython-36m-darwin.so, 2): Library not loaded: libdb2.dylib
Referenced from: /Users/myName/anaconda/envs/householding/lib/python3.6/site-packages/ibm_db.cpython-36m-darwin.so
Reason: image not found
When i run os.getcwd() I get '/Users/myName'
What I think is happening is that because my current directory is to levels down from the start of the path dlopen is looking for, it is failing. I've done some looking around but can't find a way to change where dlopen is looking
You have to actually update your environment variable
DYLD_LIBRARY_PATH
to include
/ibm_db-2.0.8-py3.6-macosx-10.6-intel.egg/clidriver/lib
If you have installed ibm_db-2.0.8 on python3.6,
On terminal write
export DYLD_LIBRARY_PATH=/Users/myName/anaconda/envs/householding/lib/python3.6/site-packages/ibm_db-2.0.8-py3.6-macosx-10.6-intel.egg/clidriver/lib
It should work like a charm after this.
For reference checkout this:Issues with MAC OS X
I was having the same error and found that the installDSDriver script creates a file at /Applications/dsdriver/db2profile stating the below:
# NAME: db2profile
#
# FUNCTION: This script sets up a default database environment for
# Bourne shell or Korn shell users.
#
# This file is tuned for IBM Data Server Driver Package only.
#
# USAGE: . db2profile
# This script can either be invoked directly as above or
# it can be added to the user's .profile file so that the
# database environment is established during login.
#
so I just added on my ~/.bash_profile the line below:
source /Applications/dsdriver/db2profile
Open a new terminal window or restart and should work.
This file exports all the environment variables needed for the db2cli command to work.
Related
After installing Spark I am trying to run PySpark from the installation folder:
opt/spark/bin/pyspark
But I get the following errors:
opt/spark/bin/pyspark: line 24: /opt/spark/bin/load-spark-env.sh: No such file or directory
opt/spark/bin/pyspark: line 68: /opt/spark/bin/spark-submit: No such file or directory
opt/spark/bin/pyspark: line 68: exec: /opt/spark/bin/spark-submit: cannot execute: No such file or directory
Why is this happening when I can see these items in their respective directories? I'm also trying to get PySpark to run standalone as a command, but I'd imagine that I must solve the former problem first.
I am running this on macOS.
This error indicates that SPARK_HOME is not set. Try this:
export SPARK_HOME=/opt/spark
pyspark
FYI, it is strongly recommended to install software on mac OS using a package manager, like https://brew.sh
This is the configuration:
export SPARK_HOME=<YOUR-PATH>/spark-2.4.4-bin-hadoop2.7
export PYTHONPATH=$SPARK_HOME/python:$PYTHONPATH
export PYTHONPATH=$SPARK_HOME/python/lib/py4j-0.10.7-src.zip:$PYTHONPATH
And if you are thinking to use notebook as well:
export PYSPARK_DRIVER_PYTHON="jupyter"
export PYSPARK_DRIVER_PYTHON_OPTS="notebook"
export PYSPARK_PYTHON=python3
export PATH=$SPARK_HOME:$PATH:~/.local/bin:$JAVA_HOME/bin:$JAVA_HOME/jre/bin
I'm trying to flesh out the workflow for a JupyterHub server in the case where a user creates an environment and wants to share it with another user. I want to test out one of the two methods.
I am trying to create an environment in a public path, and then have another user add the conda environment as a kernel. So far it looks like this.
# How the environment is created
jupyter#ip:~$ conda create -p /home/envs/test --clone root
# Current setup
nick {~}$ jupyter kernelspec list
Available kernels:
python3 /opt/conda/share/jupyter/kernels/python3
nick {~}$ conda env list
# conda environments:
#
test /home/envs/test
nenv /home/nick/.conda/envs/nenv
base * /opt/conda
nick {~}$ cat .condarc
envs_dirs:
- /home/envs
My problem is I get an error when I try and install the kernel libstdc++.so.6.0.21 does not exist. What is libstdc++.so.6.0.21?
# error when trying to install kernel
nick {~}$ jupyter kernelspec install --user /home/envs/test
[InstallKernelSpec] Removing existing kernelspec in /home/nick/.local/share/jupyter/kernels/test
Traceback (most recent call last):
File "/opt/conda/bin/jupyter-kernelspec", line 11, in <module>
sys.exit(KernelSpecApp.launch_instance())
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/jupyter_client/kernelspecapp.py", line 273, in start
return self.subapp.start()
File "/opt/conda/lib/python3.7/site-packages/jupyter_client/kernelspecapp.py", line 143, in start
replace=self.replace,
File "/opt/conda/lib/python3.7/site-packages/jupyter_client/kernelspec.py", line 346, in install_kernel_spec
shutil.copytree(source_dir, destination)
File "/opt/conda/lib/python3.7/shutil.py", line 365, in copytree
raise Error(errors)
shutil.Error: [('/home/envs/test/lib/libstdc++.so.6.0.21', '/home/nick/.local/share/jupyter/kernels/test/lib/libstdc++.so.6.0.21', "[Errno 2] No such file or directory: '/home/envs/test/lib/libstdc++.so.6.0.21'")]
Note: I found this question which is similar; however, I found through this github thread why gcc had to be removed, and I already verified that it didn't exist in my environment with:
nick {~}$ conda list --name test | grep 'gcc'
_libgcc_mutex 0.1 main
libgcc 7.2.0 h69d50b8_2
libgcc-ng 8.2.0 hdf63c60_1
I already have libgcc installed at a higher version as you can see above so I didn't think the other answer would do too much good either.
filename shows up as red with this, so I believe the link is broken.
(test) nick {~}$ ls -al /home/envs/test/lib/libstdc++.so.6.0.21
lrwxrwxrwx 1 jupyter jupyter 19 Aug 9 09:42 /home/envs/test/lib/libstdc++.so.6.0.21 -> libstdc++.so.6.0.24
I realized I was using jupyter kernelspec in a non-intended way here. It is not intended to actually create the kernel, but only to add a kernelspec if it exists (see here, at the bottom).
There are two options for writing a kernel:
You can reuse the IPython kernel machinery to handle the communications, and just describe how to execute your code. This is much simpler if the target language can be driven from Python. See Making simple Python wrapper kernels for details.
You can implement the kernel machinery in your target language. This is more work initially, but the people using your kernel might be more likely to contribute to it if it’s in the language they know.
So, in my case what I really wanted to do was use IPythons utilities (option 1 above) which is documented well here. In which case to add the shared conda environment as a kernel in a way the users can access it I just need to run.
su - <user>
conda activate test
python -m ipykernel install --user --name test --display-name "Python (test)"
Fresh install of conda. I'm trying to activate an environment (any environment) and I get the following
(base) PS C:\Users\AndrewBergan> conda activate test123
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "C:\Users\AndrewBergan\AppData\Local\Continuum\anaconda3\lib\site-packages\conda\cli\main.py", line 138, in main
return activator_main()
File "C:\Users\AndrewBergan\AppData\Local\Continuum\anaconda3\lib\site-packages\conda\activate.py", line 940, in main
print(activator.execute(), end='')
File "C:\Users\AndrewBergan\AppData\Local\Continuum\anaconda3\lib\site-packages\conda\activate.py", line 173, in execute
return getattr(self, self.command)()
File "C:\Users\AndrewBergan\AppData\Local\Continuum\anaconda3\lib\site-packages\conda\activate.py", line 150, in activate
builder_result = self.build_activate(self.env_name_or_prefix)
File "C:\Users\AndrewBergan\AppData\Local\Continuum\anaconda3\lib\site-packages\conda\activate.py", line 275, in build_activate
return self._build_activate_stack(env_name_or_prefix, False)
File "C:\Users\AndrewBergan\AppData\Local\Continuum\anaconda3\lib\site-packages\conda\activate.py", line 332, in _build_activate_stack
self._replace_prefix_in_path(old_conda_prefix, prefix))
File "C:\Users\AndrewBergan\AppData\Local\Continuum\anaconda3\lib\site-packages\conda\activate.py", line 550, in _replace_prefix_in_path
assert last_idx is not None
AssertionError
What am I doing wrong?
Turns out for some reason Continuum/anaconda3/Library/bin did not get added to my path. Adding this fixed the problem.
Edit:
As noted in the comment below, other modifications to your path might be needed. I was able to determine what was missing from my path by adding a couple print statements to activate.py and comparing the output from the conda bash window and powershell.
you can just update conda using following code
conda update -n base conda
Because conda 4.6.14 fixed this problem.
conda start to support Powershell in version 4.6. upgrade your conda, then run this:
conda init
you should be fine then.
I was too struggling with this trouble for a long time but couldn't find any solution online.
But I found it at last.
This is occurring because Windows PowerShell is not allowed to run local PowerShell scripts. It can be enabled in developer settings.
On windows 10
Settings > Update & Security > For developers > Enable PowerShell to run local PowerShell scripts without signing in.
see where to enable
I am trying to install Postgres in order to use Heroku.
I am following the instructions in the Heroku tutorial, and after Postgres installation (which was successful), it says to configure my .bash_profile to allow for Postgres command line functionality.
I am following the instructions here, but I am unable to successfully add this line:
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/latest/bin
That folder does in fact contain "psql" on my computer, so it should work. Here is my current .bash_profile:
# Setting PATH for Python 2.7
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
# The next line updates PATH for the Google Cloud SDK.
source '/Users/user/google-cloud-sdk/path.bash.inc'
# The next line enables shell command completion for gcloud.
source '/Users/user/google-cloud-sdk/completion.bash.inc'
I tried to add the Postgres line to the end of that file, but it is not working. After searching online, there does not seem to be consensus on how to add PATHs to .bash_profile. I have tried many versions listed, but none have worked.
Please let me know if I'm doing this incorrectly!
Add this line to the end of your .bash_profile:
export PATH=/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH
This makes the search of a binary to look in that location "BEFORE the rest of the PATH"
Kill all instances of terminal and open it again, then it should work.
Try which xxx where xxx is the name of some binary inside /Applications/Postgres.app/Contents/Versions/latest/bin and check if it's returning that location.
Tell me if it works.
I'm executing this code:
p = subprocess.Popen(['/path/to/my/script.sh','--flag'] , stdin=subprocess.PIPE)
p.communicate(input='Y')
p.wait()
It works when executing it on the shell using "python scriptName.py",
BUT when executing using PyDev in Eclipse, it fails, the reason:
/path/to/my/script.sh: line 111: service: command not found
This bash script "script.sh" contains the following command which causes the error:
service mysqld restart
So "service" is not recognized when running the .sh script from the context of PyDev.
I guess it has to do with some ENV VAR configurations, couldn't find how to do it.
BTW - Using "shell=True" when calling subprocess.Popen didn't solve it.
service usually is located in /usr/sbin, and that this directory isn't on the PATH. As this usually contains administrative binaries and scripts which arn't designed to be run by everyone (only by admins/root), the sbin directories arn't always added to the PATH by default.
To check this, try to print PATH in your script (or add an env command).
To fix it, you could either
set the PATH in your python script using os.setenv
pass an env dict containing the correct PATH to Popen
set the PATH in your shellscript
use the full path in your shellscript
set the PATH in eclipse