I tried to run some commands on a remote node to test if Rundeck can display its environment variables, and it's weired that Rundeck can read the default env vars but not mine :
And result is :
I have defined my custom environment variables in different places (inside /etc/profile and ~/.bash_profile) but no luck..
Non interactive bash shell reads from .bashrc not .bash_profile. You might move your env var setting to .bashrc and source that file from .bash_profile.
Related
I am running some automated Python tests with GitLab-CI on a Windows 10 machine. The GitLab-Runner on the machine used to work with executor = "shell" using the simple Windows shell. This recently stopped working (The docs say support for this shell is depreceated) and the only way to get it work again has been to use the powershell instead with adding shell = "powershell" to our config.toml file. For the tests to run, we need to activate a conda environment. Unfortunately, this seems not to work via the powershell script that GitLab-CI creates for the job.
When I open the powershell manually logged in as the user that is executing the gitlab runner jobs, changing conda environments works. I have run conda init powershell and can change the environment with conda activate myenv. Yet, when I include the following in my gitlab-ci.yml file:
script:
- conda activate myenv
- conda list
the output from conda list confirms that the environment myenv is not activated and instead the base environment is used.
Also trying the absolute path like this
script:
- conda activate C:\Users\myuser\Miniconda3\envs\myenv
- conda list
does not work.
So it seems like I can manually activate the correct conda environment in the powershell, but activating the environment via the powershell script created by GitLab-CI does not work. Is there a fix for this problem? Any help is greatly appreciated.
Looks like gitlab executes each line of the script in a separate subshell. Combine the commands into a single line.
If that doesn't work, most conda commands will accept the name of the environment as parameter -n:
conda list -n myenv
conda install -n myenv PackageName
...
As long as you're just using conda, it shouldn't be necessary to activate the environment.
As it seems to be a problem within powershell but not with cmd, one could use the following in the gitlab-ci yaml:
- cmd '/C' 'conda activate myenv && python myunittests.py'
I'm new to the fish shell, and just trying to set my $EDITOR variable so that's it's persistent across sessions and reboots. Here's what I've tried so far:
Running set -gx EDITOR vim from the command line.
Running set -Ux EDITOR vim from the command line.
Running those commands, prefixed by set -e EDITOR to unset any previous value.
Adding the above commands to my ~/.config/fish/config.fish file (it complains set: Warning: universal scope selected, but a global variable “EDITOR” exists.)
Uninstalling oh-my-fish and removing all fish configs to start from scratch.
No matter what I do, the EDITOR variable always ends up being /usr/bin/nano whenever I open a new terminal, start a new session, or reboot. What's even more strange is that in ~/.config/fish/fishd.my-hostname, I see SET_EXPORT EDITOR vim, and nothing about nano. Is this some kind of fish default? If so, how can I set this correctly?
Edit: I'm running Fish 2.6.0 on Antergos Linux.
First, the fish config file is ~/.config/fish/config.fish. Editing the file you named won't have any effect. Second, fish does not have any default for, nor does it set, the EDITOR or VISUAL variables. So whatever is setting it to /usr/bin/nano is a customization unique to your system.
If you set -Ux EDITOR vim and do not set it in config.fish it should be set to vim even if it is already set when fish starts. Run that set command then do set -U | grep EDITOR and env | grep EDITOR to see that it is set as a universal variable and exported. Now type fish to start a sub-shell. Run the previous two commands and you should see that it is still set to the same value. Now type set -U EDITOR nano in the sub-shell followed by exit. In the earlier shell you should now see that EDITOR is set to nano.
Personally I don't like to use uvars for this since at the moment they are per-machine. I just do set -gX EDITOR (type -p vim) in my ~/.config/fish/config.fish. This ensures that if I start fish on a new machine on which I've installed my ~/.config/fish directory I get my expected defaults.
The other reason not to use a uvar in this way is that the resolution order is local scope, global scope, universal scope. Since environment vars imported when fish starts running are placed in the global scope they will shadow the uvar you defined. Do not export universal variables. It is unlikely to produce the desired results. Simply set -gx the var in your config.fish script.
P.S., When asking questions of this nature you should always include pertinent facts such as the OS you're using and the fish version.
I changed the environmental variable LD_LIBRARY_PATH from the Ubuntu terminal (because I was receiving an error) and the changes were applied when I ran code (a Python code) from the terminal. But when I ran the same script from the Visual Studio Code, the error remains. How to update the environmental variable so that Visual Studio Code sees it, as well?
Environment variables are passed from parent process to child process; they are not (say) global to the system or the user. If you change a variable in one shell, the change is only seen in that shell and any processes started from that shell. So the simplest solution is to change the variable and then start VSCode from that same shell:
$ export LD_LIBRARY_PATH=/some/useful/path
$ code
If you want to keep using that shell for other things, run it in the background:
$ code >/dev/null 2>&1 &
The redirection to /dev/null is needed because otherwise VSCode prints logging information periodically, and that output will be mixed with whatever else you're doing.
If you want to set the variable permanently, see the question How do I set a user environment variable? (permanently, not session). After following those instructions, you'll need to start a new shell (and possibly even logout and login) first so the settings take effect. Then launch VSCode from the new shell.
the default location for packages is .conda folder in my home directory. however, on the server I am using, there is a very strict limit of how much space I can use, which basically avoids me from putting anything under my home directory. how can I specify the location for the virtual environment that I want to create? Thanks! server is running Ubuntu.
Use the --prefix or -p option to specify where to write the environment files. For example:
conda create --prefix /tmp/test-env python=2.7
Will create the environment named /tmp/test-env which resides in /tmp/ instead of the default .conda.
like Paul said, use
conda create --prefix=/users/.../yourEnvName python=x.x
if you are located in the folder in which you want to create your virtual environment, just omit the path and use
conda create --prefix=yourEnvName python=x.x
conda only keep track of the environments included in the folder envs inside the anaconda folder. The next time you will need to activate your new env, move to the folder where you created it and activate it with
source activate yourEnvName
While using the --prefix option works, you have to explicitly use it every time you create an environment. If you just want your environments stored somewhere else by default, you can configure it in your .condarc file.
Please see: https://conda.io/docs/user-guide/configuration/use-condarc.html#specify-environment-directories-envs-dirs
If you want to use the --prefix or -p arguments, but want to avoid having to use the environment's full path to activate it, you need to edit the .condarc config file before you create the environment.
The .condarc file is in the home directory; C:\Users\<user> on Windows. Edit the values under the envs_dirs key to include the custom path for your environment. Assuming the custom path is D:\envs, the file should end up looking something like this:
ssl_verify: true
channels:
- defaults
envs_dirs:
- C:\Users\<user>\Anaconda3\envs
- D:\envs
Then, when you create a new environment on that path, its name will appear along with the path when you run conda env list, and you should be able to activate it using only the name, and not the full path.
Command line screenshot
In summary, if you edit .condarc to include D:\envs, and then run conda env create -p D:\envs\myenv python=x.x, then activate myenv (or source activate myenv on Linux) should work.
Hope that helps!
P.S. I stumbled upon this through trial and error. I think what happens is when you edit the envs_dirs key, conda updates ~\.conda\environments.txt to include the environments found in all the directories specified under the envs_dirs, so they can be accessed without using absolute paths.
You can create it like this
conda create --prefix C:/tensorflow2 python=3.7
and you don't have to move to that folder to activate it.
# To activate this environment, use:
# > activate C:\tensorflow2
As you see I do it like this.
D:\Development_Avector\PycharmProjects\TensorFlow>activate C:\tensorflow2
(C:\tensorflow2) D:\Development_Avector\PycharmProjects\TensorFlow>
(C:\tensorflow2) D:\Development_Avector\PycharmProjects\TensorFlow>conda --version
conda 4.5.13
I ran into a similar situation. I did have access to a larger data drive. Depending on your situation, and the access you have to the server you can consider
ln -s /datavol/path/to/your/.conda /home/user/.conda
Then subsequent conda commands will put data to the symlinked dir in datavol
You can modify the default paths for environments by modifying CONDA_ENVS_PATH:
For macOS and Linux: CONDA_ENVS_PATH=~/my-envs:/opt/anaconda/envs
For Windows: set CONDA_ENVS_PATH=C:\Users\joe\envs;C:\Anaconda\envs
the documentation is here:
https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#specify-environment-directories-envs-dirs
Use -p option to specify the path to your env.
For Linux/macOS, env location info is stored in ~/.conda/environments.txt.
Use conda info --envs to list all your envs.
I'm executing this code:
p = subprocess.Popen(['/path/to/my/script.sh','--flag'] , stdin=subprocess.PIPE)
p.communicate(input='Y')
p.wait()
It works when executing it on the shell using "python scriptName.py",
BUT when executing using PyDev in Eclipse, it fails, the reason:
/path/to/my/script.sh: line 111: service: command not found
This bash script "script.sh" contains the following command which causes the error:
service mysqld restart
So "service" is not recognized when running the .sh script from the context of PyDev.
I guess it has to do with some ENV VAR configurations, couldn't find how to do it.
BTW - Using "shell=True" when calling subprocess.Popen didn't solve it.
service usually is located in /usr/sbin, and that this directory isn't on the PATH. As this usually contains administrative binaries and scripts which arn't designed to be run by everyone (only by admins/root), the sbin directories arn't always added to the PATH by default.
To check this, try to print PATH in your script (or add an env command).
To fix it, you could either
set the PATH in your python script using os.setenv
pass an env dict containing the correct PATH to Popen
set the PATH in your shellscript
use the full path in your shellscript
set the PATH in eclipse