Change Conda environment via powershell script (for Gitlab-CI) - powershell

I am running some automated Python tests with GitLab-CI on a Windows 10 machine. The GitLab-Runner on the machine used to work with executor = "shell" using the simple Windows shell. This recently stopped working (The docs say support for this shell is depreceated) and the only way to get it work again has been to use the powershell instead with adding shell = "powershell" to our config.toml file. For the tests to run, we need to activate a conda environment. Unfortunately, this seems not to work via the powershell script that GitLab-CI creates for the job.
When I open the powershell manually logged in as the user that is executing the gitlab runner jobs, changing conda environments works. I have run conda init powershell and can change the environment with conda activate myenv. Yet, when I include the following in my gitlab-ci.yml file:
script:
- conda activate myenv
- conda list
the output from conda list confirms that the environment myenv is not activated and instead the base environment is used.
Also trying the absolute path like this
script:
- conda activate C:\Users\myuser\Miniconda3\envs\myenv
- conda list
does not work.
So it seems like I can manually activate the correct conda environment in the powershell, but activating the environment via the powershell script created by GitLab-CI does not work. Is there a fix for this problem? Any help is greatly appreciated.

Looks like gitlab executes each line of the script in a separate subshell. Combine the commands into a single line.
If that doesn't work, most conda commands will accept the name of the environment as parameter -n:
conda list -n myenv
conda install -n myenv PackageName
...
As long as you're just using conda, it shouldn't be necessary to activate the environment.

As it seems to be a problem within powershell but not with cmd, one could use the following in the gitlab-ci yaml:
- cmd '/C' 'conda activate myenv && python myunittests.py'

Related

Yarn Install - Not recognized as command in Windows 11 Powershell

Yarn's not recognized as a cmd command.
I tried installing it globally with 'npm install -g yarn' and adding a path variable to "C:\Users[user]\AppData\Roaming\npm\yarn".
Any other suggestions?
First thing first, you should check you have 16+ node.js installed and, you can check it with
node -v
If it's valid,
run the npm config get prefix it'll give you a path CTRL+C this path
Open the start search and type in env and then click "Edit the system environment variables"
Modify the Environment variables
Add the copied path to you PATH as new
After this you should be good, but if the PS is cannot load the Yarn run this command
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
Here a few things you can try:
You may need to close and reopen the Powershell windows because newly added environment variables or updates to them are not automatically reflected to already open shell windows.
Check if your path is correct: it normally is C:\Users\endo\AppData\Roaming\npm on my PC, not C:\Users\endo\AppData\Roaming\npm\yarn as you mentioned.
Try running yarn.cmd or yarn.ps1.
Try CMD instead of Powershell, see if it works that way.

Execute powershell script with gitlab-runner on local windows machine

I do have following setup:
a win PC with gitlab-runner installed (working)
a powershell script running on the same PC is starting an application
a gitlab server to connect this local PC and starting the powershell script
Now when starting the powershell script directly from the local PC, the application starts and terminates after done - working as expected. When starting the same powershell script with the gitlab server (yml-file) then I can see that the application has been started (new process in taskmanager) but it is not running as well it never terminates.
When manually end the task I see that gitlab terminates again.
Question:
what could be the root cause?
is it possible to run the powershell script with gitlab-runner? I think there is a way with the command "exec". How does the command looks like when calling the powershell script?
is it possible to run the application not in the background in order to see whats going on?
others?
thanks in advance
I think there is a bug with the gitlab runner on windows.
No matter which shell you configure in the config.toml the runner
will always use cmd.exe for an exec local run.
Specify the --shell argument to override the default cmd.exe shell:
> gitlab-runner exec shell your_job --shell pwsh
If you run this locally in your project, it outputs to .builds/, so add this to your .gitignore because git will see it and think you might want to add a submodule.

How to activate conda environment from powershell?

Version Anaconda 4.8.3
What I cannot do:
I can't activate any environment from powershell.
conda activate base
What I can do:
conda env list
conda create -n xxx conda remove -n xxx
I have tried many solutions, but they turn out to be useless:
conda init powershell
restart powershell
conda activate
conda update -n base conda and redo 1
conda install -n root -c pscondaenvs pscondaenvs
Set-ExecutionPolicy RemoteSigned
Run activate base
The result of running conda activate
You don't need Admin permission.
Once you install Anaconda or Miniconda on Windows, open a Anaconda Powershell Prompt from Start Menu.
Or, If you don't see it there, then assuming you have installed
miniconda3 at path C:\miniconda3\4.9.2, do:
powershell -ExecutionPolicy ByPass -NoExit -Command "& 'C:\miniconda3\4.9.2\shell\condabin\conda-hook.ps1' ; conda activate 'C:\miniconda3\4.9.2' "
Now try:
conda init powershell
and reopen powershell.
Additional note: By default conda will autoactivate itself, when we open terminal. If you prefer not, then disable auto-activation with:
conda config --set auto_activate_base false
Open a Anaconda Powershell Prompt from Start Menu.
Now Try:
conda init powershell
Now restart the powershell, if find some error like this in powershell:
\WindowsPowerShell\profile.ps1 cannot be loaded because running scripts is disabled on this system. For more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170. At line:1 char:3
Then change the execution policy. Type this code to powershell:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
To find more with execution policy you can visit this link.
Here is my easier solution which works with Anaconda, Miniconda, and even Miniforge:
Open Powershell and browse to condabin folder in your conda installation directory, for example: C:\Users\<username>\anaconda3\condabin
Run ./conda init powershell in that folder, and re-open the powershell.
Please note: If you encountered ps1 cannot be loaded because running scripts is disabled on this system, simply run the Powershell as Administrator and enter the following: Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
Restart the Powershell & Enjoy!
When you use Anaconda or Miniconda type in Anacondaprompt:
conda init powershell
In order to activate conda on Powershell, I just executed this command:
C:\Users\<username>\anaconda3\shell\condabin\conda-hook.ps1
Then I could see all environments without needing to restart Powershell.
conda env list
conda activate base
So you need to basically just run the conda-hook.ps1 script.
Encountered the same problem. The trick is to make sure that the environment is included in the PATH variable of Windows Powershell. To view the PATH variables enter the following in the powershell:
$env:Path.split(';')
If the environment you desire doesn't show up in the output list, as in
.....
.....
..../envs/yourEnv/...
.....
run the following command on cmd:
conda init powershell
then restart _Anaconda Powershell,
then:
conda activate yourEnv
Hopefully, now the conda prompt should now show your desired environment.
To solve the problem without changing te security policy on powershell, you need to be on powershell 7. Powershell 7 will let you use a \WindowsPowerShell\profile.ps1 file as startup script with default (restricted) security policy settings. Windows Powershell is by default (on windows 10 at 14-08-2022) version 5.x.
Check your powershell version with
$PSVersionTable
If you're still on powershell 5, install powershell from the windows store. This will install powershell 7 alongside powershell 5. It will probably be called Powershell instead of Windows Powershell. Or if you're not allowed to use the windows store, ask IT to install it for you. Check the version again in the new powershell.
After this, we can follow the answer of prashant:
Open Anaconda prompt and run
conda init powershell
To stop conda starting by default run
conda config --set auto_activate_base false
& 'C:\ProgramData\Miniconda3\shell\condabin\conda-hook.ps1'; conda activate 'C:\ProgramData\Miniconda3'
This is the command found in Anaconda Powershell Prompt (Miniconda3).lnk on my computer.
On my machine, I can enter the environment anywhere by executing these two commands.

Undo Conda Powershell prompt modification

Conda 4.6.14 over-writes my default PowerShell prompt from:
C:\whatevs\okay>
to:
(env) PS>
but I ideally want what it was previously:
(env) C:\whatevs\okay>
How can I undo this change to my PowerShell prompt? I like knowing what path and Conda environment I'm using whenever I execute a command! Is there a setting I can change?
I know it's possible to modify the PowerShell prompt by editing $profile. However, I want to keep the (env). Is there some quick way to get the current Conda environment so I can include it into a custom path?
Details
The file being modified by conda init is $HOME\Documents\WindowsPowerShell\profile.ps1 who's contents are:
#region conda initialize
# !! Contents within this block are managed by 'conda init' !!
(& "C:\Users\user\Anaconda3\Scripts\conda.exe" "shell.powershell" "hook") | Out-String | Invoke-Expression
#endregion
This is an issue with Conda 4.6.14 and can be resolved by downgrading back to 4.6.12 or upgrade to the (currently) experimental 4.7.

Pyenv activate does not run activate script with Fish Shell

My pyenv is working just fine, but it does not seem to be running my activate script located at /usr/local/var/pyenv/versions/project/bin/activate.fish
When activating my environment it gives the following output, but it does not echo anything from the activate script, which indicates that it is not running.
dani#localhost ~/d/project> pyenv activate project
pyenv-virtualenv: prompt changing not working for fish.
Of course I can just source the file manually, but I'm too eager to find out why it is not running. Is there some kind of debug mode? I'm not sure how to approach.
Actually,
Virtual environment is activated but the message just says that your prompt wasn't changed. Updating prompt was intentionally removed from fish shell.
you can find detailed information here;
https://github.com/pyenv/pyenv-virtualenv/issues/153
If you want to see virtual environment is really activated or not,
run the following command;
pyenv which python
it should print something like;
.pyenv/versions/{your-virtual-env}/bin/python
try this:
set PYENV_ROOT $HOME/.pyenv
set -x PATH $PYENV_ROOT/shims $PYENV_ROOT/bin $PATH
pyenv rehash