Can't install packages using pip in Virtualenv on remote using Fabric - deployment

I have being using fabric to deploy an app with virtualenv. I was using fabric 1.4 and upgraded to 1.5.1 last week. My script stopped working.
It can't install the requirements. It seems it's not activating the virtualenv. In my code, I have:
with cd('%(path)s' % env):
with prefix('source bin/activate'):
run('pip install -U distribute')
I'm getting a permission denied error: error: could not delete '/usr/local/lib/python2.7/dist-packages/pkg_resources.py': Permission denied
The command being executed is:
Executed: /bin/bash -l -c "cd /var/www/myproject && source bin/activate && export PATH=\"\\$PATH:\\"/var/www/myproject\\" \" && pip install -U distribute"
If I ssh to the remote machine and run cd /var/www/myproject && source bin/activate && pip install -U distribute, it works just fine.
Why is my fabric script not working?
Thanks in advance

Instead of the serial approach with..
source bin/activate
pip install -U distribute
..directly use the pip executable of the virtualenv:
myenv/bin/pip install -U distribute

Although not exactly a solution, fabtools has a number of functions related to virtualenvs that are very handy. They do (almost) all of the hard work for you, and are probably worth using to check it isn't something else going wrong.
# Cut (and modified) from the fabtools documentation
from fabric.api import *
from fabtools import require
import fabtools
#task
def setup():
# Require a Python package
with fabtools.python.virtualenv('/home/myuser/env'):
require.python.package('pyramid')

Related

Create GitHub actions that use powershell scripts

I'd like to create a GitHub action that sets up an environment in Windows, running a few Powershell commands. Despite this can be done easily as a step, there does not seem to be a way to create a complete GitHub action for that. If I use this:
name: 'Rakudo Star fix for windows'
description: 'Updates zef for RakudoStar'
author: 'JJ'
runs:
using: 'node12'
main: 'upgrade.ps1'
There does not seem a way to run anything other than a JS script, or even to declare the environment. I understand that's left for later, during the job steps, but anyway it looks like a hack. Is there anything I'm missing here?
You could also run docker directly with an entrypoint for the .ps1 script
FROM ubuntu:18.04
LABEL "com.github.actions.name"="test"
LABEL "com.github.actions.description"="test."
RUN apt-get update \
&& apt-get install wget -y \
&& wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb \
&& dpkg -i packages-microsoft-prod.deb \
&& apt-get update \
&& apt-get install -y powershell
ADD test.ps1 /test.ps1
ENTRYPOINT ["pwsh", "/test.ps1"]
Update:
The using field is the application to use to execute the code specified in main. But Github Actions only support using node12 and docker. As seen from this GHActions I just ran for example's sake.
Docker won't run in most Windows environment and you'd have to use Windows Server 2019 as your base environment.

detailed guide for pgadmin deployment - Windows 7

I am an absolute new to postgresql, pgadmin and all it's tools. And with no python knowledge. I need to deploy pgadmin in as a web application on windows 7 os.
I have been trying the pgadmin's documentation from the official site , with no results. always errors, they ask u to do 1 2 3 but without where and how to check and so on....
I need a detailed guide for pgadmin 4 v 2 installation and deployment as a web application - with or without postgreSql.
Thanks...
For pgAdmin 4 v2.0 on Ubuntu, according to the download page:
Install dependencies, create a virtual environment, download, install & configure
sudo apt-get install virtualenv python-pip libpq-dev python-dev
cd
virtualenv pgadmin4
cd pgadmin4
source bin/activate
pip install https://ftp.postgresql.org/pub/pgadmin/pgadmin4/v2.0/pip/pgadmin4-2.0-py2.py3-none-any.whl
Configure
Override default paths and set it to single-user mode in the local configuration file:
nano lib/python2.7/site-packages/pgadmin4/config_local.py
Write:
import os
DATA_DIR = os.path.realpath(os.path.expanduser(u'~/.pgadmin/'))
LOG_FILE = os.path.join(DATA_DIR, 'pgadmin4.log')
SQLITE_PATH = os.path.join(DATA_DIR, 'pgadmin4.db')
SESSION_DB_PATH = os.path.join(DATA_DIR, 'sessions')
STORAGE_DIR = os.path.join(DATA_DIR, 'storage')
SERVER_MODE = False
Run
python lib/python2.7/site-packages/pgadmin4/pgAdmin4.py
Access
Access at http://localhost:5050
Exit
Exit with Ctrl-C
Run again
cd ~/pgadmin4
source bin/activate
python lib/python2.7/site-packages/pgadmin4/pgAdmin4.py
Make a shortcut
touch ~/pgadmin4/pgadmin4
chmod +x ~/pgadmin4/pgadmin4
nano ~/pgadmin4/pgadmin4
Write:
#!/bin/bash
cd ~/pgadmin4
source bin/activate
python lib/python2.7/site-packages/pgadmin4/pgAdmin4.py
Now you can just run it with a simpler command:
~/pgadmin4/pgadmin4
Ref: link

How to use virtualenv + virtualenvwrapper properly with Vagrant?

I found that the most convenient way of installing virtualenv + virtualenvwrapper is by using virtualenvburrito.
Now I can manage to automate my pip installs in a vagrant provision by the following:
Line in Vagrantfile:
config.vm.provision :shell, :path => "bootstrap.sh"
Lines in bootstrap.sh:
curl -s https://raw.github.com/brainsik/virtualenv-burrito/master/virtualenv-burrito.sh | $SHELL
source /root/.venvburrito/startup.sh
cd /vagrant
mkvirtualenv my_project
pip install -r requirements.txt
Then I run vagrant ssh but then I have to run the following to access my virtual environment:
sudo -i
source /root/.venvburrito/startup.sh
workon my_project
I don't want to always have to run sudo -i and source /root/.venvburrito/startup.sh, I just want to be able to run workon my_project directly.
But
(I.) I can't seem to append source /root/.venvburrito/startup.sh to my ~/.profile and
(II.) even if it was appended to that file I'd get a permissionerror. I can't seem to change the permissions for any protected file either.
The best way to deal with (I.) and (II.) is to set the privileged attribute in the Vagrantfile to false.
See here

AWS EC2 and rvm ssh

I have created user for my AWS ec2 VPS (deployer)
When i am logging with:
ssh -i ~/.ssh/aws/*...*.pem ubuntu#ec2*...*.amazonaws.com
command rvm use 2.0.0 is working correctly
=>
ubuntu#ip-***:~$ rvm list
rvm rubies
=* ruby-2.0.0-p247 [ x86_64 ]
# => - current
# =* - current && default
# * - default
ubuntu#ip-***:~$ rvm which
ubuntu#ip-***:~$
But when i use su - deployer i have got:
deployer#ip***:/home/ubuntu$ rvm
The program 'rvm' is currently not installed. You can install it by typing:
sudo apt-get install ruby-rvm
I would like to understand how correctly write command for ssh login.
I have tried:
ssh -i ~/.ssh/aws/*.pem *ubuntu#ec2***.amazonaws.com -t 'bash --login -c "rvm"'
but received "Connection to ec2-*.amazonaws.com closed".
Within my local machine rvm functioning correctly. I have added
[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function*
into my ~/.bash_profile
I have spent 3-5 hours studying stackoverflow topics related to this issue, but still not understand what am i doing wrong.
Any help will be highly appreciated! Thanks in advance!
I've run into this problem before and there are 2 ways to solve it.
The first way is to log in directly as the deployer user to the instance. This might mean having to create a ssh keypair (see ssh-keygen -t rsa). Then you can log in with ssh deployer#ec2.instance.address This way the rvm will be loaded directly to the deployed user's shell.
A second way is not to use the dash when su to the deployed user account.
When you use the dash then you load your own bashrc vs that particular user's bashrc.
So sudo su deployer
you nee to use:
su - deployer
it will ensure you use login shell

python and bpython using different PYTHONPATHs in Virtualenv

Something strange and unexpected is happening with the sys.path of any virtual environment I set. For example, a clean env:
$ virtualenv test
$ source test/bin/activate
(test) $
This is the expected PYTHONPATH:
(test) $ python
>>> import sys
>>> print '\n'.join(sys.path)
/home/user/test/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
/home/user/test/local/lib/python2.7/site-packages/pip-1.1-py2.7.egg
/home/user/test/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
/home/user/test/lib/python2.7/site-packages/pip-1.1-py2.7.egg
/home/user/test/lib/python2.7
/home/user/test/lib/python2.7/plat-linux2
/home/user/test/lib/python2.7/lib-tk
/home/user/test/lib/python2.7/lib-old
/home/user/test/lib/python2.7/lib-dynload
/usr/lib/python2.7
/usr/lib/python2.7/plat-linux2
/usr/lib/python2.7/lib-tk
/home/user/test/local/lib/python2.7/site-packages
/home/user/test/lib/python2.7/site-packages
But this is the one I really get:
(test) $ bpython
>>> import sys
>>> print '\n'.join(sys.path)
/usr/bin
/usr/lib/python2.7
/usr/lib/python2.7/plat-linux2
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
I can't figure out the reason of the two different sys.paths.
Because of that, no pip installation works!
I'm using Virtualenv 1.7.2, Ubuntu 12.04, Python 2.7.3.
Any help will be appreciated.
Rather than installing one copy of bpython per virtualenv, I've added this function to my shell profile (for example ~/.bashrc or ~/.zshrc). It wraps the bpython command with some logic to load the virtual environment's python path (if you have an active virtual environment).
bpython() {
if test -n "$VIRTUAL_ENV"
then
PYTHONPATH="$(python -c 'import sys; print ":".join(sys.path)')" \
command bpython "$#"
else
command bpython "$#"
fi
}
I found that I needed to deactivate and reactivate my virtualenv after installing bpython for it to work.
pip install bpython
deactivate
. bin/activate # or your equivalent activation command
My hypothesis is that you have not installed bpython after you have activated the new virtualenv.
I followed it up exactly like you mentioned:
mkvirtualenv bpython
(bpython)~ $ pip install bpython
(bpython)~ $bpython
and then ran the commands:
>>> import sys
>>> print '\n'.join(sys.path)
/Users/xxxx/.virtualenvs/bpython/bin
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/site-packages/pip-1.1-py2.7.egg
/Users/xxxx/.virtualenvs/bpython/lib/python27.zip
/Users/xxxx/.virtualenvs/bpython/lib/python2.7
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/plat-darwin
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/plat-mac
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/plat-mac/lib-scriptpackages
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/lib-tk
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/lib-old
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/lib-dynload
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/site-packages
and did the same thing again by running python under the activated virtualenv
(bpython)~ $ python
.....
>>> import sys
>>> print '\n'.join(sys.path)
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/site-packages/pip-1.1-py2.7.egg
/Users/xxxx/.virtualenvs/bpython/lib/python27.zip
/Users/xxxx/.virtualenvs/bpython/lib/python2.7
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/plat-darwin
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/plat-mac
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/plat-mac/lib-scriptpackages
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/lib-tk
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/lib-old
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/lib-dynload
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/Users/xxxx/.virtualenvs/bpython/lib/python2.7/site-packages
I saw no difference in the two results
I also discovered that if you have bpython installed locally, you need to create your virtualenv with --no-site-packages for it to work properly. If you created your virtualenv without that flag, you can create an empty file named no-global-site-packages.txt in ~/.virtualenvs/<env-name>/lib/python2.7/ as noted in this Stack Exchange answer.