uWSGI + virtualenv 'No module named site' - virtualenv

So this seems to be a really common problem with this setup, but I can't find any solutions that work on SO. I've setup a very new Ubuntu 15.04 server, then installed nginx, virtualenv (and -wrapper), and uWSGI (via apt-get, so globally, not inside the virtualenv).
My virtualenv is located at /root/Env/example. Inside of the virtualenv, I installed Django, then at /srv/www/example/app ran Django's startproject command with the project name example, so I have vaguely this structure:
-root
-Env
-example
-bin
-lib
-srv
-www
-example
-app
-example
manage.py
-example
wsgi.py
...
My example.ini file for uWSGI looks like this:
[uwsgi]
project = example
plugin = python
chdir = /srv/www/example/app/example
home = /root/Env/example
module = example.wsgi:application
master = true
processes = 5
socket = /run/uwsgi/app/example/example.socket
chmod-socket = 664
uid = www-data
gid = www-data
vacuum = true
But no matter whether I run this via uwsgi --ini /etc/uwsgi/apps-enabled/example.ini or via daemon, I get the exact same error:
Python version: 2.7.9 (default, Apr 2 2015, 15:37:21) [GCC 4.9.2]
Set PythonHome to /root/Env/example
ImportError: No module named site
I should note that the Django project works via the built-in development server ./manage.py runserver, and that when I remove home = /root/Env/example the thing works (but is obviously using the global Python and Django rather than the virtualenv versions, which means it's useless for a proper virtualenv setup).
Can anyone see some obvious path error that I'm not seeing? As far as I can tell, home is entirely correct based on my directory structure, and everything else in the ini too, so why is it not working with this ImportError?

In my case, I was seeing this issue because the django app I was trying to run was written in python 3 whereas uwsgi was configured for python 2. I fixed the problem by:
recompiling uwsgi to support both python 2 and python 3 apps
(I followed this guide)
adding this to my mydjangoproject_uwsgi.ini:
plugins = python35 # or whatever you specified while compiling uwsgi
For other folks using Django, you should also make sure you are correctly specifying the following:
# Django dir that contains manage.py
chdir = /var/www/project/myprojectname
# Django wsgi (myprojectname is the name of your top-level project)
module = myprojectname.wsgi:application
# the virtualenv you are using (full path)
home = /home/ubuntu/Env/mydjangovenv
plugins = python35

As #Freek said, site refers to a python module.
The error claims that python cannot find that package, which is because you have specified python_home to the wrong location.
I've encountered with the same problem and my uwsgi.ini is like below:
[uwsgi]
# variable
base = /home/xx/
# project settings
chdir = %(base)/
module = botservice.uwsgi:application
home = %(base)/env/bin
For this configuration uwsgi can find python executable in /env/bin but no packages could be found under this folder. So I changed home to
home = %(base)/env/
and it worked for me.
In your case, I suggest digging into home directive and point it to a location which contains both python executable and packages.

The site module is in the root of django.
First check is to activate the virtualenv manually (source /root/Env/example/bin/activate, start python and import site). If that fails, pip install django.
Assuming that django is correctly installed in the virtualenv, make sure that uWSGI activates the virtualenv. Relevant uWSGI configuration directives:
plugins = python
virtualenv = /root/Env/example
and in case you have error importing example.wsgi:
pythonpath = /srv/www/example/app/example

Related

puppet 4.0 vagrant modules missing

I am trying to use puppet modules in vagrant.
My box is running puppet 4.0
I am installing modules using:
if [ ! -d /etc/puppet/modules/ ]; then
puppet module install puppetlabs-java
fi
in site.pp
I have:
class { 'java':
distribution => 'jdk',
}
I keep getting an error about could not find declared class java
why can't puppet find my module?
/etc/puppet/modules/ is the default path isn't it?
vagrant file
Vagrant.configure(2) do |config|
config.vm.box = "bento/centos-7.2"
config.vm.provider "virtualbox" do |vb|
vb.gui = true
vb.memory = "8192"
end
config.vm.provision :shell, :path => "upgrade_puppet.sh"
config.vm.provision :shell, :path => "puppet_modules.sh"
config.vm.provision :puppet do |puppet|
puppet.options = '--verbose --debug'
puppet.environment_path = "puppet/environments"
puppet.environment = "production"
end
end
Updated answer now that Vagrantfile has been provided
Locations have changed in puppet 4 and directory environments are now in use by default.
So how you are using the puppet provisioner is correct. However, vagrant will upload all the directories it needs to the guest, based on your Vagrantfile to:
/tmp/vagrant-puppet/environments/production
When Vagrant calls the puppet apply it will be looking for the modules it requires in:
/tmp/vagrant-puppet/environments/production/modules
and that module directory does not exist on your host.
You can change your if block to be:
if [ ! -d /vagrant/puppet/environments/production/modules ]; then
puppet module install puppetlabs-java --modulepath /vagrant/puppet/environments/production/modules
fi
/vagrant is shared between host and guest. This would install the java module and its dependencies on your host machine under:
puppet
|
+--environments
+
-- production
|
+ -- manifests
| +
| -- site.pp
|
+ -- modules
+
-- java
+
-- stdlib
When you do your vagrant up, this content gets uploaded to the host under:
/tmp/vagrant-puppet
Tested and confirmed based on your Vagrantfile.
As Jaxim mentions, it's because the default directory locations have changed in the newer version of Puppet.
If you're interested in installing moduels automatically with Puppet, I'd recommend the R10K vagrant plugin, you can specify versions of modules and make updating them much easier, and allows you to download modules not on the forge, such as git repos.
https://github.com/jantman/vagrant-r10k
A little bit late, but I am switching from Chef over to Puppet (company policy, do not ask! :) ) and ran into the exact same situation and coming from Chef background I was refusing to "pollute" my project folder with so many Puppet specific stuff. In my opinion, I should only need Vagrantfile and nothing else.
I was also getting the "Could not find declared class java at /tmp/vagrant-puppet/environments/production" error message. So, after much messing around I've found that in puppet.options you can provide any arguments that you would normally provide if calling puppet apply at the command line.
So, if anything helps try modifying the puppet.options in your Vagrantfile as follows:
config.vm.provision :puppet do |puppet|
puppet.options = '--verbose --modulepath=/etc/puppetlabs/code/environments/production/modules'
puppet.environment_path = "puppet/environments"
puppet.environment = "production"
end
This will help Puppet find its own nose and not think that everything is available at the /tmp folder, but that the modules have already been installed at its own folder location.

How to import modules in IPython Clusters

I am trying to import some of my personal modules into my IPython Clusters. I am using Anacondas on Windows Vista 64 bit
from IPython.parallel import Client
rc = Client()
dview = rc[:]
with dview.sync_imports():
import lib.rf
It is giving me this error:
No module named 'lib.rf'
I can import the module in the rest of my IPython notebook, as I have this .bat file to start ipython notebook:
cd C:\Users\Jon\workspace\bf
set PYTHONPATH=%PYTHONPATH%;C:\Users\Jon\workspace\bf
C:\Anaconda\envs\p33\scripts\ipython notebook
I am using this similar code to start my ip clusters:
cd C:\Users\Jon\workspace\bf
set PYTHONPATH=%PYTHONPATH%;C:\Users\Jon\workspace\bf
C:\Anaconda\envs\p33\Scripts\ipcluster start --n=7
Why is this not working?
More info:
If I print out sys.path, I get a list that contains C:\Users\Jon\workspace\bf
If I print out the paths of my clusters, I get the same list:
%px sys.path
['',
'',
'',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\distribute-0.6.28-py3.3.egg',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\pykalman-0.9.5-py3.3.egg',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\patsy-0.2.1-py3.3.egg',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\joblib-0.8.3_r1-py3.3.egg',
'C:\\Users\\Jon\\workspace\\bf',
'C:\\Users\\Jon\\workspace\\bf\\my_numba',
'C:\\Anaconda\\envs\\p33\\python33.zip',
'C:\\Anaconda\\envs\\p33\\DLLs',
'C:\\Anaconda\\envs\\p33\\lib',
'C:\\Anaconda\\envs\\p33',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\Sphinx-1.2.3-py3.3.egg',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\win32',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\win32\\lib',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\Pythonwin',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\runipy-0.1.1-py3.3.egg',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\setuptools-7.0-py3.3.egg',
'C:\\Anaconda\\envs\\p33\\lib\\site-packages\\IPython\\extensions']
In [45]:
Further analysis:
%px lib.__path__
Out[0:11]: _NamespacePath(['C:\\Anaconda\\envs\\p33\\lib\\site-packages\\win32\\lib'])
lib.__path__
Out[57]: ['.\\lib']
Looks like the ipcluster and notebook are looking at lib in different places. I have tried renaming lib to mylib. It has not helped.
It seems that with dview.sync_imports() is being run someplace other than your IPython Notebook environment and is therefore relying a different PYTHONPATH. It is definitely not being run on one of the cluster engines and so wouldn't expect it to leverage your cluster settings of PYTHONPATH.
I'm thinking you'll need to have that directory in your PYTHONPATH (not your PATH) for the calling python environment because that is the location from which you are importing the modules.
The impact of the bit you have about setting the PYTHONPATH in the DOS shell from which you invoke ipclusters isn't clear to me. I can see that one might expect this to let the engines know about your directory, but I'm wondering if that PYTHONPATH gets initilized to the environment from which you call IPython.parallel.Client.

CoffeeScript: No output after installation

I'm running Ubuntu 13.04, after installing using:
$ sudo npm install -g coffee-script
..with output..
npm http GET https://registry.npmjs.org/coffee-script
npm http 304 https://registry.npmjs.org/coffee-script
/usr/local/bin/coffee -> /usr/local/lib/node_modules/coffee-script/bin/coffee
/usr/local/bin/cake -> /usr/local/lib/node_modules/coffee-script/bin/cake
coffee-script#1.6.3 /usr/local/lib/node_modules/coffee-script
No commands yields any result, whatsoever:
$ coffee js.coffee
$ coffee -v
$ coffee GiveMeSomeCoffeePlease
I verified that it exists:
$ which coffee
/usr/local/bin/coffee
And the file has some contents:
$ cat `which coffee`
#!/usr/bin/env node
var path = require('path');
var fs = require('fs');
var lib = path.join(path.dirname(fs.realpathSync(__filename)), '../lib');
require(lib + '/coffee-script/command').run();
Also tried version 1.6.1 which works on my laptop. No difference on this computer though. Any ideas?
I finally found the solution. I had installed the package node on Ubuntu, which is something entirely different:
Amateur Packet Radio Node program (transitional package) The
existing node package has been renamed to ax25-node. This transitional
package exists to ease the upgrade path for existing users.
I went ahead and installed the nodejs package. But seems it didn't quite create the right binding anyway, I could run nodejs but not node. So I made an alias for it and now CoffeeScript is running just fine!
cd /usr/bin; sudo ln -s nodejs node
Same here .. In my expressjs app, instead of running via
node app
now it seems I have to run it via
nodejs app
I ll either create an alias or a symlink like Mika did. I am using Ubuntu 13.10 fyi.

Does virtualenv just modify the pythonpath?

I'm deploying Django on a production server together with virtualenv, and am having trouble activating virtualenv on the server with
source .../bin/activate
I did a little research, and found that the pythonpath is changed depending on if we are or aren't in a virtualenv.
sys.path (with virtualenv activated)
['',
'/.../virtualenv/test_path/bin',
'/.../virtualenv/test_path/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
'/.../virtualenv/test_path/local/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg',
'/.../virtualenv/test_path/lib/python2.7',
'/.../virtualenv/test_path/lib/python2.7/plat-linux2',
'/.../virtualenv/test_path/lib/python2.7/lib-tk',
'/.../virtualenv/test_path/lib/python2.7/lib-old',
'/.../virtualenv/test_path/lib/python2.7/lib-dynload',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk',
'/.../virtualenv/test_path/local/lib/python2.7/site-packages',
'/.../virtualenv/test_path/local/lib/python2.7/site-packages/IPython/extensions']
sys.path (without activating virtualenv):
['',
'/usr/local/bin',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/PIL',
'/usr/lib/python2.7/dist-packages/gst-0.10',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7',
'/usr/local/lib/python2.7/dist-packages/IPython/extensions']
Is is sufficient to just change the pythonpath to point to the virtualenv
.../python2.7/site-packages
folder to get the same results as running
source .../bin/activate
?
No, it is not. virtualenv is not only about site-packages, it is about a whole isolated python environment.
Doing source /path/to/venv/bin/activate just changes your $PATH environment variable to include your virtualenv bin directory as first lookup.
If you call python directly, it is just a shortcut for:
$ /path/to/venv/bin/python myscript.py
And if you call pip in an activated virtualenv, it is the same as:
$ /path/to/venv/bin/pip install XYZ

py2app changes the location of embedded Mongodb

I designed a GUI application using wxPython that communicate with a local database (Mongodb) located in the same folder. My main application has the relative path to the database daemon to start it every time the GUI is lunched.
This is the main.py:
import mongodb
class EVA(wx.App):
# wxPython GUI here
pass
if __name__ == "__main__":
myMongodb = mongodb.Mongodb()
myMongodb.start()
myMongodb.connect()
app = EVA(0)
app.MainLoop()
This is the mongodb.py module:
from pymongo import Connection
import subprocess, os , signal
class Mongodb():
pid = 0
def start(self):
path = "/mongodb-osx-x86_64-1.6.5/bin/mongod"
data = "/data/db/"
cmd = path + " --dbpath " + data
MyCMD = subprocess.Popen([cmd],shell=True)
self.pid = MyCMD.pid
def connect(self):
try:
connection = Connection(host="localhost", port=27017)
db = connection['Example_db']
return db
except Exception as inst:
print "Database connection error: " , inst
def stop(self):
os.kill(self.pid,signal.SIGTERM)
Every thing works fine from the terminal. However, when I used py2app to make a standalone version of my program on Mac OS (OS v10.6.5, Python v2.7), I am able to lunch the GUI but can't start the database. It seems py2app changed the location of Mongodb executable folder and broke my code.
I use the following parameters with py2app:
$ py2applet --make-setup main.py
$ rm -rf build dist
$ python setup.py py2app --iconfile /icons/main_icon.icns -r /mongodb-osx-x86_64-1.6.5
How to force py2app to leave my application structure intact?
Thanks.
Py2app changes the current working directory to the foo.app/Content/Resources folder within the app bundle when it starts up. It doesn't seem to be the case from the code you show above, but if you have any paths that are dependent on the CWD (including relative pathnames) then you'll have to deal with that somehow. One common way to deal with it is to also copy the other stuff you need into that folder within the application bundle, so it will then truly be a standalone bundle that is not dependent on its location in the filesystem and hopefully also not dependent on the machine it is running upon.