Postgres/VirtualEnv/Flask - postgresql

This one may be for the experts!
So here is the problem:
I have a flask app that gets its data from Postgresql and it runs fine in my normal environment. I tried to deploy it locally using virtualenv and after pip installing all the requirements the only one that gave me trouble is psycopg2 which appears to be dependency of Postgresql.
I then used this amazing article to help me install it, by putting export PATH=/Library/PostgreSQL/9.3/bin:$PATH to the .bash_profile file.
But now I get this error:
Library not loaded: libssl.1.0.0.dylib ...Image not found
What is going on?

Related

No module named 'psycopg2' Error in Scrapy

When I use
import psycopg2
in Scrapy pipeline.py or spider.py I get the following error
ModuleNotFoundError: No module named 'psycopg2'
I know that the error is not valid, because I am running Django server with postgresql, using same venv which contains psycopg2. And also I can connect to db in a test file and enquiry as well. In addition I can run import psycopg2 in Scrapy shell with no issue. So it has puzzled me that I might be missing something,
I appreciate if someone can help me,
Since it took me hours, here is what I have found.
I tested both in windows and Linux Ubuntu, both had same error.
Apparently this is what is happening:
In windows I had Scrapy installed on Core, and in Ubuntu apparently Scrapy is already wrapped in.
I ran my crawler within venv, and it appeared that actually my Scrapy is not installed correctly in the venve. However, Since Scrapy was already in the PATH it was running from Core, where it was no psycopg2. So the above mentioned errors was actually come from come not venv.
Installing Scrapy looked like was very simple in venv at first. I checked that by trying Scrapy shell and it was running, but it was actually were running from the core again. Make sure that you have Scrapy in your pip freeze list in your venv. This is a link that has explained how to address some issues in installation.
You need to install psycopg2 in your virtualenv, issue the next command
pip install psycopg2

virtualenv can't execute on my system?

I'm trying to create a virtual environment to deploy a Flask app. However, when I try to create a virtual environment using virtualenv, I get this error:
Using base prefix '//anaconda'
New python executable in /Users/sydney/Desktop/ptproject/venv/bin/python
ERROR: The executable /Users/sydney/Desktop/ptproject/venv/bin/python is not functioning
ERROR: It thinks sys.prefix is '/Users/sydney/Desktop/ptproject' (should be '/Users/sydney/Desktop/ptproject/venv')
ERROR: virtualenv is not compatible with this system or executable
I think that I installed virtualenv using conda. When I use which virtualenv, I get this
//anaconda/bin/virtualenv
Is this an incorrect location for virtualenv? I can't figure out what else the problem would be. I don't understand the error log at all.
It turns out that virtualenv just doesn't work correctly with conda. For example:
https://github.com/conda/conda/issues/1367
(A workaround is proposed at the end of that thread, but it looks like you may be seeing a slightly different error, so maybe it won't work for you.)
Instead of deploying your app with virtualenv, why not just use a proper conda environment? Conda environments are more general (and powerful) than those provided by virtualenv.
For example, to create a new environment with python-2.7 and flask in it:
conda create -n my-new-env flask python=2.7

Problems installing/compiling DBD::mysql on OpenSuSE 10

I am getting a bunch of compile errors when I try to install the MySQL DBD::mysql Perl library. I am trying to install this library on OpenSuse linux (SUSE Linux Enterprise server 10 (x86_64) version 10, patch level 4)
The install fails when trying to compile dbdimp.h. There are hundreds of complile errors but I have reason to believe they all stem from the first 3:
dbdimp.h:23:49: error: mysql.h: No such file or directory
dbdimp.h:24:45: error: mysqld_error.h: No such file or directory
dbdimp.h:26:49: error: errmsg.h: No such file or directory
I believe that the reason I am getting the errors above is that I have no MySql Client installed. I do not know how to install mysql client (I believe I am supposed to get some version of libmysqlclient).
I am brand new to opensuse and vaguely familar with installing packages on Linux in general have used yum, yast, apt-get on ubuntu + centOS previously but the only package manager tools that seem to be on OpenSuse 10 are rpm and zypper. I have not managed to install the required mysql client using either of these. If anyone knows how to install MySql Client to resolve my issue I would greatly appreciate any recomendations
Thanks
You need to install MySQL or the MySQL client libraries, either through your system's package manager or by following the installation instructions from the source code.
It sounds like your question is really "How do I install MySQL on OpenSuse", which would be more appropriate for one of the other StackExchange sites.
I was able to resolve this to a point. In my original question, I stated that I had tried using YAST but that it had not worked. In fact, I did not fully explore the YAST install option but when I did, I managed to download install the required librarys by following an article I found (http://adminramble.com/install-mysql-yast/#chitika_close_button)
I was able to open the YAST GUI in the terminal window (not an X-window display) and navigate through the "software management" section wherein I was able to do a search for MySql. Amoung the search results were mysql client libraries, which I then selected and hit "Accept". The installation started and completed successully.
In order to complile the PERL DBD::mysql module, I had to create a new makefile with the ccflags specified so that the header files could be found:
perl Makefile.PL --cflags -I/usr/include/mysql
I say "to a point" at the top of this comment because I then ran into further problems compiling but my original issue posted has been partially resolved and getting this far may help someone else who is experiencing the same problem. Will update if I get a solution to the compile problem.

Deploying meteor app to a webserver

Does anyone know a step by step guide to deploy the own meteor app from windows to a webspace (not xxx.meteor.com).
I've found some tools like meteor.sh, but I'm a beginner and it's difficult without a guidance and without linux (needed to execute sh-files for example)
Make your project locally
Build your project locally, you could test it using meteor run or even meteor deploy xxx.meteor.com to see if its working
Bundle your app
Use meteor bundle deploy.tar.gz to make a file called deploy.tar.gz in your meteor directory containing your project
Upload your file to your server
This depends more on how your server is/what your platform is but you can use a tool to upload it for you (e.g Transmit on mac)
Install node.js & fibers on your platform if you don't have it already
This depends alot on your server platform. Have a look at http://nodejs.org/ for more detailed instructions
Extract your bundle
If on a *nix platform you could do the below in the directory where you uploaded your bundle (explanation):
tar -xzvf bundle.tar.gz
Enter the directory and install fibers
Fibers is needed for any meteor project, it helps use synchronous style code on server side javascript:
cd bundle/programs/server/node_modules
rm -r fibers
npm install fibers#1.0.1
The first line enters the directory in your bundle where fibers is installed, the second removes it, and the third reinstalls it.
Get MongoDB on another server or use a third party service like mongohq
Meteor production deployments need another mongodb. You can either install it on another server or use a third party server. It's not recommended to install it on the same server you install meteor on.
Finally check if your project is runnable
cd ../../../
node MONGO_URL=mongodb://dbuser:dbpassword#dbhost:dbport/meteor ROOT_URL=http://yourwebsite.com app.js
The first line gets you back to the bundle directory and the second runs node.js on your project with the parameters that let you connect to your mongodb database.
Install something to let it run in the background
It depends on which one you want to use, foreverjs is quite easy to use
npm install forever -g
If you get an error problem try using sudo before the npm (which lets you run as a super user).
Then you can run forever:
forever start MONGO_URL=mongodb://dbuser:dbpassword#dbhost:dbport/meteor ROOT_URL=http://yourwebsite.com app.js
And its done!
Extra notes
While its not that easy to get started from scratch this should help you get started. You still need to secure your mongodb server up if you've used your own servers.
The meteor.sh script does pretty much the same as above but very quickly if you learn to use that instead it might be faster to deploy updates
You might not have wget or a couple of commands that you might need that come up and give you Unknown command errors. Have a go at running yum or apt-get and see which one of the two you might have. You can then install the required package using one of these installer tools, i.e with yum install wget
I hope this helps you, its not that easy to deploy to a server on the first shot as a couple of things might be missing (files/packages/dependencies), you might run into other problems with permissions & stuff, but you could always ask on serverfault or here on stackoverflow on what you run into.
I recommend Meteoric.
Note, that you need to run meteoric from your development machine.
Script is self explanatory and works perfect for me.

Symbol not found: _PQbackendPID with Django project

Running on MAC os 10.6.8
with postgresSQL installed, as well django - using python2.7
Also installed psycopg2 and dj-database-url using pip in my virtual env
And added these two lines to my setting.py:
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
Based on instructions for Heroku in:
https://devcenter.heroku.com/articles/django#database_settings
When running:
python manage.py runserver
I am getting this error:
ImportError: dlopen(/Users.... venv/lib/python2.7/site-packages/psycopg2/_psycopg.so, 2): Symbol not found: _PQbackendPID
Referenced from: /Users.... venv/lib/python2.7/site-packages/psycopg2/_psycopg.so
Expected in: dynamic lookup
I kept searching for hours and tried all kind of thing including the advice on:
Mac OS X Lion Psycopg2: Symbol not found: _PQbackendPID
to no avail.
Wonder if anyone had such an issue and had any luck.
I had the same problem. Instead of installing the dependencies as Heroku suggests using
pip install Django psycopg2 dj-database-url
clone whatever repo you're hoping to run in venv, keeping its original settings.py. Then:
source venv/bin/activate
to activate the new environment, cd into your new repo, and python manage.py runserver. Should be set.
Alternatively, you could remake PostGreSQL, and run again, but that's a bit more of a task - it would work for psycopg2, though. As far as I can tell that issue comes from using an 64 or i386 build when you should be using a 32 build - but I'm not sure about this, and the above solution works well to solve the problem and use venv for what you're actually going to be using it for, most likely.
I had the same problem as you guys and I had read many pages and I couldn't find the answer in any of them. Many solution was about installing from source file and don't relate to the virtual environment.
I've found and tested following solution and it solve my problem.
1- Make sure your Postgres is NOT higher than 9.4 version according to psycopg2. Check python version as well. I use Postgres 3.9.9.
2- The problem is behind different version of Python(32/64 bit). It should comply with your operation system's bit architecture which is 64bit. Uninstall all versions of Python and pip from your system. Instruction you can find here but do NOT remove Python2.7 which is Apple-supplied system Python.
3- Install "Mac OS X 64-bit/32-bit" installer from python official website and install it.
After that install pip. Note that you should use the command "python3.5" for using Python version 3.5. You might install virtualenv from the new pip as well.
4- After all that you can go on your virtualenv and type "pip3 install -r requirement.txt" for installing all dependencies on your local machine.
Hope this can help you.