connect to postgres container from flask app inside container - postgresql

I have my flask app app.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
APP = Flask(__name__)
DB = SQLAlchemy()
if __name__ == '__main__':
APP.config.from_mapping(
SQLALCHEMY_DATABASE_URI='postgres://postgres:password#0.0.0.0:5432',
SQLALCHEMY_TRACK_MODIFICATIONS=False
)
DB.init_app(APP)
DB.create_all(app=APP)
APP.run(use_reloader=False, host='0.0.0.0', port='5000')
and I have a Dockerfile for it:
FROM python:3.6-alpine
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
WORKDIR /root
COPY app.py .
RUN pip3 install Flask==1.0.2
RUN pip3 install psycopg2-binary==2.7.6.1
RUN pip3 install Flask-SQLAlchemy==2.3.2
CMD ["python3", "app.py"]
I run:
docker build . --tag flaskapp:1
docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=password --name database postgres
docker run --rm -p 5000:5000 flaskapp:1
I then get an exception which points out:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
Is the server running on host "0.0.0.0" and accepting
TCP/IP connections on port 5432?
How do I fix this?

You have specified 0.0.0.0 as the IP address to connect to, which doesn't make sense. 0.0.0.0 is the "Any Address". You probably saw a message that postgres was listening on 0.0.0.0, which is where you got it from. In the context of a server listening on 0.0.0.0, it means that it is listening on all ipv4 interfaces. See https://en.wikipedia.org/wiki/0.0.0.0 for more information about the special 0.0.0.0 address and what it means.
If you want to connect to the postgres service, then you would need to use a valid ip address or dns name of where it is running.
In Docker, if you have multiple named containers connected to the same user-defined network, you can make use of the built-in service discovery mechanism that Docker ships with.
Here's a modified set of commands to run to take advantage of this:
docker network create mynet
docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=password --net mynet --name database postgres
docker run --rm -p 5000:5000 --net mynet flaskapp:1
Be sure to change your code to connect to postgres://postgres:password#database:5432 instead of postgres://postgres:password#0.0.0.0:5432

Related

docker container can not connect to local postgres [duplicate]

This question already has answers here:
From inside of a Docker container, how do I connect to the localhost of the machine?
(40 answers)
Closed 2 years ago.
Can't connect to Postgres from the docker container.
I do not want user docker-compose and create a Postgres container, already got Postgres app running. I think it is a bad idea to container postgres and better use system.
OSX, Postgres10, Flask
I already made
postgresql.conf
listen_addresses = '*'
port = 5432
pg_hba.conf
host all all 0.0.0.0/0 trust
I Used trust for any result, but no effect.
Dockerfile
FROM python:3.8-alpine
RUN apk update \
&& apk add --virtual build-deps gcc musl-dev \
&& apk add python3-dev \
&& apk add postgresql-dev \
&& apk add jpeg-dev zlib-dev libjpeg \
&& pip install --upgrade pip
ENV PYTHONUNBUFFERED 1 \
&& FLASK_APP app.py
#EXPOSE 5000 5432
WORKDIR /app
ADD . /app
RUN pip install -r requirements.txt
CMD ["./bin/run.sh"]
./bin/run.sh
#!/bin/sh
#python run.py
source venv/bin/activate
set -e
flask db upgrade
exec gunicorn -b --workers 4 --access-logfile --error-logfile run:app :5000
The "docker run" command I try to use:
docker run --rm -e SQLALCHEMY_DATABASE_URI=postgresql://postgres:postgres#0.0.0.0:5432/my_db --net=host -p 5000:5000 my_container:v0.1
the command leads to error
...
psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "0.0.0.0" and accepting
TCP/IP connections on port 5432?
command connect me to Postgres
psql -U postgres -h 0.0.0.0
Trying to connect to host "0.0.0.0" is technically wrong. (the address 0.0.0.0 is used to other purposes no to connect to...)
Did you tried connect using the postgres container ip address instead of "0.0.0.0"?
You can get the postgres container ip address throught the following command:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' name_of_postgres_container

How can I connect to Postgres database in the container via port 5432

I am running a postgres docker container by using the commands below: (reference: https://docs.docker.com/engine/examples/postgresql_service/)
docker build -t eg_postgresql .
docker run --rm -P --name pg_test eg_postgresql
This works but the port number is dynamic. I can connect to the database by giving the port number. (the port I see in docker ps command)
I would like to connect to this docker database from Python so I need a static port number.
I tried the parameters below:
-p 127.0.0.1:5432:5432
-p 5432:5432
In that case, the docker container's port number was set as 5432. However, I could not connect to the database. I get docker user does not exist error message.
What is your advice?
I took the Dockerfile from the link you posted. After building the container with
docker build -t eg_postgresql .
I started the container with
docker run --rm -p 5432:5432 --name pg_test eg_postgresql (which binds localhost port 5432 to the container port 5432)
and then I tried to connect with
psql -h localhost -p 5432 -d docker -U docker --password
It works like a charm. If you get a message that docker user does not exist please double check that all steps from the Dockerfile are executed succesfully during the docker build command as the creation of the docker user is done in the command RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER docker WITH SUPERUSER PASSWORD 'docker';" &&\
createdb -O docker docker. Make also sure that you have no PostgreSQL server running on your localhost so that you can be sure that you are trying to connect to PostgreSQL inside the container.

How do you send tcp/ip requests from a docker container to the host?

I'm working out how to run openproject with docker. I'm working through openproject/docker.
I've got the docker image running with an external postgres directory.
I'm now working out how to connect to an existing running instance of postgresql.
The command line I'm using looks ok according to the official documentation.
EDIT. Added in the missing -p.
docker run -p 8082:80 -p 5432:5432 --name openproject_dev -e SECRET_KEY_BASE=secret -e DATABASE_URL=postgresql://openproject:openproject-dev-
password#localhost:5432/openproject_dev \
-v /Users/admin/var/lib/openproject/logs:/var/log/supervisor \
-v /Users/admin/var/lib/openproject/static:/var/db/openproject openproject/community:5.0
I've ommitted the -d [deamon] flag so I can see any errors.
When the docker container is being created I get
-----> You're using an external database. Not initializing a local database cluster.
/usr/src/app /usr/src/app
Starting memcached: memcached.
Which I expect.
Then I get an error about connecting to the postgresql server which I don't expect.
...
PG::ConnectionBad: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
/usr/local/bundle/gems/activerecord-
4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `initialize'
I'm guessing the script initialising the container is expecting postgres to be running and it's not. How would you make the docker container port forward requests to 5432 to the host machine on the command line? ... the opposite of docker run -p 5432:5432 ... which exposes 5432 from the docker container to the host.
-e DATABASE_URL=postgresql://openproject:openproject-dev-
password#localhost:5432/openproject_dev
When you add this url, the container expects postgres to be running on localhost, i.e. in itself.
If you are running postgres on your host machine, you can let the container share the network stack with the host by passing --network host in the run command. In that case, localhost will refer to the host machine where postgres is running.
Your approach is wrong currently
docker run -p 8082:80 -p 5432:5432 --name openproject_dev -e SECRET_KEY_BASE=secret -e DATABASE_URL=postgresql://openproject:openproject-dev-password#localhost:5432/openproject_dev \
-v /Users/admin/var/lib/openproject/logs:/var/log/supervisor \
-v /Users/admin/var/lib/openproject/static:/var/db/openproject openproject/community:5.0
when you used -p 5432:5432 it means that you expect something to run inside docker on that port and you want your host machine 5432 port to map to 5432 inside docker.
Next if you were able to run that command, even though nothing is listening inside container, then that would imply that the port on your host is available. This means postgress is not listening on 5432 on host also. It may be listening on a socket. You should try to execute below command on host
psql -h 127.0.0.1
If you are not able to connect on host using this that means the postgres db is bind to a socket file and not to a IP. Now you have few options that you can exercise
Mount the socket
docker run -p 8082:80 -p 5432:5432 --name openproject_dev -e SECRET_KEY_BASE=secret -e DATABASE_URL=postgresql://openproject:openproject-dev-password#localhost:5432/openproject_dev \
-v /Users/admin/var/lib/openproject/logs:/var/log/supervisor \
-v : \
-v /Users/admin/var/lib/openproject/static:/var/db/openproject openproject/community:5.0
Bind on 0.0.0.0 on host
If you don't want to mount volume then you should change the bind address of psql to 0.0.0.0 and then change your database url to -e DATABASE_URL=postgresql://openproject:openproject-dev-password#<YOURMACHINEIP>:5432/openproject_dev
Run on host network
docker run --net host --name openproject_dev -e SECRET_KEY_BASE=secret -e DATABASE_URL=postgresql://openproject:openproject-dev-password#localhost:5432/openproject_dev \
-v /Users/admin/var/lib/openproject/logs:/var/log/supervisor \
-v : \
-v /Users/admin/var/lib/openproject/static:/var/db/openproject openproject/community:5.0

Docker Tomcat container unable to access Postgres container

I have a alpine docker with postgres, with listen address '*' and listening to 5432, which I'm deploying using
docker run -d --name postgres me/postgres:v1
and my tomcat container with oracle jre8, on which I'm deploying my rest web service using:
# Set environment
ENV CATALINA_HOME /opt/tomcat
EXPOSE 8080
# Launch Tomcat on startup
CMD ${CATALINA_HOME}/bin/catalina.sh run
RUN rm -rf ${CATALINA_HOME}/webapps/docs \
${CATALINA_HOME}/webapps/examples \
${CATALINA_HOME}/webapps/ROOT
# Deploying war file
ADD myapp.war ${CATALINA_HOME}/webapps/ROOT.war
# Restarting server after deploying
CMD ${CATALINA_HOME}/bin/catalina.sh run
And deploying it with
docker run -d -p 8080:8080 --name tomcat --link postgres:postgres me/tomcat:v1
Both are being executed on my laptop, with IP address 192.168.x.x, and I checked the port is listening.
Unfortunately my web service on tomcat cannot connect to the postgres service using
jdbc:postgresql://192.168.x.x:5432/dBName
Alternate I already tried: I launched postgres on it's own port using,
docker run -d -p 5432:5432 --name postgres me/postgres:v1
docker run -d -p 8080:8080 --name tomcat me/tomcat:v1
Then used
jdbc:postgresql://192.168.x.x:5432/dBName
and
jdbc:postgresql://localhost:5432/dBName
but neither seems to work.
In both cases I can see my web server running in tomcat manager, and I am able to access my dB using
psql -h localhost -p 5432 -d dBName -U myUser
as well as pgAdmin.
Any help in resolving this is appreciated.
Solution Update: While using --link, point to postgres (i.e., your postgresql container name) instead of IP
jdbc:postgresql://postgres:5432/dBName
Many thanks to #larsks for pointing it out.
While using --link, point to postgres (i.e., your postgresql container name) instead of IP
jdbc:postgresql://postgres:5432/dBName
So for a full solution, run your postgresql and tomcat container
docker run -d --name postgres me/postgresql:v1
docker run -d -p 8080:8080 --name tomcat --link postgres:postgres me/tomcat:v1
(Notice here I didn't put port for postgres container since it will already have 5432 exposed internally, unless you want to hit it from outside your host, you don't need to specify a port here)
And your server war file will the jdbc address above, postgres will automatically resolve to the container's IP address when they are linked.
Many thanks to #larsks for pointing it out.

Docker container for Postgres 9.1 not exposing port 5432 to host

I'm trying to use a Docker container to run a PostgreSQL server, and connect with it from my host machine.
My configuration is:
Host machine: Mac OS X 10.10.5
Docker 1.10.1
I've done this:
Step 1: create a volume for permanent postgres data
docker volume create --name postgres_data
Step 2: Start the postgres instance
UPDATE: As suggested in comments, I specified port mapping when running the container
docker run --name my_postgres_container -e POSTGRES_PASSWORD=my_password -v postgres_data:/var/lib/postgresql/data -p 5432:5432 -d postgres:9.1
Step 3: connect to Docker instance by doing this:
docker run -it --link my_postgres_container:postgres --rm postgres:9.1 sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'
But I want to connect to that instance just by:
psql -h localhost -p 5432 -U postgres
Like if I had installed Postgres locally in my host machine.
The problem is port 5432 is not exposed. So, I can't connect with it:
sudo lsof -i -P | grep -i "listen" --> no port 5432 open
UPDATE 2: Even stranger. I've also done this:
Stop Docker. Then, run a normal PostgreSQL 9.4.4 instance in my host machine (no docker involved here, just postgres running in my Mac OS X host, listening on port 5432). Everything is normal:
sudo lsof -i -P | grep -i "postgres"
postgres 14100 jorge 6u IPv4 0x780274158cebef01 0t0 TCP localhost:5432 (LISTEN)
I can connect with my local postgres instance without any problem (look the output of the command: is the postgres compiled for Mac OS X, my host):
psql -h localhost -U postgres -c "select version()"
version
---------------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 9.4.4 on x86_64-apple-darwin14.3.0, compiled by Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn), 64-bit
(1 row)
Now the fun part. I start my Docker instance again, while the host PostgreSQL instance is running.
It starts! (and it shouldn't). I can even connect using docker run...
docker run -it --link my_postgres_instance:postgres --rm postgres:9.1 sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'
If I run select version() now, it shows postgres running inside my docker instance at the same time postgres is running in my host, out of docker, using the same 5432 port. (Look at the ouput, is postgres compiled for Debian, the OS inside the postgres:9.1 container)
postgres=# select version();
version
------------------------------------------------------------------------------------------------
PostgreSQL 9.1.20 on x86_64-unknown-linux-gnu, compiled by gcc (Debian 4.9.2-10) 4.9.2, 64-bit
(1 row)
Why?
Does it make sense? My final goal is to run a Django app in another Docker container and connect with my Postgres instance. How could I do that?
It's 2018 and I just had a similar problem. The solution for me seemed to be with the order of props to docker. e.g. this resulted in no port being exposed;
docker run -d --name posttest postgres:alpine -e POSTGRES_PASSWORD=fred -p 5432:5432
while this worked fine (image exposed port 5432 as expected);
docker run --name posttest -d -p 5432:5432 -e POSTGRES_PASSWORD=fred postgres:alpine
Run the postgre image with the correct Port Mapping using -p <host_port>:<container_port>:
docker run --same-options-as-step-one -d -p 5432:5432 postgres:9.1
Your docker host is a virtual machine, which has it's own IP adddres.
You can detect this IP address by entering the following command:
docker-machine ip
The answer will be something like 192.168.99.100
When you have mapped the ports using the -p 5432:5432 switch, you will be able to connect to postgres with any tool from your dev machine using the IP address mentioned.
I was able to connect using container IP or host IP, except localhost (127.0.0.1).
To get container id run
docker ps
Find required container id and run
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <container_id>
Port must be exposed.
Here is an example of docker-compose.yml which starts two containers postgres and adminer, which is database management tool you can use to connect to postgres:
version: '3'
services:
adminer:
image: adminer
restart: always
ports:
- 8080:8080
postgres:
image: postgres:11.4-alpine
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
I had a similar issue. My problem was simply 0.0.0.0 not mapping to localhost so I just had to add to psql
psql --host=0.0.0.0
This is presuming
docker port <container name>
outputs
5432/tcp -> 0.0.0.0:5432
Other answers work, but don't explain why they work.
Given the command:
psql -h localhost -p 5432:5432 -U postgres
localhost is actually a special value that tells psql to look for a unix socket connection, instead of going over TCP. We can't use unix sockets to connect to docker services.
Changing the command like so fixes it, by forcing TCP:
psql -h 127.0.0.1 -p 5432:5432 -U postgres
That will work as long as you docker run ... -p 5432:5432 .... Specifying the IP returned by docker-machine ip also forces TCP, so that also works.
I had a similar problem working in a VMWare virtual machine with Lubuntu. The VM had been paused and then was restarted. The PostgreSQL Docker container was correctly mapped and listening on localhost:5432, but I always got:
psql: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Restarting the VM solved the problem in my case.
Try this to install postgresql
docker run --name postgres -d -p 5432:5432 -e POSTGRES_PASSWORD=fred postgres:alpine
Or Change port host machine Heree (mac)
docker run --name postgres -d -p 5436:5432 -e POSTGRES_PASSWORD=fred postgres:alpine
Tip:
Install pgadmin4
docker run -p 5050:80 -e "PGADMIN_DEFAULT_EMAIL=name#example.com" -e "PGADMIN_DEFAULT_PASSWORD=admin" -d dpage/pgadmin4