I was previously running the Docker container for my web application below:
echo Starting postgres container...
docker run -d -p 5432:5432 --name db my_db
echo Postgres container started
echo Starting memcached container...
docker run -d -p 11211:11211 --name mem memcached
echo Memcached container started
echo Starting web container...
docker run -d -p 8000:8080 --name web --link db:postgres --link mem:memcache my_web_app
echo Web container started
echo Waiting for tomcat to initialize...
sleep 5
echo curling logon.jsf
curl http://111.111.11.111:8000/app/logon.jsf #Real URL redacted
echo Application started
Which became the below docker-compose script:
version: '2'
services:
web:
image: tomcat:7.0.69-jre8
build: ./ci
ports:
- "8000:8080"
depends_on:
- db
- mem
db:
image: postgres:9.4.7
build: ./db
ports:
- "5432:5432"
environment:
POSTGRES_USER: admin
POSTGRES_PASSWORD: admin
mem:
image: memcached
ports:
- "11211:11211"
When I launch the container, it seems like postgres is no longer running (Connection refused, netstat -al | grep postgres doesn't show it), and thus, the app doesn't work. I reviewed the change I have made, and realized that the docker-compose.yml has no analog for the tomcat sleep and curl command I had given in the run script - should it? I couldn't find any information online regarding a proper analog. Is something else potentially afoot here? Please advise if any further details are required.
EDIT: docker ps yields:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a942be02fc22 tomcat:7.0.69-jre8 "catalina.sh run" 5 minutes ago Up About a minute 0.0.0.0:8000->8080/tcp memcachedocker_web_1
fd6074cf8ec9 postgres:9.4.7 "/docker-entrypoint.s" 5 minutes ago Up About a minute 0.0.0.0:5432->5432/tcp memcachedocker_db_1
ce18771fee28 memcached "/entrypoint.sh memca" 5 minutes ago Up About a minute 0.0.0.0:11211->11211/tcp memcachedocker_mem_1
and the logs for the container:
Success. You can now start the database server using:
postgres -D /var/lib/postgresql/data
or
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....LOG: database system was shut down at 2016-06-01 13:09:23 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
CREATE DATABASE
CREATE ROLE
/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
CREATE DATABASE
LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
waiting for server to shut down....LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
LOG: database system was shut down at 2016-06-01 13:09:25 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
and thus inside the container, :
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 127.0.0.11:44095 0.0.0.0:* LISTEN
tcp 0 0 172.18.0.4:38868 132.246.2.23:80 TIME_WAIT
tcp 0 0 172.18.0.4:36424 5.153.231.35:80 TIME_WAIT
tcp6 0 0 127.0.0.1:8005 :::* LISTEN
tcp6 0 0 :::8009 :::* LISTEN
tcp6 0 0 :::8080 :::* LISTEN
tcp6 0 0 172.18.0.4:47522 172.18.0.2:11211 ESTABLISHED
tcp6 0 0 172.18.0.4:47518 172.18.0.2:11211 ESTABLISHED
tcp6 0 0 172.18.0.4:47524 172.18.0.2:11211 ESTABLISHED
tcp6 0 0 172.18.0.4:47520 172.18.0.2:11211 ESTABLISHED
tcp6 0 0 172.18.0.4:47516 172.18.0.2:11211 ESTABLISHED
tcp6 0 0 172.18.0.4:47514 172.18.0.2:11211 ESTABLISHED
udp 0 0 172.18.0.4:48360 10.0.2.3:53 ESTABLISHED
udp 0 0 127.0.0.11:40276 0.0.0.0:*
We can see that the memcached container is up and running, however postgres is absent. The only thing running on the localhost interface is bound to 8005, and it isn't postgres. I should add finally that I am able to telnet memcached on its port from the web container and confirm its operation, however I can't do the same with postgres, and I'm not sure why.
There are a number of things that can probably be removed from your
docker-compose.yml file, assuming that the database and memcached
services are dedicated to your web application (that is, they don't
need to be accessed from anywhere else).
It doesn't make sense to specify both image and build in a service
definition, so I've made a few assumptions about what you actually
want.
I would start with something like this:
version: '2'
services:
web:
build: ./ci
ports:
- '8000:8080'
depends_on:
- db
- mem
db:
image: postgres
environment:
POSTGRES_USER: admin
POSTGRES_PASSWORD: admin
mem:
image: memcached
You don't need ports directives for either the db or mem
services, because you don't need to expose these services to your host
(or elsewhere). A docker container can access services on another
container on the same network without any special directives.
Within the web container, you can access either the db or mem
container by name, because this is how newer versions of Docker
operate when using a non-default network: Docker creates an internal
DNS service and registers the container names there. So, for example,
from inside the web container I can use the psql command line
postgresql client to connect to the database server like this:
$ docker exec -it myservice_web_1 sh
web_1# psql -h db -U admin
Password for user admin:
psql (9.4.6, server 9.5.1)
WARNING: psql major version 9.4, server major version 9.5.
Some psql features might not work.
admin=#
You would configure your web application to access the database server
at host db, port 5432.
With this environment up and running, you could connect to your web service from your host at http://localhost:8000, or from either your host or another host on the network at http://<your_hostname_or_ip_here>:8000.
Related
I am running a postgres image using a docker-compose.yml file (not in detached mode). I have created two users and two databases there. In my bash terminal I logged into both users and when I typed \conninfo it said that
You are connected to database "db1" as user "user1" on host "localhost" (address "127.0.0.1") at port "5432".
and "db2" and "user2" for the other one.
When I typed the command "sudo netstat -plunt | grep postgres" it returned the following
tcp 0 0 127.0.0.1:5433 0.0.0.0:* LISTEN 953/postgres
I have a few questions.
1\ Why does it say port 5432 in the first one and 5433 in the second??
2\ When I type localhost:5432 in the url I don't see any page. It says
This page isn’t working
localhost didn’t send any data.
ERR_EMPTY_RESPONSE
Why can't I see anything in the web page at the above address??
3\ In my vscode terminal I am also getting these two error messages
2021-12-25 22:03:51.887 UTC [45] LOG: invalid length of startup packet
ERROR: relation public.databasechangeloglock does not exist at character 22
Do you know why I am getting these errors?
My docker-compose.yml file is as follows (shown only partly).
version: "3.4"
services:
sumo-db:
image: postgres-sumo:1.4.0
environment:
- POSTGRES_PASSWORD=password
- DB_NAME_1=db1
- DB_USER_1=user1
- DB_PASS_1=pass1
- DB_NAME_2=db2
- DB_USER_2=user2
- DB_PASS_2=pass2
volumes:
- sumo-db-data:/var/lib/postgresql/data
restart: on-failure
ports:
- 5432:5432
networks:
- sumo
If you need further information feel free to ask me. As always thank you.
Inshaf
If the docker hypervisor is capturing connections to 5432 on the host and tunneling them into the container, netstat will attribute this to the hypervisor process, not to a PostgreSQL process. netstat doesn't know who is on the other end of the tunnel, only its own end. Now it could recognize 5432 as being a traditional PostgreSQL port and label it that way, but -n tells it not to. Apparently you also have a 2nd instance of PostgreSQL, this one running directly on the host and on port 5433.
When I type localhost:5432 in the url I don't see any page.
A URL is something you can type, not something you type other things into. If you are putting that into the address bar of a web browser, that is not how you connect (successfully) to databases.
What is a vscode terminal? Does it have anything to do with either PostgreSQL or docker? Does the message mean that this terminal committed those errors, or someone else committed those errors against it?
I have a postgresql server up and running under root. It's in a docker container and the exposed port is 5434. My API docker container is able to connect to this server, but I can't access it from my local machine, the connection is getting timed out.
Here's the output of the command ps -elf|grep root
4 S root 15264 5623 0 80 0 - 2128 - Jan25 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 5434 -container-ip 172.18.0.5 -container-port 5432
Now that Postgres is running in a Docker container, your IP is not local anymore, so you need to update the pg_hba.conf for the server to allow connections from the network from which you are accessing Postgres.
I used to use a script (/docker-entrypoint-initdb.d/db_init.sh) to loop through database dumps in a folder copied into the postgres container and restoring them. It used to work nicely but recently it stopped working.
I get following error:
postgres_server_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/db_init.sh
postgres_server_1 | --> db_init.sh: Found /dumps/dataset_1.dump as dataset_1
postgres_server_1 | psql: could not connect to server: Connection refused
postgres_server_1 | Is the server running on host "localhost" (127.0.0.1) and accepting
postgres_server_1 | TCP/IP connections on port 5432?
The db_init.sh script loops through a folder containing database dumps and checks if the database exists already. If not it restores the dump.
/docker-entrypoint-initdb.d/db_init.sh content:
shopt -s nullglob
for i in /dumps/*.dump;
do
db_name=${i##*/}
db_name=${db_name%.dump}
echo "--> db_init.sh: Found $i as $db_name"
if psql -U postgres -h localhost -lqt | cut -d \| -f 1 | grep -qw ${db_name}; then
echo "--> db_init.sh: Database ${db_name} already exists."
else
echo "--> db_init.sh: Setting up database: ${db_name}"
createdb -U postgres -h localhost-T template0 ${db_name}
psql -U postgres -h localhost ${db_name} < ${i}
fi
done
echo "--> db_init.sh: Setup complete."
I am using docker-compose to start the postgres container (and some others).
The docker-compose.yml content:
version: '3'
services:
postgres_server:
image: postgres
volumes:
- /etc/localtime:/etc/localtime:ro
- ./data/dumps:/dumps:ro
- ./scripts/db_init.sh:/docker-entrypoint-initdb.d/db_init.sh
environment:
- TZ=Europe/Berlin
restart: always
volumes:
postgres_data:
driver: local
Now what I don't understand is why there seems to be a connection error usually associated with trying to connect to a postgres database from a different machine or container. But the script itself is running in the postgres container and a volume connects the directory containing the dumps into the container.
Running the psql command from within the container using docker exec -it container_name bash works fine and the dumps are there. Why do the psql commands work when executing them manually from within the container but not when executed via /docker-entrypoint-initdb.d/db_init.sh?
It looks like this commit has broken your script.
Explanation:
PostgreSQL may accept connections not only via TCP/IP, but also via UNIX socket. The -h localhost argument tells psql to use TCP connections rather than UNIX socket.
If you look into the current docker-entrypoint.sh version, you will see, that during the execution of scripts in /docker-entrypoint-initdb.d PostgreSQL listens only on the UNIX socket, and the startup log says:
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
This means that psql -h localhost will not connect to the database, as PostgreSQL does not listen on IP socket. You must use psql without -h localhost option to make it use UNIX socket instead of TCP connections.
But why running psql -h localhost manually works?
If you look into the docker-entrypoint.sh again, you will see that when all init scripts are executed, PostgreSQL is being stopped and then started again in normal (operational) mode, in which it listens both on UNIX and IP sockets:
LOG: listening on IPv4 address "0.0.0.0", port 5432
LOG: listening on IPv6 address "::", port 5432
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
So, when the startup process is complete, you may connect to PostgreSQL using TCP connections, thus, entering into the container and running psql -h localhost succeeds.
I recently installed postgresql on my ubuntu 14.04 vm, but I can't seem to be able to connect to my database from localhost: postgres://127.0.0.1:5432/mydb?sslmode=disable is not found, although psql mydb brings me to the correct terminal, sudo netstat -plunt |grep postgres returns nothing, but su postgres -c "netstat -plunt |grep postgres" prints:
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 31/postgres
tcp6 0 0 ::1:5432 :::* LISTEN 31/postgres
So it's clearly there, just somehow under the postgres ubuntu user (but with the database belonging to root). I know that there is a way to reconfigure my postgresql installation such that it can be reached through localhost (postgres://127.0.0.1:5432/mydb?sslmode=disable works), and if anyone could tell me what I need to do, that would be awesome. Thank you!
I have containerized a web application running on Tomcat and started the container using the command,
docker run -d -p 8080:8080 tveuser/tve-repository:tve-services
I am also running a PostgreSQL container on the same host using the following command:
docker run -d -p 80:80 -p 5432:5432 tveuser/tve-repository:tve-postgresql
i verified that the PostgreSQL is running by using phpPgAdmin but could not get tomcat to connect to it. 'docker ps' also tells me that both the containers are up and running.
I connect the web application with the database through tomcat context.xml which has an entry like
<Parameter name="abc.connection.url" value="jdbc:postgresql://1.2.3.4:5432/dbname" />
where 1.2.3.4 is the docker host Ip in which the container is running.But i get the following error when i run the tomcat container:
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:225)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136)
...............
Caused by: java.net.NoRouteToHostException: No route to host
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
Any help appreciated.
TL;DR : check the firewall on your host
You need to connect to your host's IP address on the port that you've exposed for postgres on your postgres container, not to the IP address of the docker container.
E.g.,
<Parameter name="abc.connection.url" value="jdbc:postgresql://192.168.x.x:5432/dbname" />
You'll also need to configure postgres inside your postgres docker container to listen for connections from somewhere other than 127.0.0.1, which is the default.
E.g., in pg_hba.conf:
host all all 0.0.0.0/0 md5
This tells postgres to accept incoming connections from any IP address (you'll probably want to lock this down to something like 192.168.0.0/16.
You'll also need to change the value of listen_addresses in postgres.conf:
listen_addresses = '*' # what IP address(es) to listen on;
Once you've made these configuration changes, you can check to see if postgres is listening to all requests from all IPs inside your docker container:
netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 816/postgres
tcp6 0 0 :::5432 :::* LISTEN 816/postgres
And then from your host, make sure you can telnet into port 5432:
telnet 192.168.x.x 5432
Trying 192.168.x.x...
Connected to 192.168.x.x.
Escape character is '^]'.
As Chris McKinnel points out, forwarding ports to the host doesn't work like that. I wanted to give you a second solution as an alternative to his: Links.
https://docs.docker.com/userguide/dockerlinks/
So you could do something like:
docker run -d -P --name tomcat --link postgres:postgres apache/tomcat apachectl start
... which, assuming your PostgreSQL container was named "postgres", would allow you to connect directly from the Tomcat container.
Note that this only works if both containers are on the same machine, but it sounds like they are.