I'm struggling to get the Pact Broker running in a docker container to connect to my local installation of PostgreSQL on Windows.
This is what my docker run command looks like...
docker run pactfoundation/pact-broker
-e PACT_BROKER_DATABASE_USERNAME=pact_broker
-e PACT_BROKER_DATABASE_PASSWORD={Password}
-e PACT_BROKER_DATABASE_HOST=host.docker.internal
-e PACT_BROKER_DATABASE_NAME=pact_broker
-e PACT_BROKER_DATABASE_PORT=5432
-p 9292:9292
Which returns the following error...
Unable to load application: Sequel::DatabaseConnectionError: PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
bundler: failed to load command: puma (/pact_broker/vendor/bundle/ruby/2.6.0/bin/puma)
Sequel::DatabaseConnectionError: PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
Here is my database and user in pgAdmin 4
Here is the command I'm using to test my connection
psql postgresql://pact_broker:{Password}#localhost/pact_broker
Which returns the following...
psql (13.3)
WARNING: Console code page (65001) differs from Windows code page (1252)
8-bit characters might not work correctly. See psql reference
page "Notes for Windows users" for details.
Type "help" for help.
pact_broker=>
I've also tried running the following command which returns the same error
docker run pactfoundation/pact-broker -e PACT_BROKER_DATABASE_URL=postgresql://pact_broker:{Password}#host.docker.internal/pact_broker -p 9292:9292
My postgres.conf file contains listen_addresses = '*' and my pg_hba.conf contains the following
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all scram-sha-256
# IPv4 local connections:
host all all 127.0.0.1/32 scram-sha-256
# IPv6 local connections:
host all all ::1/128 scram-sha-256
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all scram-sha-256
host all all 0.0.0.0/0 trust
host replication all ::1/128 scram-sha-256
host all all ::/0 md5
I'm having similar problems with an instance hosted with Azure Database for PostgreSQL
My connection string looks like the following...
psql "host=pactbrokerserver.postgres.database.azure.com port=5432 dbname={your_database} user=postgres#pactbrokerserver password={your_password} sslmode=require"
Which I've attempted to connect to via the pact broker locally with...
docker run pactfoundation/pact-broker
-e PACT_BROKER_DATABASE_USERNAME=postgres#pactbrokerserver
-e PACT_BROKER_DATABASE_PASSWORD={Password}
-e PACT_BROKER_DATABASE_HOST=pactbrokerserver.postgres.database.azure.com
-e PACT_BROKER_DATABASE_NAME=pactbroker
-e PACT_BROKER_DATABASE_PORT=5432
-e PACT_BROKER_DATABASE_CONNECT_MAX_RETRIES=5
-p 9292:9292
And it returns exactly the same error.
What could be the issues here?
I think what's happening here is that you've put the container name before the environment argument list to the docker run command.
So instead of setting the PACT_BROKER_DATABASE_NAME and other environment variables for the running container with your custom values, they are simply being discarded by the runtime.
Try this instead:
docker run -e PACT_BROKER_DATABASE_USERNAME=pact_broker \
-e PACT_BROKER_DATABASE_PASSWORD={Password} \
-e PACT_BROKER_DATABASE_HOST=host.docker.internal \
-e PACT_BROKER_DATABASE_NAME=pact_broker \
-e PACT_BROKER_DATABASE_PORT=5432 \
-p 9292:9292 \
pactfoundation/pact-broker
Debugging Options
In case I'm wrong, it seems likely that the issue could be network path related. To rule this out, we can get a shell into the docker container by replacing the entrypoint (which is usually the script to start and run the web server) with /bin/sh:
docker run -it --entrypoint /bin/sh --user root -e PACT_BROKER_DATABASE_USERNAME=pact_broker -e PACT_BROKER_DATABASE_PASSWORD=Penwater858 -e PACT_BROKER_DATABASE_HOST=host.docker.internal -e PACT_BROKER_DATABASE_NAME=pact_broker -e PACT_BROKER_DATABASE_PORT=5432 -p 9292:9292 pactfoundation/pact-broker
# once in the container...
apk update
apk add busybox-extras
# see if there is a network path
telnet $PACT_BROKER_DATABASE_HOST $PACT_BROKER_DATABASE_PORT
Related
I've created a PostGIS docker container with the following code:
docker run --name=h4d -d -e POSTGRES_USER=h4d_user -e POSTGRES_PASS=password -e POSTGRES_DBNAME=gis -e ALLOW_IP_RANGE=0.0.0.0/0 -p 5432:5432 -v h4d_data:/var/lib/postgresql --restart=always kartoza/postgis:latest
I can connect to the docker from my localhost, but I can't from another terminal. The error message says "could not connect to server: Connextion timed out (0x0000274C/10060) Is the server running on host "" and accepting TCP/IP connections on port 5432?
I'm not sure if maybe I must edit some firewall settings or something else. I'm working on Windows 10
Maybe this will help ..
Go to the postgresql.conf and change the parameter of listen_addresses to the ip address you wish or just place * for all ips, e.g:
listen_addresses = '*'
In the pg_hba.conf file you also have to add which ips and users may access a certain database, e.g.
host my_db my_user 128.176.1.1 md5
In an Ubuntu machine these files are normally found at: /etc/postgresql/10/main
I've got a simple Code Build project for a Django App and I want to start a Postgres database in a docker container for testing.
The commands are:
docker run --name django-test-db -d -p 5432:5432 -e POSTGRES_PASSWORD=djangotest -e POSTGRES_USER=configuration-service-master -e POSTGRES_DB=configuration-service postgres:12-alpine
docker exec django-test-db psql -U configuration-service-master -c \"\\dn\"
On my local machine everything works quite fine, but in AWS CodeBuild I'm receiving the error:
psql: error: could not connect to server: could not connect to server:
No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Try two things:
Connect with '-h' flag to specify host so the connection goes through TCP/IP instead of Unix Socket
psql -h localhost -p 5432
Enable privileged mode on Build project:
https://docs.aws.amazon.com/codebuild/latest/userguide/change-project.html#change-project-console
I'm trying to extend the postgres Docker image to potentially (via an environment variable flag) execute flyway DB migrations on DB init. My Dockerfile is here:
FROM postgres:9.6
# Install curl and java (for Flyway)
RUN set -x \
&& apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl openjdk-8-jre
# Install Flyway
ENV FLYWAY_VERSION 4.2.0
ENV FLYWAY_INSTALL_DIR /usr/src/flyway
ENV FLYWAY_CONF ${FLYWAY_INSTALL_DIR}/flyway-${FLYWAY_VERSION}/conf/flyway.conf
ENV FLYWAY_EXE ${FLYWAY_INSTALL_DIR}/flyway-${FLYWAY_VERSION}/flyway
RUN mkdir -p ${FLYWAY_INSTALL_DIR} && \
curl -L https://repo1.maven.org/maven2/org/flywaydb/flyway-commandline/${FLYWAY_VERSION}/flyway-commandline-${FLYWAY_VERSION}.tar.gz | \
tar -xzC ${FLYWAY_INSTALL_DIR} && \
chmod +x ${FLYWAY_EXE}
# Copy migration scripts
ENV MIGRATIONS_LOCATION /flyway/migrations
COPY migrations $MIGRATIONS_LOCATION
COPY init_db.sh /docker-entrypoint-initdb.d/init_db.sh
With my init_db.sh startup script:
#!/bin/bash
set -e
RUN_MIGRATIONS="${RUN_MIGRATIONS:-false}"
DB_URL="jdbc:postgresql://localhost:5432/$DB_NAME"
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE DATABASE $DB_NAME;
EOSQL
if [ "$RUN_MIGRATIONS" == "true" ]; then
echo "running migrations ..."
${FLYWAY_EXE} -user=$POSTGRES_USER -password=$POSTGRES_PASSWORD -url=$DB_URL -locations="filesystem:$MIGRATIONS_LOCATION" migrate
fi
However, when running the container with RUN_MIGRATIONS=true, flyway fails to connect to postgres:
docker build . -t postgres-flyway && docker run -e DB_NAME=db -e RUN_MIGRATIONS=true -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres postgres-flyway
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....LOG: database system was shut down at 2018-08-06 02:19:32 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: autovacuum launcher started
LOG: database system is ready to accept connections
done
server started
ALTER ROLE
/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/init_db.sh
CREATE DATABASE
running migrations ...
Flyway 4.2.0 by Boxfuse
ERROR:
Unable to obtain Jdbc connection from DataSource (jdbc:postgresql://localhost:5432/db) for user 'postgres': Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State : 08001
Error Code : 0
Message : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
The postgres image runs postgres on port 5432 (as usual) so I'm at a loss on why flyway is unable to connect to postgres over localhost:5432.
I also noticed that within this context, pg_isready states that postgres is accepting connections but when specifying the hostname as localhost or 127.0.0.1 it is unable to reach postgres either. That is, by inserting a few pg_isready commands in my init_db.sh script:
...
pg_isready
pg_isready -p 5432
pg_isready -h localhost -p 5432
...
I see the following log output on postgres init:
...
/var/run/postgresql:5432 - accepting connections
/var/run/postgresql:5432 - accepting connections
localhost:5432 - no response
...
I'm suspicious that I've reached a limitation of postgres' initialize context, but I would like to understand why postgres is unreachable over localhost/127.0.0.1:5432 at this point of initialization.
I had the same problem running flyway when creating a docker image for my database based on the postgres:10.5 image. I added the following to my entrypoint.sh before running flyway, to confirm that the problem I was seeing was caused by the docker-entrypoint.sh change #Nick Maraston posted in his answer:
echo "$(date) - waiting for database to start"
while ! pg_isready -h localhost -p 5432 -d $POSTGRES_DB
do
echo "$(date) - waiting for database to start"
sleep 10
done
The result was that the above code looped for ever. I then replaced it with the following code to restart the database listening for TCP/IP connections on localhost:
pg_ctl -D "$PGDATA" -m fast -w stop
pg_ctl -D "$PGDATA" \
-o "-c listen_addresses='localhost'" \
-w start
Rather than restarting the database like this, a cleaner solution would be to use the JDBC -socketFactory option explained here.
I discovered the problem while digging through the images entry point script. A recent change to the image restricts postgres to only listen for connections over a unix domain socket during internal initialization: https://github.com/docker-library/postgres/pull/440
It is true that postgres docker is listening to a unix socket, eg. /var/run/postgresql/.s.PGSQL.5432. But it is not necessary to force the server to switch its listening address. Postgres database URI allows a connection string to point to a unix socket.
Reference: Connect to a database over a unix socket using SQLAlchemy
The example provided:
export DATABASE_URL=postgres://user:password#/dbname?host=/path/to/unix/socket
I was able to omit the host, and decided to use this environment variable in my /docker-entrypoint-initdb.d/*.sh script instead. Note that no string follows the # symbol, nor is there a host query string here. You may need to explicitly define the host depending on your application.
Solution:
export DATABASE_URL="postgres://$POSTGRES_USER:$POSTGRES_PASSWORD#/$POSTGRES_DB"
Spent quite sometime on this but no breakthrough. This is my container creation statement:
> docker create --name aasaan_db_dev -e POSTGRES_USER=aasaan -e
> POSTGRES_PASSWORD=aasaan_admin --expose 5432 postgres:9.4.0
>
> > deepakkt#deepakkt-ubuntu:~$ docker ps
>
> CONTAINER ID IMAGE COMMAND
> CREATED STATUS PORTS NAMES
> 15b21deb19a0 postgres:9.4.0 "/docker-entrypoin..." About
> an hour ago Up About an hour 5432/tcp aasaan_db_dev
The following works:
> deepakkt#deepakkt-ubuntu:~$ docker exec -it aasaan_db_dev psql -U
> aasaan psql (9.4.0) Type "help" for help.
>
> aasaan=#
However, from the host (same error from pgAdmin as well)
> deepakkt#deepakkt-ubuntu:~$ psql -h 127.0.0.1 -U aasaan
Password for user aasaan:
psql: FATAL: password authentication failed for user
> "aasaan" FATAL: password authentication failed for user "aasaan"
Contents of pg_hba.conf inside the container:
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local replication postgres trust
#host replication postgres 127.0.0.1/32 trust
#host replication postgres ::1/128 trust
host all all 0.0.0.0/0 md5
What am I missing?
It is needed to map the container port to host port.
Have a look at below docker documentation:
The EXPOSE instruction does not actually publish the port. It
functions as a type of documentation between the person who builds the
image and the person who runs the container, about which ports are
intended to be published. To actually publish the port when running
the container, use the -p flag on docker run to publish and map one or
more ports, or the -P flag to publish all exposed ports and map them
to high-order ports.
Source: docker docs
docker ps is showing 5432/tcp for your port section.
This means the port 5432 is opened on your postgres container. (This is by default because it's exposed in the Dockerfile of postgres so you don't need the --expose command).
The name of your container is aasaan_db_dev. Other containers which are deployed in the same bridge network can access your container using aasaan_db_dev:5432.
But your container is not mapping it's port to the outside which means you can not access the container from outside the bridge network. (for example not from eth0).
To map your port use the following command:
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d -p 5432:5432 postgres
In this case the -p option will map 5432 from your container on 5432 of your host. Now you should be able to connect to the container from your host network (for example localhost).
You should not use 127.0.0.1 as it will refer to your localhost. But actually, it is running on a container.
You should use
psql -h <imagename> -U <username>
You need to provide port as well.
I want to remotely connect to a Postgres instance. I know we can do this using the psql command passing the hostname
I tried the following:
psql -U postgres -p 5432 -h hostname
I modified the /etc/postgresql/9.3/main/pg_hba.conf file on the target machine to allow remote connections by default
I added the following line to the file
host all all source_ip/32 trust
I restarted the cluster using
pg_ctlcluster 9.2 mycluster stop
pg_ctlcluster 9.2 mycluster start
However, when I try to connect from the source_ip, I still get the error
Is the server running on host "" and accepting TCP/IP connections on port 5432?
What am I doing wrong here?
I resolved this issue using below options:
Whitelist your DB host from your network team to make sure you have access to remote host
Install postgreSQL version 4 or above
Run below command:
psql -h <REMOTE HOST> -p <REMOTE PORT> -U <DB_USER> <DB_NAME>
psql -h <IP_Address> -p <port_no> -d <database_name> -U <DB_username> -W
-W option will prompt for password. For example:
psql -h 192.168.1.50 -p 5432 -d testdb -U testuser -W
I figured it out.
Had to set listen_addresses='*' in postgresql.conf to allow for incoming connections from any ip / all ip
Step Wise below
Opening the Port - Make sure the PSQL Port is open to all remote connections or connections from a specific set of IPs as per your requirement. PSQL, in general, runs at port 5432, and it is configurable, so expose relevant Port accordingly.
Update Remote Server PSQL Configuration - Set listen_addresses = '*' in postgresql.conf file, path in general is /etc/postgresql/psql_version/main/postgresql.conf
Connect remotely - psql -U <db_username> -h <IP_address> - in case psql is running on a port other than 5432 on the remote server, specify port by adding -p <port_number>
A little plus below -
In case the IP has been mapped to a domain name, you can connect by replacing <IP_address> with <host_name>. To do this, add a new connection rule in pg_hba.conf file
Note -
All above explained can cause security issues - best practice always is to either keep your psql port closed, or only allow a list of IPs to connect through the port.
Note that "ident" in pg_hba.conf requires a "ident server" to be running on the client.