Add custom pg_hba.conf in postgres docker container? - postgresql

I want to expose 5432 port to be accessible externally, but before doing that I want to limit it to only specific IPs, so I want to do that via pg_hba.conf
If I use docker default settings, communication between containers, works fine (by specified networks).
But if I specify pg_hba.conf:
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all md5
And try to use this pg_hba.conf file (which is actually straight copy from container itself):
psycopg2.OperationalError: could not translate host name "db" to address: Temporary failure in name resolution (where db is postgres db service in docker-compose.yml).
My compose has this for postgres:
other: # other service that connects to postgres using psycopg2
networks:
- backend
db:
image: postgres:13
command: ["postgres", "-c", "config_file=/etc/postgresql.conf"]
ports:
- 5432:5432 # for external communications
volumes:
- db-data:/var/lib/postgresql/data
- path/to/cfg:/etc/postgresql.conf
- path/to/hba:/etc/postgresql/pg_hba.conf
networks:
- backend
networks:
backend:
And then in postgresql.conf I have edited this line:
hba_file = '/etc/postgresql/pg_hba.conf'
Also config has this set: listen_addresses = '*'
Inspecting docker network:
docker network inspect bridge
[
{
"Name": "bridge",
"Id": "fe4fe8b8eb4e0c06d43428eadd2bb3f44a2ac581fe55618b4c70c5c28c107b8d",
"Created": "2022-11-26T09:34:03.461094956+02:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {
"com.docker.network.bridge.default_bridge": "true",
"com.docker.network.bridge.enable_icc": "true",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
"com.docker.network.bridge.name": "docker0",
"com.docker.network.driver.mtu": "1500"
},
"Labels": {}
}
]
P.S. I checked
PostgreSQL in Docker - pg_hba.conf to allow access from host to container
And How can I allow connections by specifying docker-compose host names in postgres's pg_hba.conf file?
But none seem to make any difference.
UPDATE:
I retried again, using same thing. Just specifying pg_hba.conf and now I don't get that error. Not sure how it was solved. But pg_hba.conf is picked up just fine and I can limit access with it.

Try to keep both config files on the default place (in /var/lib/postgresql/data) and revert the change for hba_file location. Make sure the files have the same permission and ownership as in container: 600 postgres:postgres (999:999)
EDIT:
Just looked in the docs. They start pg with custom config like this:
$ docker run -d --name some-postgres -v "$PWD/my-postgres.conf":/etc/postgresql/postgresql.conf -e POSTGRES_PASSWORD=mysecretpassword postgres -c 'config_file=/etc/postgresql/postgresql.conf'
where postgres before "-c" is the image name. I think you should redirect "-c" part to entrypoint.
EDIT2:
You're right, according to the entrypoint script this call with "postgres" as a parameter also works.

You need to add the POSTGRES_PASSWORD as an environment variable on the docker-compose.yml for the the db container, otherwise the postgres will failed with:
tmp-db-1 | Error: Database is uninitialized and superuser password is not specified.
tmp-db-1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
tmp-db-1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
tmp-db-1 |
tmp-db-1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
tmp-db-1 | connections without a password. This is *not* recommended.
tmp-db-1 |
tmp-db-1 | See PostgreSQL documentation about "trust":
tmp-db-1 | https://www.postgresql.org/docs/current/auth-trust.html
If it still fails could you share the logs of the db container?

Related

Docker-compose container postgresql not available at localhost:5432 [duplicate]

Newbie with docker, I am trying to connect throught localhost my pgAdmin container to the postgres one.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0b00555238ba dpage/pgadmin4 "/entrypoint.sh" 43 minutes ago Up 43 minutes 0.0.0.0:80->80/tcp, 443/tcp pedantic_turing
e79fb6440a95 postgres "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp pg-docker
I succeed connecting with psql command.
psql -h localhost -U postgres -d postgres
But when I create the server on pgAdmin with the same parameters as psql I got the following error.
Unable to connect to server:
could not connect to server: Connection refused Is the server running
on host "localhost" (127.0.0.1) and accepting TCP/IP connections on
port 5432? could not connect to server: Address not available Is the
server running on host "localhost" (::1) and accepting TCP/IP
connections on port 5432?
I succeed to connect throught the IPAddress given by docker inspect on the container.
By the way, I checked postgresql.conf and assert that listen_addresses = '*' and also that pg_hba.conf contain host all all all md5.
But I don't get it, why shouldn't I be able to use the localhost address ? And why does docker even give me an address that is not local ?
In this case:
Pgadmin fails to connect to localhost, but psql works from outside docker.
both pgadmin & Postgres are running as Containers
Although you haven't indicated if you are doing so, ideally both containers could be part of a custom bridge network for automatic DNS resolution.
If not added explicitly they will be part of the default bridge network.
To find out the networks created in your docker runtime, type:
$ docker network ls
Some networks will be listed in the console, maybe you'll find a [name]_default it should be your network.
Execute
docker network inspect [name]_default
it'll show up a bunch of information, for us the most important is IPv4Address, something like this:
"7c3cd7532ab8aacc70830afb74adad7296d9c8ddd725c498af2d7ee2d2c2aadd": {
"Name": "intime_postegres_1",
"EndpointID": "56a9cb574469f22259497b72719f9f4a3e555b09f95058fcf389ef5287381f28",
"MacAddress": "02:42:ac:12:00:02",
"IPv4Address": "172.18.0.2/16",
"IPv6Address": ""
}
Instead of using localhost for the server name/ip in the pgAdmin new server dialog, connect to the postgres instance's "IPv4Address".
In my case connecting at 172.18.0.2:5432, worked like a charm.
I too had the case when you're able to connect with psql command on terminal but not with pgAdmin4.
The following solution worked for me.
First -
docker ps
From there you'll get the container name/ID of the postgres container, then do -
docker inspect name_of_container_here
It'll give you something like this -
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "f35dbe66b36cd38673aad6e1278ad33031ef9d5abd34582d4b91955e288f855e",
"EndpointID": "2e63ea59e9d0de7526bb02ba29bc0b16bcad51b8963127ad380416d15da994db",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:02",
"DriverOpts": null
}
}
Note the IPAddress and provide that while creating a new server in pgAdmin4 -
I had the same issue and I solved it in a non-canonical way, but very satisfactory to my local workflow.
I always start my postgresql container with a docker-compose.yaml file. So I just added the pgAdmin to the same compose file:
version: "3.7"
services:
my_awesome_db:
image: postgres:latest
ports:
- "5432:5432"
container_name: postgresql-local
volumes:
- "/var/run/postgres.sock:/var/run/postgres/postgres.sock"
- "/home/myuser/docker-apps/volumes/postgres-data:/var/lib/postgresql/data"
pg_admin:
image: dpage/pgadmin4:latest
container_name: pgadmin4
ports:
- "15432:80"
environment:
- GUNICORN_THREADS=1
- PGADMIN_DEFAULT_EMAIL=my_awesome_email#email.com
- PGADMIN_DEFAULT_PASSWORD=does_not_matter
depends_on:
- my_awesome_db
So I access pgAdmin on my localhost:15432 (just because it's easy to remember), and I've configured pgAdmin with:
Host: my_awesome_db
Port: 5432
Username: postgres
For me worked like this:
Add Server -> Connection -> Hostname/address
Set field to 172.17.0.1
More info at Docker Network doc: https://docs.docker.com/network/network-tutorial-standalone
You just need to specify in your connection string the, server parameter, the name of your container name and not the localhost or 127.0.0.1
first you need to get container information
$docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0b00555238ba dpage/pgadmin4 "/entrypoint.sh" 43 minutes ago Up 43 minutes 0.0.0.0:80->80/tcp, 443/tcp pedantic_turing
e79fb6440a95 postgres "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp pg-docker
then extract IP Address
$sudo docker inspect e79fb6440a95 | grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "192.168.64.2",
And when you create your server put your IP Address
Because I was connecting to a server on the hose, it was as simple as adding
--net host
to the docker run command, so that the container gets access to the host's ports.

Docker - Postgres and pgAdmin 4 : Connection refused

Newbie with docker, I am trying to connect throught localhost my pgAdmin container to the postgres one.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0b00555238ba dpage/pgadmin4 "/entrypoint.sh" 43 minutes ago Up 43 minutes 0.0.0.0:80->80/tcp, 443/tcp pedantic_turing
e79fb6440a95 postgres "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp pg-docker
I succeed connecting with psql command.
psql -h localhost -U postgres -d postgres
But when I create the server on pgAdmin with the same parameters as psql I got the following error.
Unable to connect to server:
could not connect to server: Connection refused Is the server running
on host "localhost" (127.0.0.1) and accepting TCP/IP connections on
port 5432? could not connect to server: Address not available Is the
server running on host "localhost" (::1) and accepting TCP/IP
connections on port 5432?
I succeed to connect throught the IPAddress given by docker inspect on the container.
By the way, I checked postgresql.conf and assert that listen_addresses = '*' and also that pg_hba.conf contain host all all all md5.
But I don't get it, why shouldn't I be able to use the localhost address ? And why does docker even give me an address that is not local ?
In this case:
Pgadmin fails to connect to localhost, but psql works from outside docker.
both pgadmin & Postgres are running as Containers
Although you haven't indicated if you are doing so, ideally both containers could be part of a custom bridge network for automatic DNS resolution.
If not added explicitly they will be part of the default bridge network.
To find out the networks created in your docker runtime, type:
$ docker network ls
Some networks will be listed in the console, maybe you'll find a [name]_default it should be your network.
Execute
docker network inspect [name]_default
it'll show up a bunch of information, for us the most important is IPv4Address, something like this:
"7c3cd7532ab8aacc70830afb74adad7296d9c8ddd725c498af2d7ee2d2c2aadd": {
"Name": "intime_postegres_1",
"EndpointID": "56a9cb574469f22259497b72719f9f4a3e555b09f95058fcf389ef5287381f28",
"MacAddress": "02:42:ac:12:00:02",
"IPv4Address": "172.18.0.2/16",
"IPv6Address": ""
}
Instead of using localhost for the server name/ip in the pgAdmin new server dialog, connect to the postgres instance's "IPv4Address".
In my case connecting at 172.18.0.2:5432, worked like a charm.
I too had the case when you're able to connect with psql command on terminal but not with pgAdmin4.
The following solution worked for me.
First -
docker ps
From there you'll get the container name/ID of the postgres container, then do -
docker inspect name_of_container_here
It'll give you something like this -
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "f35dbe66b36cd38673aad6e1278ad33031ef9d5abd34582d4b91955e288f855e",
"EndpointID": "2e63ea59e9d0de7526bb02ba29bc0b16bcad51b8963127ad380416d15da994db",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:02",
"DriverOpts": null
}
}
Note the IPAddress and provide that while creating a new server in pgAdmin4 -
I had the same issue and I solved it in a non-canonical way, but very satisfactory to my local workflow.
I always start my postgresql container with a docker-compose.yaml file. So I just added the pgAdmin to the same compose file:
version: "3.7"
services:
my_awesome_db:
image: postgres:latest
ports:
- "5432:5432"
container_name: postgresql-local
volumes:
- "/var/run/postgres.sock:/var/run/postgres/postgres.sock"
- "/home/myuser/docker-apps/volumes/postgres-data:/var/lib/postgresql/data"
pg_admin:
image: dpage/pgadmin4:latest
container_name: pgadmin4
ports:
- "15432:80"
environment:
- GUNICORN_THREADS=1
- PGADMIN_DEFAULT_EMAIL=my_awesome_email#email.com
- PGADMIN_DEFAULT_PASSWORD=does_not_matter
depends_on:
- my_awesome_db
So I access pgAdmin on my localhost:15432 (just because it's easy to remember), and I've configured pgAdmin with:
Host: my_awesome_db
Port: 5432
Username: postgres
For me worked like this:
Add Server -> Connection -> Hostname/address
Set field to 172.17.0.1
More info at Docker Network doc: https://docs.docker.com/network/network-tutorial-standalone
You just need to specify in your connection string the, server parameter, the name of your container name and not the localhost or 127.0.0.1
first you need to get container information
$docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0b00555238ba dpage/pgadmin4 "/entrypoint.sh" 43 minutes ago Up 43 minutes 0.0.0.0:80->80/tcp, 443/tcp pedantic_turing
e79fb6440a95 postgres "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp pg-docker
then extract IP Address
$sudo docker inspect e79fb6440a95 | grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "192.168.64.2",
And when you create your server put your IP Address
Because I was connecting to a server on the hose, it was as simple as adding
--net host
to the docker run command, so that the container gets access to the host's ports.

Docker postgres image: no pg_hba.conf entry for host

I have a Docker image for DB (postgres is installed) and I try to run it but I get the following:
db | 2018-11-27 08:29:28.849 UTC [2808] FATAL: no
pg_hba.conf entry for host "172.x.x.x", user "postgres", database
"postgres", SSL off
I tried the following:
To change the IP and put my current machine's ip
When setting up Postgres: echo -e "host all all 0.0.0.0/0 trust" I put that
None of those work.
I ran into the same problem getting my application container to talk to my postgres container. I fixed this with three steps:
define the network ip address range for the docker network
# +----------------------+
# | docker-compose.yml |
# +----------------------+
networks:
my_docker_network:
driver: bridge
ipam:
config:
- subnet: 10.5.0.0/16
whitelist the same network ip address range in the postgres pg_hba.conf file
# +---------------+
# | pg_hba.conf |
# +---------------+
# TYPE DATABASE USER ADDRESS METHOD
# IPv4 local connections:
host all all 127.0.0.1/32 trust
host all all 10.5.0.0/16 trust
I used trust to avoid authentication hassles, but you could specify md5, cert, or any other valid authentication method.
allow postgres to listen to external hosts in the postgresql.conf file
# +-------------------+
# | postgresql.conf |
# +-------------------+
# - Connection Settings -
listen_addresses = '*' # what IP address(es) to listen on;

Docker connection to PostgreSQL failing from host

Spent quite sometime on this but no breakthrough. This is my container creation statement:
> docker create --name aasaan_db_dev -e POSTGRES_USER=aasaan -e
> POSTGRES_PASSWORD=aasaan_admin --expose 5432 postgres:9.4.0
>
> > deepakkt#deepakkt-ubuntu:~$ docker ps
>
> CONTAINER ID IMAGE COMMAND
> CREATED STATUS PORTS NAMES
> 15b21deb19a0 postgres:9.4.0 "/docker-entrypoin..." About
> an hour ago Up About an hour 5432/tcp aasaan_db_dev
The following works:
> deepakkt#deepakkt-ubuntu:~$ docker exec -it aasaan_db_dev psql -U
> aasaan psql (9.4.0) Type "help" for help.
>
> aasaan=#
However, from the host (same error from pgAdmin as well)
> deepakkt#deepakkt-ubuntu:~$ psql -h 127.0.0.1 -U aasaan
Password for user aasaan:
psql: FATAL: password authentication failed for user
> "aasaan" FATAL: password authentication failed for user "aasaan"
Contents of pg_hba.conf inside the container:
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local replication postgres trust
#host replication postgres 127.0.0.1/32 trust
#host replication postgres ::1/128 trust
host all all 0.0.0.0/0 md5
What am I missing?
It is needed to map the container port to host port.
Have a look at below docker documentation:
The EXPOSE instruction does not actually publish the port. It
functions as a type of documentation between the person who builds the
image and the person who runs the container, about which ports are
intended to be published. To actually publish the port when running
the container, use the -p flag on docker run to publish and map one or
more ports, or the -P flag to publish all exposed ports and map them
to high-order ports.
Source: docker docs
docker ps is showing 5432/tcp for your port section.
This means the port 5432 is opened on your postgres container. (This is by default because it's exposed in the Dockerfile of postgres so you don't need the --expose command).
The name of your container is aasaan_db_dev. Other containers which are deployed in the same bridge network can access your container using aasaan_db_dev:5432.
But your container is not mapping it's port to the outside which means you can not access the container from outside the bridge network. (for example not from eth0).
To map your port use the following command:
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d -p 5432:5432 postgres
In this case the -p option will map 5432 from your container on 5432 of your host. Now you should be able to connect to the container from your host network (for example localhost).
You should not use 127.0.0.1 as it will refer to your localhost. But actually, it is running on a container.
You should use
psql -h <imagename> -U <username>
You need to provide port as well.

fe_sendauth: no password supplied

database.yml:
# SQLite version 3.x
# gem install sqlite3
#
# Ensure the SQLite 3 gem is defined in your Gemfile
# gem 'sqlite3'
development:
adapter: postgresql
encoding: utf8
database: sampleapp_dev #can be anything unique
#host: localhost
#username: 7stud
#password:
#adapter: sqlite3
#database: db/development.sqlite3
pool: 5
timeout: 5000
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
adapter: postgresql
encoding: utf8
database: sampleapp_test #can be anything unique
#host: localhost
#username: 7stud
#password:
#adapter: sqlite3
#database: db/test.sqlite3
pool: 5
timeout: 5000
production:
adapter: postgresql
database: sampleapp_prod #can be anything unique
#host: localhost
#username: 7stud
#password:
#adapter: sqlite3
#database: db/production.sqlite3
pool: 5
timeout: 5000
pg_hba.conf:
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local replication postgres md5
#host replication postgres 127.0.0.1/32 md5
#host replication postgres ::1/128 md5
I changed the METHOD in the first three lines from md5 to trust, but I still get the error.
And no matter what combinations of things I try in database.yml, when I do:
~/rails_projects/sample_app4_0$ bundle exec rake db:create:all
I always get the error:
fe_sendauth: no password supplied
I followed this tutorial to get things setup:
https://pragtob.wordpress.com/2012/09/12/setting-up-postgresql-for-ruby-on-rails-on-linux
Mac OSX 10.6.8
PostgreSQL 9.2.4 installed via enterpriseDB installer
Install dir: /Library/PostgreSQL/9.2
After making changes to the pg_hba.conf or postgresql.conf files, the cluster needs to be reloaded to pick up the changes.
From the command line: pg_ctl reload
From within a db (as superuser): select pg_reload_conf();
From PGAdmin: right-click db name, select "Reload Configuration"
Note: the reload is not sufficient for changes like enabling archiving, changing shared_buffers, etc -- those require a cluster restart.
I just put --password flag into my command and after hitting Enter it asked me for password, which I supplied.
psql -U postgres --password
Password:<Enter your password>
Then this will appear. postgres=#
This worked for me
This occurs if the password for the database is not given.
default="postgres://postgres:password#127.0.0.1:5432/DBname"
I also had same issue. But my application was running on docker daemon. I changed those environment in docker-compose.yml file. And also changed in database.yml file. These changes resolved my issue.
DATABASE_HOST= db
POSTGRES_PASSWORD= password
What worked for me was making sure that I had these four lines after development, test, and production in the database.yml file. I leave username and password blank here for security purposes, but fill these out in the actual application
username:
password:
host: localhost
port: 5432