Create a postgres database within a docker-compose.yml file - postgresql

I want to create a postgres database named bank within a docker-compose.yml file just after the postgres container has started but when i run docker-compose --env-file .env -f docker-compose.yaml up -d i get this error: /var/run/postgresql:5432 - no response...
when i remove the line with the command: option, everything start correctly and i get: /var/run/postgresql:5432 - accepting connections
But now, i have to run this steps by steps in the terminal:
docker exec -it postgres bash
psql -U my_user_name
create database bank;
and exit
And i really don't want it to work like that, instead, i want the database to be created within the docker-compose file. (Note that, when i remove the command: option, and i run until pg_isready; do sleep 1; done; echo accepting; inside the container, it ouput accepting almost immediately)
The POSTGRES_DB env variable doesn't work, The username is still used as default
This is my docker-compose file:
services:
db:
container_name: postgres
image: postgres
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- PGDATA=/data/postgres
volumes:
- db:/data/postgres
ports:
- "5332:5432"
networks:
- db
restart: unless-stopped
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d postgres" ]
interval: 30s
timeout: 10s
retries: 5
command: /bin/bash -c "until pg_isready -U ${POSTGRES_USER} -p 5432; do sleep 1; done; psql -U ${POSTGRES_USER} -c 'CREATE DATABASE bank;'"
networks:
db:
driver: bridge
volumes:
db:
The most important line is the one with command: :
command: /bin/bash -c "until pg_isready -U ${POSTGRES_USER} -p 5432; do sleep 1; done; psql -U ${POSTGRES_USER} -c 'CREATE DATABASE bank;'"
Please help me with the correct command to execute so that the database will be created automatically when running docker-compose --env-file .env -f file up -d

Have you considered using the POSTGRES_DB environment variable ?
services:
db:
container_name: postgres
image: postgres
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
PGDATA: /data/postgres
POSTGRES_DB: bank
volumes:
- db:/data/postgres
ports:
- "5332:5432"
networks:
- db
restart: unless-stopped
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d postgres" ]
interval: 30s
timeout: 10s
retries: 5
networks:
db:
driver: bridge
volumes:
db:

Related

docker mongo for single (primary node only) replica set (for development)?

This is the portion of the dockerfile that has served us well to date. However, now I need to convert this to be a single node replica set (for transactions to work). I don't want any secondary or arbiter - just the primary node. What am I missing to get this working?
mongo:
image: mongo:4.4.3
container_name: mongo
restart: unless-stopped
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: myPass
command: mongod --port 27017
ports:
- '27017:27017'
volumes:
- ./data/mongodb:/data/db
- ./data/mongodb/home:/home/mongodb/
- ./configs/mongodb/mongo-init.sh:/docker-entrypoint-initdb.d/mongo-init.sh:ro
Got it working. I inserted the following into the block in my question above:
hostname: mongodb
volumes:
- ./data/mongodb/data/log/:/var/log/mongodb/
# the healthcheck avoids the need to initiate the replica set
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u root -p imagiaRoot --quiet) -eq 1
interval: 10s
start_period: 30s
I was unable to initiate the replica set via the healthcheck. I used the bash script below instead. For Windows users, be sure to call your DB with the name of your computer. For example:
mongodb://DESKTOP-QPRKMN2:27017
run-test.sh
#!/bin/bash
echo "Running docker-compose"
docker-compose up -d
echo "Waiting for DB to initialize"
sleep 10
echo "Initiating DB"
docker exec mongo_container mongo --eval "rs.initiate();"
echo "Running tests"
# test result
if go test ./... -v
then
echo "Test PASSED"
else
echo "Test FAILED"
fi
# cleanup
docker-compose -f docker-compose.test.yml down
docker-compose file
version: '3.8'
services:
mongo:
hostname: $HOST
container_name: mongo_container
image: mongo:5.0.3
volumes:
- ./test-db.d
expose:
- 27017
ports:
- "27017:27017"
restart: always
command: ["--replSet", "test", "--bind_ip_all"]
This forum post was very helpful: https://www.mongodb.com/community/forums/t/docker-compose-replicasets-getaddrinfo-enotfound/14301/4

Dump mongo database running on a docker container

I have this docker-compose.yml file where I run a mongo container
version: '3'
services:
appapi:
container_name: appapi
image: strapi/strapi:3.1.3
environment:
DATABASE_CLIENT: ${APPAPI_DATABASE_CLIENT}
DATABASE_HOST: ${APPAPI_DATABASE_HOST}
DATABASE_PORT: ${APPAPI_DATABASE_PORT}
DATABASE_NAME: ${APPAPI_DATABASE_NAME}
DATABASE_USERNAME: ${APPAPI_DATABASE_USERNAME}
DATABASE_PASSWORD: ${APPAPI_DATABASE_PASSWORD}
ports:
- 1337:1337
volumes:
- ./app:/srv/app
depends_on:
- appmongo
appmongo:
container_name: appmongo
image: mongo:4.4.0
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: ${APPDB_MONGO_INITDB_ROOT_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${APPDB_MONGO_INITDB_ROOT_PASSWORD}
ports:
- "27027:27017"
volumes:
- ./data/db:/data/db
I want to backup the database running a dump
docker run -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=admin --rm mongo mongodump --host mongoapp:27027 --archive --gzip | cat > ./mongodumps/dump_$(date '+%d-%m-%Y_%H-%M-%S').gz
I tried to modify the previous command but I am not able to connect and do the dump, I am getting
2020-08-15T19:27:04.870+0000 Failed: can't create session: could not connect to server: server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: mongoapp:27027, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : dial tcp: lookup mongoapp on 192.168.65.1:53: no such host }, ] }
I was able to dump/restore with the following commands
dump
docker exec defymongo sh -c 'mongodump --archive -u {{mongouser}} -p {{mongopass}}' > ./mongodumps/dump_$(date '+%d-%m-%Y_%H-%M-%S').gz
restore
docker exec -i defymongo sh -c 'mongorestore --archive -u {{mongouser}} -p {{mongopass}}' < ./mongodumps/dump_$(date '+%d-%m-%Y_%H-%M-%S').gz
The difference is here the commands use sh -c to execute mongorestore and pass parameters with authentication values.
This is not enough to backup Strapi. Probably there are some values inside the /src/app folder in Strapi that should also be backed up
Hopefully this will helps someone else

Mongo container with a replica set with only one node in docker-compose

I want to create a Docker container with an instance of Mongo. In particular, I would like to create a replica set with only one node (since I'm interested in transactions and they are only available for replica sets).
Dockerfile
FROM mongo
RUN echo "rs.initiate();" > /docker-entrypoint-initdb.d/replica-init.js
CMD ["--replSet", "rs0"]
docker-compose.yml
version: "3"
services:
db:
build:
dockerfile: Dockerfile
context: .
ports:
- "27017:27017"
If I use the Dockerfile alone everything is fine, while if I use docker-compose it does not work: in fact if I then log to the container I got prompted as rs0:OTHER> instead of rs0:PRIMARY>.
I consulted these links but the solutions proposed are not working:
https://github.com/docker-library/mongo/issues/246#issuecomment-382072843
https://github.com/docker-library/mongo/issues/249#issuecomment-381786889
This is the compose file I have used for a while now for local development. You can remove the keyfile pieces if you don't need to connect via SSL.
version: "3.8"
services:
mongodb:
image : mongo:4
container_name: mongodb
hostname: mongodb
restart: on-failure
environment:
- PUID=1000
- PGID=1000
- MONGO_INITDB_ROOT_USERNAME=mongo
- MONGO_INITDB_ROOT_PASSWORD=mongo
- MONGO_INITDB_DATABASE=my-service
- MONGO_REPLICA_SET_NAME=rs0
volumes:
- mongodb4_data:/data/db
- ./:/opt/keyfile/
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
interval: 10s
start_period: 30s
command: "--bind_ip_all --keyFile /opt/keyfile/keyfile --replSet rs0"
volumes:
mongodb4_data:
It uses Docker's health check (with a startup delay) to sneak in the rs.initiate() if it actually needs it after it's already running.
To create a keyfile.
Mac:
openssl rand -base64 741 > keyfile
chmod 600 keyfile
Linux:
openssl rand -base64 756 > keyfile
chmod 600 keyfile
sudo chown 999 keyfile
sudo chgrp 999 keyfile
The top answer stopped working for me in later MongoDB and Docker versions. Particularly because rs.initiate().ok would throw an error if the replica set was already initiated, causing the whole command to fail. In addition, connecting from another container was failing because the replica set's sole member had some random host, which wouldn't allow the connection. Here's my new docker-compose.yml:
services:
web:
# ...
environment:
DATABASE_URL: mongodb://root:root#db/?authSource=admin&tls=false
db:
build:
context: ./mongo/
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: root
ports:
- '27017:27017'
volumes:
- data:/data/db
healthcheck:
test: |
test $$(mongosh --quiet -u root -p root --eval "try { rs.initiate({ _id: 'rs0', members: [{ _id: 0, host: 'db' }] }).ok } catch (_) { rs.status().ok }") -eq 1
interval: 10s
start_period: 30s
volumes:
data:
Inside ./mongo/, I have a custom Dockerfile that looks like:
FROM mongo:6
RUN echo "password" > /keyfile \
&& chmod 600 /keyfile \
&& chown 999 /keyfile \
&& chgrp 999 /keyfile
CMD ["--bind_ip_all", "--keyFile", "/keyfile", "--replSet", "rs0"]
This Dockerfile is suitable for development, but you'd definitely want a securely generated and persistent keyfile to be mounted in production (and therefore strike the entire RUN command).
You still need to issue replSetInitiate even if there's only one node in the RS.
See also here.
I had to do something similar to build tests around ChangeStreams which are only available when running mongo as a replica set. I don't remember where I pulled this from, so I can't explain it in detail but it does work for me. Here is my setup:
Dockerfile
FROM mongo:5.0.3
RUN echo "rs.initiate({'_id':'rs0', members: [{'_id':1, 'host':'127.0.0.1:27017'}]});" > "/docker-entrypoint-initdb.d/init_replicaset.js"
RUN echo "12345678" > "/tmp/key.file"
RUN chmod 600 /tmp/key.file
RUN chown 999:999 /tmp/key.file
CMD ["mongod", "--replSet", "rs0", "--bind_ip_all", "--keyFile", "/tmp/key.file"]
docker-compose.yml
version: '3.7'
services:
mongo:
build: .
restart: always
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u admin -p pass --quiet) -eq 1
interval: 10s
start_period: 30s
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: pass
MONGO_INITDB_DATABASE: test
Run docker-compose up and you should be good.
Connection String: mongodb://admin:pass#localhost:27017/test
Note: You shouldn't use this in production obviously, adjust the key "12345678" in the Dockerfile if security is a concern.
If you just need single node replica set of MongoDB via docker-compose.yml you can simply use this:
mongodb:
image: mongo:5
restart: always
command: ["--replSet", "rs0", "--bind_ip_all"]
ports:
- 27018:27017
healthcheck:
test: mongo --eval "rs.initiate()"
start_period: 5s
This one works fine for me:
version: '3.4'
services:
ludustack-db:
container_name: ludustack-db
command: mongod --auth
image: mongo:latest
hostname: mongodb
ports:
- '27017:27017'
env_file:
- .env
environment:
- MONGO_INITDB_ROOT_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
- MONGO_INITDB_ROOT_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
- MONGO_INITDB_DATABASE=${MONGO_INITDB_DATABASE}
- MONGO_REPLICA_SET_NAME=${MONGO_REPLICA_SET_NAME}
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
interval: 60s
start_period: 60s

Docker-compose postgresql password authentication failed

I'm trying to get a setup going with a webservice that consumes a postgres database. Should be simple to setup, but I'm getting errors. So, first thing I want to make sure is that the database I set up is actually there and running.
To test this I substitute the "consumer" or "client" for an alpine interactive shell like so:
version: '3'
services:
db:
image: postgres:10.1-alpine
container_name: db
expose:
- 5432
volumes:
- "dbdata:/var/lib/postgresql/data"
environment:
- POSTGES_USER=user
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=db
web:
image: alpine:latest
stdin_open: true
tty: true
entrypoint: /bin/sh
depends_on:
- db
volumes:
dbdata:
Then I run the following command to get into the interactive shell:
docker-compose run web
and the following command to get in the database:
apk --update add postgresql-client && rm -rf /var/cache/apk/*
psql -h db -U user db
I get a plain denial from postgresql:
psql: FATAL: password authentication failed for user "user"
Same error message for each combo of username/password/databasename I try. Not much helpful.
What am I doing wrong here?
You have a typo in your docker-compose file. You mispelled POSTGRES here:
POSTGES_USER=user
That means the user user isn't being created. If I correct that typo, so that I have:
version: '3'
services:
db:
image: postgres:10.1-alpine
expose:
- 5432
volumes:
- "dbdata:/var/lib/postgresql/data"
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=db
web:
image: alpine:latest
stdin_open: true
tty: true
entrypoint: /bin/sh
depends_on:
- db
volumes:
dbdata:
Start the environment:
docker-compose up -d
Attach to the web contained and install the postgresql client:
$ docker attach project_web_1
/ # apk add --update postgresql-client
Then I can connect without a problem:
/ # psql -h db -U user db
Password for user user:
psql (11.2, server 10.1)
Type "help" for help.
db=#

Docker postgress User "postgres" has no password assigned

I keep getting
User "postgres" has no password assigned.
updated
.env
POSTGRES_PORT=5432
POSTGRES_DB=demo_db2
POSTGRES_USER=postgres
POSTGRES_PASSWORD=password
Even though the postgres password is set.
I'm trying to use the same variables from the following command
docker run --name demo4 -e POSTGRES_PASSWORD=password -d postgres
Could this be an issue with volumes ? im very confused.
I ran this command as well
docker run -it --rm --name demo4 -p 5432:5432 -e POSTGRES_PASSWORD=password -e POSTGRES_USER=postgress postgres:9.4
docker-compose.yml
# docker-compose.yml
version: "3"
services:
app:
build: .
depends_on:
- database
ports:
- 8000:8000
environment:
- POSTGRES_HOST=database
database:
image: postgres:9.6.8-alpine
volumes:
- pgdata:/var/lib/postgresql/pgdata
ports:
- 8002:5432
react_client:
build:
context: ./client
dockerfile: Dockerfile
image: react_client
working_dir: /home/node/app/client
volumes:
- ./:/home/node/app
ports:
- 8001:8001
env_file:
- ./client/.env
volumes:
pgdata:
You are missing the inclusion of the .env file...
Docker composer:
database:
environment:
- ENV_VAR=VALUE
or
database:
env_file:
- .env
Plain Docker:
docker run options --env ENV_VAR=VALUE ...
or
docker run options --env-file .env ...`