I want to establish a gitlab test stage that uses a gitlab service postgres database. The problem is, that everytime I try to access the database via a script call in the pipeline I get the following error:
psql: error: could not connect to server: Connection refused
Is the server running on host "postgres" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
yaml looks the following
image: some_image:latest
stages:
- test
tests:
image: node:latest
stage: test
services:
- postgres:latest
before_script:
- apt-get update && apt-get install -y postgresql-client libpq-dev
# access database script from another repo here through git clone
- psql -U postgres -h postgres < ./create-database.sql
script:
- npm install
- npm run tests
only:
- master
Am I missing something - is the database maybe not created and I am calling to soon?
You're missing the configuration properties of the DB. They can not be configured on the UI - or they can but won't be passed to the actual DB service.
So please add the following properties to the script:
variables:
POSTGRES_DB: <db name>
POSTGRES_USER: <db user>
POSTGRES_PASSWORD: <some PW>
POSTGRES_HOST_AUTH_METHOD: trust
Documentation link: https://docs.gitlab.com/ee/ci/services/postgres.html
Related
I have been spending 3-4 hours on this and still have not found a solution.
I can successfully run the docker container and use psql from the container bash, however, when I try to call the db from my local machine I continue to get this error message:
error role "postgres" does not exist
I have already tried editing "listen_addresses" in the postgresql.conf file from the container bash
My setup:
I am using a macbook - Monterey 12.4
my docker compose file:
version: '3.4'
services:
postgres:
image: postgres:latest
ports:
- "5432:5432"
environment:
- POSTGRES_DB=postgres_db
- POSTGRES_USER=testUser
- POSTGRES_PASSWORD=testPW
volumes:
- postgres-data:/var/lib/postgresql/db
but this issue occurs if I do it through the standard CLI command as well, i.e:
docker run -d -p 5432:5432 --name my-postgres -e POSTGRES_PASSWORD=mysecretpassword postgres
I tried to follow this tutorial but it didnt work:
[https://betterprogramming.pub/connect-from-local-machine-to-postgresql-docker-container-f785f00461a7][1]
when I try this command:
psql -h localhost -p 5432 -U postgres -W
it doesnt work:
psql: error: connection to server at "localhost" (::1), port 5432 failed: FATAL: role "postgres" does not exist
Also for reference, the user "postgres" does exist in postgres - as a superuser
Replace POSTGRES_USER=testUser with POSTGRES_USER=postgres in the compose configuration. Also use the password defined in POSTGRES_PASSWORD. Delete the old container and create a new one.
Thank you all for your help on this.
It turns out the issue was that I was running postgres on my local machine as well.
so once I turn that off I was able to connect.
I appreciate your time!
I'm running a docker container with the vanilla Postgres image on my local machine. I'd like to connect to the database from my local machine (i.e., not from "within the container". However, on trying to connect, I get an error.
Here's my docker-compose.yml file:
version: "3.8"
services:
db:
image: postgres
restart: always
ports:
- 5432:5432
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: mypassword
Here's how I start up:
docker-compose run db
Here's how I connect:
psql -h localhost -p 5432 -U postgres
This produces the error:
could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?
If I spin up the database without Docker Compose, the same connection command works as expected:
docker run --name mypg -p 5432:5432 -e POSTGRES_PASSWORD=password postgres
I could just go with the flow and use the command above. But this seems to be pointing to a flaw in how I think about Docker/Compose. For example, maybe Docker Compose's internal DNS resolver makes this approach fail.
Any ideas?
Version info:
psql --version
psql (PostgreSQL) 13.3
I have read through several SO posts, including these, but they don't address or fix the problem I'm seeing:
docker-compose: accessing postgres' shell (psql)
Can't connect to postgres when using docker-compose
Try docker-compose up db instead of run. Using run will run a one-off command against your container, whereas up will turn on the container and leave it running, so another application should be able to access it.
https://docs.docker.com/compose/faq/#whats-the-difference-between-up-run-and-start
Error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user "username"
Hello, I am trying to run a program locally using Docker and am getting the error in the title, even though it was working before.
I've tried reinstalling Docker, re-cloning the repo, reinstalling PostgresSQL (the problem started when I installed it for the first time). From reading similar questions, I ensured that the password matches. The password is 'password' for the Docker Postgres Database and I've tried changing it but it still hasn't worked.
I'm using 'docker-compose up -d' and then running tests but I get the error in the title. I've tried running 'docker-compose down' and then redoing it, but I still get the error.
.env file:
FLASK_ENV=development
REDIS_URL=redis://localhost:6379
DATABASE_URL=postgresql+psycopg2://username:password#localhost:5432/programname
PROGRAM_API_APP_NAME=test-prod.compute.random.com
docker-compose.yml:
version: '3'
services:
postgresql:
image: postgres:10-alpine
environment:
- POSTGRES_DB=programname
- POSTGRES_PASSWORD=password
- POSTGRES_USER=username
ports:
- "5432"
redis:
image: redis:5.0.3
ports:
- "6379:6379"
(I don't have the port as 5432:5432 because it didn't work with that and I found an answer to remove the second 5432.)
Boss helped me with this. Apparently, when I installed Postgres, it created a postgres user that had a process that was using port 5432 and even when we killed it, it automatically restarted. To solve this specific problem, we changed the docker-compose file to use port 5433 locally and 5432 in the container. Still have to find out how to get rid of the postgres user.
I wanted to build my springboot project. Then I want to dockerize my code. But when I built, I got error. I think this occured caused by postgresql setting. But I could not find reason.
Could you please help me?
docker-compose.yml file;
version: '2'
services:
web:
build: .
ports:
- 8080:8080
db:
container_name: productdb
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=bright
- POSTGRES_USER=postgres
- POSTGRES_DB=productdb
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- "5432:5432"
volumes:
productdb: {}
application.yml file;
server:
port: 8761
eureka:
client:
registerWithEureka: false
fetchRegistry: false
server:
enableSelfPreservation: false
waitTimeInMsWhenSyncEmpty: 0
spring:
application:
name: product-service
datasource:
url: jdbc:postgresql://db:5432/productdb
username: postgres
password: xxxx
initialization-mode: always
jpa:
show-sql: true
hibernate:
ddl-auto:
properties:
hibernate:
temp:
use_jdbc_metadata_defaults: false
Error looks like;
org.postgresql.util.PSQLException: The connection attempt failed.
Thank you
If your docker-compose.yml file is well configured, it should be start two containers:
docker ps
source: https://intelligentbee.com/2017/09/18/setup-docker-symfony-project/
One for app and one for db.
These containers are in the same host, so if your web need to connect to the database, you must the ip instead : localhost, 127.0.0.1 or 0.0.0.0
You cat get the ip with this
hostname -I| awk '{printf $1}'
If your web and your database would be in different host, you can use the public ip where is hosted the database. But as you are using docker-compose this is not the case.
I suggest you to test if your database is ready and available, before using it in your web app.
In order to test your database , You can following one of these approaches:
Check db status with telnet
There are several way , but the easiest option is the telnet command. For instance, in order to test if mysql container is ready to use in the same machine where was started:
telnet localhost 3306
If your mysql is ready, telnet must show you a result like the following picture:
Any other negative result, would indicate that your mysql container is exited or wrong.
Note:Change 3306 for the correct postgress port
Check db status with Database IDE
Other option for UI users is testing the database connection using some Database IDE. Just download one of the several postgress client IDEs and testing your database.
Don't hardcode parameters
It is a good practice to externalize configuration using environment variables. Spring and docker know and allow us to use them.
So, modify your application.yml :
From
datasource:
url: jdbc:postgresql://db:5432/productdb
To
datasource:
url: jdbc:postgresql://${DATABASE_HOST}:5432/productdb
For development, in your eclipse use run as configurations >> environment section
For production you can:
export variable before run
pass it to your docker run sentence...
docker run -d \
--name my_funny_api \
-p 8080:8080 \
-e "DATABASE_HOST=10.10.01.52" \
-i -t my_funny_api_image
or
export HOST_IP=$(hostname -I| awk '{printf $1}')
docker run -d \
--name my_funny_api \
-p 8080:8080 \
-e "DATABASE_HOST=${DATABASE_HOST}" \
-i -t my_funny_api_image
Finally to avoid manually task to manage your variables, you can use : http://github.com/jrichardsz/tachikoma-ops
Using DataGrip software and DB in DigitalOcean. Got error
[08001] The connection attempt failed. java.net.SocketTimeoutException: connect timed out.
Made sure my current IP was one of the allowed inbound connections and that worked. (Even though the error should probably have been different.)
Hope this is useful to someone eventually.
Your DB should accept connections outside of the container
sudo docker run --name pg -p 5432:5432 -v pg_data:/var/lib/postgres/data -e POSTGRES_DB=mydb -e POSTGRES_USER=pg_user -e POSTGRES_PASSWORD=pg_password -d postgres -c "listen_addresses=*"
"listen_addresses=" It will accept connection outside of the container*
You can use follow credential to connect your spring boot project
db_user=pg_user
db_password=pg_password
db_url=jdbc:postgresql://localhost:5432/mydb
I started a postgresql server in docker and exposed the 5432 port by sudo docker run -it -p 5432:5432 9c421f1a239c bash and start the postgres server manually inside the docker container, but cannot connect to it with command: psql -h 172.17.0.63 -U venti. 172.17.0.63 is a right IP, and venti is my pg username. But get error:
psql: could not connect to server: Connection refused
Is the server running on host "172.17.0.63" and accepting
TCP/IP connections on port 5432?
My pg_hba.conf looks like this:
local all postgres peer
host all all 0.0.0.0/0 trust
local all all trust
Connecting to pg server inside container works successfully.
Dockerfile:
FROM ubuntu:12.04
RUN apt-get update
RUN apt-get install -y gcc libc-dev-bin libc6 libc6-dev libssl-dev libkrb5-dev comerr-dev
RUN apt-get install -y postgresql-common libpq-dev postgresql-9.1-postgis --fix-missing
RUN apt-get install -y postgresql postgresql-client
USER postgres
ENV PGDATA /etc/postgresql/9.1/main
ENV LOGDIR /etc/postgresql/9.1/main/postgresql.log
WORKDIR /usr/lib/postgresql/9.1/bin
USER root
RUN apt-get install -y vim
USER postgres
RUN sed -e '90d' -i /etc/postgresql/9.1/main/pg_hba.conf
RUN sed -e '91d' -i /etc/postgresql/9.1/main/pg_hba.conf
RUN echo "host all all 0.0.0.0/0 trust" >> '/etc/postgresql/9.1/main/pg_hba.conf'
RUN echo "local all all trust" >> '/etc/postgresql/9.1/main/pg_hba.conf'
RUN ./pg_ctl start && sleep 8 && ./createdb pg && ./createdb bloodstone \
&& createuser -Upostgres -s venti \
&& createdb -Uventi -Oventi venti
# ENTRYPOINT ./pg_ctl start && bash -c "while true; do echo "" > /dev/null; sleep 1; done"
VOLUME $PGDATA
EXPOSE 5432
It's merely a misconfiguration. I should have set pg to listen on public addresses or make a port mapping. I fixed this by editing pg config file with sed:
RUN sed -e "s/[#]\?listen_addresses = .*/listen_addresses = '*'/g" -i '/etc/postgresql/9.1/main/postgresql.conf'
Add this to a proper place in your Dockerfile, and you should be OK.
To troubleshoot postgres auth problems in general, look at the postgres log/stderr to see verbose reasons why it failed. (I see your problem is solved but if anyone runs into similar problems)
To find the postgres log location: "show log_destination;" if you have a working psql (eg locally on the postgres server box)
I see yours is set to "/etc/postgresql/9.1/main/postgresql.log". You can connect to the container to see the log with "docker exec -it container_name bash".
Alternately, if a container runs postgres directly, "docker attach db_container_name" views postgres stderr messages.
Note that by default user postgres does not have a password and uses only ident auth.
I also faced similar issue. Try running docker-compose up twice.
The reason is, that in my docker-compose.yml file the DB service was below the Web service in which it was trying to connect to DB which does not exists yet.
**
version: '2'
services:
nginx:
image: nginx:latest
container_name: ng01
ports:
- "8000:8000"
volumes:
- ./src:/src
- ./config/nginx:/etc/nginx/conf.d
- /static:/static <--- HERE
depends_on:
- web
web:
build: .
container_name: dg01
command: bash -c "python /src/IMS/manage.py makemigrations && python /src/IMS/manage.py migrate && gunicorn IMS.wsgi -b
0.0.0.0:8000"
depends_on:
- db
volumes:
- ./src:/src
- /static:/static
expose:
- "8000"
db:
image: postgres:latest
container_name: ps01**