Objective
I'm trying to configure Hashicorp Terraform to utilize a PostgreSQL database as the state storage backend. I'm running Terraform version 1.3.3 on Ubuntu Studio 64-bit, and also have Docker Engine installed locally.
Repro Steps
Spin up Postgres with Docker docker run --detach --net=host --env POSTGRES_PASSWORD=xxxxx postgres
Exec into container and create database with psql: docker exec -it <containerID> psql --user=postgres
Configure Terraform backend, as shown below
Run terraform init command
terraform {
backend "pg" {
conn_str = "postgres://xxxxxxxxxx:xxxxxx#localhost/terraform_remote_state?sslmode=disable"
}
}
Actual Result
The pq library for Golang is throwing an error saying "Invalid Credentials."
I am able to use database clients, like psql or DBeaver Community Edition to connect to the database server.
Is the Postgres driver for Terraform simply broken? How can I get it to authenticate properly?
Related
I tried to connect my Postgres database which is running inside a Digitalocean droplet, with a Google Data studio (Looker studio).
Tried to use the Data studio connector. but I received an error message
Sorry, we encountered an error and were unable to complete your request.
Error ID: a69bfcec
Here's the command I used to create a Postgres container in the droplet :
docker run -d --name postgres -v postgres_data:/var/lib/postgresql/data -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -p 25060:5432 postgres:latest
I checked the container running with docker ps command, and it is running just fine.
I was also able to connect it the database from my terminal.
To summarize, I am trying to run a Chainlink node via Docker on an Azure VM. I also created an Azure Postgresql DB and verified the VM is able to connect via psql cli.
Steps I took to get the node running (following this link):
Create Azure VM
Install docker
mkdir ~/.chainlink-rinkeby
Created .env file
Set ETH_URL via an External Provider
Create Postgres SQL Database following this link
Set Remote Database_Url config using sslmode=disable
Start the node with:
cd ~/.chainlink-rinkeby && docker run -p 6688:6688 -v ~/.chainlink-rinkeby:/chainlink -it --env-file=.env smartcontract/chainlink local n
My .env file:
"ROOT=/chainlink LOG_LEVEL=debug ETH_CHAIN_ID=4 MIN_OUTGOING_CONFIRMATIONS=2 LINK_CONTRACT_ADDRESS=0x01BE23585060835E02B77ef475b0Cc51aA1e0709 CHAINLINK_TLS_PORT=0 SECURE_COOKIES=false GAS_UPDATER_ENABLED=true ALLOW_ORIGINS=*"
"ETH_URL=wss://cl-rinkeby.fiews.io/v1/MY_API_KEY"
"DATABASE_URL=postgresql://MY_USER_NAME:MY_PASSWORD#MY_DATABASE_nAME.postgres.database.azure.com:5432/postgres?sslmode=disable"
Error:
[ERROR] unable to lock ORM: dial tcp 127.0.0.1:5432: connect: connection refused logger/default.go:139 stacktrace=github.com/smartcontractkit/chainlink/core/logger.Error
I've also tried giving a version of 0.10.8 in the chainlink startup command but the error I get for that is:
[ERROR] failed to initialize database, got error failed to connect to `host=/tmp user=root database=`: dial error
You are trying to connect your Chainlink node to a remote PostgreSQL database. This database is managed by AZUR's cloud service and hosted and administered there internally. The issue with the connection is that the Chainlink node wants to establish a connection to 127.0.0.1 and is therefore convinced that the Postgres is located locally in your Chainlink docker container.
A docker container is an own environment and represent an own host, so 127.0.0.1 will loopback into the container itself. I recommend you to have a look here on the official network documentation of docker: https://docs.docker.com/config/containers/container-networking/
With the version 0.10.8 you established a connection. The issue here is now related to the USER and the DATABASE. Please ensure that you create a database and an own USER for it and not use the admin credentials (root) like the superuser.
You can enter the postgres via the azur cli and type in the following lines:
CREATE DATABASE <yourdbname>;
CREATE USER <youruser> WITH ENCRYPTED PASSWORD '<yourpass>';
In addition you can have a look at this post related to the connection to your postgres database:
https://stackoverflow.com/a/67992586/15859805
I have a Postgres server running inside Docker. Inside this Postgres server, I have a database named 'aa'.
I also have a Docker image of a Spring Boot Application. When this image is executed in Docker, database tables should be created inside database 'aa'.
In order to achieve this, I executed the following steps:
Run the Postgres Server inside Docker
docker run --name PostgresServer --e POSTGRES_PASSWORD=*** -d Postgres
Enter PostgresServer then Create database 'aa'
sudo docker exec -it PostgresServer psql -U postgres
CREATE DATABASE AA;
3. Run the Sprint Boot Docker Image (this is where the problem happens)
docker run -v /Users/juancesard.pineda/Desktop/brapi:/home/brapi/properties -d brapicoordinatorselby/brapi-java-server:v2
I checked the logs: It says database 'aa' does not exist wherein clearly it exists in the Postgres Server
org.postgresql.util.PSQLException: FATAL: database "aa" does not exist
Some additional info:
Docker listens at port 8080
Postgres server listens at port 5432
My application.properties file looks like this:
server.port = 8080
server.servlet.context-path=/Users/juancesard.pineda/Desktop/brapi/application.properties/germplasm
spring.datasource.url=jdbc:postgresql://host.docker.internal:5432/aa
spring.datasource.username=****
spring.datasource.password=****
spring.datasource.driver-class-name=org.postgresql.Driver
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.show-sql=false
spring.jpa.properties.hibernate.hbm2ddl.import_files=sql/crops.sql, sql/lists.sql, sql/locations.sql, sql/people.sql, sql/programs.sql, sql/trials.sql, sql/seasons.sql, sql/studies.sql, sql/breeding_methods.sql, sql/germplasm.sql, sql/attribute_defs.sql, sql/attribute_values.sql, sql/seed_lots.sql, sql/observation_units.sql, sql/crosses.sql, sql/pedigree.sql, sql/events.sql, sql/images.sql, sql/observation_variables.sql, sql/observations.sql, sql/samples.sql, sql/allele_calls.sql, sql/genome_maps.sql, sql/references.sql, sql/vendor.sql
spring.mvc.dispatch-options-request=true
Am I missing some config in my application.properties file?
Thank you in advance
The problem is in your application.properties file. The datasource url is pointing to host.docker.internal. This should only be used when connecting something from inside a docker container to something running outside docker on the host machine (documentation). You need a different solution because your Spring server and Postgres server are both running inside docker containers.
You need a Docker Network, specifically a bridge network, to connect your different containers. Docker Bridge Networks
Try something like this
Run this command to create a network in docker called “brapi_net”
$> docker network create --driver bridge network_name
Run the postgres container and link it to the network. Create your database schema as normal.
$> docker run --name PostgresServer --network=network_name --e POSTGRES_PASSWORD=*** -d Postgres
$> sudo docker exec -it PostgresServer psql -U postgres
$> CREATE DATABASE aa;
With a Docker network, container names act as server names. Modify your application.properties to reflect this
spring.datasource.url=jdbc:postgresql://PostgresServer:5432/aa
Run the server container and link it to the network.
$> docker run -v /Users/juancesard.pineda/Desktop/brapi:/home/brapi/properties --network=network_name -d brapicoordinatorselby/brapi-java-server:v2
I’m currently trying to run fossology in Gitlab CI. Fossology requires an external database that can be set up from a schema created using pg_dump. When I'm trying to use psql I get the title error.
At the moment, I have a script that sets up a container that runs the required version of postgres (9.6). It then tries to run an .sql script via psql in the postgres container via docker exec. Upon doing so it gets the title error.
I have tried specifying both a port and a host when issuing the psql statement, neither of which worked. I have tried using localhost, 127.0.0.1, the IP address of the postgres container and the name of the container as a host. I have tried rewriting things in different scripts, but nothing seems to work.
After extensive google searching, many people seem to have the same error message but not for the same reasons and not usually when using a docker container to host the database.
When I have run the contents of my script in the command line, i do not get this error, the script works fine and I can connect to Fossology. The issue only arises when trying to do the same in Gitlab CI.
The sequence of steps (i.e. pasted line by line) that works when using the command line on Mac:
# creates blank database and hosts it in a docker container
docker run -d --name fossdb -p 5432:5432 postgres:9.6
docker cp /fossology_db_schema.sql fossdb:/fossy.sql
docker exec -it fossdb bash
psql postgres -U postgres
# creates user needed for database to work with fossology
create user fossy with password 'fossy';
create database fossology;
grant all privileges on database fossology to fossy;
\q
# builds the fossology database in the hosted blank database
psql fossology < fossy.sql
psql postgres -U postgres
\connect fossology
exit
What I am attempting in GitLab CI:
# creates container with postgres image
docker run -d --name fossdb -p 5432:5432 --network foss-net postgres:9.6
# creates blank database (error occurs here)
docker exec fossdb psql -h $(docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' fossdb) -f ./createBlank.sql -U postgres
# builds fossology database from schema
docker exec fossdb psql -h $(docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' fossdb) fossology < ./schema.sql -U postgres
createBlank.sql:
create user fossy with password 'fossy';
create database fossology;
grant all privileges on database fossology to fossy;
Expected results: runs createBlank.sql to create a blank database called fossology, then builds fossology database from schema
Actual results: psql: could not connect to server: Connection refused
Is the server running on host "172.19.0.2" and accepting
TCP/IP connections on port 5432?
Are you sure you set up postgres completely?
A few quick checks you can perform:
(Excuse me, you DID do that. goto suggestion 2)
suggestion 1: Did you tell postgres there is a user with a password? (createuser command)
https://www.postgresql.org/docs/9.2/app-createuser.html
suggestion 2: Did you tell postgres that user can connect, and how? (tcp or local sockets)
https://www.postgresql.org/docs/9.2/auth-pg-hba-conf.html
Operating System :- windows 10
I am using Docker toolbox and Install MongoDB Image on my local system.
Image installation done successfully.
after that I am using command docker exec -it taxlien bash to connect with mongo db and it connected successfully but I am unable to connect through our local using MongoDB Compass Community and also through our web application.
Now Finally I got solution using cmd docker-machine ip default , I got IP address
after that I will execute cmd
docker run -p :27017 --name test -d mongo:3.6
then i will connect with my local application and MongoDB commuinty.
Try this docker-machine ip default, and replace localhost by the result
See How do I connect to a container hosted in Docker Toolbox?
And check this doc https://docs.docker.com/toolbox/toolbox_install_windows/#optional-add-shared-directories