I started a process on the host that listens on a Unix socket:
/cloud_sql_proxy -enable_iam_login -dir=/var/run/cloudsql -instances=project:region:server
I confirmed I can make a connection with psql: psql "sslmode=disable host=/var/run/cloudsql/project:region:server user=myuser#project.iam dbname=mydb
I need to connect to Postgres over this socket with npgsql but inside a container.
I'm using this connection string:
string DBConnectionString = #"
User ID=myuser#project.iam;
Host=/var/run/cloudsql/project:region:server;
Database=mydb;
Port=5432
";
using (var connection = new NpgsqlConnection(DBConnectionString))
connection.Query("SELECT * FROM mytable ORDER BY zzz");
Running this application locally on the host with this connection string works as expected- it can connect to the Unix socket and query the DB without issue. From inside a container it is not working.
I start the container trying to mount the file for the Unix socket:
docker run \
--mount type=bind,source="/var/run/cloudsql",target="/var/run/cloudsql" \
"myimage"
But I'm getting this error:
[00:13:46 FTL] Npgsql.NpgsqlException (0x80004005): Exception while connecting
---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (111): Connection refused /var/run/cloudsql/project:region:server/.s.PGSQL.5432
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
When I look at the process using the socket (cloud SQL proxy) I see no output during the attempt. When to connect to the socket from the host with the app or psql I see logs for login attempts. So maybe the socket isn't mounted correctly?
Update: also works fine with k8s
Just to add to this, this also works in k8s. In my manifest I just create a shared volume. I have a container running the app and another container running the cloud SQL proxy and mount the volume to both and it just works. So I'm wondering if this is some local docker perms issue or something?
Related
I'm running a simple Docker Compose script to instantiate Postgres and pgAdmin for local development.
It works fine on Windows via Power Shell as administrator, but in Linux I'm getting this error:
Unable to connect to server connection to server at '172.18.0.3' port 5432 failed: timeout expired
How to solve it?
This question already has answers here:
From inside of a Docker container, how do I connect to the localhost of the machine?
(40 answers)
Closed 1 year ago.
I pulled pgAdmin4 docker image into my linux debian machine and followed the process specified here to configure the container. I run docker run -p 8000:8000 --env-file ./pgadmin_docker_env.list -d dpage/pgadmin4. For clarity, the pgadmin_docker_env.list specified in the command contains the environmental variables:: PGADMIN_DEFAULT_EMAIL=my_email#example.com PGADMIN_DEFAULT_PASSWORD=my_password. With the container running in detached mode, I run localhost:8080 in my web browser to access pgAdmin 4 in server mode. However, I was unable to create a server connection to the localhost postgres database from inside the pgadmin. I got the following error after input of the connection parameters (shown in the screenshot attached below)
Unable to connect to server: connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections?
UPDATE
I used host.docker.internal in place of localhost but I still got an error
Unable to connect to server: could not translate host name "host.docker.internal" to address: Name does not resolve
You can skip a step if you've already done it
Using psql, alter the authentication credential of default postgres user, postgres with the following commands
sudo -u postgres psql
ALTER USER postgres PASSWORD 'newPassword';
Optionally, you can also create a user for your current account as a superuser with CREATE ROLE user_name WITH LOGIN SUPERUSER CREATEDB CREATEROLE REPLICATION;
Modify /etc/postgresql/13/main/pg_hba.conf and add
host all all 0.0.0.0/0 md5 to the end of the file
Modify the pgadmin_docker_env.list file to include your choice port
PGADMIN_LISTEN_PORT=8000
Stop the previously running container pgadmin docker stop pgadmin and remove the containerdocker rm pgadmin. Then run docker run --env-file ./pgadmin_docker_env.list --network="host" --name pgadmin dpage/pgadmin4 to run the container in host network mode. See more on host network mode
Run localhost:8000 in your web browser and create a server connection using the same connection parameter as in the screenshot.
localhost in this scenario refers to the PgAdmin container, where there is not a Postgres instance running.
You want to connect to Postgres running on the host machine from the container (from what I can tell anyway?) so use host.docker.internal instead of localhost.
To summarize, I am trying to run a Chainlink node via Docker on an Azure VM. I also created an Azure Postgresql DB and verified the VM is able to connect via psql cli.
Steps I took to get the node running (following this link):
Create Azure VM
Install docker
mkdir ~/.chainlink-rinkeby
Created .env file
Set ETH_URL via an External Provider
Create Postgres SQL Database following this link
Set Remote Database_Url config using sslmode=disable
Start the node with:
cd ~/.chainlink-rinkeby && docker run -p 6688:6688 -v ~/.chainlink-rinkeby:/chainlink -it --env-file=.env smartcontract/chainlink local n
My .env file:
"ROOT=/chainlink LOG_LEVEL=debug ETH_CHAIN_ID=4 MIN_OUTGOING_CONFIRMATIONS=2 LINK_CONTRACT_ADDRESS=0x01BE23585060835E02B77ef475b0Cc51aA1e0709 CHAINLINK_TLS_PORT=0 SECURE_COOKIES=false GAS_UPDATER_ENABLED=true ALLOW_ORIGINS=*"
"ETH_URL=wss://cl-rinkeby.fiews.io/v1/MY_API_KEY"
"DATABASE_URL=postgresql://MY_USER_NAME:MY_PASSWORD#MY_DATABASE_nAME.postgres.database.azure.com:5432/postgres?sslmode=disable"
Error:
[ERROR] unable to lock ORM: dial tcp 127.0.0.1:5432: connect: connection refused logger/default.go:139 stacktrace=github.com/smartcontractkit/chainlink/core/logger.Error
I've also tried giving a version of 0.10.8 in the chainlink startup command but the error I get for that is:
[ERROR] failed to initialize database, got error failed to connect to `host=/tmp user=root database=`: dial error
You are trying to connect your Chainlink node to a remote PostgreSQL database. This database is managed by AZUR's cloud service and hosted and administered there internally. The issue with the connection is that the Chainlink node wants to establish a connection to 127.0.0.1 and is therefore convinced that the Postgres is located locally in your Chainlink docker container.
A docker container is an own environment and represent an own host, so 127.0.0.1 will loopback into the container itself. I recommend you to have a look here on the official network documentation of docker: https://docs.docker.com/config/containers/container-networking/
With the version 0.10.8 you established a connection. The issue here is now related to the USER and the DATABASE. Please ensure that you create a database and an own USER for it and not use the admin credentials (root) like the superuser.
You can enter the postgres via the azur cli and type in the following lines:
CREATE DATABASE <yourdbname>;
CREATE USER <youruser> WITH ENCRYPTED PASSWORD '<yourpass>';
In addition you can have a look at this post related to the connection to your postgres database:
https://stackoverflow.com/a/67992586/15859805
I installed the Metabase application, created a Docker container (on one Linux host) and I want to connect to a remote Postgres database (on a second Linux host) from the Docker container. The setup of Metabase allows entry of connection parameters to the Postgres database. Each time I enter the proper connection parameters in Metabase I get a database connection error. I can connect to the Postgres database from the host running the container using PSQL with no problem. My question is, is there something within Docker or the remote Postgres server that I must setup in order to allow a connection between a Docker container and a remote database? I realize this is normally caused by incorrect host/port information, problems with DNS, firewall blocking or other network problems. I am new to Docker and I do not know where to look or what to setup to make this work. Does anyone have suggestions about how to make this connection work? Thank you!
I have a PostgreSQL DB sitting on my local machine (Windows) and I would like to import it into my Hortonworks Sandbox using Apache Sqoop. While something like this sounds great, the complicating factor is that my Sandbox is sitting in a Docker container, so statements such as sqoop list-tables --connect jdbc:postgresql://127.0.0.1/ambari --username ambari -P seem to run into authentication errors. I believe the issue comes from trying to connect to the local host from inside the docker container.
I looked at this post on connecting to a MySQL DB from within a container and this one to try to use PostgreSQL instead, but have so far been unsuccessful. I have tried connecting to '127.0.0.1' and '172.17.0.1' (the host's IP) in order to connect to my local host from within Docker. I have also adjusted PostgreSQL's configuration file to listen for connections on all IP addresses. However, I still get the following error messages when I run sqoop list-tables --connect jdbc:postgresql://<ip>:5432/<db_name> --username postgres -P (where <ip> is either 127.0.0.1 or 172.17.0.1, and <db_name> is the name of my database)
For connecting with 127.0.0.1:
ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: org.postgresql.util.PSQLException: FATAL: Ident authentication failed for user "postgres"
For connecting with 172.17.0.1:
Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Any suggestions would be very helpful!
If this is just for local testing and not for production level coding, you can enable all trusted connections to your database by updating the pg_hba.conf file
Locate your pg_hba.conf file inside your postgres data directory
Vim the file and update it with the following lines:
#TYPE DATABASE USER ADDRESS METHOD
local all all trust
host all all 0.0.0.0/0 trust
host all all ::1/128 trust
Restart your postgres service
If you do this, your first use case (using 127.0.0.1) should work