Gorm + Docker error while connection - postgresql

I trying connection to my postgresql database with Docker:
package main
import (
"fmt"
"log"
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
)
type Product struct {
gorm.Model
Code string
Price uint
}
var db *gorm.DB
func init() {
connection := fmt.Sprintf("host=db sslmode=disable user=dnz-dev password=dnz-dev")
db, err := gorm.Open("postgres", connection)
if err != nil {
log.Fatalln(err)
}
defer db.Close()
}
func main() {
// Migrate the schema
db.AutoMigrate(&Product{})
}
and docker-compose
version: "3.3"
services:
db:
build: ./dnz-db
container_name: dnz-database
ports:
- "6000:5432"
volumes:
- ./dnz-db/data:/var/lib/postgresql/data
environment:
- POSTGRES_USER=dnz-dev
- POSTGRES_PASSWORD=dnz-dev
api:
build: ./dnz-api
container_name: dnz-api
volumes:
- ./dnz-api:/go/src/app
ports:
- "5000:3000"
depends_on:
- db
I run docker-compose up --build and I get this error:
Attaching to dnz-database, dnz-api dnz-api | 2017/11/05 10:23:46 dial
tcp 172.21.0.2:5432: getsockopt: connection refused dnz-api | exit
status 1 dnz-api exited with code 1
What I doing wrong?

You are unable to connect to your psql container because you haven't linked it. Take a look at Docker-Compose documentation on links.
Also, I'm assuming you aren't scheduling your container startup order. Psql container must start before golang's one. Take a look at Docker Compose documentation on startup order. You can use wait-for-it to achieve such objective ast stated in the docs. Just wget it and save it in your project.
I don't know the contents of your Dockerfile but I'll assume it's something like:
FROM golang:1.9
RUN mkdir -p /go/src/github.com/pavel/gorm-psql
WORKDIR /go/src/github.com/pavel/gorm-psql
ADD . /go/src/github.com/pavel/gorm-psql
RUN go get -v
So, your docker-compose.yml should be edited to first run wait-for-it.sh and link psql container to your app with something like:
version: '3.3'
services:
db:
image: postgres
environment:
POSTGRES_DB: dnz-dev
POSTGRES_USER: dnz-dev
POSTGRES_PASSWORD: dnz-dev
ports:
- 6000:5432
api:
build: .
command: ["./wait-for-it.sh", "db:6000", "--", "go", "run", "main.go"]
volumes:
- .:/go/src/github.com/pavel/gorm-psql
ports:
- "5000:3080"
depends_on:
- db
links:
- db
If your main has another name just change it. Edit your volumes to point whatever path you need. I've set a standard one with /go/src/github.com/pavel/gorm-psql. I'm assuming you created a db named dnz-dev, if the name is different just edit it.
$ go env:
. . .
GOPATH="/home/pavel/go"
GOROOT="/usr/lib/go"
. . .
Just run docker-compose up and it should work just fine. I'm relying on postgresql and golang latest images.

Related

Golang project and postgres image with docker compose up doesn't fail, but doesn't work either

not sure what I did wrong here. I'm trying to make a golang/postgres dockerized project with a persistent db. Below are the files. When I run go run main.go then curl http://localhost:8081/ I get the expected output, but when I try this with docker compose up I'm having issues, but everything seems to be working because I don't see any error messages postgres-1 | 2022-08-29 05:31:59.703 UTC [1] LOG: database system is ready to accept connections, but when I try curl http://localhost:8081/ I'm getting an error curl: (56) Recv failure: Connection reset by peer. I tried removing the postgres part entirely and I'm still getting the same problem. I can see that docker is up and running and the port is listening
sudo lsof -i -P -n | grep 8081
docker-pr 592069 root 4u IPv4 1760430 0t0 TCP *:8081 (LISTEN)
I'm using this on Ubuntu 22.04
main.go:
package main
import (
"fmt"
"log"
"net/http"
)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "this can be anything")
})
http.HandleFunc("/try-it", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "it worked!")
})
log.Fatal(http.ListenAndServe(":8081", nil))
}
Dockerfile:
FROM golang:1.19-alpine3.16
WORKDIR /app
COPY go.mod ./
RUN go mod download
COPY . .
RUN go build -o main .
RUN go build -v -o /usr/local/bin/app ./...
EXPOSE 8081
CMD [ "./main" ]
docker-compose.yml:
version: '3.9'
services:
api:
restart: unless-stopped
build:
context: .
dockerfile: Dockerfile
ports:
- "8081:8080"
depends_on:
- postgres
networks:
- backend
postgres:
image: postgres
restart: unless-stopped
ports:
- "5001:5432"
volumes:
- psqlVolume:/var/lib/postgresql/data
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${POSTGRES_DB}
networks:
- backend
networks:
backend:
volumes:
psqlVolume:
As #DavidMaze and #Brits explained in the comments the docker container is up and running, but the ports need to be mapped in both the container and the application. For example in main.go this method http.ListenAndServe(":8084", nil) in the app would need to map with the container port (in this case :8084)
version: '3.9'
services:
api:
image: api
restart: unless-stopped
build:
context: .
dockerfile: Dockerfile
ports:
- "8083:8084" #host_port:container_port
a curl request could be made on the host port where docker is listening (in this case it's :8083). For example curl http://localhost:8083/. This would make a request to the host machine and that request would be captured by docker which is listening on port :8083 then transmit the request to the container which is listening on :8084 as specified in the docker-compose.yml. If the port mapping isn't correct then curl will return curl: (56) Recv failure: Connection reset by peer.Thank you for the learning experience and I appreciate all your help.

Spring Boot connection to Postgres fails when deploying with docker compose [duplicate]

I have a Java Spring Boot app which works with a Postgres database. I want to use Docker for both of them. I initially put just the Postgres in Docker, and I had a docker-compose.yml file defined like this:
version: '2'
services:
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
Then, when I issued the commands sudo dockerd and sudo docker-compose -f docker-compose.yml up, it was starting the database. I could connect using pgAdmin for example, by using localhost as server and port 5432. Then, in my Spring Boot app, inside the application.properties file I defined the following properties.
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
spring.datasource.username=sample
spring.datasource.password=sample
spring.jpa.generate-ddl=true
At this point I could run my Spring Boot app locally through Spring Suite, and it all was working fine. Then, I wanted to also add my Spring Boot app as Docker image. I first of all created a Dockerfile in my project directory, which looks like this:
FROM java:8
EXPOSE 8080
ADD /target/manager.jar manager.jar
ENTRYPOINT ["java","-jar","manager.jar"]
Then, I entered to the directory of the project issued mvn clean followed by mvn install. Next, issued docker build -f Dockerfile -t manager . followed by docker tag 9c6b1e3f1d5e myuser/manager:latest (the id is correct). Finally, I edited my existing docker-compose.yml file to look like this:
version: '2'
services:
web:
image: myuser/manager:latest
ports:
- 8080:8080
depends_on:
- db
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
But, now if I issue sudo docker-compose -f docker-compose.yml up command, the database again starts correctly, but I get errors and exit code 1 for the web app part. The problem is the connection string. I believe I have to change it to something else, but I don't know what it should be. I get the following error messages:
web_1 | 2017-06-27 22:11:54.418 ERROR 1 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : Unable to create initial connections of pool.
web_1 |
web_1 | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections
Any ideas?
Each container has its own network interface with its own localhost. So change how Java points to Postgres:
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
To:
spring.datasource.url=jdbc:postgresql://db:5432/sample
db will resolve to the proper Postgres IP.
Bonus. With docker-compose you don't need to build your image by hand. So change:
web:
image: myuser/manager:latest
To:
web:
build: .
I had the same problem and I lost some time to understand and solve this problem:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
I show all the properties so that everyone understands.
application.properties:
spring.datasource.url=jdbc:postgresql://localhost:5432/testdb
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL82Dialect
spring.jpa.hibernate.ddl-auto=update
docker-compose.yml:
version: "3"
services:
springapp:
build: .
container_name: springapp
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
ports:
- 8000:8080
restart: always
depends_on:
- db
db:
image: postgres
container_name: db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=testdb
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5000:5432
volumes:
- pgdata:/var/lib/postgresql/data
restart: always
volumes:
pgdata:
For start spring application with local database we use url localhost.
For connect to container with database we need change 'localhost' on your database service, in my case 'localhost' to 'db'.
Solution: add SPRING_DATASOURCE_URL environment in docker-compose.yml wich rewrite spring.datasource.url value for connect:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
I hope this helps someone save his time.
You can use this.
version: "2"
services:
sample_db-postgresql:
image: postgres:9.5
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
volumes:
- sample_db:/var/lib/postgresql/data
volumes:
sample_db:
You can use ENV variable to change the db address in your docker-compose.
Dockerfile:
FROM java:8
EXPOSE 8080
ENV POSTGRES localhost
ADD /target/manager.jar manager.jar
ENTRYPOINT exec java $JAVA_OPTS -jar manager.jar --spring.datasource.url=jdbc:postgresql://$POSTGRES:5432/sample
docker-compose:
`
container_name: springapp
environment:
- POSTGRES=db`

Can't connect to databases from devcontainer

I'm working on building an environment for an educational setting, but for some reason my dev container isn't able to connect to the databases that are being generated by the docker-compose.yml. See below:
# docker-compose.yml
version: '3.9'
services:
conda:
image: continuumio/anaconda3:latest
build:
context: ..
dockerfile: .devcontainer/Dockerfile
command: sleep infinity
volumes:
- ..:/workspace:cached
mongo:
image: mongo:latest
restart: unless-stopped
volumes:
- dbs:/data/db
postgres:
image: postgres:latest
restart: unless-stopped
volumes:
- dbs:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
volumes:
dbs:
When I attempt to connect to the PostgreSQL database using SQLTools on port 5432 on localhost, I get the following:
[1642491269885] ERROR (ls): Connecting error: {"code":-32001,"data":{"driver":"PostgreSQL","driverOptions":{}},"name":"Error"}
ns: "conn-manager"
[1642491269886] ERROR (ext): ERROR: Error opening connection connect ECONNREFUSED 127.0.0.1:5432, {"code":-32001,"data":{"driver":"PostgreSQL","driverOptions":{}}}
ns: "error-handler"
As a possibly important aside, I am also noticing that the PostgreSQL instance is constantly restarting. Here is what the docker container set looks like:
However, I should note that I can't connect to the mongodb instance either, using the Mongo for VS Code extension.
Let me know how I can make this work!

server and postgresql cannot connect when both running on Docker `getaddrinfo ENOTFOUND`

after creating docker containers with docker compose file (below), I call
$ docker run myApp
However, I get
Error: getaddrinfo ENOTFOUND main_db
this only happens when both server and postgresql are in docker containers (I am able to connect to postgresql on localhost)
I'm running a NestJS app using TypeOrm to connect to a postgresql server
inside the app.module.ts where it boots up the connection my config should match my docker postgresql config. the host points to the container I created on docker main_db and I declared this as a dependency of my server, the main service. Everything should be on the same network webnet.:
TypeOrmModule.forRoot({
type: 'postgres',
host: 'main_db',
port: +process.env.POSTGRES_PORT,
username: process.env.POSTGRES_USER,
password: process.env.POSTGRES_PASSWORD,
database: process.env.POSTGRES_DB,
autoLoadEntities: true,
synchronize: true,
logging: dbLogging,
}),
docker-compose.yml
version: '3.7'
services:
main:
container_name: main
build:
context: .
target: development
volumes:
- .:/usr/src/app
- /usr/src/app/node_modules
ports:
- ${SERVER_PORT}:${SERVER_PORT}
- 9229:9229
command: npm run start:dev
env_file:
- .env
networks:
- webnet
depends_on:
- main_db
main_db:
container_name: main_db
image: postgres:12
restart: always
networks:
- webnet
environment:
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
PG_DATA: /var/lib/postgresql/data
ports:
- '${POSTGRES_PORT}:${POSTGRES_PORT}'
volumes:
- pgdata:/var/lib/postgresql/data
networks:
webnet:
volumes:
pgdata:
.env file
POSTGRES_PORT=5432
POSTGRES_USER=test
POSTGRES_PASSWORD=test
POSTGRES_DB=test
SERVER_PORT=3001
When creating a Dockerfile and a docker-compose.yml file and you call docker run myApp for the app defined inside of the Dockerfile instead of calling docker-compose up, you will see the app running; however, it will not start the containers defined in the docker-compose file. In the case of the NestJS server, it was running the server, but could not find the container with the database, since this container was not being spun up. Although the distinction between the app setup in the Dockerfile and the definition of the containers in the docker-compose.yml was clear, I didn't realize that the docker command didn't reference the docker-compose.yml. Thus, posting here in case anyone else has a similar confusion.

Extend docker postgres image to create extra database

I've already seen this topic : https://stackoverflow.com/a/26599273/2323245
But I have the following problem :
postgres_1 | FATAL: role "docker" does not exist
app_1 | Error: Could not establish a connection with the database
This is my docker-compose.yml file
version: "2"
services:
app:
build:
context: .
dockerfile: Dockerfiles/app.dockerfile
links:
- postgres
depends_on:
- postgres
ports:
- "8080:8080"
environment:
PORT: 8080
networks:
- neo_dev
restart: always
postgres:
build:
context: .
dockerfile: Dockerfiles/postgres.dockerfile
networks:
- neo_dev
volumes:
- postgresql:/var/lib/postgresql
# This needs explicit mapping due to https://github.com/docker-library/postgres/blob/4e48e3228a30763913ece952c611e5e9b95c8759/Dockerfile.template#L52
- postgresql_data:/var/lib/postgresql/data
ports:
- "5430:5432" #in case you already have postgres running in your host machine
networks:
neo_dev:
driver: bridge
volumes:
postgresql:
postgresql_data:
My postgres.dockerfile is
FROM library/postgres
MAINTAINER Samir Bouaked "sbouaked#neocasesoftware.com"
ADD Dockerfiles/init.sql /docker-entrypoint-initdb.d/
And this is my init.sql file
CREATE USER docker with password 'docker';
CREATE DATABASE docker;
GRANT ALL PRIVILEGES ON DATABASE docker TO docker;
In my golang app, i'm trying to reach the database with
router.GET("/db", func(c *gin.Context) {
db, err := sql.Open("postgres", "host=postgres port=5432 user=docker dbname=docker password=docker sslmode=disable")
if err != nil {
log.Fatal("Error: The data source arguments are not valid - " + err.Error())
}
err = db.Ping()
if err != nil {
log.Println("Error: Could not establish a connection with the database")
c.String(http.StatusOK, "hotstName : %s\nDbStatus : %s", os.Getenv("HOSTNAME"), err)
} else {
c.String(http.StatusOK, "\nhotstName : %s\nDbStatus : connection ok !\n", os.Getenv("HOSTNAME"))
}
defer db.Close()
})
I tried different solutions like passing directly in the dockerfile env variables like that
ENV POSTGRES_USER docker
ENV POSTGRES_PASSWORD docker
ENV POSTGRES_DB docker
But have always the same result...
For more information you can check this repo https://github.com/neocase/docker-go-starter-kit
I don't understand the problem
It was a problem of docker volume.
I have to docker volume rm postgresvolume and do again a docker-compose up --build
It wasn't a problem of environment variables, but juste the fact that the init.sql was in cash and not run at the begining. It's due to some test I make a the start of the project
If you want to create the DB by passing ENV variables using docker-compose you can specify something like
postgres:
container_name: postgres
image: library/postgres
ports:
- "5432:5432"
environment:
- POSTGRES_DATABASE=docker
- POSTGRES_USER=docker
- POSTGRES_PASSWORD=docker
- POSTGRES_ADMIN_PASSWORD=docker
in your compose YML file.
This should create you a postgres container with an DB docker with user/pwd docker, no need for your own dockerfile anymore (so you can use any postgres image that supports this behavior)
After running compose (and if it doesn't work as you expect) you should have a look at the logs if your postgres container is working at all, or if there is an issue with your app.