I'm unable to connect an AWS SAM lambda function to a local postgres instance. I'll be specific in a moment, but first let me say that the same code works for my coworker and that I've tried using Postgres within a docker container as well as installed manually. To simplify the problem I started over with the SAM helloworld golang example and added the following:
import (
_ "github.com/lib/pq"
"database/sql"
)
.
.
.
const (
host = "localhost"
port = 5432
user = "postgres"
password = "postgres"
dbname = "postgres"
)
.
.
.
.
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=disable",
host, port, user, password, dbname)
db, err := sql.Open("postgres", psqlInfo)
if err != nil {
panic(err)
}
defer db.Close()
err = db.Ping()
if err != nil {
panic(err)
}
fmt.Println("Successfully connected!")
And I get the following error:
dial tcp 127.0.0.1:5432: connect: connection refused: OpError
[{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/function.go","line":35,"label":"(*Function).Invoke.func1"},{"path":"runtime/panic.go","line":969,"label":"gopanic"},{"path":"hello-world/main.go","line":67,"label":"handler"},{"path":"reflect/value.go","line":475,"label":"Value.call"},{"path":"reflect/value.go","line":336,"label":"Value.Call"},{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/handler.go","line":124,"label":"NewHandler.func1"},{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/handler.go","line":24,"label":"lambdaHandler.Invoke"},{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/function.go","line":67,"label":"(*Function).Invoke"},{"path":"reflect/value.go","line":475,"label":"V
alue.call"},{"path":"reflect/value.go","line":336,"label":"Value.Call"},{"path":"net/rpc/server.go","line":377,"label":"(*service).call"},{"path":"runtime/asm_amd64.s","line":1374,"label":"goexit"}]
SAM CLI, version 1.21.1
I've now tried this on two machines and get the same error. Is there there something that I'm just missing there? I can confirm that the the database is receiving connections fine. I hit it every day from a node app. Also, I can build the golang code outside of SAM and it will work fine. So there's something with my SAM setup that's wrong.
Figured it out. Within the lambda I needed to set the postgres connection string to hit host.docker.internal
I spend a lot of time figuring out how to connect using AWS SAM to a local Postgres and finally found that changing the dbhostname from localhost to host.docker.internal works.
connectionString = 'postgresql://dbuser:dbpassword#host.docker.internal:5432/dbname';
Related
I'm new to GoLang and I`ve an issue with connecting my Go webserver with Postgres Database.
Can someone tell me what I'm doing wrong here?
All these credentials are correct btw. User exists, Password is correct, DB exists and belongs to the user.
package app
import (
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
)
func connectDB(){
db, err := gorm.Open("postgres", "host=localhost port=5432 user=power_user dbname=local_db password=power_user")
if err != nil {
log.Fatal("DB Connection failed")
}
}
If you're sure about psql server is running, try to modify pg_hba.conf, locates at /etc/postgresql/${version}/main/pg_hba.conf usually:
# TYPE DATABASE USER ADDRESS METHOD
local all all trust
I tried to connect to Google CloudSQL PostgresSQL using Gorm golang and seems like it is not working.
Here's the code
func InitDB() *gorm.DB {
psqlInfo := fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=disable", os.Getenv("DB_HOST"), os.Getenv("DB_PORT"), os.Getenv("DB_USER"), os.Getenv("DB_PASSWORD"), os.Getenv("DB_NAME"))
db, err := gorm.Open("postgres", psqlInfo)
if err != nil {
fmt.Println("Failed to connect to the Database")
}
fmt.Println("Connected to the Database")
DB = db
return DB
}
If im using the localhost config everything works fine. See my .env file for cloudSQL config
DB_HOST=trader-234509:us-central1:testv1
DB_PORT=5432
DB_USER=username
DB_NAME=testv1
DB_PASSWORD=
Error is saying
dial tcp: lookup trader-234509:us-central1:testv1: no such host
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x164681d]
My local config (This one works fine)
DB_HOST=localhost
DB_PORT=5432
DB_USER=username
DB_NAME=test
DB_PASSWORD=
Did i do anything wrong?
Here's roughly how I connect from AppEngine:
import (
_ "github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/postgres"
"fmt"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func New() (*gorm.DB, error) {
user := "theuser"
password := "apassword"
dbHost := "my-project:the-region:my-db-instance-name"
databaseName := "mydbname"
connString := fmt.Sprintf("host=%s user=%s dbname=%s sslmode=disable password=%s", dbHost, user, databaseName, password)
return gorm.Open(postgres.New(postgres.Config{
DriverName: "cloudsqlpostgres",
DSN: connString,
}))
}
Cloud SQL doesn't support direct connection to instance name for 3rd party application yet, for details: https://cloud.google.com/sql/docs/mysql/connect-external-app
Based on my experience, there are 2 solutions:
As per above instruction, you can setup a Cloud Proxy following the steps and the connections flow would be: Golang app -> Cloud Proxy -> Cloud SQL
This approach is flexible and you're able to control the connection using firewall.
But you have to spend extra $$ to maintain the server and this cloud proxy instance, I've also heard that the cloud proxy may have some performance issues but I don't have exact evidence so far
Assign a Private IP to the Cloud SQL instance, refer to https://cloud.google.com/sql/docs/mysql/private-ip, so that you application can access the DB directly using this IP.
The trade-off is obvious that the app must be in the same network of the project, but this is a much more convenient approach, esp. all your applications are hosted in the same network
I didn't try out the public IP access approach as Cloud Proxy is just what I need if remote network connection is needed
In short, you need to deploy your code in a VM or other Google Managed service using option 2, or setup the cloud proxy to support your local debugging for option 1
Hope it helps
For a project i'm trying to connection a cloud function to a cloud sql database setup as described in this quickstart guide.
The function is configured in the same region, the service account has the Role Cloud SQL-Client. I called the function through my computer like this:
gcloud functions call <function-name> --region=<region> --data '{"recipient":"hello","requester":"hello","message":"test"}'
The connection to the function is working, it seems like just the authentication to the database doesn't work but i don't get where i failed.
I checked the password, user and connection name multiple times, reset the password and it still doesn't work.
I found the issue here related to connecting cloud functions to cloud sql.
I tried surrounding the password in the dsn-string with single-quotes just to be sure escaping of characters in the password isn't a problem.
I also checked the environment variables coming in and they are the ones i entered in the configuration.
The function just pings the database for test purposes:
package receiver
import (
"database/sql"
"fmt"
"net/http"
"os"
// Import Postgres SQL driver
_ "github.com/lib/pq"
)
// Receives a message and stores it
func Receive(w http.ResponseWriter, r *http.Request) {
connectionName := os.Getenv("POSTGRES_INSTANCE_CONNECTION_NAME")
dbUser := os.Getenv("POSTGRES_USER")
dbPassword := os.Getenv("POSTGRES_PASSWORD")
dsn := fmt.Sprintf("user=%s password='%s' host=/cloudsql/%s dbname=messages", dbUser, dbPassword, connectionName)
var err error
db, err := sql.Open("postgres", dsn)
if err != nil {
fmt.Fprintf(w, "Could not open db: %v \n", err)
}
// Only allow 1 connection to the database to avoid overloading
db.SetMaxIdleConns(1)
db.SetMaxOpenConns(1)
defer db.Close()
if pingerror := db.Ping(); pingerror != nil {
fmt.Fprintf(w, "Failed to ping database: %s \n", pingerror)
return
}
}
The variable POSTGRES_INSTANCE_CONNECTION_NAME is formatted as described here as ProjectID:Region:InstanceID.
Expected is a success message or no error and i'm actually getting this message:
pq: password authentication failed for user "postgres"
Note: I also created a function containing the demo code from here with my sql database settings and the error is the same. It seems like i missed some step while setting up the user or sql instance. But i can't find out which.
Feels strange to answer my own question but here it is: For some reason connecting with the postgres user doesn't work. Finally i created a new database user for the function and a password containing only alphanumeric characters.
The unix socket at /cloudsql/{connectionName} is only provided in the GCF runtime. When running locally, you either need change your connection string or use the Cloud SQL proxy to simulate a unix socket at the same path.
I'm trying to connect to my localhost postgresql server without SSL and I'm getting this error:
pq: SSL is not enabled on the server
That's fine, I know how to fix it:
type App struct {
Router *mux.Router
DB *sql.DB
}
func (a *App) Initialize(dbname string) {
connectionString := fmt.Sprintf("dbname=%s sslmode=disable", dbname)
var err error
a.DB, err = sql.Open("postgres", connectionString)
if err != nil {
log.Fatal(err)
}
defer a.DB.Close()
}
However I'm still getting the error!
I was able to recreate your error with a fresh install of postgres. While the error output was
pq: SSL is not enabled on the server
the real error was not having any databases created. To create a testdb let's run
createdb testdb
in your terminal with postgres already running in the background.
I think the answer comes more just to assign a parameter flag to your Postgres connection string url.
I was connecting to an AWS RDS by URL and I've just change to localhost on 5432 default port.
http://127.0.0.1:5432?sslmode=disable
or
http://localhost:5432?sslmode=disable
I stumbled upon this trying to connect to Postgresql server from my Go program with wrong password.
I spin up the docker container for postgres
docker run -i -t -v=":/var/lib/postgresql" -p 5432:5432 my_image/postgresql:9.3
And verify that it is reachable from host using
psql -h my_docker_ip -p 5432 -U pguser -W pgdb // passowrd: pguser
Now I want to connect to the container postgres using go in my host machine.
import (
"database/sql"
_ "github.com/lib/pq"
"fmt"
)
func main() {
db, err := sql.Open("postgres", "user=pguser password='pguser' host=192.168.99.100 port=5432 sslmode=verify-full")
if err != nil {
fmt.Println(err)
}
rows, err := db.Query("SELECT * FROM test")
fmt.Println(rows)
}
While there were no error on initializing the db instance reference, the test query itself print out
<nil>
This should not happen because I created table test and multiple rows in table test prior to running the go code.
Can someone tell me what I am doing wrong?
Thanks
I saw the same error when using mysql client in golang:
Failed to connect to database: x509: cannot validate certificate for 10.111.202.229 because it doesn't contain any IP SANs
The solution in https://stackoverflow.com/a/54636760/8645590 worked for me.