How to connect to Google CloudSQL PostgresSQL - postgresql

I tried to connect to Google CloudSQL PostgresSQL using Gorm golang and seems like it is not working.
Here's the code
func InitDB() *gorm.DB {
psqlInfo := fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=disable", os.Getenv("DB_HOST"), os.Getenv("DB_PORT"), os.Getenv("DB_USER"), os.Getenv("DB_PASSWORD"), os.Getenv("DB_NAME"))
db, err := gorm.Open("postgres", psqlInfo)
if err != nil {
fmt.Println("Failed to connect to the Database")
}
fmt.Println("Connected to the Database")
DB = db
return DB
}
If im using the localhost config everything works fine. See my .env file for cloudSQL config
DB_HOST=trader-234509:us-central1:testv1
DB_PORT=5432
DB_USER=username
DB_NAME=testv1
DB_PASSWORD=
Error is saying
dial tcp: lookup trader-234509:us-central1:testv1: no such host
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x164681d]
My local config (This one works fine)
DB_HOST=localhost
DB_PORT=5432
DB_USER=username
DB_NAME=test
DB_PASSWORD=
Did i do anything wrong?

Here's roughly how I connect from AppEngine:
import (
_ "github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/postgres"
"fmt"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func New() (*gorm.DB, error) {
user := "theuser"
password := "apassword"
dbHost := "my-project:the-region:my-db-instance-name"
databaseName := "mydbname"
connString := fmt.Sprintf("host=%s user=%s dbname=%s sslmode=disable password=%s", dbHost, user, databaseName, password)
return gorm.Open(postgres.New(postgres.Config{
DriverName: "cloudsqlpostgres",
DSN: connString,
}))
}

Cloud SQL doesn't support direct connection to instance name for 3rd party application yet, for details: https://cloud.google.com/sql/docs/mysql/connect-external-app
Based on my experience, there are 2 solutions:
As per above instruction, you can setup a Cloud Proxy following the steps and the connections flow would be: Golang app -> Cloud Proxy -> Cloud SQL
This approach is flexible and you're able to control the connection using firewall.
But you have to spend extra $$ to maintain the server and this cloud proxy instance, I've also heard that the cloud proxy may have some performance issues but I don't have exact evidence so far
Assign a Private IP to the Cloud SQL instance, refer to https://cloud.google.com/sql/docs/mysql/private-ip, so that you application can access the DB directly using this IP.
The trade-off is obvious that the app must be in the same network of the project, but this is a much more convenient approach, esp. all your applications are hosted in the same network
I didn't try out the public IP access approach as Cloud Proxy is just what I need if remote network connection is needed
In short, you need to deploy your code in a VM or other Google Managed service using option 2, or setup the cloud proxy to support your local debugging for option 1
Hope it helps

Related

AWS SAM golang lambda localhost connection refused to postgres

I'm unable to connect an AWS SAM lambda function to a local postgres instance. I'll be specific in a moment, but first let me say that the same code works for my coworker and that I've tried using Postgres within a docker container as well as installed manually. To simplify the problem I started over with the SAM helloworld golang example and added the following:
import (
_ "github.com/lib/pq"
"database/sql"
)
.
.
.
const (
host = "localhost"
port = 5432
user = "postgres"
password = "postgres"
dbname = "postgres"
)
.
.
.
.
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=disable",
host, port, user, password, dbname)
db, err := sql.Open("postgres", psqlInfo)
if err != nil {
panic(err)
}
defer db.Close()
err = db.Ping()
if err != nil {
panic(err)
}
fmt.Println("Successfully connected!")
And I get the following error:
dial tcp 127.0.0.1:5432: connect: connection refused: OpError
[{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/function.go","line":35,"label":"(*Function).Invoke.func1"},{"path":"runtime/panic.go","line":969,"label":"gopanic"},{"path":"hello-world/main.go","line":67,"label":"handler"},{"path":"reflect/value.go","line":475,"label":"Value.call"},{"path":"reflect/value.go","line":336,"label":"Value.Call"},{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/handler.go","line":124,"label":"NewHandler.func1"},{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/handler.go","line":24,"label":"lambdaHandler.Invoke"},{"path":"github.com/aws/aws-lambda-go#v1.13.3/lambda/function.go","line":67,"label":"(*Function).Invoke"},{"path":"reflect/value.go","line":475,"label":"V
alue.call"},{"path":"reflect/value.go","line":336,"label":"Value.Call"},{"path":"net/rpc/server.go","line":377,"label":"(*service).call"},{"path":"runtime/asm_amd64.s","line":1374,"label":"goexit"}]
SAM CLI, version 1.21.1
I've now tried this on two machines and get the same error. Is there there something that I'm just missing there? I can confirm that the the database is receiving connections fine. I hit it every day from a node app. Also, I can build the golang code outside of SAM and it will work fine. So there's something with my SAM setup that's wrong.
Figured it out. Within the lambda I needed to set the postgres connection string to hit host.docker.internal
I spend a lot of time figuring out how to connect using AWS SAM to a local Postgres and finally found that changing the dbhostname from localhost to host.docker.internal works.
connectionString = 'postgresql://dbuser:dbpassword#host.docker.internal:5432/dbname';

Gorm can't connect to local postgres db

I'm new to GoLang and I`ve an issue with connecting my Go webserver with Postgres Database.
Can someone tell me what I'm doing wrong here?
All these credentials are correct btw. User exists, Password is correct, DB exists and belongs to the user.
package app
import (
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
)
func connectDB(){
db, err := gorm.Open("postgres", "host=localhost port=5432 user=power_user dbname=local_db password=power_user")
if err != nil {
log.Fatal("DB Connection failed")
}
}
If you're sure about psql server is running, try to modify pg_hba.conf, locates at /etc/postgresql/${version}/main/pg_hba.conf usually:
# TYPE DATABASE USER ADDRESS METHOD
local all all trust

How do you connect to MongoDB Atlas with the official mongo-go-driver [duplicate]

This question already has answers here:
How to use new URL from mongodb 3.6 to connect from golang
(2 answers)
Closed 3 years ago.
I'm looking at the tutorial offered in conjunction with the release of the official mongo-go-driver and the connection example uses a MongoDB server on localhost
// Set client options
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
However, the new hosted MongoDB service Atlas requires username and password to login. The connection string takes the format
mongodb://[username:password#]host1[/[database][?options]]
but there is no Golang example in the driver examples for Atlas.
So I'm wondering, what is the best way to log into Atlas without hard coding a password into a source file that will be posted on Github?
I am hosting my test Atlas cluster on AWS so I wanted to have similar credential management to the AWS process. From the AWS credentials page:
The default provider chain looks for credentials in the following order:
Environment variables.
Shared credentials file.
If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.
Therefore, I wanted to implement the environment veriable for my simple login to Atlas example. Code below assumes that the following line has been issued at the command line
export MONGO_PW='<your Atlas admin user password>'
Then the following program will verify your connection
package main
import (
"context"
"fmt"
"os"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
var username = "<username>"
var host1 = "<atlas host>" // of the form foo.mongodb.net
func main() {
ctx := context.TODO()
pw, ok := os.LookupEnv("MONGO_PW")
if !ok {
fmt.Println("error: unable to find MONGO_PW in the environment")
os.Exit(1)
}
mongoURI := fmt.Sprintf("mongodb+srv://%s:%s#%s", username, pw, host1)
fmt.Println("connection string is:", mongoURI)
// Set client options and connect
clientOptions := options.Client().ApplyURI(mongoURI)
client, err := mongo.Connect(ctx, clientOptions)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
err = client.Ping(ctx, nil)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println("Connected to MongoDB!")
}
From here the rest of the tutorial linked in my original question goes smoothly.

How to connect to Cloud-SQL from Cloud Function in Go?

For a project i'm trying to connection a cloud function to a cloud sql database setup as described in this quickstart guide.
The function is configured in the same region, the service account has the Role Cloud SQL-Client. I called the function through my computer like this:
gcloud functions call <function-name> --region=<region> --data '{"recipient":"hello","requester":"hello","message":"test"}'
The connection to the function is working, it seems like just the authentication to the database doesn't work but i don't get where i failed.
I checked the password, user and connection name multiple times, reset the password and it still doesn't work.
I found the issue here related to connecting cloud functions to cloud sql.
I tried surrounding the password in the dsn-string with single-quotes just to be sure escaping of characters in the password isn't a problem.
I also checked the environment variables coming in and they are the ones i entered in the configuration.
The function just pings the database for test purposes:
package receiver
import (
"database/sql"
"fmt"
"net/http"
"os"
// Import Postgres SQL driver
_ "github.com/lib/pq"
)
// Receives a message and stores it
func Receive(w http.ResponseWriter, r *http.Request) {
connectionName := os.Getenv("POSTGRES_INSTANCE_CONNECTION_NAME")
dbUser := os.Getenv("POSTGRES_USER")
dbPassword := os.Getenv("POSTGRES_PASSWORD")
dsn := fmt.Sprintf("user=%s password='%s' host=/cloudsql/%s dbname=messages", dbUser, dbPassword, connectionName)
var err error
db, err := sql.Open("postgres", dsn)
if err != nil {
fmt.Fprintf(w, "Could not open db: %v \n", err)
}
// Only allow 1 connection to the database to avoid overloading
db.SetMaxIdleConns(1)
db.SetMaxOpenConns(1)
defer db.Close()
if pingerror := db.Ping(); pingerror != nil {
fmt.Fprintf(w, "Failed to ping database: %s \n", pingerror)
return
}
}
The variable POSTGRES_INSTANCE_CONNECTION_NAME is formatted as described here as ProjectID:Region:InstanceID.
Expected is a success message or no error and i'm actually getting this message:
pq: password authentication failed for user "postgres"
Note: I also created a function containing the demo code from here with my sql database settings and the error is the same. It seems like i missed some step while setting up the user or sql instance. But i can't find out which.
Feels strange to answer my own question but here it is: For some reason connecting with the postgres user doesn't work. Finally i created a new database user for the function and a password containing only alphanumeric characters.
The unix socket at /cloudsql/{connectionName} is only provided in the GCF runtime. When running locally, you either need change your connection string or use the Cloud SQL proxy to simulate a unix socket at the same path.

Go postgres connection SSL not enabled

I'm trying to connect to my localhost postgresql server without SSL and I'm getting this error:
pq: SSL is not enabled on the server
That's fine, I know how to fix it:
type App struct {
Router *mux.Router
DB *sql.DB
}
func (a *App) Initialize(dbname string) {
connectionString := fmt.Sprintf("dbname=%s sslmode=disable", dbname)
var err error
a.DB, err = sql.Open("postgres", connectionString)
if err != nil {
log.Fatal(err)
}
defer a.DB.Close()
}
However I'm still getting the error!
I was able to recreate your error with a fresh install of postgres. While the error output was
pq: SSL is not enabled on the server
the real error was not having any databases created. To create a testdb let's run
createdb testdb
in your terminal with postgres already running in the background.
I think the answer comes more just to assign a parameter flag to your Postgres connection string url.
I was connecting to an AWS RDS by URL and I've just change to localhost on 5432 default port.
http://127.0.0.1:5432?sslmode=disable
or
http://localhost:5432?sslmode=disable
I stumbled upon this trying to connect to Postgresql server from my Go program with wrong password.