For a project i'm trying to connection a cloud function to a cloud sql database setup as described in this quickstart guide.
The function is configured in the same region, the service account has the Role Cloud SQL-Client. I called the function through my computer like this:
gcloud functions call <function-name> --region=<region> --data '{"recipient":"hello","requester":"hello","message":"test"}'
The connection to the function is working, it seems like just the authentication to the database doesn't work but i don't get where i failed.
I checked the password, user and connection name multiple times, reset the password and it still doesn't work.
I found the issue here related to connecting cloud functions to cloud sql.
I tried surrounding the password in the dsn-string with single-quotes just to be sure escaping of characters in the password isn't a problem.
I also checked the environment variables coming in and they are the ones i entered in the configuration.
The function just pings the database for test purposes:
package receiver
import (
"database/sql"
"fmt"
"net/http"
"os"
// Import Postgres SQL driver
_ "github.com/lib/pq"
)
// Receives a message and stores it
func Receive(w http.ResponseWriter, r *http.Request) {
connectionName := os.Getenv("POSTGRES_INSTANCE_CONNECTION_NAME")
dbUser := os.Getenv("POSTGRES_USER")
dbPassword := os.Getenv("POSTGRES_PASSWORD")
dsn := fmt.Sprintf("user=%s password='%s' host=/cloudsql/%s dbname=messages", dbUser, dbPassword, connectionName)
var err error
db, err := sql.Open("postgres", dsn)
if err != nil {
fmt.Fprintf(w, "Could not open db: %v \n", err)
}
// Only allow 1 connection to the database to avoid overloading
db.SetMaxIdleConns(1)
db.SetMaxOpenConns(1)
defer db.Close()
if pingerror := db.Ping(); pingerror != nil {
fmt.Fprintf(w, "Failed to ping database: %s \n", pingerror)
return
}
}
The variable POSTGRES_INSTANCE_CONNECTION_NAME is formatted as described here as ProjectID:Region:InstanceID.
Expected is a success message or no error and i'm actually getting this message:
pq: password authentication failed for user "postgres"
Note: I also created a function containing the demo code from here with my sql database settings and the error is the same. It seems like i missed some step while setting up the user or sql instance. But i can't find out which.
Feels strange to answer my own question but here it is: For some reason connecting with the postgres user doesn't work. Finally i created a new database user for the function and a password containing only alphanumeric characters.
The unix socket at /cloudsql/{connectionName} is only provided in the GCF runtime. When running locally, you either need change your connection string or use the Cloud SQL proxy to simulate a unix socket at the same path.
Related
From code
The code bellow outputs the following:
2022/06/21 16:01:07 Failed to connect to db: failed to connect to 'host=localhost user=postgres database=local': server error (FATAL: Ident authentication failed for user "postgres" (SQLSTATE 28000)) exit status 1
import (
"context"
"log"
"github.com/jackc/pgx/v4"
)
func main() {
dbCtx := context.Background()
db, err := pgx.Connect(
dbCtx,
"postgres://postgres:smashthestate#localhost:5432/local",
)
if err != nil {
log.Fatalf("Failed to connect to db: %v\n", err)
}
defer db.Close(dbCtx)
// do stuff with db...
}
From terminal
However, connection to db is possible from terminal. For example, this command if run with the same parameters (db name, user name, password) will give correct output:
psql -d local -U postgres -W -c 'select * from interest;'
Notes
Command works correctly even if sent by user that is neither root nor postgres.
Authentication method for local connection is set to trust inside pg_hba.conf:
local all all trust
So, what am I missing here? Why everything works fine from the command line but doesn't work from code?
Go's defaults are different from psql's. If no hostname is given, Go defaults to using localhost, while psql defaults to using the Unix domain sockets.
To specify a Unix domain socket to Go, you need to do it oddly to get it to survive URI validation:
postgres://postgres:smashthestate#:5432/local?host=%2Ftmp
Though the end might need to be more like ?host=%2Fvar%2Frun%2Fpostgresql, depending on how the server was configured.
But why are you specifying a password which is not needed? That is going to cause pointless confusion.
I'm new to GoLang and I`ve an issue with connecting my Go webserver with Postgres Database.
Can someone tell me what I'm doing wrong here?
All these credentials are correct btw. User exists, Password is correct, DB exists and belongs to the user.
package app
import (
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
)
func connectDB(){
db, err := gorm.Open("postgres", "host=localhost port=5432 user=power_user dbname=local_db password=power_user")
if err != nil {
log.Fatal("DB Connection failed")
}
}
If you're sure about psql server is running, try to modify pg_hba.conf, locates at /etc/postgresql/${version}/main/pg_hba.conf usually:
# TYPE DATABASE USER ADDRESS METHOD
local all all trust
This question already has answers here:
How to use new URL from mongodb 3.6 to connect from golang
(2 answers)
Closed 3 years ago.
I'm looking at the tutorial offered in conjunction with the release of the official mongo-go-driver and the connection example uses a MongoDB server on localhost
// Set client options
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
However, the new hosted MongoDB service Atlas requires username and password to login. The connection string takes the format
mongodb://[username:password#]host1[/[database][?options]]
but there is no Golang example in the driver examples for Atlas.
So I'm wondering, what is the best way to log into Atlas without hard coding a password into a source file that will be posted on Github?
I am hosting my test Atlas cluster on AWS so I wanted to have similar credential management to the AWS process. From the AWS credentials page:
The default provider chain looks for credentials in the following order:
Environment variables.
Shared credentials file.
If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.
Therefore, I wanted to implement the environment veriable for my simple login to Atlas example. Code below assumes that the following line has been issued at the command line
export MONGO_PW='<your Atlas admin user password>'
Then the following program will verify your connection
package main
import (
"context"
"fmt"
"os"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
var username = "<username>"
var host1 = "<atlas host>" // of the form foo.mongodb.net
func main() {
ctx := context.TODO()
pw, ok := os.LookupEnv("MONGO_PW")
if !ok {
fmt.Println("error: unable to find MONGO_PW in the environment")
os.Exit(1)
}
mongoURI := fmt.Sprintf("mongodb+srv://%s:%s#%s", username, pw, host1)
fmt.Println("connection string is:", mongoURI)
// Set client options and connect
clientOptions := options.Client().ApplyURI(mongoURI)
client, err := mongo.Connect(ctx, clientOptions)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
err = client.Ping(ctx, nil)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println("Connected to MongoDB!")
}
From here the rest of the tutorial linked in my original question goes smoothly.
I tried to connect to Google CloudSQL PostgresSQL using Gorm golang and seems like it is not working.
Here's the code
func InitDB() *gorm.DB {
psqlInfo := fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=disable", os.Getenv("DB_HOST"), os.Getenv("DB_PORT"), os.Getenv("DB_USER"), os.Getenv("DB_PASSWORD"), os.Getenv("DB_NAME"))
db, err := gorm.Open("postgres", psqlInfo)
if err != nil {
fmt.Println("Failed to connect to the Database")
}
fmt.Println("Connected to the Database")
DB = db
return DB
}
If im using the localhost config everything works fine. See my .env file for cloudSQL config
DB_HOST=trader-234509:us-central1:testv1
DB_PORT=5432
DB_USER=username
DB_NAME=testv1
DB_PASSWORD=
Error is saying
dial tcp: lookup trader-234509:us-central1:testv1: no such host
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x164681d]
My local config (This one works fine)
DB_HOST=localhost
DB_PORT=5432
DB_USER=username
DB_NAME=test
DB_PASSWORD=
Did i do anything wrong?
Here's roughly how I connect from AppEngine:
import (
_ "github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/postgres"
"fmt"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func New() (*gorm.DB, error) {
user := "theuser"
password := "apassword"
dbHost := "my-project:the-region:my-db-instance-name"
databaseName := "mydbname"
connString := fmt.Sprintf("host=%s user=%s dbname=%s sslmode=disable password=%s", dbHost, user, databaseName, password)
return gorm.Open(postgres.New(postgres.Config{
DriverName: "cloudsqlpostgres",
DSN: connString,
}))
}
Cloud SQL doesn't support direct connection to instance name for 3rd party application yet, for details: https://cloud.google.com/sql/docs/mysql/connect-external-app
Based on my experience, there are 2 solutions:
As per above instruction, you can setup a Cloud Proxy following the steps and the connections flow would be: Golang app -> Cloud Proxy -> Cloud SQL
This approach is flexible and you're able to control the connection using firewall.
But you have to spend extra $$ to maintain the server and this cloud proxy instance, I've also heard that the cloud proxy may have some performance issues but I don't have exact evidence so far
Assign a Private IP to the Cloud SQL instance, refer to https://cloud.google.com/sql/docs/mysql/private-ip, so that you application can access the DB directly using this IP.
The trade-off is obvious that the app must be in the same network of the project, but this is a much more convenient approach, esp. all your applications are hosted in the same network
I didn't try out the public IP access approach as Cloud Proxy is just what I need if remote network connection is needed
In short, you need to deploy your code in a VM or other Google Managed service using option 2, or setup the cloud proxy to support your local debugging for option 1
Hope it helps
I'm trying to connect to my localhost postgresql server without SSL and I'm getting this error:
pq: SSL is not enabled on the server
That's fine, I know how to fix it:
type App struct {
Router *mux.Router
DB *sql.DB
}
func (a *App) Initialize(dbname string) {
connectionString := fmt.Sprintf("dbname=%s sslmode=disable", dbname)
var err error
a.DB, err = sql.Open("postgres", connectionString)
if err != nil {
log.Fatal(err)
}
defer a.DB.Close()
}
However I'm still getting the error!
I was able to recreate your error with a fresh install of postgres. While the error output was
pq: SSL is not enabled on the server
the real error was not having any databases created. To create a testdb let's run
createdb testdb
in your terminal with postgres already running in the background.
I think the answer comes more just to assign a parameter flag to your Postgres connection string url.
I was connecting to an AWS RDS by URL and I've just change to localhost on 5432 default port.
http://127.0.0.1:5432?sslmode=disable
or
http://localhost:5432?sslmode=disable
I stumbled upon this trying to connect to Postgresql server from my Go program with wrong password.