I'm writing a golang application and using the golang postgres driver - https://github.com/lib/pq/
I use a connection string like this
'name:pass#host:port/dbname'
I try to add aplication_name param in conn string, but this doesn't work
'name:pass#host:port/dbname?application_name=myapp'
Is it possible to set the application name from golang? (standard way)
Even though it's not mentioned in the documentation, if you look at the lib/pq source code you will find that application_name is supported.
This style connection works as desired:
connstring := fmt.Sprintf("user='%s' password='%s' dbname='%s' host='%s' application_name='%s'", user, password, dbname, host, application_name)
db, err := sql.Open("postgres", connstring)
If you look to the documentation a application_name option is not suuported. Maybe you could use:
fallback_application_name - An application_name to fall back to if
one isn't provided.
name:pass#host:port/dbname?fallback_application_name=myapp
You can set the application name one of two ways.
// Application name as space-separated list of options.
sql.Open("postgres", "user=myuser password=mypass host=localhost port=5432 dbname=mydb sslmode=disable application_name=myapp")
or
// Application name as query param.
sql.Open("postgres", "postgres://myuser:mypass#localhost:5432/mydb?sslmode=disable&application_name=myapp")
Related
I cannot connect to an Azure PostgreSQL database in a Golang application
Error message :
FATAL: Invalid Username specified. Please check the Username and retry connection. The Username should be in <username#hostname> format. (SQLSTATE 28000))
I do not have an # in my username.
I am using gorm as a ORM and connecting like this
dbUrl := fmt.Sprintf("postgres://%s:%s#%s:5432/%s", dbUser, dbPass, dbHost, dbName)
db, err := gorm.Open(postgres.Open(dbUrl), &gorm.Config{})
My variables look like this
POSTGRES_PASSWORD="password!"
DB_HOST="some-url-with-dashes.postgres.database.azure.com"
APP_PORT="8080"
POSTGRES_USER="postgresuser"
DB_NAME="file"
My code does see them and correctly creates dbUrl like :
postgres://postgresuser:password!#some-url-with-dashes.postgres.database.azure.com:5432/file
I have tried changing the username and password
You must have configured the parameter db_user_namespace to on. Don't do that.
I tried to connect to Google CloudSQL PostgresSQL using Gorm golang and seems like it is not working.
Here's the code
func InitDB() *gorm.DB {
psqlInfo := fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=disable", os.Getenv("DB_HOST"), os.Getenv("DB_PORT"), os.Getenv("DB_USER"), os.Getenv("DB_PASSWORD"), os.Getenv("DB_NAME"))
db, err := gorm.Open("postgres", psqlInfo)
if err != nil {
fmt.Println("Failed to connect to the Database")
}
fmt.Println("Connected to the Database")
DB = db
return DB
}
If im using the localhost config everything works fine. See my .env file for cloudSQL config
DB_HOST=trader-234509:us-central1:testv1
DB_PORT=5432
DB_USER=username
DB_NAME=testv1
DB_PASSWORD=
Error is saying
dial tcp: lookup trader-234509:us-central1:testv1: no such host
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x164681d]
My local config (This one works fine)
DB_HOST=localhost
DB_PORT=5432
DB_USER=username
DB_NAME=test
DB_PASSWORD=
Did i do anything wrong?
Here's roughly how I connect from AppEngine:
import (
_ "github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/postgres"
"fmt"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func New() (*gorm.DB, error) {
user := "theuser"
password := "apassword"
dbHost := "my-project:the-region:my-db-instance-name"
databaseName := "mydbname"
connString := fmt.Sprintf("host=%s user=%s dbname=%s sslmode=disable password=%s", dbHost, user, databaseName, password)
return gorm.Open(postgres.New(postgres.Config{
DriverName: "cloudsqlpostgres",
DSN: connString,
}))
}
Cloud SQL doesn't support direct connection to instance name for 3rd party application yet, for details: https://cloud.google.com/sql/docs/mysql/connect-external-app
Based on my experience, there are 2 solutions:
As per above instruction, you can setup a Cloud Proxy following the steps and the connections flow would be: Golang app -> Cloud Proxy -> Cloud SQL
This approach is flexible and you're able to control the connection using firewall.
But you have to spend extra $$ to maintain the server and this cloud proxy instance, I've also heard that the cloud proxy may have some performance issues but I don't have exact evidence so far
Assign a Private IP to the Cloud SQL instance, refer to https://cloud.google.com/sql/docs/mysql/private-ip, so that you application can access the DB directly using this IP.
The trade-off is obvious that the app must be in the same network of the project, but this is a much more convenient approach, esp. all your applications are hosted in the same network
I didn't try out the public IP access approach as Cloud Proxy is just what I need if remote network connection is needed
In short, you need to deploy your code in a VM or other Google Managed service using option 2, or setup the cloud proxy to support your local debugging for option 1
Hope it helps
For a project i'm trying to connection a cloud function to a cloud sql database setup as described in this quickstart guide.
The function is configured in the same region, the service account has the Role Cloud SQL-Client. I called the function through my computer like this:
gcloud functions call <function-name> --region=<region> --data '{"recipient":"hello","requester":"hello","message":"test"}'
The connection to the function is working, it seems like just the authentication to the database doesn't work but i don't get where i failed.
I checked the password, user and connection name multiple times, reset the password and it still doesn't work.
I found the issue here related to connecting cloud functions to cloud sql.
I tried surrounding the password in the dsn-string with single-quotes just to be sure escaping of characters in the password isn't a problem.
I also checked the environment variables coming in and they are the ones i entered in the configuration.
The function just pings the database for test purposes:
package receiver
import (
"database/sql"
"fmt"
"net/http"
"os"
// Import Postgres SQL driver
_ "github.com/lib/pq"
)
// Receives a message and stores it
func Receive(w http.ResponseWriter, r *http.Request) {
connectionName := os.Getenv("POSTGRES_INSTANCE_CONNECTION_NAME")
dbUser := os.Getenv("POSTGRES_USER")
dbPassword := os.Getenv("POSTGRES_PASSWORD")
dsn := fmt.Sprintf("user=%s password='%s' host=/cloudsql/%s dbname=messages", dbUser, dbPassword, connectionName)
var err error
db, err := sql.Open("postgres", dsn)
if err != nil {
fmt.Fprintf(w, "Could not open db: %v \n", err)
}
// Only allow 1 connection to the database to avoid overloading
db.SetMaxIdleConns(1)
db.SetMaxOpenConns(1)
defer db.Close()
if pingerror := db.Ping(); pingerror != nil {
fmt.Fprintf(w, "Failed to ping database: %s \n", pingerror)
return
}
}
The variable POSTGRES_INSTANCE_CONNECTION_NAME is formatted as described here as ProjectID:Region:InstanceID.
Expected is a success message or no error and i'm actually getting this message:
pq: password authentication failed for user "postgres"
Note: I also created a function containing the demo code from here with my sql database settings and the error is the same. It seems like i missed some step while setting up the user or sql instance. But i can't find out which.
Feels strange to answer my own question but here it is: For some reason connecting with the postgres user doesn't work. Finally i created a new database user for the function and a password containing only alphanumeric characters.
The unix socket at /cloudsql/{connectionName} is only provided in the GCF runtime. When running locally, you either need change your connection string or use the Cloud SQL proxy to simulate a unix socket at the same path.
I am looking to connect and query to a PostgreSQL. But I only want to connect to a particular Schema.
As per the doc (JDBC) we can use
jdbc:postgresql://localhost:5432/mydatabase?searchpath=myschema
or update As of 9.4 you can specify the url with the new currentSchema parameter like so:
jdbc:postgresql://localhost:5432/mydatabase?currentSchema=myschema
But I am unable to do so with golang SQL driver;
As per the documents, we can also use SET search_path TO myschema,public;
But I only want to declare it for once during initializing but I think this needs to be executed every time for new connection.
Also I am using following code please help me identify the correct parameters to be passed to this in order to only connect with schema
db, err := sql.Open("postgres", `dbname=`+s.settings.Database+
` user=`+s.settings.Username+` password=`+s.settings.Password+
` host=`+s.settings.Url+` sslmode=disable`)
Adding currentSchema=myschema or searchpath=myschema is not working!
Is there a way I can only connect to a particular database-schema in GO
You should add search_path=myschema to dataSourceName
P.S. better use fmt.Sprintf("host=%s port=%d dbname=%s user=%s password='%s' sslmode=disable search_path=%s", ...) instead ``+``
Set Search_path is right and you do it once. ie:
db, err := sql.Open("postgres",
"host=localhost dbname=Test sslmode=disable user=postgres password=secret")
if err != nil {
log.Fatal("cannot connect ...")
}
defer db.Close()
db.Exec(`set search_path='mySchema'`)
rows, err := db.Query(`select blah,blah2 from myTable`)
...
I'm trying to connect to a postresql database with the pq driver in Go. When I do it on a local copy of the database, with a connection string like
DB, err = sql.Open("postgres", "user=user password=pwd dbname=mydb sslmode=disable")
it all works well.
However, when I switch to a production server where the connection goes through pgbouncer:
DB, err = sql.Open("postgres", "user=user password=pwd host=/var/run/pgbouncer port=port dbname=mydb sslmode=disable")
I keep getting the same error for all queries, however simple:
Database error: pq: S:"ERROR" M:"prepared statement \"1\" does not exist" C:"26000" F:"prepare.c" L:"519" R:"FetchPreparedStatement"
(it's always "prepared statement \"1\"", independent of the query I'm trying to pass)
The query in both cases is run simply as follows:
res_rows, err := DB.Query(query)
if err != nil {
log.Printf("Database error: %s\n", err)
}
for res_rows.Next() {
...
}
Googling suggests to turn off prepared statements, but I don't know how to do that in Go and I'm not sure it is supported at all. Any help (even a suggestion to use something else entirely) would be greatly appreciated.
Package driver
type Queryer
type Queryer interface {
Query(query string, args []Value) (Rows, error)
}
Queryer is an optional interface that may be implemented by a
Conn.
If a Conn does not implement Queryer, the sql package's
DB.Query will first prepare a query, execute the statement, and then
close the statement.
I don't see where the lib/pq PostgreSQL driver implements Queryer. Therefore, the DB.Query query is prepared before execution.
PgBouncer doesn't support the PREPARE feature for all pooling methods: Feature matrix for pooling modes.
The Postgres driver now has a solution to fix this issue: https://github.com/lib/pq/issues/389
It isn't in the documentation but works as expected, including with PgBouncer and Transaction pooling enabled.
If you're using PgBouncer you need to set binary_parameters=yes to your database dsn connection as a query parameter
try this:
DB, err = sql.Open("postgres", "user=user password=pwd dbname=mydb sslmode=disable, binary_parameters=yes")