I'm trying to connect to a postresql database with the pq driver in Go. When I do it on a local copy of the database, with a connection string like
DB, err = sql.Open("postgres", "user=user password=pwd dbname=mydb sslmode=disable")
it all works well.
However, when I switch to a production server where the connection goes through pgbouncer:
DB, err = sql.Open("postgres", "user=user password=pwd host=/var/run/pgbouncer port=port dbname=mydb sslmode=disable")
I keep getting the same error for all queries, however simple:
Database error: pq: S:"ERROR" M:"prepared statement \"1\" does not exist" C:"26000" F:"prepare.c" L:"519" R:"FetchPreparedStatement"
(it's always "prepared statement \"1\"", independent of the query I'm trying to pass)
The query in both cases is run simply as follows:
res_rows, err := DB.Query(query)
if err != nil {
log.Printf("Database error: %s\n", err)
}
for res_rows.Next() {
...
}
Googling suggests to turn off prepared statements, but I don't know how to do that in Go and I'm not sure it is supported at all. Any help (even a suggestion to use something else entirely) would be greatly appreciated.
Package driver
type Queryer
type Queryer interface {
Query(query string, args []Value) (Rows, error)
}
Queryer is an optional interface that may be implemented by a
Conn.
If a Conn does not implement Queryer, the sql package's
DB.Query will first prepare a query, execute the statement, and then
close the statement.
I don't see where the lib/pq PostgreSQL driver implements Queryer. Therefore, the DB.Query query is prepared before execution.
PgBouncer doesn't support the PREPARE feature for all pooling methods: Feature matrix for pooling modes.
The Postgres driver now has a solution to fix this issue: https://github.com/lib/pq/issues/389
It isn't in the documentation but works as expected, including with PgBouncer and Transaction pooling enabled.
If you're using PgBouncer you need to set binary_parameters=yes to your database dsn connection as a query parameter
try this:
DB, err = sql.Open("postgres", "user=user password=pwd dbname=mydb sslmode=disable, binary_parameters=yes")
Related
I am having an issue with Go and performing standard operations on my PostgreSQL database.
I first started coding with GORM, and was getting the following error message while connecting:
dial tcp: lookup tcp/fullstack_api: getaddrinfow: The specified class was not found.
After switching to the standard "database/sql" package with the _ "github.com/lib/pq" postgreSQL dialect, connecting was no longer throwing this error. However, now I get this error when trying to perform any query on the connected database, which I assume GORM was doing initially.
The following code causes this error on my system:
// Connect initiates a DB connection.
func (dbConn *PostgresConnection) Connect() error {
handle, connErr := sql.Open("postgres", dbConn.getConnectionString())
if connErr != nil { // Does NOT cause an error
return connErr
}
if pingErr := handle.Ping(); pingErr != nil { // Causes the above error
return pingErr
}
dbConn.handle = handle
return nil
}
I have checked that the PostgreSQL service is running, and the database exists.
While writing this question, I checked my connection string / env variables again.
I realized that I had a stupid copy/paste error from the day before that I hadn't validated:
Connection string: host=127.0.0.1 port=5432 port=new_database user=db_user password=XXXXXX
As you can see, there is an additional port variable that should have been the dbname. After fixing this issue, everything worked as expected.
Connection string: host=127.0.0.1 port=5432 dbname=new_database user=db_user password=XXXXXX
TLDR: Always re-validate every piece of your connection information when getting this (cryptic) error!
For a project i'm trying to connection a cloud function to a cloud sql database setup as described in this quickstart guide.
The function is configured in the same region, the service account has the Role Cloud SQL-Client. I called the function through my computer like this:
gcloud functions call <function-name> --region=<region> --data '{"recipient":"hello","requester":"hello","message":"test"}'
The connection to the function is working, it seems like just the authentication to the database doesn't work but i don't get where i failed.
I checked the password, user and connection name multiple times, reset the password and it still doesn't work.
I found the issue here related to connecting cloud functions to cloud sql.
I tried surrounding the password in the dsn-string with single-quotes just to be sure escaping of characters in the password isn't a problem.
I also checked the environment variables coming in and they are the ones i entered in the configuration.
The function just pings the database for test purposes:
package receiver
import (
"database/sql"
"fmt"
"net/http"
"os"
// Import Postgres SQL driver
_ "github.com/lib/pq"
)
// Receives a message and stores it
func Receive(w http.ResponseWriter, r *http.Request) {
connectionName := os.Getenv("POSTGRES_INSTANCE_CONNECTION_NAME")
dbUser := os.Getenv("POSTGRES_USER")
dbPassword := os.Getenv("POSTGRES_PASSWORD")
dsn := fmt.Sprintf("user=%s password='%s' host=/cloudsql/%s dbname=messages", dbUser, dbPassword, connectionName)
var err error
db, err := sql.Open("postgres", dsn)
if err != nil {
fmt.Fprintf(w, "Could not open db: %v \n", err)
}
// Only allow 1 connection to the database to avoid overloading
db.SetMaxIdleConns(1)
db.SetMaxOpenConns(1)
defer db.Close()
if pingerror := db.Ping(); pingerror != nil {
fmt.Fprintf(w, "Failed to ping database: %s \n", pingerror)
return
}
}
The variable POSTGRES_INSTANCE_CONNECTION_NAME is formatted as described here as ProjectID:Region:InstanceID.
Expected is a success message or no error and i'm actually getting this message:
pq: password authentication failed for user "postgres"
Note: I also created a function containing the demo code from here with my sql database settings and the error is the same. It seems like i missed some step while setting up the user or sql instance. But i can't find out which.
Feels strange to answer my own question but here it is: For some reason connecting with the postgres user doesn't work. Finally i created a new database user for the function and a password containing only alphanumeric characters.
The unix socket at /cloudsql/{connectionName} is only provided in the GCF runtime. When running locally, you either need change your connection string or use the Cloud SQL proxy to simulate a unix socket at the same path.
I am looking to connect and query to a PostgreSQL. But I only want to connect to a particular Schema.
As per the doc (JDBC) we can use
jdbc:postgresql://localhost:5432/mydatabase?searchpath=myschema
or update As of 9.4 you can specify the url with the new currentSchema parameter like so:
jdbc:postgresql://localhost:5432/mydatabase?currentSchema=myschema
But I am unable to do so with golang SQL driver;
As per the documents, we can also use SET search_path TO myschema,public;
But I only want to declare it for once during initializing but I think this needs to be executed every time for new connection.
Also I am using following code please help me identify the correct parameters to be passed to this in order to only connect with schema
db, err := sql.Open("postgres", `dbname=`+s.settings.Database+
` user=`+s.settings.Username+` password=`+s.settings.Password+
` host=`+s.settings.Url+` sslmode=disable`)
Adding currentSchema=myschema or searchpath=myschema is not working!
Is there a way I can only connect to a particular database-schema in GO
You should add search_path=myschema to dataSourceName
P.S. better use fmt.Sprintf("host=%s port=%d dbname=%s user=%s password='%s' sslmode=disable search_path=%s", ...) instead ``+``
Set Search_path is right and you do it once. ie:
db, err := sql.Open("postgres",
"host=localhost dbname=Test sslmode=disable user=postgres password=secret")
if err != nil {
log.Fatal("cannot connect ...")
}
defer db.Close()
db.Exec(`set search_path='mySchema'`)
rows, err := db.Query(`select blah,blah2 from myTable`)
...
I'm trying to connect to my localhost postgresql server without SSL and I'm getting this error:
pq: SSL is not enabled on the server
That's fine, I know how to fix it:
type App struct {
Router *mux.Router
DB *sql.DB
}
func (a *App) Initialize(dbname string) {
connectionString := fmt.Sprintf("dbname=%s sslmode=disable", dbname)
var err error
a.DB, err = sql.Open("postgres", connectionString)
if err != nil {
log.Fatal(err)
}
defer a.DB.Close()
}
However I'm still getting the error!
I was able to recreate your error with a fresh install of postgres. While the error output was
pq: SSL is not enabled on the server
the real error was not having any databases created. To create a testdb let's run
createdb testdb
in your terminal with postgres already running in the background.
I think the answer comes more just to assign a parameter flag to your Postgres connection string url.
I was connecting to an AWS RDS by URL and I've just change to localhost on 5432 default port.
http://127.0.0.1:5432?sslmode=disable
or
http://localhost:5432?sslmode=disable
I stumbled upon this trying to connect to Postgresql server from my Go program with wrong password.
I'm writing a golang application and using the golang postgres driver - https://github.com/lib/pq/
I use a connection string like this
'name:pass#host:port/dbname'
I try to add aplication_name param in conn string, but this doesn't work
'name:pass#host:port/dbname?application_name=myapp'
Is it possible to set the application name from golang? (standard way)
Even though it's not mentioned in the documentation, if you look at the lib/pq source code you will find that application_name is supported.
This style connection works as desired:
connstring := fmt.Sprintf("user='%s' password='%s' dbname='%s' host='%s' application_name='%s'", user, password, dbname, host, application_name)
db, err := sql.Open("postgres", connstring)
If you look to the documentation a application_name option is not suuported. Maybe you could use:
fallback_application_name - An application_name to fall back to if
one isn't provided.
name:pass#host:port/dbname?fallback_application_name=myapp
You can set the application name one of two ways.
// Application name as space-separated list of options.
sql.Open("postgres", "user=myuser password=mypass host=localhost port=5432 dbname=mydb sslmode=disable application_name=myapp")
or
// Application name as query param.
sql.Open("postgres", "postgres://myuser:mypass#localhost:5432/mydb?sslmode=disable&application_name=myapp")