I am trying to connect to my GAE Postgres SQL db using Go+Gin+PGX. I have the Postgres SQL api activated and this program runs on my local machine but does not run on GAE. I think it is not connecting the db via pgx but I am not sure.
main.go works on my local machine and instance of psql but after deploying to GAE cannot connect to db via the Public IP in GCP SQL instance. I have modified my working code to fix the MWE.
package main
import (
"fmt"
"os"
"github.com/gin-gonic/gin"
"github.com/jackc/pgx/v4"
"golang.org/x/net/context"
)
func main() {
conn, err := connectDB()
if err != nil {
os.Exit(1)
}
defer conn.Close(context.Background())
router := gin.Default()
router.GET("/", func(c *gin.Context) {
c.JSON(200, gin.H{
"message": "pong",
})
})
router.Run(":8080")
}
func connectDB() (c *pgx.Conn, err error) {
postgres := "postgresql://postgres:root#35.236.60.144:5432/goapi" //35.236.60.144
conn, err := pgx.Connect(context.Background(), postgres)
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to connect to database:\n\t%v\n", err.Error())
return nil, err
}
err = conn.Ping(context.Background())
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to ping:\n\t%v\n", err.Error())
return nil, err
}
return conn, err
}
If I use Postman to perform a GET to GAE website then it I get
<html>
<head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>500 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered an error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body>
</html>
The corresponding GAE error stack trace sample is
runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xaabec7]
at
github.com/jackc/pgx/v4.(*Conn).exec (conn.go:413)
at
github.com/jackc/pgx/v4.(*Conn).Exec (conn.go:396)
at
github.com/jackc/pgx/v4.(*Conn).Ping (conn.go:347)
at
main.connectDB (main.go:41)
at
main.main (main.go:15)
When I view the log I see...
Unable to connection to database: failed to connect to `host=35.236.60.144 user=postgres database=goapi`: dial error (dial tcp 35.236.60.144:5432: connect: connection timed out)
Related
Folks,
I'm trying to run gRPC python server with go stub.
It's working properly locally and as dockerized, however, when I'm converting to deployments and run it on kubernetes GKE cluster, I'm getting frequent "connection refused" error as below
I'm client! --> working
Retrieving ... --> working
id:1 username:"admin" email:"admin#admin.admin" --> working
id:2 username:"user1" --> working
I'm client! --> working
Retrieving ... --> working
2021/11/27 00:55:22 Retrive error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 10.96.3.78:50051: connect: connection refused"
Also tried to add WithKeepaliveParams() dial option and close connection explicitly at the end without defer but still getting the same error.
Here is my stub code
package main
import (
"context"
"fmt"
// "io"
"log"
"time"
"github.com/nurhun/grpc_django_go_client/accountpb"
"google.golang.org/grpc"
"google.golang.org/grpc/keepalive"
)
func main() {
var kacp = keepalive.ClientParameters{
Time: 10 * time.Second, // send pings every 10 seconds if there is no activity
Timeout: time.Second, // wait 1 second for ping back
PermitWithoutStream: true, // send pings even without active streams
}
for {
fmt.Println("I'm client!")
cc, err := grpc.Dial("crud:50051", grpc.WithInsecure(), grpc.WithDisableHealthCheck(), grpc.WithKeepaliveParams(kacp))
if err != nil {
log.Fatal("Connection error", err)
}
// defer cc.Close()
c := accountpb.NewUserControllerClient(cc)
// ##### Using Retrive with User ID to get user data #####
fmt.Println("Retrieving ...")
res1, err := c.Retrieve(context.Background(), &accountpb.UserRetrieveRequest{Id: 1})
if err != nil {
log.Fatal("Retrive error: ", err)
}
fmt.Println(res1)
res2, err := c.Retrieve(context.Background(), &accountpb.UserRetrieveRequest{Id: 2})
if err != nil {
log.Fatal("Retrive error: ", err)
}
fmt.Println(res2)
time.Sleep(10 * time.Second)
cc.Close()
}
}
Any idea where is this error comes from ?
I am trying to connect to a mongo database hosted in azure using the .crt file.
I am successfully able to connect from my linux machine terminal using command:
mongo mongodb://username:password#prod-replicaset-0.com:27017,prod-replicaset-1.com:27017,prod-replicaset-2.com:27017/ --tls --tlsCAFile rootca.crt --tlsAllowInvalidCertificates
I am also able to connect from mongo UI client like robo3T by setting "Use SSL protocol" and using Auth Mechanism as "SCRAM-SHA-256".
[If I set Auth Mechanism to any other value, results in Authentication Failure]
But I am not able to connect to that database in Go lang code.
Here is a sample of code I am using:
package main
import (
"crypto/tls"
"crypto/x509"
"io/ioutil"
"log"
"net"
"github.com/globalsign/mgo"
)
func InitMongo() error {
rootCerts := x509.NewCertPool()
ca, err := ioutil.ReadFile("./rootca.crt")
if err != nil {
log.Fatalf("failed to read file : %s", err.Error())
return err
}
success := rootCerts.AppendCertsFromPEM(ca)
if !success {
log.Printf("rootcert failed")
}
connStr := "mongodb://username:password#prod-replicaset-0.com:27017,prod-replicaset-1.com:27017,prod-replicaset-2.com:27017/?ssl=true"
dbDialInfo, err := mgo.ParseURL(connStr)
if err != nil {
log.Fatal("unable to parse url - " + err.Error())
}
dbDialInfo.DialServer = func(addr *mgo.ServerAddr) (net.Conn, error) {
return tls.Dial("tcp", addr.String(), &tls.Config{
RootCAs: rootCerts,
InsecureSkipVerify: true,
})
}
// dbDialInfo.Mechanism = "SCRAM-SHA-256"
_session, err := mgo.DialWithInfo(dbDialInfo)
if err != nil {
log.Fatalf("failed to creating db session : %s", err.Error())
return err
}
log.Printf("Created session - %v", _session)
return nil
}
When I run this code, I get error:
failed to creating db session : "server returned error on SASL authentication step: Authentication failed."
If I set [dbDialInfo.Mechanism = "SCRAM-SHA-256"] before creating session, I get error:
failed to creating db session : "SASL support not enabled during build (-tags sasl)"
Please let me know what is causing this issue, how can I connect to the database.
Currently I am using "github.com/globalsign/mgo", if it required to use any other library, that's totally fine for me.
I just want to get connected to the db.
rootca.crt file looks something like:
-----BEGIN CERTIFICATE-----
MIIGLjCCBBagAwIBAgIUbxINX1qe6W+7kolWGp+MX8NbYj8wDQYJKoZIhvcNAQEL
<blah> <blah> <blah> <blah> <blah> <blah> <blah> <blah> <blah>
jCZAGGHmbrR3zeIsOY8yKau0IXqRp5Wy6NQ0poOTcma9BfwNUVc4/ixsCkEVYbgW
eMs=
-----END CERTIFICATE-----
Thank you.
After researching a lot, I was not able to find a way to connect to mongodb using .crt file using globalsign library.
However I was successfully able to do this using mongo-driver library.
here connection string can be of format:
mongodb://user:password#replicaset-0.com:27017,replicaset-1.com:27017,replicaset-2.com:27017/?ssl=true&tlsCAFile=./ca.crt&tlsCertificateKeyFile=./ca.pem&authSource=admin&replicaSet=replicaset
Sample code:
import (
"context"
"log"
"os"
// "github.com/globalsign/mgo"
mgo "go.mongodb.org/mongo-driver/mongo"
mongoOptions "go.mongodb.org/mongo-driver/mongo/options"
)
func InitMongo() (error) {
connStr := os.Getenv("MONGODB_CONN_STR")
dbName := os.Getenv("MONGODB_DATABASE")
clientOpts := mongoOptions.Client().ApplyURI(connStr)
if err := clientOpts.Validate(); err != nil {
log.Print("unable to parse url")
log.Fatal(err)
}
client, err := mgo.Connect(context.TODO(), clientOpts)
if err != nil {
log.Print("unable to connect into database")
log.Fatal(err)
}
if err := client.Ping(context.TODO(), nil); err != nil {
log.Print("database ping failed")
log.Fatal(err)
}
//client.Database(dbName)
return nil
}
I'm trying to connect to my mongo db atlas database, but I'm getting an error that I can't solve it within mongo+go forums.
The code:
package main
import (
"context"
"log"
"time"
"fmt"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
"go.mongodb.org/mongo-driver/bson"
)
func main(){
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://my-user:<my-pass>#datalake0-lesz0.a.query.mongodb.net/my-db?ssl=true&authSource=admin"))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
defer client.Disconnect(ctx)
err = client.Ping(ctx, readpref.Primary())
if err != nil {
log.Fatal(err)
}
databases, err := client.ListDatabaseNames(ctx, bson.M{})
if err != nil {
log.Fatal(err)
}
fmt.Println(databases)
}
The error:
connection() error occured during connection handshake: auth error: sasl conversation error: unable to authenticate using mechanism "SCRAM-SHA-1": (AuthenticationFailed) authentication failed, correlationID = 167bc5ba18415510a4144b7a
exit status 1
If the URI in the connect string is verbatim, then this is your problem:
mongodb://my-user:<my-pass> is incorrect, <my-pass> should be substituted with your password
You've probably cut and pasted the provided URI on completion of DB setup but this connect string does not include items like your password, also you gave my-user which, unless you set up the username my-user that also needs changing.
sql.Open() wouldn't error:
if db, err = sql.Open("postgres", url); err != nil {
return nil, fmt.Errorf("Postgres connect error : (%v)", err)
}
but db.Ping() would error:
if err = db.Ping(); err != nil {
return nil, fmt.Errorf("Postgres ping error : (%v)", err)
}
and it was simply because the lib/pq connection string wouldn't connect from within a docker container with the seperated connection parameters.
For example:
url := fmt.Sprintf("user=%v password=%v host=%v port=%v dbname=%v",
rs.conf.Redshift.User,
rs.conf.Redshift.Password,
rs.conf.Redshift.Host,
rs.conf.Redshift.Port,
rs.conf.Redshift.DB)
Using the connection string as a URL worked:
url := fmt.Sprintf("postgres://%v:%v#%v:%v/%v?sslmode=disable",
pql.conf.Postgres.User,
pql.conf.Postgres.Password,
pql.conf.Postgres.Host,
pql.conf.Postgres.Port,
pql.conf.Postgres.DB)
See lib/pq docs here:
https://godoc.org/github.com/lib/pq
I was stuck on this for more than a day and I owe the fix to Nikolay Sandalov's comment here in GitHub:
https://github.com/coreos/clair/issues/134#issuecomment-491300639
Thank you, Nikolay 🙇🏻♂️
I'm trying to connect a go client to mongodb server running with ssl enabled. I get a clear error message indicating that the hand shake failed due to ssl error. I use a self signed certificate on the client side.
Got below from the mongodb server:
2017-05-13T04:38:53.910+0000 I NETWORK [thread1] connection accepted from 172.17.0.1:51944 #10 (1 connection now open)
2017-05-13T04:38:53.911+0000 E NETWORK [conn10] SSL: error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate
2017-05-13T04:38:53.911+0000 I - [conn10] end connection
Error from Go client:
Could not connect to mongodb_s1.dev:27017 x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "XYZ")
Tried multiple options, but didn't help
You can skip TLS security checks using InsecureSkipVerify = true. This allows you to use self-signed certificates. See the code from compose help below.
Instead of skipping security checks, it is advisable to add the CA used to sign your certificates to the list of trusted CAs of the system.
package main
import (
"crypto/tls"
"fmt"
"net"
"os"
"strings"
"gopkg.in/mgo.v2"
)
func main() {
uri := os.Getenv("MONGODB_URL")
if uri == "" {
fmt.Println("No connection string provided - set MONGODB_URL")
os.Exit(1)
}
uri = strings.TrimSuffix(uri, "?ssl=true")
Here:
tlsConfig := &tls.Config{}
tlsConfig.InsecureSkipVerify = true
dialInfo, err := mgo.ParseURL(uri)
if err != nil {
fmt.Println("Failed to parse URI: ", err)
os.Exit(1)
}
And here:
dialInfo.DialServer = func(addr *mgo.ServerAddr) (net.Conn, error) {
conn, err := tls.Dial("tcp", addr.String(), tlsConfig)
return conn, err
}
session, err := mgo.DialWithInfo(dialInfo)
if err != nil {
fmt.Println("Failed to connect: ", err)
os.Exit(1)
}
defer session.Close()
dbnames, err := session.DB("").CollectionNames()
if err != nil {
fmt.Println("Couldn't query for collections names: ", err)
os.Exit(1)
}
fmt.Println(dbnames)
}