Im trying to Get current open Postgres DB connections using golang.
We are running our application on aws lambda, and many DB transactions are failing to begin.
Error that we see is -- pq: PAM authentication failed for user "username"
How can i log the current number of open postgres DB connections using golang so i can see if it reaches the max limit ?
The error you get is probably for incorrect user/password and not for reaching max connections. For max connections, you should get an error like "too many connections for ..."
To get the current number of connections, you need to query pg_stat_database:
func getNumberOfConnections(db *sql.DB) int {
queryStmt := "SELECT sum(numbackends) FROM pg_stat_database"
rows, err := db.Query(queryStmt)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
var sum int
for rows.Next() {
rows.Scan(&sum)
}
return sum
}
Related
I went through the documentation to insert multiple records into postgres using the package pg https://pkg.go.dev/github.com/go-pg/pg/v10#example-DB.Model-BulkInsert .
db := modelDB()
book1 := &Book{
Title: "new book 1",
}
book2 := &Book{
Title: "new book 2",
}
_, err := db.Model(book1, book2).Insert()
if err != nil {
panic(err)
}
fmt.Println(book1, book2)
Honestly i dont like this solution since it does not allow me to pass an array of books. Cause the use case i have is that i wont know the number of books i need to insert.
Should i be using transactions here cause i might have to insert more than 20 record at once. If yes, please help as am not finding good examples for this one.
PS: Must use pg library.
Just put your book, in array of slices, and insert it, go-pg support insert batch
I am wondering if this mongoDB driver for golang is susceptible to an injection attack:
https://github.com/mongodb/mongo-go-driver
Not able to find it documented, or glean it from the internal workings of the package.
It uses bson.D and bson.M internal map types for filtering, so it should be fairly easy to scrub the params and secure, just wondering if anyone knows for sure:
https://pkg.go.dev/go.mongodb.org/mongo-driver#v1.2.1/bson?tab=doc
For example, can we safely do something like this?
filter := bson.D{{"token", token}}
result := []struct {
AccountID string `bson:"account_id"`
Token string `bson:"token"`
}{}
// coll is *mondo.Collection
cur, err := d.coll.Find(ctx, filter)
err = cur.All(ctx, &result);
Thank you!
I am passing batchSize using changestreamopt.BatchSize.
But this not working this error occurred: BSON field '$changeStream.batchSize' is an unknown field
Sample API call
// coll is *mongo.Collection
// ctx is context
cur, err := coll.Watch(ctx, nil, changestreamopt.BatchSize(1000))
This looks like a bug in the current mongo-go-driver (v0.0.16) where the batchSize option is passed into the $changestream pipeline stage instead to the cursor.
The code change_stream.go#L66-L73 seems to only use maxAwaitTime to the cursor.
I've opened a ticket GODRIVER-591 with the MongoDB Go driver team.
Rarely I see a EOF error from mgo in my logs. Searching about this issue I came across this discussion where it is suggested that it would be safe to put a session.Refresh() at the start of the loop to handle this issue other issues like socket error, timeout etc.
However I couldn't find if my loop should be like this where the collection (*mgo.Collection) is re-assigned after each Refresh():
session := // create mgo session
var collection *mgo.Collection
for{
session.Refresh()
collection := session.DB("dbname").C("collectionName")
....
}
OR like below where collection is assigned once outside the loop:
session := // create mgo session
collection := session.DB("dbname").C("collectionName")
for{
session.Refresh()
....
}
Posting this since I am not able to simulate this issue at will
I am using maxTime to terminate the queries in mongo if it exceeds 1000ms.
Is there a way to find out if the query got terminated due to maxtime or it couldn't return any results due to match issues.
MongoDB will throw exception if query timeout.
Like this:
error: { "$err" : "operation exceeded time limit", "code" : 50 }
Hence you are able to distinguish between the exception case and no result case.
Read more here.