Can the MongoDB client be reused? [duplicate] - mongodb

This question already has answers here:
Is mongodb client driver concurrent safe?
(1 answer)
goroutine create multiple mongodb connection
(1 answer)
Closed 21 days ago.
I've managed to connect to MongoDB and I've got this client object, can this be passed around and reused?
Or does it need to be created each time I want to interact with the database?
If it can be reused then ill move the database connection out to its own function and run it just once at startup.
package main
import (
"context"
"fmt"
"log"
"main/api"
"os"
"time"
"github.com/joho/godotenv"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type User struct {
Username string
Password string `bson:"password,omitempty"`
}
func main() {
if err := godotenv.Load(); err != nil {
log.Println("No .env file found")
}
uri := os.Getenv("MONGODB_URI")
serverAPIOptions := options.ServerAPI(options.ServerAPIVersion1)
clientOptions := options.Client().
ApplyURI(uri).
SetServerAPIOptions(serverAPIOptions)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, clientOptions)
if err != nil {
log.Fatal(err)
}
usersColl := client.Database("sampleShop").Collection("users")
newUser := User{Username: "8282", Password: "Korean"}
result, err := usersColl.InsertOne(context.TODO(), newUser)
if err != nil {
panic(err)
}
fmt.Println(result)
fmt.Println(uri)
api.GetPaymentSession()
}

Related

How to list collections in mongodb

In golang..
I get list mongodb database name below..
filter := bson.D{{}}
dbs, _ := client.ListDatabaseNames(context.TODO(), filter)
fmt.Printf("%+v\n", dbs)
But, I want to get list collections name.
How about ListCollectionNames?
Here's an example from the documentation. I added some placeholder lines that you need to replace by your client connection code and getting a database:
package main
import (
"context"
"fmt"
"log"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
)
func main() {
// Placeholder:
//
// Connect to MongoDB, handle error
client, err := mongo.connect(....)
if err != nil {
log.Fatal(err)
}
// Obtain the DB, by name. db will have the type
// *mongo.Database
db := client.Database("name-of-your-DB")
// use a filter to only select capped collections
result, err := db.ListCollectionNames(
context.TODO(),
bson.D{{"options.capped", true}})
if err != nil {
log.Fatal(err)
}
for _, coll := range result {
fmt.Println(coll)
}
}

How to Manage MongoDB Client across Packages in Golang

I am currently starting out with MongoDB on GoLang. My current use case is this.
I wish to initialize connection to my MongoDB Database in a particular package and use the returned client across several other local packages. Here is what I have tried,
I have initialized the connection to MongoDB inside a package called dataLayer as below
package dataLayer
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
func InitDataLayer() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, options.Client().ApplyURI(credentials.MONGO_DB_ATLAS_URI))
if err != nil {
log.Fatal(err)
} else {
log.Println("Connected to Database")
}
}
Now if I wish to use the returned client in other packages, is it production safe to keep on calling initDataLayer over and over again to get back a client?
Thanks alot.
You don't have to call InitDataLayer over and over again. You need to create only one client, you can use the same client to connect to Mongo DB from various places.
Mongo client internally maintains a pool of connections, so you don't have to create this client again and again.
It's a good design to store this client as the struct field and keep reusing the same.
Edit:
Create Connection
package dataLayer
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
func InitDataLayer()(*mongo.Client) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, options.Client().ApplyURI(credentials.MONGO_DB_ATLAS_URI))
if err != nil {
log.Fatal(err)
} else {
log.Println("Connected to Database")
}
return client
}
main.go,
Close connection at the end of usage, otherwise connection would be leaked.
func main(){
//...
client = dataLayer.InitDataLayer()
defer client.Disconnect(context.Background())
...
}
Repository/DAL, store client in Repository struct
Interface
type BookRepository interface {
FindById( ctx context.Context, id int) (*Book, error)
}
Repository Implementation that store client in struct
type bookRepository struct {
client *mongo.Client
}
func (r *bookRepository) FindById( ctx context.Context, id int) (*Book, error) {
var book Book
err := r.client.DefaultDatabase().Collection("books").FindOne(ctx, bson.M{"_id": id }).Decode(&book)
if err == mongo.ErrNoDocuments {
return nil, nil
}
if err != nil {
return nil, err
}
return &book, nil
}

Why I get an error "client disconnected" when trying to get documents from mongo collection in go?

I have mongo capped collection and a simple API, written on Go. I built and run it. When I try to sent Get request or simply go localhost:8000/logger in browser - my process closes. Debug shows this happens, while executing "find" in collection. It produces error "client is disconnected". Collection has 1 document, and debug shows it is connected with my helper.
Go version 1.13
My code:
func main() {
r := mux.NewRouter()
r.HandleFunc("/logger", getDocs).Methods("GET")
r.HandleFunc("/logger", createDoc).Methods("POST")
log.Fatal(http.ListenAndServe("localhost:8000", r))
}
func getDocs(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var docs []models.Logger
//Connection mongoDB with helper class
collection := helper.ConnectDB()
cur, err := collection.Find(context.TODO(), bson.M{})
if err != nil {
helper.GetError(err, w)
return
}
defer cur.Close(context.TODO())
for cur.Next(context.TODO()) {
var doc models.Logger
err := cur.Decode(&doc)
if err != nil {
log.Fatal(err)
}
docs = append(docs, doc)
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
json.NewEncoder(w).Encode(docs)
}
func ConnectDB() *mongo.Collection {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://127.0.0.1:27017"))
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to MongoDB!")
logCollection := client.Database("local").Collection("loggerCollection")
return logCollection
}
According to the documentation, the call to mongo.NewClient doesn't ensure that you can connect the Mongo server. You should first call mongo.Client.Ping() to verify if you can connect to the database or not.
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://127.0.0.1:27017"))
if err != nil {
log.Fatal(err)
}
if err := client.Ping(context.TODO(), readpref.Primary()); err != nil {
// Can't connect to Mongo server
log.Fatal(err)
}
There could be several reasons behind failing to connect, the most obvious one is incorrect setup of ports. Is your mongodb server up and listening on port 27017? Is there any change you're running mongodb with Docker and it's not forwarding to the correct port?
I faced similar issue , read #Jay answer it definitely helped , as I checked my MongoDB was running using "MongoDB Compass" , then I changed the location of my insert statement , previously I was calling before the call of "context.WithTimeout". Below is working code.
package main
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type Book struct {
Name string `json:"name,omitempty"`
PublisherID string `json:"publisherid,omitempty"`
Cost string `json:"cost,omitempty"`
StartTime string `json:"starttime,omitempty"`
EndTime string `json:"endtime,omitempty"`
}
func main() {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
defer client.Disconnect(ctx)
testCollection := client.Database("BooksCollection").Collection("BooksRead")
inserRes, err := testCollection.InsertOne(context.TODO(), Book{Name: "Harry Potter", PublisherID: "IBN123", Cost: "1232", StartTime: "2013-10-01T01:11:18.965Z", EndTime: "2013-10-01T01:11:18.965Z"})
log.Println("InsertResponse : ", inserRes)
log.Println("Error : ", err)
}
I can see document inserted in console as well as in "MongoDB Comapass."
In heiper function "ConnectDB" after "NewClient" I must use "client.Connect(context.TODO())"
before any other use of client

mongodb getting 10_000 rows at a time

I'm trying to get 10000 document at a time in mongodb, but i got :
Information :
Driver https://github.com/mongodb/mongo-go-driver
opt.SetBatchSize(15_000)
opt.SetAllowPartialResults(false)
index on timestamp
Code :
package main
import (
"context"
"fmt"
"net/http"
"os"
"time"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
var database *mongo.Database
func main() {
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://20.20.20.43:27017"))
if err != nil {
panic(err)
}
database = client.Database("chat_data")
chatText := make([]chat, 0)
now := time.Now().Unix()
ctx, _ = context.WithTimeout(context.Background(), 30*time.Second)
// mongodb batch option
opt := options.Find()
opt.SetBatchSize(15_000)
opt.SetAllowPartialResults(false)
// mongodb filter
filter := bson.M{"timestamp": bson.M{"$gte": now - 108000}}
cur, err := database.Collection("chat").Find(ctx, filter, opt)
if err != nil {
// fmt.Fprint(w, err)
fmt.Println(err)
return
}
defer cur.Close(ctx)
for cur.Next(ctx) {
var result chat
err := cur.Decode(&result)
if err != nil {
fmt.Println(err)
continue
}
// do something with result....
// fmt.Println(result)
chatText = append(chatText, result)
}
if err := cur.Err(); err != nil {
// fmt.Fprint(w, cur.Err())
fmt.Println(err)
return
}
fmt.Println("done")
fmt.Println(len(chatText))
}
can i achieve this with mongodb & go driver ?, 30 second timeout are always reached
Edit 1
i try in python (with pymongo) it's only need 0m2.159s to query 36k doc with that filter
Try 7000, if it works try 12000, if it doesn't work try 4000, etc.
Make note of how long these requests take to figure out if the execution time is proportional to batch size.
You are querying on just the timestamp field. If you create an index on that collection with the timestamp field first, you should get faster results, and get a free sort in the process.

How to solve "command find requires authentication" using Golang and MongoDriver

I need help to fix the error. I'm using IBM mongo services.
go version go1.13.6 darwin/amd64
mongo driver version 1.2.1
The connection is working, I can read and write but sometimes it returns
: command find requires authentication and command insert requires authentication
MONGO_DB_URI=mongodb://username:password:port,host/dbname?authSource=admin&replicaSet=replset&connect=direct&alias=default
Connect:
func ConnectDatabase() *mongo.Client {
clientOptions := options.Client().ApplyURI(os.Getenv("MONGO_DB_URI"))
ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
var err error
client, err = mongo.Connect(ctx, clientOptions)
if err != nil {
log.Fatal(err)
}
ctx, _ = context.WithTimeout(context.Background(), 10*time.Second)
err = client.Ping(ctx, nil)
if err != nil {
log.Fatal(err)
return nil
}
fmt.Println("Connected to MongoDB!")
return client
}
Read:
func FindAll(collectionName string, query bson.M) (*mongo.Cursor, error) {
collection := client.Database("dbname").Collection(collectionName)
singleResult, err := collection.Find(context.TODO(), query)
return singleResult, err
}
Read:
ctx, _ := context.WithTimeout(context.Background(), 20*time.Second)
cur, err := mongo.GetCollection("collection_name").Find(ctx, createQuery())
if err != nil {
log.Println(err.Error())
}
I'm using the same database and the same configurations at our another Python Project. No exceptions.
There is a difference between connecting to a DB and performing operations on the DB.
Mongo lets you connect without authentication because you have to be able to connect to be able to authenticate.
var cred options.Credential
cred.AuthSource = YourAuthSource
cred.Username = YourUserName
cred.Password = YourPassword
// set client options
clientOptions := options.Client().ApplyURI(os.Getenv("MONGO_DB_URI")).SetAuth(cred)
//... the rest of your code
Hope this helps.