Golang: fail to connect with MongoDB - mongodb

I am trying to connect Go application with MongoDB server while running the database using docker.
I'm able to connect to the DB using the shell and perform different actions. However, the Go app fails when connecting to the DB. I'm using mgo driver and below is the code in use where am trying to implement a db middleware that can be used by all routes:
middleware code:
package db
import (
"net/http"
"os"
"github.com/gorilla/context"
"github.com/urfave/negroni"
mgo "gopkg.in/mgo.v2"
)
const key = "dbkey"
func GetDb(r *http.Request) *mgo.Database {
if rv := context.Get(r, key); rv != nil {
return rv.(*mgo.Database)
}
return nil
}
func SetDb(r *http.Request, val *mgo.Database) {
context.Set(r, key, val)
}
func MongoMiddleware() negroni.HandlerFunc {
database := os.Getenv("DB_NAME")
session, err := mgo.Dial("127.0.0.1:27017")
if err != nil {
println(err) // error message below
}
return negroni.HandlerFunc(func(rw http.ResponseWriter, r *http.Request, next http.HandlerFunc) {
reqSession := session.Clone()
defer reqSession.Close()
db := reqSession.DB(database)
SetDb(r, db)
next(rw, r)
})
}
the error am getting is :
panic: runtime error: invalid memory address or nil pointer dereference
route and main package code:
package main
import (
gmux "github.com/gorilla/mux"
"github.com/urfave/negroni"
"github.com/mypro/db"
"github.com/mypro/hub"
)
func main() {
router := gmux.NewRouter()
router.HandleFunc("/name", hub.Create).
Methods("GET")
n := negroni.Classic()
n.Use(db.MongoMiddleware())
n.UseHandler(router)
n.Run(":9000")
}
method that consume the db middleware to find a collection:
type Name struct {
Id bson.ObjectId `bson:"_id"`
Name string `bson:"name"`
}
func Create(w http.ResponseWriter, r *http.Request) {
var aName Name
db := db.GetDb(r)
names := db.C("x")
err := names.Find(bson.M{"name": "sam"}).One(&aName)
if err != nil {
log.Print(err)
}
fmt.Println(&aName)
json.NewEncoder(w).Encode(&aName)
}

Related

Getting "_id : 000000000" for every get request

I have recently started working with GO and tried to create an API. This was a basic API with few get and post endpoints. Every endpoint is working just fine but one. I am doing the same thing as in the other get endpoints but don't understand why am I getting the empty instance back from the mongoDB. And as far as I can guess, I think this is the problem due to data type. This is my Post struct.
package models
import (
"time"
"go.mongodb.org/mongo-driver/bson/primitive"
)
type Posts struct {
Id primitive.ObjectID `json:"id" bson:"_id" validate:"nil=false"`
Caption string `json:"caption" bson:"caption"`
ImageUrl string `json:"imageUrl" bson:"imageUrl" validate:"nil=false"`
Author string `json:"author" bson:"author" validate:"nil:false"`
Time time.Time `json:"time" bson:"time"`
}
This is the controller getPost
package controller
import (
"insta/models"
"log"
"net/http"
"github.com/gin-gonic/gin"
"go.mongodb.org/mongo-driver/bson"
// "go.mongodb.org/mongo-driver/bson/primitive"
)
func GetPost(c *gin.Context) {
var post models.Posts
postId := c.Param("postId")
client, ctx, cancel := getConnection()
defer cancel()
defer client.Disconnect(ctx)
err := client.Database("instagram").Collection("posts").FindOne(ctx, bson.M{"_id": postId}).Decode(&post)
if err != nil {
log.Printf("Couldn't get the Post")
}
c.JSON(http.StatusOK, gin.H{"post": post})
}
This is my main
package main
import (
"insta/controller"
"github.com/gin-gonic/gin"
)
func main() {
router := gin.Default()
router.GET("/posts/:postId" , controller.GetPost)
router.Run()
}
I am getting this response.
PostId is valid
The problem is the postId from the param is of type string while the postId in mongodb is of type primitive.ObjectID which are not the same.
The solution is to convert it to a MongoDB ObjectID before querying.
func GetPost(c *gin.Context) {
var post models.Posts
postId := c.Param("postId")
postObjectId, err := primitive.ObjectIDFromHex(postId)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"message": "PostID is not a valid ObjectID"})
return
}
client, ctx, cancel := getConnection()
defer cancel()
defer client.Disconnect(ctx)
err = client.Database("instagram").Collection("posts").FindOne(ctx, bson.M{"_id": postObjectId}).Decode(&post)
// Check if document exists return 404 error
if errors.Is(err, mongo.ErrNoDocuments) {
c.JSON(http.StatusNotFound, gin.H{"message": "Post with the given id does not exist"})
return
}
// Mongodb network or server error
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"message": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"post": post})
}

Go: client is disconnected

It's been few weeks since I joined in Gophers team. So far so good.
I started a new project using a fiber web framework to build backend APIs.
I am using MongoDB as my database.
database/db.go
package database
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
var DB *mongo.Database
// InitMongo : Initialize mongodb...
func connectToMongo() {
log.Printf("Initializing database")
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
if err != nil {
log.Fatal("Could not able to connect to the database, Reason:", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal("Context error, mongoDB:", err)
}
//Cancel context to avoid memory leak
defer cancel()
defer client.Disconnect(ctx)
// Ping our db connection
err = client.Ping(context.Background(), readpref.Primary())
if err != nil {
log.Fatal("Ping, mongoDB:", err)
}
log.Printf("Database connected!")
// Create a database
DB = client.Database("golang-test")
return
}
// In Golang, init() functions always initialize whenever the package is called.
// So, whenever DB variable called, the init() function initialized
func init() {
connectToMongo()
}
controllers/mongo.controller/mongo.controller.go
package mongocontroller
import (
"log"
"github.com/gofiber/fiber/v2"
service "gitlab.com/.../services/mongoservice"
)
// GetPersons godoc
// #Summary Get persons.
// #Description Get persons
// #Tags persons
// #Produce json
// #Success 200 {object} []service.Person
// #Failure 400 {object} httputil.HTTPError
// #Failure 404 {object} httputil.HTTPError
// #Failure 500 {object} httputil.HTTPError
// #Router /v1/persons [get]
func GetPersons(c *fiber.Ctx) error {
res, err := service.GetPersons()
if err != nil {
log.Fatal("ERROR: in controller...", err)
}
return c.JSON(res)
}
services/mongoservice/mongo.service.go
package mongoservice
import (
"context"
"log"
database "gitlab.com/.../database"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/primitive"
)
// Person : ...
type Person struct {
ID primitive.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
Name string `bson:"name,omitempty" json:"name,omitempty"`
Age int `bson:"age,omitempty" json:"age,omitempty"`
}
func GetPersons() ([]Person, error) {
ctx := context.Background()
persons := []Person{}
log.Printf("mongo data...", ctx)
cur, err := database.DB.Collection("persons").Find(ctx, bson.M{})
if err != nil {
log.Fatal(err)
}
// Iterate through the returned cursor.
for cur.Next(ctx) {
var person Person
cur.Decode(&person)
persons = append(persons, person)
}
defer cur.Close(ctx)
return persons, err
}
Here is my data stored in the database:
The problem is, the line cur, err := database.DB.Collection("persons").Find(ctx, bson.M{}) from service always throwing Client is disconnected.
Any help is appreciated!
Thank you.
You are calling defer client.Disconnect(ctx) in the same function which is creating the connection (connectToMongo ) so it will close the connection after calling the function. You should return the connection and close after finishing your task. I mean these parts:
defer cancel()
defer client.Disconnect(ctx)

Cannot read time_stamp with type `strfmt.DateTime` from MongoDB

I'm trying to write timestamp with strfmt.DateTime type (https://godoc.org/github.com/go-openapi/strfmt#DateTime) into the mongodb
I can successfully write this format of date into the DB, which looks like this:
{ "_id" : ObjectId("5bcb58f7540ac6d0bc946e22"), "status" : "test", "time_stamp" : {
"data" : "2018-10-21T00:33:59.699+08:00" } }
But I just can't retrieve it from the mongodb, the value of the time_stamp always shows 0001-01-01T00:00:00.000Z, I just don't see why.
Here is my code, any suggestions or opinions are welcome! Thanks
package main
import (
"fmt"
"time"
"github.com/go-openapi/strfmt"
"github.com/op/go-logging"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
type TxStatus struct {
Status string `json:"status" bson:"status"`
TimeStamp *strfmt.DateTime `json:"time_stamp" bson:"time_stamp"`
}
type MongoDBOperations struct {
mongoSession *mgo.Session
database string
collection string
}
var log = logging.MustGetLogger("example")
func main() {
mo := MongoDBOperations{}
mo.database = Database
mo.collection = Collection
// We need this object to establish a session to our MongoDB.
mongoDBDialInfo := &mgo.DialInfo{
Addrs: []string{MongoDBHosts},
Timeout: 60 * time.Second,
Database: AuthDatabase,
Username: AuthUserName,
Password: AuthPassword,
}
// Create a session which maintains a pool of socket connections
// to our MongoDB.
var err error
mo.mongoSession, err = mgo.DialWithInfo(mongoDBDialInfo)
if err != nil {
log.Fatalf("CreateSession: %s\n", err)
}
mo.mongoSession.SetMode(mgo.Eventual, true)
write(mo)
read(mo)
}
func write(mo MongoDBOperations) {
log.Info("write operation")
session := mo.mongoSession.Copy()
defer session.Close()
c := session.DB(Database).C(Collection)
timestamp := strfmt.DateTime(time.Now())
txStatus := TxStatus{
Status: "test",
TimeStamp: &timestamp,
}
if err2 := c.Insert(txStatus); err2 != nil {
panic(err2)
}
}
func read(mo MongoDBOperations) {
log.Info("write operation")
session := mo.mongoSession.Copy()
defer session.Close()
c := session.DB(Database).C(Collection)
// Find and Count
var status []TxStatus
err2 := c.Find(bson.M{"status": "test"}).All(&status)
if err2 != nil {
panic(err2)
}
for _, elem := range status {
fmt.Printf("%+v\n", elem)
}
}
I dug through the code of github.com/go-openapi/strfmt and what I've found out is that they are using github.com/globalsign/mgo instead of gopkg.in/mgo.v2. Changing imports to use github.com/globalsign/mgo and github.com/globalsign/mgo/bson has fixed the issue. Globalsign package is fork of mgo.v2 so the methods remain the same and there is no need to change any code besides import.

Errors Getting polls by id from mongoDB with go-gin and mgo

How do i query polls by id with go-gin and MongoDB, i have tried several methods but i still get errors (not found), can't seem to find a walk around below is my code, with my database on mongoDB:
type Poll struct {
//ID string `json:"_id,omitempty"`
ID bson.ObjectId `json:"id,omitempty" bson:"_id,omitempty"`
Firstname string `json:"firstname,omitempty"`
Lastname string `json:"lastname,omitempty"`
Poll string `json:"poll,omitempty"`
// Address *Address `json:"address,omitempty"`
}
var (
// Session stores mongo session
Session *mgo.Session
// Mongo stores the mongodb connection string information
Mongo *mgo.DialInfo
)
const (
// MongoDBUrl is the default mongodb url that will be used to connect to the
// database.
MongoDBUrl = "mongodb://localhost:27017/smartpoll"
// CollectionPoll holds the name of the poll collection
CollectionPoll = "polls"
)
// Connect connects to mongodb
func Connect() {
uri := os.Getenv("MONGODB_URL")
if len(uri) == 0 {
uri = MongoDBUrl
}
mongo, err := mgo.ParseURL(uri)
s, err := mgo.Dial(uri)
if err != nil {
fmt.Printf("Can't connect to mongo, go error %v\n", err)
panic(err.Error())
}
s.SetSafe(&mgo.Safe{})
fmt.Println("Connected to", uri)
Session = s
Mongo = mongo
}
func init() {
Connect()
}
func main() {
port := os.Getenv("PORT")
if port == "" {
log.Fatal("$PORT must be set")
}
router := gin.Default()
router.Use(ConnectMiddleware)
router.GET("/", func (c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"message": "OK"})
})
router.GET("/polls/:_id", pollsByID)
router.Run(":" + port)
}
func ConnectMiddleware(c * gin.Context) {
c.Set("db", Session.DB(Mongo.Database))
c.Next()
}
func pollsByID(c * gin.Context) {
db := c.MustGet("db").(*mgo.Database)
id := c.Param("id")
poll := []Poll{}
// err := db.C(CollectionPoll).Find(id).One(&poll)
err := db.C(CollectionPoll).Find(bson.M{"_id": id}).One(&poll)
if err != nil {
//c.Error(err)
//panic(err)
log.Println(err)
}
result := gin.H{"payload": poll}
c.Writer.Header().Set("Content-Type", "application/json")
c.JSON(200, result)
}
my DB is as follows:
{
"_id" : ObjectId("58d9cf1cdf353f3d2f5951b4"),
"id" : "1",
"firstname" : "Sam",
"lastname" : "Smith",
"poll" : "Who is the Richest in the World"
}
Your ID is an ObjectId, but your input is a string. You need to use bson.ObjectIdHex to parse the string into an ObjectId:
err := db.C(CollectionPoll).FindId(bson.ObjectIdHex(id)).One(&poll)
Change polls from an array:
polls := []Poll{}
TO:
polls := Poll{}

split routes into separate package in Golang and MongoDB

I'm new to Golang, i have been created an api in Golang and MongoDB.
After a hard struggle successfully separate the controller and model packages ,Now i want to define routes in a separate package of routers and access them in main package same like controllers and models.I'm using gorilla/mux package for routing.Anyone can help me please, thanks in Advance!
and here is all of my code:
RESTMONGOMVC/main.go
package main
import (
"RESTMONGOMVC/controllers"
"log"
"net/http"
"github.com/gorilla/mux"
"gopkg.in/mgo.v2"
)
var (
session *mgo.Session
collection *mgo.Collection
err error
)
func getSession() *mgo.Session {
// Connect to our local mongo
s, err := mgo.Dial("mongodb://localhost")
// Check if connection error, is mongo running?
if err != nil {
panic(err)
}
// Deliver session
return s
}
func main() {
var err error
r := mux.NewRouter()
uc := controllers.NewNoteController(getSession())
r.HandleFunc("/api/notes", uc.GetNotes).Methods("GET")
r.HandleFunc("/api/notes", uc.CreateNote).Methods("POST")
r.HandleFunc("/api/notes/{id}", uc.UpdateNote).Methods("PUT")
r.HandleFunc("/api/notes/{id}", uc.DeleteNote).Methods("DELETE")
http.Handle("/api/", r)
http.Handle("/", http.FileServer(http.Dir(".")))
log.Println("Starting Mongodb Session")
session, err = mgo.Dial("localhost")
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
collection = session.DB("notesdb").C("notes")
log.Println("Listening on 8080")
http.ListenAndServe(":8080", nil)
}
controllers/note.go
package controllers
import (
"RESTMONGOMVC/models"
"encoding/json"
"log"
"net/http"
"time"
"github.com/gorilla/mux"
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
var (
session *mgo.Session
collection *mgo.Collection
err error
)
type (
// UserController represents the controller for operating on the User resource
NoteController struct {
session *mgo.Session
}
)
// NewUserController provides a reference to a UserController with provided mongo session
func NewNoteController(s *mgo.Session) *NoteController {
return &NoteController{s}
}
func (uc NoteController) GetNotes(w http.ResponseWriter, r *http.Request) {
var notes []models.Note
iter := collection.Find(nil).Iter()
result := models.Note{}
for iter.Next(&result) {
notes = append(notes, result)
}
w.Header().Set("Content-Type", "application/json")
j, err := json.Marshal(models.NotesResource{Notes: notes})
if err != nil {
panic(err)
}
w.Write(j)
}
func (uc NoteController) CreateNote(w http.ResponseWriter, r *http.Request) {
var noteResource models.NoteResource
err := json.NewDecoder(r.Body).Decode(&noteResource)
if err != nil {
panic(err)
}
note := noteResource.Note
//get a new Id
obj_id := bson.NewObjectId()
note.Id = obj_id
note.CreatedOn = time.Now()
//Insert into document collection
err = collection.Insert(&note)
if err != nil {
panic(err)
} else {
log.Printf("Inserted New Record with Title :%s", note.Title)
}
j, err := json.Marshal(models.NoteResource{Note: note})
if err != nil {
panic(err)
}
w.Header().Set("Content-Type", "application/json")
w.Write(j)
}
func (uc NoteController) UpdateNote(w http.ResponseWriter, r *http.Request) {
var err error
//get id from incoming url
vars := mux.Vars(r)
id := bson.ObjectIdHex(vars["id"])
//decode the incoming Note into json
var noteResource models.NoteResource
err = json.NewDecoder(r.Body).Decode(&noteResource)
if err != nil {
panic(err)
}
//partial update on mongodb
err = collection.Update(bson.M{"_id": id},
bson.M{"$set": bson.M{
"title": noteResource.Note.Title,
"decription": noteResource.Note.Description,
}})
if err == nil {
log.Printf("Updated Note : %s", id, noteResource.Note.Title)
} else {
panic(err)
}
w.WriteHeader(http.StatusNoContent)
}
func (uc NoteController) DeleteNote(w http.ResponseWriter, r *http.Request) {
var err error
vars := mux.Vars(r)
id := vars["id"]
//Remove from database
err = collection.Remove(bson.M{"_id": bson.ObjectIdHex(id)})
if err != nil {
log.Printf("Could not find the Note %s to delete", id)
}
w.WriteHeader(http.StatusNoContent)
}
models/note.go
package models
import (
"time"
"gopkg.in/mgo.v2/bson"
)
type Note struct {
Id bson.ObjectId `bson:"_id" json:"id"`
Title string `json:"title"`
Description string `json:"description"`
CreatedOn time.Time `json:"craetedOn"`
}
type NoteResource struct {
Note Note `json:"note"`
}
type NotesResource struct {
Notes []Note `json:"notes"`
}
Not an programming expert but this is how I manage my routes/handlers.
routes/routes.go
package routes
import (
"github.com/gorilla/mux"
)
//NewRouter is main routing entry point
func NewRouter() *mux.Router {
r := mux.NewRouter()
indexHandler(r) // Index handler
fileServer(r) // Fileserver to serve static files
otherLogicalHandler(r) // Other domain/business logic scoped handler
return r
}
routes/indexHandler.go
package routes
import (
"fmt"
"net/http"
"github.com/gorilla/mux"
"github.com/myusername/project/models"
)
func indexHandler(r *mux.Router) {
r.HandleFunc("/", indexMainHandler).Methods("GET")
// Other endpoints goes there if you want to list it in this current indexHandler.go file
// Example: r.HandleFunc("/signup", signupMainHandler).Methods("GET")
}
// Handlers
func indexMainHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=UTF-8")
// Call your model/s there
mydata, err := models.GetMyDataFunction()
if err != nil {
// Handle your error there
return
}
utils.ExecuteTemplate(w, "index.html", struct {
Title string
// Use your model data for templates there
MyData []models.MyData
// Other models/data can go there if multiple data objects used per page.
}{
Title: "Main Page",
MyData: mydata,
})
}
// func signupMainHandler(w http.ResponseWriter, r *http.Request) ...
// Basically repeat the same logic as in indexMainHandler function
routes/fileServer.go
package routes
import (
"net/http"
"github.com/gorilla/mux"
)
func fileServer(r *mux.Router) {
fs := http.FileServer(http.Dir("static"))
r.PathPrefix("/static/").Handler(http.StripPrefix("/static/", fs))
}
routes/otherLogicalHandler.go
... and so on.
As you can see, all they belong to package routes but are divided into multiple files. File names doesn't actually matter. You can name them as you want.
Models lives in models directory and also belongs to single package models package.
Every time you create new routes file, remember to call it in routes.go file.
Hope this will help for somebody.