I'm having a bit of trouble creating a query using the bson functionality of mgo. I'm simply trying to do {'search_id': {'$in': [1,2,4,7,9]}}, but I can't work out how to do it in mgo.
I have a slice of ints, and tried passing that directly:
toRemove := []int{1,2,4,7,9}
err = coll.Remove(bson.M{"search_id": bson.M{"$in": toRemove}})
I saw another post which suggested I needed to use []interface{}, but that doesn't work either:
toRemoveI := make([]interface{}, len(toRemove))
for idx, val := range toRemove {
toRemoveI[idx] = val
}
err = coll.Remove(bson.M{"search_id": bson.M{"$in": toRemoveI}})
I've looked through he docs and other questions here and on gh, but most questions involving slices seem to be about getting data into a slice as opposed to what I'm trying to achieve.
Any help would be most appreciated.
Your original proposal (passing an []int value) has no flaws, it's valid to do that.
What the problem is is that you use Collection.Remove() which finds and removes a single document matching the provided selector document. So your proposed solution will remove exactly 1 document, one whose search_id is contained in the slice you passed. If no such document is found (and the session is in safe mode, see Session.SetSafe()), mgo.ErrNotFound is returned.
Instead use Collection.RemoveAll() which finds and removes all documents matching the selector:
toRemove := []int{1,2,4,7,9}
info, err := c.RemoveAll(bson.M{"search_id": bson.M{"$in": toRemove}})
if err != nil {
log.Printf("Failed to remove: %v", err)
} else {
log.Printf("Removed %d documents.", info.Removed)
}
Related
suppose I have code like this:
var result bson.M
err := coll.FindOne(context.TODO(), filter).Decode(&result)
if err != nil {
panic(err)
}
// somehow I can change the _id before I do insertOne
if _, err := coll.InsertOne(context.TODO(), result); err != nil {
panic(err)
}
is there a way I can insertOne without knowing the data struct? But I have to change the _id before I insert it.
The ID field in MongoDB is always _id, so you may simply do:
result["_id"] = ... // Assign new ID value
Also please note that bson.M is a map, and as such does not retain order. You may simply change only the ID and insert the clone, but field order may change. This usually isn't a problem.
If order is also important, use bson.D instead of bson.M which retains order, but finding the _id is a little more complex: you have to use a loop as bson.D is not a map but a slice.
This is how it could look like when using bson.D:
var result bson.D
// Query...
// Change ID:
for i := range result {
if result[i].Key == "_id" {
result[i].Value = ... // new id value
break
}
}
// Now you can insert result
Struggling with unmarshalling data in Golang from mongo, may be cause I am new to this. Just started learning golang with MongoDB
Tried with map[string]interface{} to avoid any struct related errors
var data map[string]interface{}
filter := bson.M{"profile.username": username}
singleResult := u.getCollection(client).FindOne(u.ctx, filter)
err := singleResult.Decode(data)
This fails to unmarshall with error cannot Decode to nil value
Tried with exact struct structure too.
var result *models.UserData
filter := bson.M{"profile.username": username}
singleResult := u.getCollection(client).FindOne(u.ctx, filter)
err := singleResult.Decode(result)
Fails with same error cannot Decode to nil value
Tried to find all with map[string]interface{}
var result []models.UserData
cursor, _ := u.getCollection(client).Find(u.ctx, bson.M{})
err := cursor.All(u.ctx, &result)
Works perfectly as expected
Tried to find all with exact struct structure
var data []map[string]interface{}
cursor, _ := u.getCollection(client).Find(u.ctx, bson.M{})
err := cursor.All(u.ctx, &result)
Works perfectly as expected
Now I thought may be I am not finding the data in mongo but then
filter := bson.M{"profile.username": username}
singleResult := u.getCollection(client).FindOne(u.ctx, filter)
raw, _ := singleResult.DecodeBytes()
log.Print("\n\n" + raw.String()+"\n\n")
This prints the data as expected. Although one thing I noticed all non-string values are formatted as {"$numberLong":"1"}. Still don't know if it is correct or cause of the issue.
In your first 2 examples that fail, the data passed to Decode() are both nil:
// data == nil
var data map[string]interface{}
// ...
// result == nil
var result *models.UserData
Try like
var result = &models.UserData{} // init the pointer with a block of valid allocated memory
// ...
err := singleResult.Decode(result)
In order for Decode() to write the document(s) into the passed value, it must be a (non-nil) pointer. Passing any value creates a copy, and if you pass a non-pointer, only the copy could be modified. If you pass a pointer, a copy is still made, but Decode() will modify the pointed value, not the pointer.
In your first 2 examples that fail, you pass a non-pointer (or a nil pointer):
err := singleResult.Decode(result)
Modify it to pass a (non-nil) pointer:
err := singleResult.Decode(&result)
Your last 2 examples work because you're already passing (non-nil) pointers:
err := cursor.All(u.ctx, &result)
I'm stuck at a probably simple problem: If I filter this in mongodb compass (filter {dateTime:{$gt: new Date("2020-11-23T12:31:38")}}):
It returns 556 documents.
Trying to create a cursor in Go that have those documents is proving to be quite hard!
I've this right now:
cursor, err := coll.Find(context.Background(), bson.M{"dateTime": bson.M{"$gt": "new Date("+ date + ")"}}, opt)
if err != nil {
fmt.Println("Err creting database: ", err)
return nil, err
}
if cursor.Next(context.Background()) {
fmt.Println("Cursor0!")
cursor.Next(context.Background())
}
cursor1, err := coll.Find(context.Background(), bson.M{}, opt)
if err != nil {
fmt.Println("Err creting database: ", err)
return nil, err
}
if cursor1.Next(context.Background()) {
fmt.Println("Cursor1!")
cursor.Next(context.Background())
}.
I've tried, along other different tries, to put the filter just as bson.M{"dateTime": bson.M{"$gt": date}}, along other similar tryes, but they also returned 0 documents. The date variable have exacly the date used in the mongodb compass filter.
I created another cursor, with no filter, just to control if the connection with mongo is ok, and to see if it returns any documents when it has no filter, and it does return documents. Does anyone knows the answer to this one?
Thanks!!
new Date("2020-11-23T12:31:38") is JavaScript syntax. You need to use the proper Go syntax for creating timestamps.
The problem was that I was dealling with more than 1 Collection, and in one the date was saved as string, and in the other, as date. In the one that the date s saved as string, no surprise, we have to send the date as string too, some logic to when date is in mongo as Date
I'm just getting started with golang and I'm attempting to read several rows from a Postgres users table and store the result as an array of User structs that model the row.
type User struct {
Id int
Title string
}
func Find_users(db *sql.DB) {
// Query the DB
rows, err := db.Query(`SELECT u.id, u.title FROM users u;`)
if err != nil { log.Fatal(err) }
// Initialize array slice of all users. What size do I use here?
// I don't know the number of results beforehand
var users = make([]User, ????)
// Loop through each result record, creating a User struct for each row
defer rows.Close()
for i := 0; rows.Next(); i++ {
err := rows.Scan(&id, &title)
if err != nil { log.Fatal(err) }
log.Println(id, title)
users[i] = User{Id: id, Title: title}
}
// .... do stuff
}
As you can see, my problem is that I want to initialize an array or slice beforehand to store all the DB records, but I don't know ahead of time how many records there are going to be.
I was weighing a few different approaches, and wanted to find out which of the following was most used in the golang community -
Create a really large array beforehand (e.g. 10,000 elements). Seems wasteful
Count the rows explicitly beforehand. This could work, but I need to run 2 queries - one to count and one to get the results. If my query is complex (not shown here), that's duplicating that logic in 2 places. Alternatively I can run the same query twice, but first loop through it and count the rows. All this would work, but it just seems unclean.
I've seen examples of expanding slices. I don't quite understand slices well enough to see how it could be adapted here. Also if I'm constantly expanding a slice 10k times, it definitely seems wasteful.
Go has a built-in append function for exactly this purpose. It takes a slice and one or more elements and appends those elements to the slice, returning the new slice. Additionally, the zero value of a slice (nil) is a slice of length zero, so if you append to a nil slice, it will work. Thus, you can do:
type User struct {
Id int
Title string
}
func Find_users(db *sql.DB) {
// Query the DB
rows, err := db.Query(`SELECT u.id, u.title FROM users u;`)
if err != nil {
log.Fatal(err)
}
defer rows.Close()
var users []User
for rows.Next() {
err := rows.Scan(&id, &title)
if err != nil {
log.Fatal(err)
}
log.Println(id, title)
users = append(users, User{Id: id, Title: title})
}
if err := rows.Err(); err != nil {
log.Fatal(err)
}
// ...
}
User appending to a slice:
type DeviceInfo struct {
DeviceName string
DeviceID string
DeviceUsername string
Token string
}
func QueryMultiple(db *sql.DB){
var device DeviceInfo
sqlStatement := `SELECT "deviceName", "deviceID", "deviceUsername",
token FROM devices LIMIT 10`
rows, err := db.Query(sqlStatement)
if err != nil {
panic(err)
}
defer rows.Close()
var deviceSlice []DeviceInfo
for rows.Next(){
rows.Scan(&device.DeviceID, &device.DeviceUsername, &device.Token,
&device.DeviceName)
deviceSlice = append(deviceSlice, device)
}
fmt.Println(deviceSlice)
}
I think that what you are looking for is the capacity.
The following allocates an array that can hold 10,000 items:
users := make([]User, 0, 10_000)
but the array is, itself, still empty (len(users) == 0).
Now you can add up to at least 10,000 items before the array needs to be grown. For that purpose the append() works as expected:
users = append(users, User{...})
Maps are grown with a x2 of the size starting with 1. So it remains a power of two. I'm not sure whether slices are grown the same way (in powers of two). If so, then the allocated size above would be:
math.Pow(2, math.Ceil(math.Log(10_000)/math.Log(2)))
which is 2^14 which is 16,384.
Note: if your query uses a compatible INDEX, i.e. the WHERE clause matches the INDEX one to one, then an extra SELECT COUNT(*) ... is free since the number of elements is known and it will return that number without the need to scan all the rows of your tables.
Mgo and golang question.
I run into problem again. I try to update record in the database, but running simple command visitors.UpdateId(v.Id, bson.M{"$set": zscore}); where zscore is a variable of type Zscore, does not work. However if I manually convert zscore to bson.M structure, everything works fine.
Does anybody know how to update the record in mongodb using mgo, without manually dumping structure values into bson.M?
Example:
type Zscore struct {
a float64 `bson:"a,omitempty" json:"a"`
b float64 `bson:"b,omitempty" json:"b"`
c float64 `bson:"c,omitempty" json:"c"`
}
v := Visitor{}
zscore := Zscore{}
visitors := updater.C("visitors")
for result.Next(&v) {
zscore.a = 1
zscore.b = 2
zscore.c = 0
//does not work
if err := visitors.UpdateId(v.Id, bson.M{"$set": zscore}); err != nil {
log.Printf("Got error while updating visitor: %v\n", err)
}
//works
set := bson.M{
"zscore.a": zscore.a,
"zscore.b": zscore.b,
"zscore.c": zscore.c,
}
if err := visitors.UpdateId(v.Id, bson.M{"$set": set}); err != nil {
log.Printf("Got error while updating visitor: %v\n", err)
}
}
All Go marshaling packages I'm aware of, including the bson package, will not marshal fields that are private (start with a lowercase letter). To fix the issue, just export the relevant fields by uppercasing the first letter of their name.
Also note that, besides the issue mentioned above, the first part of your example will not marshal in an equivalent way to the second part. bson.M{"$set": zscore} is equivalent to bson.M{"$set": bson.M{"a": ... etc ...}}.