I have this fairly simple Chat object:
{
_id: ObjectId('a4bd8c2f5g19b0a1d1'),
chatId: 1,
userId1: 100,
userId2: 234,
messages: [
{msgId: 1, senderId: 100, msgSentOnUnix: 1652046779, msgReadOnUnix: 1652046787, content: 'hello'},
{msgId: 2, senderId: 100, msgSentOnUnix: 1652046786, msgReadOnUnix: 1652046787, content: 'world'},
{msgId: 3, senderId: 234, msgSentOnUnix: 1652046795, msgReadOnUnix: -1, content: 'right back...'},
{msgId: 4, senderId: 234, msgSentOnUnix: 1652046802, msgReadOnUnix: -1, content: 'at you'},
]
}
It's a chat between 2 people, with a chatId, both users' IDs, and all their messages.
I wish to update many sub-documents, and get the count of how many sub-documents were updated, in one single update query.
To be more specific:
I want to update user XXX's messages as "read" once user YYY enters the "chatroom", and I want to know how many "unread" messages were there. I don't want to do this with 2 queries, as I am concerned with concurrency issues...
Right now, for the update alone (without the count) I use this update command:
db.chats.updateOne(
{
userID1: 100,
userID2: 234,
},
{
$set: {
'messages.$[i].msgReadOnUnix': $$NOW
}
},
{
arrayFilters: [
{
$and: [
{ 'i.senderID': 234 }, // user 100 is reading users 234's messages
{ 'i.msgReadOnUnix': -1 }, // -1 Marks that a message is unread
]
}
]
},
)
Could a count be returned as a result here? to state how many sub-documents were modified?
So I eventually overcame the issue the way #prasad_ had mentioned,
the approach of running the test twice!
Once inside mongoDB, and once on the application level.
I.E. I used findOneAndUpdate({}) to return the document, without the { new: true } property, in order to get the document prior to the update, and then ran the same check that mongo ran:
(i am using node JS)
const { error, errorMsg, data } = await Chat.findOneAndUpdate(
{
userID1: 100,
userID2: 234,
},
{
$set: {
'messages.$[i].msgReadOnUnix': new Date().getTime(),
},
},
{
arrayFilters: [
{
$and: [{ 'i.senderID': 234}, { 'i.msgReadOnUnix': -1 }],
},
],
},
)
.lean()
.then((resultSet) => ({ error: false, data: resultSet }))
.catch((errorMsg) => ({ error: true, errorMsg }));
// running the check again here on the application level ↓
for(let i=0; i < ... ; i++){
// write check logic here and count
}
Since it's the same document coming in it solves the concurrency issue that I was so afraid of. While this works great, can anyone think of a better way? One where I don't have to run my check twice?
Feels like there should be a better way query-wise to be able to solve this...
Related
It's been 2 days (or nights should I say) since I am trying to figure out following so would appreciate your help guys.
in mongodb I have number of orders (I will simplify documents for the case).
I want to group all documents by $campRoundId and where there are installments, push all object to installments variable.
The problem I am facing is when document.installments array is empty it pushes it to the array too.
Initial documents (same campRoundId to group by - first with no instalmments, second with 2)
[
{
_id: ObjectId("62792d8a519af6ae8cdff779"),
campRoundId: ObjectId("620a790b2cbc52006c83115a"),
installments: [],
},
{
_id: ObjectId("62792d8a519af6ae8cdff77a"),
campRoundId: ObjectId("620a790b2cbc52006c83115a"),
installments: [
{
payment: 100,
paymentStatus: false,
},
{
payment: 20,
paymentStatus: false,
},
],
},
];
my aggregation
/**
* _id: The id of the group.
* fieldN: The first field name.
*/
{
_id : "$campRoundId",
installments: {
$push: {
$cond:[
{ $gte: ["$installments.length", 1] },
"$installments",
null
]
}
}
}
I want to get rid of empty object, so if there are no installments nothing will be pushed. (dotted lines)
I have a document which is structured like this:
{
'item_id': '12345'
'total_score': 100,
'user_scores': {
'ABC': 40,
'DEF': 60
}
}
I'm using PyMongo, but documentation of MongoDB seems easily translatable across different distributions. With PyMongo, I could update user scores with:
collection.update_one(
{ 'item_id': '12345' },
{ '$set': { 'user_scores.GHI': 20 } },
upsert=True
)
Which results in this:
{
'item_id': '12345'
'total_score': 100,
'user_scores': {
'ABC': 40,
'DEF': 60,
'GHI': 20
}
}
The issue is of course that the total_score is now incorrect. I want that total score to update, so that in a future query, I can quickly ascertain the score of each result, and even sort by score.
One solution could be to find an existing document using find_one({'item_id: '12345'}), (create if it doesn't exist), then update with new scores, and update total score. The problem there is that I want to run thousands of these at the same time, and it's far more efficient to call bulk_write on a series of requests.
So, a better solution would be to do two sequential update requests:
request1 = UpdateOne(
{ 'item_id' : '12345' },
{ '$set': { 'user_scores.GHI': 20 } },
upsert = True
)
request2 = UpdateOne(
{ 'item_id' : '12345' },
{ '$set': { 'total_score': { '$sum': { '$values': 'user_scores' } } } },
upsert = True
)
The first request updates the user scores, same as before. The second request, there are two concepts going on. The syntax for this isn't correct, but here's what I'm trying to do:
I need to get the values from the user_scores dictionary. { '$values': 'user_scores' } is how I've tried to convey this.
That gives me an array of values. I know these are all numeric, so I now need to sum those, conveyed with { '$sum': { '$values': 'user_scores' } }.
I can run these batch updates consecutively, so there's no risk of summing the wrong thing. The danger with having a total_score field will always be that it isn't updated and thus doesn't contain the correct number. I'd imagine this is a common case with document-based models?
If you're using Mongo version 4.2+ they introduced a new feature: pipelined updates, Meaning now you can do what you want in one go:
db.collection.updateOne({ 'item_id' : '12345' },
[
{ '$set': { 'user_scores.GHI': 20 } },
{ '$set': { 'total_score': { '$sum': [ "$user_scores.GHI", "$user_scores.ABC", "$user_scores.GHI"] } } },,
]);
Unfortunately this is not possible for lesser Mongo versions hence if that is the case you'll have to keep using your solution which is splitting this into 2 actions.
EDIT:
For dynamic update we can use $map and $objectToArray like so:
db.collection.updateOne(
{'item_id': '12345'},
[
{'$set': {'user_scores.GHI': 20}},
{
'$set':
{
'total_score': {
'$sum': {
'$map': {
'input': {'$objectToArray': '$user_scores'},
'as': 'score',
'in': '$$score.v'
}
}
}
}
}
]);
Given a collection of Users:
db.users.insertMany(
[
{
_id: 1,
name: "sue",
points: [
{ points: 85, bonus: 20 },
{ points: 85, bonus: 10 }
]
},
{
_id: 2,
name: "bob",
points: [
{ points: 85, bonus: 20 },
{ points: 64, bonus: 12 }
]
}]);
How do I add an attribute bonus_raw in every points, with a copy of the value of bonus value? I tried:
db.getCollection('users').update({ },
{$set:{ 'points.$.bonus_raw' : 'points.$.bonus' }}, false, true)
but I get:
The positional operator did not find the match needed from the query. Unexpanded update: points.$.bonus_raw
Updating multiple items in an array is not possible as of now in MongoDB.
To get this done, you will have to query the document, loop over all of your nested documents, and then save it back to MongoDB.
In your case, this can help:-
db.users.find({points: { $exists: true } }).forEach(function (doc){
doc.points.forEach(function (points) {
points.bonus_raw = points.bonus;
});
db.users.save(doc)
});
Also, take care of race conditions while doing an update in this way. See this
I made a csv import from a huge ms excel sheet table. So the collection is one level. I need to change a few column so they are 2 levels.
Example, this is from the csv-import.
{
title: 'House A',
energyElectricity: 55,
energyHeat: 35,
energyCooling:45
}
This is not good. I want this in the following format:
{
title: 'House A',
energy: {
electricity: 55,
heat: 35,
cooling:45
}
}
Is there anyway to do this with an update query?
I tried some stuff but no luck.
Some pseudo code here:
db.consumers.update({}, {energy.electricity: energyElectricity, energy.heat:energyHeat}, {multi:true});
There really is no other way to do this other than looping the results as it is presently not possible to refer to any existing fields of a document during an update operation.
So your basic construct needs to look something like ( in whatever language ):
db.collection.find({}).forEach(function(doc) {
db.collection.update(
{ "_id": doc._id },
{
"title": doc.title,
"energy": {
"electricity": doc.energyElectricty,
"heat": doc.energyHeat,
"cooling": doc.energyCooling
}
}
);
});
You could do this a little more efficiently with "bulk updates" as available from MongoDB 2.6 and upwards:
var batch = [];
var count = 0;
db.collection.find({}).forEach(function(doc) {
batch.push({
"q": { "_id": doc._id },
"u": {
"title": doc.title,
"energy": {
"electricity": doc.energyElectricty,
"heat": doc.energyHeat,
"cooling": doc.energyCooling
}
}
});
count++;
if ( count % 500 == 0 ) {
db.runCommand({ "update": "collection", "updates": batch });
batch = [];
}
});
if ( batch.length > 0 ) {
db.runCommand({ "update": "collection", "updates": batch });
}
So while all updates are still being done over the wire, this does actually only send over the wire once per 500 ( or how many your feel comfortably sits under the 16MB BSON limit ) items.
Of course though, since you mention this came from a CSV import, you can always re-shape your input and import the collection again if that turns out to be a reasonable option.
Consider the following posts collection:
{
_id: 1,
title: "Title1",
category: "Category1",
comments: [
{
title: "CommentTitle1",
likes: 3
},
{
title: "CommentTitle2",
likes: 4
}
]
}
{
_id: 2,
title: "Title2",
category: "Category2",
comments: [
{
title: "CommentTitle3",
likes: 1
},
{
title: "CommentTitle4",
likes: 4
}
]
}
{
_id: 3,
title: "Title3",
category: "Category2",
comments: [
{
title: "CommentTitle5",
likes: 1
},
{
title: "CommentTitle6",
likes: 3
}
]
}
I want to retrieve all the posts, and if one post has a comment with 4 likes I want to retrieve this comment only under the "comments" array. If I do this:
db.posts.find({}, {comments: { $elemMatch: {likes: 4}}})
...I get this (which is exactly what I want):
{
_id: 1,
comments: [
{
title: "CommentTitle2",
likes: 4
}
]
}
{
_id: 2,
comments: [
{
title: "CommentTitle4",
likes: 4
}
]
}
{
_id: 3
}
But how can I retrieve the remaining fields of the documents without having to declare each of them like below? This way if added more fields to the post document, I wouldn't have to change the find query
db.posts.find({}, {title: 1, category: 1, comments: { $elemMatch: {likes: 4}}})
Thanks
--EDIT--
Sorry for the misread of your question. I think you'll find my response to this question here to be what you are looking for. As people have commented, you cannot project this way in a find, but you can use aggregation to do so:
https://stackoverflow.com/a/21687032/2313887
The rest of the answer stands as useful. So I think I'll leave it here
You must specify all of the fields you want or nothing at all when using projection.
You are asking here essentially that once you choose to alter the output of the document and limit how one field is displayed then can I avoid specifying the behavior. The bottom line is thinking of the projection part of a query argument to find just like SQL SELECT.It behaves in that * or all is the default and after that is a list of fields and maybe some manipulation of the fields format. The only difference is for _id which is always there by default unless specified otherwise by excluding it, i.e { _id: 0 }
Alternately if you want to filter the collection you nee to place your $elemMatch in thequery itself. The usage here in projection is to explicitly limit the returned document to only contain the matching elements in the array.
Alter your query:
db.posts.find(
{ comments: { $elemMatch: {likes: 4}}},
{ title: 1, category: 1, "comments.likes.$": 1 }
)
And to get what you want we use the positional $ operator in the projection portion of the find.
See the documentation for the difference between the two usages:
http://docs.mongodb.org/manual/reference/operator/query/elemMatch/
http://docs.mongodb.org/manual/reference/operator/projection/elemMatch/
This question is pretty old, but I just faced the same issue and I didn't want to use the aggregation pipeline as it was simple query and I only needed to get all fields applying an $elemMatch to one field.
I'm using Mongoose (which was not the original question but it's very frequent these days), and to get exactly what the question said (How can I retrieve all the fields when using $elemMatch?) I made this:
const projection = {};
Object.keys(Model.schema.paths).forEach(key => {
projection[key] = 1;
});
projection.subfield = { $elemMatch: { _id: subfieldId } };
Model.find({}, projection).then((result) => console.log({ result });