MongoDB $addToSet inside another $addToSet - mongodb

I am trying to achieve the below response from query.
{
users:[
user:{
name:"",
email:[
]
},
....user2
]
}
I have something similar to below.
I will put this way. One user can have n number of devices. He may have more than one email per device. I want to group at devices. And user field should have common information for that device. As we think, name will always be same. Device specific attributes also needed like whatKindOfDevice, howManyIssuesAreThereInThatDevice, howManyCanBeAddressedByUpgrade, howManyAreRare,etc.. along with this i need to get all the emails used in that device like owner,user,associate - all emails put.
Think my document Id is not associated with single user, single device. One user can have any number of devices. One device can have n Number of documents.
$group:{
"_id":"user_device_id",
"user": {
"$addToSet": {
"name":"$name",
"deviceName":"$deviceName",
"email": {"$addToSet":{}} //Something Similar I am expecting
}
}
}
If I add email outer user it works - but it affects the response format required.
Is it possible or any other way to get it the similar response through the query?
Let's assume one user can have more than one document. In each doc, there could be same or duplicate email IDs. I am trying to get that together.
Please advise.
Sample Doc:
{
_id:ObjectId,
name:"user1",
email:"a#yahoo.com"
},
{
_id:ObjectId,
name:"user1",
email:"a#device.com"
},
..user2Doc
..user1Doc with another category, duplicate email i.e a#yahoo.com
..user2Doc with new email
..

Well, it seems like you want to get all the email for the particular user and then group all the users.
So, to achieve that you have to do consecutive grouping stages.
I used the below documents:
[
{
name:"user1",
email:"a#yahoo.com"
},
{
name:"user1",
email:"a#device.com"
},
{
name:"user2",
email:"b#yahoo.com"
},
{
name:"user1",
email:"c#device.com"
}
]
Here is the query:
db.collection.aggregate([
{
$group:{
"_id":"$name",
"emails":{
$addToSet:"$email"
},
"name":{
$first:"$name"
}
}
},
{
$group:{
"_id":null,
"users":{
$addToSet:{
"name":"$name",
"email":"$emails"
}
}
}
},
{
$project:{
"_id":0
}
}
]).pretty()
Output:
{
"users" : [
{
"name" : "user1",
"email" : [
"c#device.com",
"a#device.com",
"a#yahoo.com"
]
},
{
"name" : "user2",
"email" : [
"b#yahoo.com"
]
}
]
}
For mare about $group refer here.
Hope this will help :)

Related

How to extract grouped results from array in $group stage and return as separate fields?

I'm running an aggregation query, and the $group stage is as follows
$group:
{
_id:
{
year_month: { $dateToString: { "date": "$updated_at", "format": "%Y-%m" } }
,client_name: "$clients_docs.client_name"
,client_label: "$clients_docs.client_label"
,client_code: "$clients_docs.client_code"
,client_country: "$clients_docs.client_country"
,base_curr: "$clients_docs.client_base_currency"
,inv_curr: "$clients_docs.client_invoice_currency"
,dest_curr: "$store.destination_currency"
}
,total_vol: { $sum: "$USD_Value" }
,total_tran: { $sum: 1 }
}
It returns the correct results, and returns all the grouped results in the _id:{} array.
I now want to extract all those fields from the array and return them not within the array so I can more easily export the output to a spreadsheet.
I tried using this stage:
{
$project:
{
year_month: 1
,client_name: 1
,client_label: 1
,client_code: 1
,client_country: 1
,base_curr: 1
,inv_curr: 1
,dest_curr: 1
,total_vol: 1
,total_tran : 1
}
},
But that returned the same results as the $group stage:
{
"_id" : {
"year_month" : "2022-01",
"client_name" : "client A",
"client_label" : "client A",
"client_code" : NumberInt(0000),
"client_country" : "TH",
"base_curr" : "USD",
"inv_curr" : "USD",
"dest_curr" : "HKD"
},
"total_vol" : 100000,
"total_tran" : 100.0
}
I want the "year_month" through "dest_curr" fields at the same level as the "total_vol" and "total_tran", so that when the data is exported they all appear as separate columns (now it's all captured as one "_id" column, and a "total_vol" and "total_tran" column). What's the best way to do this?
From a terminology perspective, you currently have an embedded document (or nested fields) rather than an array.
The straightforward way to do this is to simply enumerate each field, eg:
"year_month": "$_id.year_month",
There are fancier ways to do this, but as you only have a handful of fields this should suffice. Working playground example here.
Edit
An alternative ("fancier") approach is to leverage the $replaceWith stage using the $mergeObjects operator inside of it. Then you can $unset the previous _id field afterwards. It would look like this:
db.collection.aggregate([
{
"$replaceWith": {
"$mergeObjects": [
"$$ROOT",
"$_id"
]
}
},
{
$unset: "_id"
}
])
Playground link here
I also fixed the earlier playground link that had a typo for the client_label field.

How to query an object in an array embedded in mongodb?

I am currently working on an application that takes control of Projects, which have Meetings and that these meetings have Participants.
I want to consult a Participant by his nomina field.
Structure for a project document object:
{
"id":"5c1b0616a0441f27f022bfdc",
"name":"Project Test",
"area":"Area",
"date":"2019-01-01",
"meetings":[
{
"id":"5c1b073d445707834699ce97",
"objetive":"Objetive",
"fecha":"2019-01-01",
"participants":[
{
"nomina":1,
"name":"Person 1",
"role":"Rol1",
"area":"area1",
"signature":null
},
{
"nomina":2,
"name":"Person 2",
"role":"rol 2",
"area":"área 2",
"signature":null
}
]
}
]
}
Expected behavior
I want to consult a Participant by nomina field knowing the id of the Project and also knowing the id of the Meeting.
Expected output
Having:
id Project = 5c1b0616a0441f27f022bfdc
id Meeting = 5c1b073d445707834699ce97
nomina Participant = 1
It's expected that the query will return me:
{
"nomina":1,
"name":"Person 1",
"role":"Rol1",
"area":"area1",
"signature":null
}
For not so huge number of meetings in every document if you want to get the exact document stated, you can do this pipeline, it is straight forward:
db.collection.aggregate(
[
{
$match: {
id:"5c1b0616a0441f27f022bfdc"
}
}, {
$unwind: {
path : "$meetings"
}
},
{
$unwind: {
path : "$meetings.participants"
}
},
{
$match: {
"meetings.id":"5c1b073d445707834699ce97",
"meetings.participants.nomina":1
}
},
{
$replaceRoot: {
newRoot: "$meetings.participants"
}
}
]);
If you would have over thousands of elements in meetings then I'd suggest adding another match to meetings or grouping meetings and project IDs.
But if you just want to get the document containing what you want it is just a simple find query:
db.collection.find({id:"5c1b0616a0441f27f022bfdc","meetings.id":"5c1b073d445707834699ce97","meetings.participants.nomina":1 });

How is findById() + save() different from update() in MongoDB

While trying to update a MongoDB document using Mongoose, can I use a findById() with a save() in the callback, or should I stick with traditional update methods such as findByIdAndModify, findOneAndModify, update(), etc.? Say I want to update the name field of the following document (please see a more elaborate example in the edit at the end, which motivated my question):
{
"_id": ObjectId("123"),
"name": "Development"
}
(Mongoose model name for the collection is Category)
I could do this:
Category.update({ "_id" : "123" }, { "name" : "Software Development" }, { new: true })
or I could do this:
Category.findById("123", function(err, category) {
if (err) throw err;
category.name = "Software Development";
category.save();
});
For more elaborate examples, it feels easier to manipulate a JavaScript object that can simply be saved, as opposed to devising a relatively complex update document for the .update() operation. Am I missing something fundamentally important?
Edited 7/21/2016 Responding to the comment from #Cameron, I think a better example is warranted:
{
"_id": ObjectId("123"),
"roles": [{
"roleId": ObjectId("1234"),
"name": "Leader"
}, {
"roleId": ObjectId("1235"),
"name": "Moderator"
}, {
"roleId": ObjectId("1236"),
"name": "Arbitrator"
}]
}
What I am trying to do is remove some roles as well as add some roles in the roles array of sub-documents in a single operation. To add role sub-documents, $push can be used and to remove role sub-documents, $pull is used. But if I did something like this:
Person.update({
"_id": "123"
}, {
$pull : {
"roles" : {
"roleId" : {
$in : [ "1235", "1236" ]
}
}
},
$push : {
"roles" : {
$each: [{
"roleId" : ObjectId("1237"),
"name" : "Developer"
}]
}
}
}
When I try to execute this, I get the error Cannot update 'roles' and 'roles' at the same time, of course. That's when I felt it is easier to find a document, manipulate it any way I want and then save it. In that scenario, I don't know if there is really any other choice for updating the document.
I typically like to use findById() when I am performing more elaborate updates and don't think you are missing anything fundamentally important.
However one method to be aware of in mongoose is findByIdAndUpdate(), this issues a mongodb findAndModify update command and would allow you to perform your first example with the following code: Category.findByIdAndUpdate("123", function(err, savedDoc) {...}).

mongoDB find document greatest date and check value

I have a Conversation collection that looks like this:
[
{
"_id": "QzuTQYkGDBkgGnHrZ",
"participants": [
{
"id": "YymyFZ27NKtuLyP2C"
},
{
"id": "d3y7uSA2aKCQfLySw",
"lastVisited": "2016-02-04T02:59:10.056Z",
"lastMessage": "2016-02-04T02:59:10.056Z"
}
]
},
{
"_id": "e4iRefrkqrhnokH7Y",
"participants": [
{
"id": "d3y7uSA2aKCQfLySw",
"lastVisited": "2016-02-04T03:26:33.953Z",
"lastMessage": "2016-02-04T03:26:53.509Z"
},
{
"id": "SRobpwtjBANPe9hXg",
"lastVisited": "2016-02-04T03:26:35.210Z",
"lastMessage": "2016-02-04T03:15:05.779Z"
}
]
},
{
"_id": "twXPHb76MMxQ3MQor",
"participants": [
{
"id": "d3y7uSA2aKCQfLySw"
},
{
"id": "SRobpwtjBANPe9hXg",
"lastMessage": "2016-02-04T03:27:35.281Z",
"lastVisited": "2016-02-04T03:57:51.036Z"
}
]
}
]
Each conversation (document) can have a participant object with the properties of id, lastMessage, lastVisited.
Sometimes, depending on how new the conversation is, some of these values don't exist just yet (such as lastMessage, lastVisited).
What I'm trying to do is compare each participant in each individual conversation (document) and see if out of the all the participants, the greatest lastMessage field value belongs to the logged in user. Otherwise, I'm assuming that the conversation has messages that the logged in user hasn't seen yet. I want to get that count of messages that the user possibly hasn't seen yet.
In the example above, say we're logged in as d3y7uSA2aKCQfLySw. We can see that he was the last person to send a message for conversation 1, 2 BUT not 3. The count returning for how many updated conversations that d3y7uSA2aKCQfLySw hasn't seen should be 1.
Can someone point me in the right direction? I haven't the slightest clue as to how to approach the issue. My apologies for the lengthy question.
It is always advisable to store dates as ISODate rather than strings to leverage the flexibility provided by various date operators in the aggregation framework.
One way of getting the count is to,
$match the conversations in which the user is involved.
$unwind the participants field.
$sort by the lastMessage field in descending order
$group by the _id to get back the original conversations intact, and get the latest message per group(conversation) using the $first operator.
$project a field with value 0, for each group where the top most record is of the user we are looking for and 1 for others.
$group again to get the total count of the conversations in which he has not been the last one to send a message.
sample code:
var userId = "d3y7uSA2aKCQfLySw";
db.t.aggregate([
{
$match:{"participants.id":userId}
},
{
$unwind:"$participants"
},
{
$sort:{"participants.lastMessage":-1}
},
{
$group:{"_id":"$_id","lastParticipant":{$first:"$$ROOT.participants"}}
},
{
$project:{
"hasNotSeen":{$cond:[
{$eq:["$lastParticipant.id",userId]},
0,
1
]},
"_id":0}
},
{
$group:{"_id":null,"count":{$sum:"$hasNotSeen"}}
},
{
$project:{"_id":0,"numberOfConversationsNotSeen":"$count"}
}
])
I'd like to try this function.
function findUseen(uId) {
var numMessages = db.demo.aggregate(
[
{
$project: {
"participants.lastMessage": 1,
"participants.id": 1
}
},
{$unwind: "$participants"},
{$sort: {"participants.lastMessage": -1}},
{
$group: {
_id: "$_id",
participantsId: {$first: "$participants.id"},
lastMessage: {$max: "$participants.lastMessage"}
}
},
{$match: {participantsId: {$ne: uId}}},
]
).toArray().length;
return numMessages;
}
calling findUnseen("d3y7uSA2aKCQfLySw") will return 1.
I have adopted this function just to return count, but as you see it's easy to tweak it to return all unseen message metadata too.

MongoDB conditionally $addToSet sub-document in array by specific field

Is there a way to conditionally $addToSet based on a specific key field in a subdocument on an array?
Here's an example of what I mean - given the collection produced by the following sample bootstrap;
cls
db.so.remove();
db.so.insert({
"Name": "fruitBowl",
"pfms" : [
{
"n" : "apples"
}
]
});
n defines a unique document key. I only want one entry with the same n value in the array at any one time. So I want to be able to update the pfms array using n so that I end up with just this;
{
"Name": "fruitBowl",
"pfms" : [
{
"n" : "apples",
"mState": 1111234
}
]
}
Here's where I am at the moment;
db.so.update({
"Name": "fruitBowl",
},{
// not allowed to do this of course
// "$pull": {
// "pfms": { n: "apples" },
// },
"$addToSet": {
"pfms": {
"$each": [
{
"n": "apples",
"mState": 1111234
}
]
}
}
}
)
Unfortunately, this adds another array element;
db.so.find().toArray();
[
{
"Name" : "fruitBowl",
"_id" : ObjectId("53ecfef5baca2b1079b0f97c"),
"pfms" : [
{
"n" : "apples"
},
{
"n" : "apples",
"mState" : 1111234
}
]
}
]
I need to effectively upsert the apples document matching on n as the unique identifier and just set mState whether or not an entry already exists. It's a shame I can't do a $pull and $addToSet in the same document (I tried).
What I really need here is dictionary semantics, but that's not an option right now, nor is breaking out the document - can anyone come up with another way?
FWIW - the existing format is a result of language/driver serialization, I didn't choose it exactly.
further
I've gotten a little further in the case where I know the array element already exists I can do this;
db.so.update({
"Name": "fruitBowl",
"pfms.n": "apples",
},{
$set: {
"pfms.$.mState": 1111234,
},
}
)
But of course that only works;
for a single array element
as long as I know it exists
The first limitation isn't a disaster, but if I can't effectively upsert or combine $addToSet with the previous $set (which of course I can't) then it the only workarounds I can think of for now mean two DB round-trips.
The $addToSet operator of course requires that the "whole" document being "added to the set" is in fact unique, so you cannot change "part" of the document or otherwise consider it to be a "partial match".
You stumbled on to your best approach using $pull to remove any element with the "key" field that would result in "duplicates", but of course you cannot modify the same path in different update operators like that.
So the closest thing you will get is issuing separate operations but also doing that with the "Bulk Operations API" which is introduced with MongoDB 2.6. This allows both to be sent to the server at the same time for the closest thing to a "contiguous" operations list you will get:
var bulk = db.so.initializeOrderedBulkOp();
bulk.find({ "Name": "fruitBowl", "pfms.n": "apples": }).updateOne({
"$pull": { "pfms": { "n": "apples" } }
});
bulk.find({ "Name": "fruitBowl" }).updateOne({
"$push": { "pfms": { "n": "apples", "state": 1111234 } }
})
bulk.execute();
That pretty much is your best approach if it is not possible or practical to move the elements to another collection and rely on "upserts" and $set in order to have the same functionality but on a collection rather than array.
I have faced the exact same scenario. I was inserting and removing likes from a post.
What I did is, using mongoose findOneAndUpdate function (which is similar to update or findAndModify function in mongodb).
The key concept is
Insert when the field is not present
Delete when the field is present
The insert is
findOneAndUpdate({ _id: theId, 'likes.userId': { $ne: theUserId }},
{ $push: { likes: { userId: theUserId, createdAt: new Date() }}},
{ 'new': true }, function(err, post) { // do the needful });
The delete is
findOneAndUpdate({ _id: theId, 'likes.userId': theUserId},
{ $pull: { likes: { userId: theUserId }}},
{ 'new': true }, function(err, post) { // do the needful });
This makes the whole operation atomic and there are no duplicates with respect to the userId field.
I hope this helpes. If you have any query, feel free to ask.
As far as I know MongoDB now (from v 4.2) allows to use aggregation pipelines for updates.
More or less elegant way to make it work (according to the question) looks like the following:
db.runCommand({
update: "your-collection-name",
updates: [
{
q: {},
u: {
$set: {
"pfms.$[elem]": {
"n":"apples",
"mState": NumberInt(1111234)
}
}
},
arrayFilters: [
{
"elem.n": {
$eq: "apples"
}
}
],
multi: true
}
]
})
In my scenario, The data need to be init when not existed, and update the field If existed, and the data will not be deleted. If the datas have these states, you might want to try the following method.
// Mongoose, but mostly same as mongodb
// Update the tag to user, If there existed one.
const user = await UserModel.findOneAndUpdate(
{
user: userId,
'tags.name': tag_name,
},
{
$set: {
'tags.$.description': tag_description,
},
}
)
.lean()
.exec();
// Add a default tag to user
if (user == null) {
await UserModel.findOneAndUpdate(
{
user: userId,
},
{
$push: {
tags: new Tag({
name: tag_name,
description: tag_description,
}),
},
}
);
}
This is the most clean and fast method in the scenario.
As a business analyst , I had the same problem and hopefully I have a solution to this after hours of investigation.
// The customer document:
{
"id" : "1212",
"customerCodes" : [
{
"code" : "I"
},
{
"code" : "YK"
}
]
}
// The problem : I want to insert dateField "01.01.2016" to customer documents where customerCodes subdocument has a document with code "YK" but does not have dateField. The final document must be as follows :
{
"id" : "1212",
"customerCodes" : [
{
"code" : "I"
},
{
"code" : "YK" ,
"dateField" : "01.01.2016"
}
]
}
// The solution : the solution code is in three steps :
// PART 1 - Find the customers with customerCodes "YK" but without dateField
// PART 2 - Find the index of the subdocument with "YK" in customerCodes list.
// PART 3 - Insert the value into the document
// Here is the code
// PART 1
var myCursor = db.customers.find({ customerCodes:{$elemMatch:{code:"YK", dateField:{ $exists:false} }}});
// PART 2
myCursor.forEach(function(customer){
if(customer.customerCodes != null )
{
var size = customer.customerCodes.length;
if( size > 0 )
{
var iFoundTheIndexOfSubDocument= -1;
var index = 0;
customer.customerCodes.forEach( function(clazz)
{
if( clazz.code == "YK" && clazz.changeDate == null )
{
iFoundTheIndexOfSubDocument = index;
}
index++;
})
// PART 3
// What happens here is : If i found the indice of the
// "YK" subdocument, I create "updates" document which
// corresponds to the new data to be inserted`
//
if( iFoundTheIndexOfSubDocument != -1 )
{
var toSet = "customerCodes."+ iFoundTheIndexOfSubDocument +".dateField";
var updates = {};
updates[toSet] = "01.01.2016";
db.customers.update({ "id" : customer.id } , { $set: updates });
// This statement is actually interpreted like this :
// db.customers.update({ "id" : "1212" } ,{ $set: customerCodes.0.dateField : "01.01.2016" });
}
}
}
});
Have a nice day !