matching fields internally in mongodb - mongodb

I am having following document in mongodb
{
"_id" : ObjectId("517b88decd483543a8bdd95b"),
"studentId" : 23,
"students" : [
{
"id" : 23,
"class" : "a"
},
{
"id" : 55,
"class" : "b"
}
]
}
{
"_id" : ObjectId("517b9d05254e385a07fc4e71"),
"studentId" : 55,
"students" : [
{
"id" : 33,
"class" : "c"
}
]
}
Note: Not an actual data but schema is exactly same.
Requirement: Finding the document which matches the studentId and students.id(id inside the students array using single query.
I have tried the code like below
db.data.aggregate({$match:{"students.id":"$studentId"}},{$group:{_id:"$student"}});
Result: Empty Array, If i replace {"students.id":"$studentId"} to {"students.id":33} it is returning the second document in the above shown json.
Is it possible to get the documents for this scenario using single query?

If possible, I'd suggest that you set the condition while storing the data so that you can do a quick truth check (isInStudentsList). It would be super fast to do that type of query.
Otherwise, there is a relatively complex way of using the Aggregation framework pipeline to do what you want in a single query:
db.students.aggregate(
{$project:
{studentId: 1, studentIdComp: "$students.id"}},
{$unwind: "$studentIdComp"},
{$project : { studentId : 1,
isStudentEqual: { $eq : [ "$studentId", "$studentIdComp" ] }}},
{$match: {isStudentEqual: true}})
Given your input example the output would be:
{
"result" : [
{
"_id" : ObjectId("517b88decd483543a8bdd95b"),
"studentId" : 23,
"isStudentEqual" : true
}
],
"ok" : 1
}
A brief explanation of the steps:
Build a projection of the document with just studentId and a new field with an array containing just the id (so the first document it would contain [23, 55].
Using that structure, $unwind. That creates a new temporary document for each array element in the studentIdComp array.
Now, take those documents, and create a new document projection, which continues to have the studentId and adds a new field called isStudentEqual that compares the equality of two fields, the studentId and studentIdComp. Remember that at this point there is a single temporary document that contains those two fields.
Finally, check that the comparison value isStudentEqual is true and return those documents (which will contain the original document _id and the studentId.
If the student was in the list multiple times, you might need to group the results on studentId or _id to prevent duplicates (but I don't know that you'd need that).

Unfortunately it's impossible ;(
to solve this problem it is necessary to use a $where statement
(example: Finding embeded document in mongodb?),
but $where is restricted from being used with aggregation framework

db.data.find({students: {$elemMatch: {id: 23}} , studentId: 23});

Related

Add object to object array if an object property is not given yet

Use Case
I've got a collection band_profiles and I've got a collection band_profiles_history. The history collection is supposed to store a band_profile snapshot every 24 hour and therefore I am using MongoDB's recommended format for historical tracking: Each month+year is it's own document and in an object array I will store the bandProfile snapshot along with the current day of the month.
My models:
A document in band_profiles_history looks like this:
{
"_id" : ObjectId("599e3bc406955db4cbffe0a8"),
"month" : 7,
"tag_lowercased" : "9yq88gg",
"year" : 2017,
"values" : [
{
"_id" : ObjectId("599e3bc41c073a7418fead91"),
"profile" : {
"_id" : ObjectId("5989a65d0f39d9fd70cde1fe"),
"tag" : "9YQ88GG",
"name_normalized" : "example name1",
},
"day" : 1
},
{
"_id" : ObjectId("599e3bc41c073a7418fead91"),
"profile" : {
"_id" : ObjectId("5989a65d0f39d9fd70cde1fe"),
"tag" : "9YQ88GG",
"name_normalized" : "new name",
},
"day" : 2
}
]
}
And a document in band_profiles:
{
"_id" : ObjectId("5989a6190f39d9fd70cddeb1"),
"tag" : "9V9LRGU",
"name_normalized" : "example name",
"tag_lowercased" : "9v9lrgu",
}
This is how I upsert my documents into band_profiles_history at the moment:
BandProfileHistory.update(
{ tag_lowercased: tag, year, month},
{ $push: {
values: { day, profile }
}
},
{ upsert: true }
)
My problem:
I only want to insert ONE snapshot for every day. Right now it would always push a new object into the object array values no matter if I already have an object for that day or not. How can I achieve that it would only push that object if there is no object for the current day yet?
Putting mongoose aside for a moment:
There is an operation addToSet that will add an element to an array if it doesn't already exists.
Caveat:
If the value is a document, MongoDB determines that the document is a duplicate if an existing document in the array matches the to-be-added document exactly; i.e. the existing document has the exact same fields and values and the fields are in the same order. As such, field order matters and you cannot specify that MongoDB compare only a subset of the fields in the document to determine whether the document is a duplicate of an existing array element.
Since you are trying to add an entire document you are subjected to this restriction.
So I see the following solutions for you:
Solution 1:
Read in the array, see if it contains the element you want and if not push it to the values array with push.
This has the disadvantage of NOT being an atomic operation meaning that you could end up would duplicates anyways. This could be acceptable if you ran a periodical clean up job to remove duplicates from this field on each document.
It's up to you to decide if this is acceptable.
Solution 2:
Assuming you are putting the field _id in the subdocuments of your values field, stop doing it. Assuming mongoose is doing this for you (because it does, from what I understand) stop it from doing it like it says here: Stop mongoose from creating _id for subdocument in arrays.
Next you need to ensure that the fields in the document always have the same order, because order matters when comparing documents in the addToSet operation as stated in the citation above.
Solution 3
Change the schema of your band_profiles_history to something like:
{
"_id" : ObjectId("599e3bc406955db4cbffe0a8"),
"month" : 7,
"tag_lowercased" : "9yq88gg",
"year" : 2017,
"values" : {
"1": { "_id" : ObjectId("599e3bc41c073a7418fead91"),
"profile" : {
"_id" : ObjectId("5989a65d0f39d9fd70cde1fe"),
"tag" : "9YQ88GG",
"name_normalized" : "example name1"
}
},
"2": {
"_id" : ObjectId("599e3bc41c073a7418fead91"),
"profile" : {
"_id" : ObjectId("5989a65d0f39d9fd70cde1fe"),
"tag" : "9YQ88GG",
"name_normalized" : "new name"
}
}
}
Notice that the day field became the key for the subdocuments on the values. Notice also that values is now an Object instead of an Array.
No you can run an update query that would update values.<day> only if values.<day> didn't exist.
Personally I don't like this as it is using the fact that JSON doesn't allow duplicate keys to support the schema.
First of all, sadly mongodb does not support uniqueness of a field in an array of a collection. You can see there is major bug opened for 7 years and not closed yet(that is a shame in my opinion).
What you can do from here is limited and all is on application level. I had same problem and solve it in application level. Do something like this:
First read your document with document _id and values.day.
If your reading in step 1 returns null, that means there is no record on values array for given day, so you can push the new value(I assume band_profile_history has record with _id value).
If your reading in step 1 returns a document, that means values array has a record for given day. In that case you can use setoperation with $operator.
Like others said, they will be not atomic but while you are dealing with your problem in application level, you can make whole bunch of code synchronized. There will be 2 queries to run on mongodb among of 3 queries. Like below:
db.getCollection('band_profiles_history').find({"_id": "1", "values.day": 3})
if returns null:
db.getCollection('band_profiles_history').update({"_id": "1"}, {$push: {"values": {<your new band profile history for given day>}}})
if returns not null:
db.getCollection('band_profiles_history').update({"_id": "1", "values.day": 3}, {$set: {"values.$": {<your new band profile history for given day>}}})
To check if object is empty
{ field: {$exists: false} }
or if it is an array
{ field: {$eq: []} }
Mongoose also supports field: {type: Date} so you can use it instead counting a days, and do updates only for current date.

removing object from nested array of objects mongodb

I've got collection with volunteer information in it, and it lists the volunteers as an array of objects. I can display all the shifts for each volunteer, but removing one from the array is proving difficult for me:
Sample data:
"_id" : ObjectId("59180305c19dbaa4ecd9ee59"),
"where" : "Merchandise tent",
"description" : "Sell gear at the merchandise tent.",
"shifts" : [
{
"dateNeeded" : ISODate("2017-06-23T00:00:00Z"),
"timeslot" : "8:00 - NOON",
"needed" : 2,
"_id" : ObjectId("591807546a71c3a57d1a2105"),
"volunteers" : [
{
"fullname" : "Mary Mack",
"phone" : "1234567890",
"email" : "mary#gmail.com",
"_id" : ObjectId("591ce45bc7e8a8c7b742474c")
}
]
},
The data I have available for this is:
_id, where, shifts.timeslot, shifts.dateNeeded, volunteers.email
Can someone help me? Lets say Mary Mack wants to unVolunteer for the 8 - Noon shift at the merchandise tent. She may be listed under other shifts as well, but we only want to remove her from this shift.
You can do this by specifying something to match the "document" and then the required "shifts" array entry as the query expression for an .update(). Then apply the positional $ operator for the matched array index with $pull:
db.collection.update(
{ "_id": ObjectId("59180305c19dbaa4ecd9ee59"), "shifts.timeslot": "8:00 - NOON" },
{ "$pull": { "shifts.$.volunteers": { "fullname": "Mary Mack" } } }
)
That is okay in this instance since you are only trying to "match" on the "outer" array in the nested structure and the $pull has query arguments of it's own to identify the array entry to remove.
You really should be careful using "nested arrays" though. As whilst a $pull operation like this works, updates to the "inner" array are not really possible since the positional $ operator will only match the "first" element that meets the condition. So your example of "Mary Mack" in multiple shifts would only ever match in the first "shifts" array entry found.
Try this
db.example.update(
{},
{ $unset: {"Mary Mack":1}},
false, true
)

Insert and return ID of sub-document in MongoDB document sub-document array

My node.js application will insert a sub-document into a nested sub-document array field of the following MongoDB document, and I need to determine the ID of the newly inserted sub-document:
{
"_id" : ObjectId("578d5a52cc13117022e09def"),
"name" : "Grade 5 - Section A",
"scores" : [{
"studentId" : ObjectId("5776bd36ffc8227405d364d2"),
"performance" : [{
"_id" : ObjectId("57969b8fc164a21c20698261"),
"subjectId" : ObjectId("577694ecbf6f3a781759c54a"),
"score" : 86,
"maximum" : 100,
"grade" : "B+"
}]
}]
}
The sub-document looks like this:
{
"subjectId" : ObjectId("5776ffe1804540e29c602a62"),
"score" : 74,
"maximum" : 100,
"grade" : "A-"
}
I am adding the sub-document using the following Mongoose code:
Class.update({
_id: '578d5a52cc13117022e09def',
'scores.studentId': '5776bd36ffc8227405d364d2'
}, {
$addToSet: {
'scores.$.performance': {
'subjectId' : '5776ffe1804540e29c602a62',
'score' : 74,
'maximum' : 100,
'grade' : 'A-'
}
}
}, function(err, result) {
if (err) {
throw err;
}
console.log(result);
});
The subject sub-document gets added in the performance sub-document array which is itself nested in the scores sub-document array. Notice that the newly inserted sub-document is assigned with its own ID, as instituted by the defined schema. Even if I get back the entire document, that's not very helpful. I specifically need the ID of that newly inserted sub-document. What is the recommended approach to this problem?
In this case I prefer pre-assign the ID to the sub-document (i.e. sub._id = ObjectId() or use uuid package if you prefer uuid): is clear and predictable.
Also remember that if you frequent query by a subdoc id is good to add (using ensureIndex()) an index for this use case in the collection.
There is a good solution for that, try to use the methods create and push of the MongooseArrays.
In your code you could return the Student and do something like this:
const newPerformance = student.performance.create(newData);
student.performance.push(newPerformance);
const updatedStudent = await student.save();
if (updatedStudent) return newPerformance._id;
This is just a simple example.
Using the create method of MongooseArrays, link for doc, the mongoose will create an _id and do all the validations and casts it needs, so if the save process is fine the created subdocument you could just use the _id of the subdocument you got with the create method.

MongoDB Why this error : can't append to array using string field name: comments

I have a DB structure like below:
{
"_id" : 1,
"comments" : [
{
"_id" : 2,
"content" : "xxx"
}
]
}
I update a new subdocument in the comments feild. It is OK.
db.test.update(
{"_id" : 1, "comments._id" : 2},
{$push : {"comments.$.comments" : {_id : 3, content:"xxx"}}}
)
after that the DB structure:
{
"_id" : 1,
"comments" : [
{
"_id" : 2,
"comments" : [
{
"id" : 3,
"content" : "xxx"
}
],
"content" : "xxx"
}
]
}
But when I update a new subdocument in the comment field that _id is 3, There is a error:
db.test.update(
{"_id" : 1, "comments.comments.id" : 3},
{$push : {"comments.comments.$.comments" : {id : 4, content:"xxx"}}}
)
error message:
can't append to array using string field name: comments
Well, it makes total sense if you think about it. MongoDb has the advantage and the disadvantage of solving magically certain things.
When you query the database for a specific regular field like this:
{ field : "value" }
The query {field:"value"} makes total sense, it wouldn't in case value is part of an array but Mongo solves it for you, so in case the structure is:
{ field : ["value", "anothervalue"] }
Mongo iterates through all of them and matches "value" into the field and you don't have to think about it. It works perfectly.. at only one level, because it's impossible to guess what you want to do if you have multiple levels
In your case the first query works because it's the case in this example:
db.test.update(
{"_id" : 1, "comments._id" : 2},
{$push : {"comments.$.comments" : {_id : 3, content:"xxx"}}}
)
Matches _id in the first level, and comments._id at the second level, it gets an array as a result but Mongo is able to solve it.
But in the second case, think what you need, let's isolate the where clause:
{"_id" : 1, "comments.comments.id" : 3},
"Give me from the main collection records with _id:1" (one doc)
"And comments which comments inside have and id=3" (array * array)
The first level is solved easily, comments.id, the second is not possible due comments returns an array, but one more level is an array of arrays and Mongo gets an array of arrays as a result and it's not possible to push a document into all the records of the array.
The solution is to narrow your where clause to obtain an unique document in comments (could be the first one) but it's not a good solution because you never know what is the position of the document you're looking for, using the shell I think the only option to be accurate is to do it in two steps. Check this query that works (not the solution anyway) but "solves" the multiple array part fixing it to the first record:
db.test.update(
{"_id" : 1, "comments.0.comments._id" : 3},
{$push : {"comments.0.comments.$.comments" : {id : 4, content:"xxx"}}}
)

MongoDB aggregation group

I need few values to pass through the pipeline(without changes) to the subsequent pipeline aggregator while using a group
For eg if my input is
{ "_id" : 1, "tags" : [ "a", "b", "b", "c" ], "text" : "a" },
{ "_id" : 2, "tags" : [ "a", "c" ], "text" : "b" }
and I like the output to be
{ _id: 1, tags : [a,b,c], text : a} , { _id: 2, tags : [a,c], text : b}
I did an unwind and group and do $addToSet to remove the dups. My question is how do I make
text appear in the output as is. the text key and values need to just pass through this pipeline, However i am forced to use accumulators
I tried this code so far
use test
var unwind = { $unwind : "$tags"};
var group = { $group : { _id : "$_id" , tags : { $addToSet : "$tags" }
, text : { $first : "$text" }}};
db.test.aggregate(unwind,group)
and it works fine but that is not the intention of $first and it is suggested to be used with sort. What is the right way to do this ?
There's nothing wrong with using $first for this. As an alternative, you could add text to your _id, but that's hardly better as then you'd need to add a $project to the end of your pipeline to move it back out of _id.
You have to do the same sort of mildly awkward thing with SQL group bys.
All fields but the _id field must use an accumulator with $group, as specified in the $group documentation page. The problem is that it's unclear what it would mean to "pass" a value through the pipeline with $group, since MongoDB has no way of knowing that the value of that field is going to be the same for all documents you are grouping together. You have to choose what value you want for that field with the accumulator.