Sort sub-arrays in an Aggregate Result - mongodb

I have three documents:
{
"id_user": "t57092501745ad6285ac58c22",
"name": "Day #1",
"date": {
"$date": "2016-04-21T20:50:00.190Z"
},
"text": "My text"
}
{
"id_user": "t57092501745ad6285ac58c22",
"name": "Day #2",
"date": {
"$date": "2016-04-22T20:50:00.190Z"
},
"text": "My text"
}
{
"id_user": "t57092501745ad6285ac58c22",
"name": "Day #3",
"date": {
"$date": "2016-04-22T20:51:00.190Z"
},
"text": "My text"
}
and I need to group these for day, so I do:
{
"$match": {
"id_user": "t57092501745ad6285ac58c22"
}
}, {
"$sort": {
"date": -1
}
}, {
"$group": {
"_id": {
$dayOfYear: "$date"
},
"data": {
"$push": {
"id_user": "$id_user",
"name": "$name",
"date": "$date",
"text": "$text"
},
},
}
}
and the result is:
{
{
_id: 113,
data: [{
"id_user": "t57092501745ad6285ac58c22",
name: "Day #1",
date: "2016-04-22T20:51:00.190Z",
text: "My text"
}]
}, {
_id: 114,
data: [{
"id_user": "t57092501745ad6285ac58c22",
name: "Day #3",
date: "2016-04-23T20:51:00.190Z",
text: "My text"
}, {
"id_user": "t57092501745ad6285ac58c22",
name: "Day #2",
date: "2016-04-23T20:50:00.190Z",
text: "My text"
}]
}
}
and it's ok, but the order is not what I need:
{ Day #1 }, { Day #3, Day #2 }
if I change the sort to { "date": 1 } I can invert the order of the 2 groups, this way:
{ Day #3, Day #2 }, { Day #1 }
but I don't know how tho change also the order inside the sub-array, to obtain the correct:
{ Day #1 }, { Day #2, Day #3 }
What's the correct way?

If you want the items in the "array" to be in "ascending" order, then your $sort order is wrong, and you need to reverse it. Also output from $group as a "document" is not ordered in any way. So if you want a specific order then you need to actually $sort on the returned _id as well:
[
{ "$match": {
"id_user": "t57092501745ad6285ac58c22"
}},
{ "$sort": { "date": 1 } }
{ "$group": {
"_id": { "$dayOfYear": "$date" },
"data": {
"$push": {
"id_user": "$id_user",
"name": "$name",
"date": "$date",
"text": "$text"
}
}
}},
{ "$sort": { "_id": 1 } }
]
Then both orders are then correct

Related

MongoError: aggreation, $match, $filter, $elemMatch, unknown operator: $dateToString

I want to retrieve all the users who have communicated with the page on the date entered from the front, the data structure below,
[
{
"_id": "1",
"createdAt": "2019-02-20T21:34:17.634Z",
"updatedAt": "2022-03-01T20:47:55.100Z",
"fullName": "Jennifer Lieutard",
"firstName": "Jennifer",
"lastName": "Lieutard",
"conversations": [{
"page": "1000",
"messages": [
{
"content": "lorem ipsum",
"date": "2019-09-23T10:40:59.394Z"
},
{
"content": "lorem not ipsum",
"date": "2019-09-23T10:51:56.165Z"
},
{...}
]
}]
},
{
"_id": "2",
"createdAt": "2019-02-20T21:34:17.634Z",
"updatedAt": "2022-03-01T20:47:55.100Z",
"fullName": "Peter Pan",
"firstName": "Peter",
"lastName": "Pan",
"conversations": [{
"page": "1001",
"lastMessage": "Yes they can",
"messages": [
{
"content": "lorem ipsum",
"date": "2019-09-23T10:40:59.394Z"
},
{
"content": "lorem not ipsum",
"date": "2019-09-23T10:51:56.165Z"
},
{...}
]
}]
}
]
/*PAGES collections*/
{
"_id": "5dfb43c3bc1ca6634d559685",
"name": "All mighty",
"updatedAt": "2022-03-20T11:29:30.049Z"
}
what i did
$match: {
$expr: {
$and: [
{ $eq: [{ $month: "$createdAt" }, { $month: searchByDate == 'undefined' ? new Date() : new Date(searchByDate)}] },
{ $eq: [ { $year: "$createdAt"}, { $year: searchByDate == 'undefined' ? new Date() : new Date(searchByDate)}] }
]
},
"conversations": {
$elemMatch: {
"page": ObjectId(pageId),
"$and": [
{formatedDate: {$dateToString: { format: "%Y-%m", date: "$createdAt" }}},
"2019-02"
]
}
}
}
},
{
$group: {
_id: {
$dayOfMonth: "$createdAt"
},
"users": {
$push: {
fullName: "$fullName",
createdDate: "$createdAt",
formatedDate: {$dateToString: { format: "%Y-%m", date: "$messages.0.date" }},
}
},
"dates": {
$push: {
"createdAt": "$createdAt",
"month": {
$month: "$createdAt"
},
"day": {
$dayOfMonth: "$createdAt"
}
}
},
count:{$sum:1}
}
}
searchByDate is (YYYY-MM) value input from the frontend, example : 2019-20,
therefore must convert the date in base of the type 2020-12-08T11:05:54.609+00:00 in 2020-12 and compared to searchByDate, and here is what I got as an error
MongoError: unknown operator: $dateToString
Basically the task that I want to achieve is the following
filter to retrieve only the users who spoke to the page in the
desired month, that is to say the conversations.page == pageId
Retrieve the first message in this conversation of the page, ie the message.0
Convert the date of this message to a Year-Month format (YYYY-MM)
compare this converted date to the entered value 2019-02

How to group and a count fields in a mongoDB collection

I'm really new to mongodb coming from a sql background and struggling to work out how to run a simple report that will group a value from a nested document with a count and in a sort order with highest count first.
I've tried so many ways from what I've found online but I'm unable to target the exact field that I need for the grouping.
Here is the collection.
{
"_id": {
"$oid": "6005f95dbad14c0308f9af7e"
},
"title": "test",
"fields": {
"6001bd300b363863606a815e": {
"field": {
"_id": {
"$oid": "6001bd300b363863606a815e"
},
"title": "Title Two",
"datatype": "string"
},
"section": "Section 1",
},
"6001bd300b363863423a815e": {
"field": {
"_id": {
"$oid": "6001bd3032453453606a815e"
},
"title": "Title One",
"datatype": "string"
},
"section": "Section 1",
},
"6001bd30453534863423a815e": {
"field": {
"_id": {
"$oid": "6001bd300dfgdfgdf06a815e"
},
"title": "Title One",
"datatype": "string"
},
"section": "Section 1",
}
},
"sections": ["Section 1"]
}
The result I need to get from the above example would be:
"Title One", 2
"Title Two", 1
Can anyone please point me in the right direction? Thank you so much.
Having dynamic field names is usually a poor design.
Try this one:
db.collection.aggregate([
{ $set: { fields: { $objectToArray: "$fields" } } },
{ $unwind: "$fields" },
{ $group: { _id: "$fields.v.field.title", count: { $count: {} } } },
{ $sort: { count: -1 } }
])
Here's another way to do it. The $project throws away everything except for the deep-dive to "title". Then just $unwind and $sortByCount.
db.collection.aggregate([
{
"$project": {
"titles": {
"$map": {
"input": {
"$objectToArray": "$fields"
},
"in": "$$this.v.field.title"
}
}
}
},
{
"$unwind": "$titles"
},
{
"$sortByCount": "$titles"
}
])
Try it on mongoplayground.net.

Multiple nested grouping in MongoDB Aggregation

I have more than thousand of documents in my mongoDB collection where each document represents the data for a song like this:
{
"_id": ObjectID("612e03a790c3d3189845589b"),
"artist": "3 Doors Down",
"title": "Landning in London",
"album": "Another 700 Miles",
"genre": "Seventeen Days",
"trackNumber": 5,
"year": 2005,
"rating": 3,
"duration": "4:32",
"country": "United States",
}
Now with the help of the aggregation pipeline I want to group my collections the following way. So Far I could do grouping for 1 stage but I struggle to push through the complete root document until end and do this nested grouping.
List all genres --> list all artists within this genre --> list all albums of this artist within this genre --> list the COMPLETE root document for all songs of this album:
{
[
{
"genreName": "Alternative Rock",
"artists": [
{
"artistName": "3 Doors Down",
"albums": [
{
"albumName": "Seventeen Days",
"songs": [
{
"_id": ObjectID(612e03a790c3d3189845589b),
"artist": "3 Doors Down",
"title": "Landning in London",
"album": "Another 700 Miles",
"genre": "Seventeen Days",
"trackNumber": 5,
"year": 2005,
"rating": 3,
"duration": "4:32",
"country": "United States",
}
]
}
]
}
]
}
]
}
This one should work:
db.collection.aggregate([
{
$group: {
_id: { genreName: "$genre", artistName: "$artist", albumName: "$album" },
songs: { $push: "$$ROOT" }
}
},
{
$group: {
_id: { genreName: "$_id.genreName", artistName: "$_id.artistName" },
albums: { $push: { albumName: "$_id.albumName", songs: "$songs" } }
}
},
{
$group: {
_id: "$_id.genreName",
artists: { $push: { artistsName: "$_id.artistsName", albums: "$albums" } }
}
},
{ $project: { _id: 0, genreName: "$_id", artists: 1 } }
])

How to get the last value of every month in MongoDB?

I have three fields, id, date and qty in my DB. I want to group them by the id and find the qty which is the latest quantity of every month. So for every month, the date with the latest day of every month, the qty will be returned for it.
If the input is
[
{
"id": "ABC",
"date": "2020-10-02 15:03:00.00",
"qty": 500,
},
{
"id": "ABC",
"date": "2020-10-31 20:22:00.00",
"qty": 100,
},
{
"id": "ABC",
"date": "2020-11-03 04:22:00.00",
"qty": 200,
},
{
"id": "ABC",
"date": "2020-11-18 04:22:00.00",
"qty": 50,
},
{
"id": "ABC1",
"date": "2020-11-05 04:22:00.00",
"qty": 5000,
},
{
"id": "ABC1",
"date": "2020-11-15 04:22:00.00",
"qty": 4580,
},
]
then the output should be
[
{
"id": "ABC",
"qtys": [
{
"date": "2020-10-31 20:22:00.00",
"qty": 100
},
{
"date": "2020-11-18 04:22:00.00",
"qty": 50
}
]
},
{
"id": "ABC1",
"qtys": [
{
"date": "2020-11-15 04:22:00.00",
"qty": 4580
}
]
},
]
$addFields to convert date field from string type to date type, if its already date type then ignore this stage
$sort by date in descending order
$group by id, month and year after extracting from date field
using $year and $month to get first document
$group by only id and construct array of quantities in qtys
db.collection.aggregate([
{ $addFields: { date: { $toDate: "$date" } } },
{ $sort: { date: -1 } },
{
$group: {
_id: {
id: "$id",
month: { $month: "$date" },
year: { $year: "$date" }
},
qtys: { $first: { date: "$date", qty: "$qty" } }
}
},
{
$group: {
_id: "$_id.id",
qtys: { $push: "$qtys" }
}
}
])
Playground

Aggregate and unwind array, but keep top level key

Lets say I have the following document in my collection Classes Collecton
{
"_id": ObjectId("5df58d45244a850d54b922c8"),
"mentors" : {
"numOfMentors" : NumberInt(1),
"mentorList" : [
ObjectId("5c9ba636347bb645e0865283")
]
},
"lessons": [
{
"_id": ObjectId("5db221be211d7b68ac8be618"),
"mentorData": {
"objective": "Ensuring students that making mistakes is normal and that it is a part of life",
"task": "Post a video explaining obstacles that you had to overcome as a programmer",
"dueDate": "2019-12-14T15:26:10.000+0000"
},
"studentData": {
"objective": "Learning that failures help you grow",
"task": "Post a video explaining obstacles that you have overcame",
"dueDate": "2020-01-14T22:26:10.000+0000" <---- CHECKING THIS DATE
},
"title": "How to overcome obstacles",
"comments": []
}
]
}
And I am aggregating as follows:
Class.aggregate([
{ $match: { "students.studentList": req.user._id } },
{ $unwind: "$lessons" },
{
$addFields: {
date: {
$dateToString: {
format: "%Y-%m-%d",
date: "$lessons.studentData.dueDate"
}
}
}
},
{
$match: {
$and: [
{ date: { $gte: startDate } },
{ date: { $lte: endDate } }
]
}
},
{ $group: { _id: "$_id", lessons: { $push: "$lessons" } } }
])
Which returns this...
{
"lessonImage": {
"url": "https://stemsandbox.blob.core.windows.net/stemuli/lesson-picture-a406bd19-0677-4b60-909e-7de2ca3c6f93.jpg",
"originalname": "nintendo-switch-console.jpg",
"mimetype": "image/jpeg",
"blobName": "lesson-picture-a406bd19-0677-4b60-909e-7de2ca3c6f93.jpg",
"container": "stemuli",
"blob": "lesson-picture-a406bd19-0677-4b60-909e-7de2ca3c6f93.jpg",
"size": "147469",
"etag": "\"0x8D797B8F1EC9C39\"",
"createdOn": "2020-01-12T23:41:28.588Z"
},
"_id": "5db221be211d7b68ac8be619",
"mentorData": {
"objective": "Learn to make a single web page web app",
"task": "Create a short video instructing how to setup environment",
"dueDate": "2019-02-02T22:26:10.000Z"
},
"studentData": {
"objective": "Program a single page web app in React",
"task": "Program a single page web app in React and submit by october 30th",
"dueDate": "2020-01-22T22:26:10.000Z"
},
"title": "Learning to program in React",
}
I want it to be like this
Note mentors field
{
"lessonImage": {
"url": "https://stemsandbox.blob.core.windows.net/stemuli/lesson-picture-a406bd19-0677-4b60-909e-7de2ca3c6f93.jpg",
"originalname": "nintendo-switch-console.jpg",
"mimetype": "image/jpeg",
"blobName": "lesson-picture-a406bd19-0677-4b60-909e-7de2ca3c6f93.jpg",
"container": "stemuli",
"blob": "lesson-picture-a406bd19-0677-4b60-909e-7de2ca3c6f93.jpg",
"size": "147469",
"etag": "\"0x8D797B8F1EC9C39\"",
"createdOn": "2020-01-12T23:41:28.588Z"
},
"_id": "5db221be211d7b68ac8be619",
"mentorData": {
"objective": "Learn to make a single web page web app",
"task": "Create a short video instructing how to setup environment",
"dueDate": "2019-02-02T22:26:10.000Z"
},
"studentData": {
"objective": "Program a single page web app in React",
"task": "Program a single page web app in React and submit by october 30th",
"dueDate": "2020-01-22T22:26:10.000Z"
},
"title": "Learning to program in React",
"classId": "5e1baea87fcee8639cbce29d",\
//FIELD BELOW ADDED
"mentors": [
ObjectId("5c9ba636347bb645e0865283")
]
}
You can add $push to your $group and then run $reduce as a next stage to flatten an array of arrays:
{ $group: { _id: "$_id", lessons: { $push: "$lessons" }, mentors: { $push: "$mentors.mentorList" } } },
{ $addFields: { mentors: { $reduce: { input: "$mentors", initialValue: [], in: { $setUnion: [ "$$value", "$$this" ] } } } } }
Mongo Playground
EDIT: $setUnion will remove any potential duplicates