I'm trying to group all distinct fields and then lookup into another table , the problem is that I'm adding too many results for the same user inside the lookup, so I wonder how can I group those results internally and then in the next stage add another lookup to a different table.
Currently I have the following code:
Post.aggregate([
{ "$group" : {"_id" : { "id_fanpage": "$id_fanpage" },"total": { $sum: 1 } } },
{
"$lookup": {
"from": "v3_post_reaccion_angry",
"localField": "_id.id_fanpage",
"foreignField": "id_fanpage",
"as": "resultingAngries"
}
},
{
"$lookup": {
"from": "v3_post_reaccion_compartir",
"localField": "_id.id_fanpage",
"foreignField": "id_fanpage",
"as": "resultingCompartir"
}
}
]).exec(function(err, results){
console.log(results);
return results
}).then(success(res))
What I'm trying to do is distinct Post rows by "id_fanpage", then join another table by that returned id_fanpage, which is basically working as expected, the results are this:
{
"_id": {
"id_fanpage": "5a63a96d1aa940ca0ada1c82"
},
"total": 831,
"resultingAngries": [
{
"_id": "5a71ccda1aa940dd154caf83",
"id_post": "5a71ccda1aa940dd154caf82",
"id_fanpage": "5a63a96d1aa940ca0ada1c82",
"facebook_usuario_id": 345346565665,
"facebook_usuario_nombre": "AAAAAAAAAAAAAAAAA"
},
{
"_id": "5a71cd6b1aa940dd154ddb43",
"id_post": "5a71cd6b1aa940dd154ddb40",
"id_fanpage": "5a63a96d1aa940ca0ada1c82",
"facebook_usuario_id": "44444444444444444",
"facebook_usuario_nombre": "XXXXXXXXXXXXXXXX"
},
{
"_id": "5a71d7f61aa940dd156b60f4",
"id_post": "5a71d7f61aa940dd156b60e9",
"id_fanpage": "5a63a96d1aa940ca0ada1c82",
"facebook_usuario_id": "asdf12345232421",
"facebook_usuario_nombre": "YYYYYYYYYYYYYY"
},
{
"_id": "5a71cde11aa940dd154ec514",
"id_post": "5a71cde11aa940dd154ec513",
"id_fanpage": "5a63a96d1aa940ca0ada1c82",
"facebook_usuario_id": "asdf2345234555",
"facebook_usuario_nombre": "ZZZZZZZZZZZZZZ"
},
{
"_id": "5a71cdcb1aa940dd154e917a",
"id_post": "5a71cdcb1aa940dd154e9178",
"id_fanpage": "5a63a96d1aa940ca0ada1c82",
"facebook_usuario_id": "44444444444444444",
"facebook_usuario_nombre": "XXXXXXXXXXXXXXX"
}
]
}
Now the problem is, I want to group all results inside resultingAngries by facebook_usuario_id and also be able to add the the same to the next lookup, but as far as I know I cannot use a group after the first lookup without affecting the next lookup.
So the result should be:
{
"_id": {
"id_fanpage": "5a63a96d1aa940ca0ada1c82"
},
"total": 831,
"resultingAngries": [
{
"_id": "this is my user id 1111111111",
"count": N,
},
{
"_id": "this is my user id 2222222222",
"count": Y,
}
],
"resultingLikes": [
{
"_id": "this is my user id 1111111111",
"count": N,
},
{
"_id": "this is my user id 2222222222",
"count": Y,
}
],
},
{
"_id": {
"id_fanpage": "5a63a96d1aa942345wedf23455s"
},
"total": 20,
"resultingLikes": [
{
"_id": "this is my user id 1111111111",
"count": N,
},
{
"_id": "this is my user id 2222222222",
"count": Y,
}
],
},
"resultingAngries": [
{
"_id": "this is my user id 1111111111",
"count": N,
},
{
"_id": "this is my user id 2222222222",
"count": Y,
}
],
And so on.
I already have seen aggregate, and lookups and not sure if $addToSet can make it, because I cannot group internally so far so applying addToSet is giving me hard times. Help would be appreciate.
Insert below three stages between your $lookup's.
The below steps $unwind resultingAngries array followed by first $group to count the distinct combination of facebook_usuario_id & id_fanpage and second $group to $push the count and facebook_usuario_id back into resultingAngries array.
{"$unwind":"$resultingAngries"},
{"$group":{
"_id":{
"id_fanpage":"$_id.id_fanpage",
"facebook_usuario_id":"$resultingAngries.facebook_usuario_id"
},
"total":{"$first":"$total"},
"count":{"$sum":1}
}},
{"$group":{
"_id":{"id_fanpage":"$_id.id_fanpage"},
"total":{"$first":"$total"},
"resultingAngries":{
"$push":{
"_id":"$_id.facebook_usuario_id",
"count":"$count"
}
}
}}
Related
I want to join two mongodb collections, collectionA and collectionB.
For each document in collectionA I want to check if exists a coincidence in collectionB.
If I do it in a $lookup, it returns all the documents joined, but I would like the search in collectionB stops as soon as one coincidence is found (kind of a mongodb findOne). My concern is the performance, I know I could get just the element 0 from the array.
Is there a way to do it using the mongodB aggregation framework?
Example:
collectionA:
[
{
"_id": 1,
"item": "almonds"
},
{
"_id": 2,
"item": "pecans"
}
]
colectionB:
[
{
"_fid": 1,
"date": "2021-01-10"
},
{
"_fid": 1,
"date": "2021-01-11"
},
{
"_fid": 1,
"date": "2021-01-12"
},
{
"_fid": 2,
"date": "2021-01-03"
}
]
$lookup mongoDb
db.colectionA.aggregate([
{
"$lookup": {
"from": "colectionB",
"localField": "_id",
"foreignField": "_fid",
"as": "matches"
}
}
])
Result
[
{
"_id": 1,
"item": "almonds",
"matches": [
/* I don't want this array, with 1 element would be enough */
{
"_fid": 1,
"_id": ObjectId("5a934e000102030405000002"),
"date": "2021-01-10"
},
{
"_fid": 1,
"_id": ObjectId("5a934e000102030405000003"),
"date": "2021-01-11"
},
{
"_fid": 1,
"_id": ObjectId("5a934e000102030405000004"),
"date": "2021-01-12"
}
]
},
{
"_id": 2,
"item": "pecans",
"matches": [
{
"_fid": 2,
"_id": ObjectId("5a934e000102030405000005"),
"date": "2021-01-03"
}
]
}
]
You can test on this mongo playground.
Thanks in advance
If you're using at least MongoDB 3.6, you can execute an aggregation pipeline on a joined collection. It might look like this:
db.colectionA.aggregate([
{
"$lookup": {
"from": "colectionB",
"as": "matches",
"let": {
"fid": "$_id"
},
"pipeline": [
{
"$match": {
"$expr": {
"$eq": [
"$_fid",
"$$fid"
]
}
}
},
{
"$limit": 1
}
]
}
}
])
Working Mongo playground
I'm trying to Group By a embedded document.
This is my document
[{"_id":{"$oid":"610acbb68c1149ae85333753"},"numero_demande":"ERA-1624381087","client_datas":{"cotisation":"10000","qualite":["M."],"nom":"SABOM","prenom":"Manourou","email":"test#gmail.com","adresse":"Ouaga","prof":["366"],"activite":["700"],"date_effet":"2021-06-25T16:53:41.042Z","duree_contrat":"3","simulation_id":"60d2169fc31934a15cca1185","token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJmcmVzaCI6ZmFsc2UsImlhdCI6MTYyNDM4MTIxOSwianRpIjoiNGNiMDIyY2QtY2YyYy00NzkxLTk1MTItMjk4YTA3MDQ3ZDgyIiwidHlwZSI6ImFjY2VzcyIsInN1YiI6IjYwOTNlNDE2MzhmMDcxNTgzY2E0OTBjZiIsIm5iZiI6MTYyNDM4MTIxOSwiZXhwIjoxNjI0MzgxNTE5fQ.iI-Lz0Zh9FFiKBSusXsZaIkw9-kcFa5WcNcxiNSl0NZYnziHE2rOO0_-xIXEvaWbFIsaW2lY6iRczDY7jGOHTA"},"type":{"$oid":"60b4f154e077bf0f9da94376"},"user_id":{"$oid":"60d2169fc31934a15cca1183"},"status_id":{"$oid":"60ba7de4c57cffaa5598811a"},"result_simulation":{"montant_prime":330725},"selected_pack":{"montant_prime":330725},"date_created":{"$date":"2021-06-22T14:41:57.087Z"},"updated_date":{"$date":"2021-06-28T19:12:55.499Z"},"moyen_paiement":{"$oid":"610abe695c637c75446fa551"},"numero_police":"TEST01"}, {"_id":{"$oid":"610bfb41f76fcb2599219f17"},"numero_demande":"ERA-1628174841","client_datas":{"cotisation":"6800","valeur_verrerie":"","qualite":["M."],"nom":"hgvhgvgv","prenom":"jjjgjgjhg","email":"lkkjh#kjhkjhk.cjfk","adresse":"kjhgjhgjg","prof":["730"],"activite":["1055"],"date_effet":"2021-08-09T14:22:56.322Z","duree_contrat":"10","beneficiaires":[],"simulation_id":"610bf9f984467cd196219f10","token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJmcmVzaCI6ZmFsc2UsImlhdCI6MTYyODE3NDg3MCwianRpIjoiZTYxNjI5M2EtNjU3NC00YWExLTlmOTYtYTQzYWNmNjA5ZTAzIiwidHlwZSI6ImFjY2VzcyIsInN1YiI6IjYwOTNlNDE2MzhmMDcxNTgzY2E0OTBjZiIsIm5iZiI6MTYyODE3NDg3MCwiZXhwIjoxNjI4MTc1MTcwfQ.upmeD6MgzacDai_2IelKPNBDb4dmbQteJpZYjomJ9H0Mb1B5CXW1LhI8iE5Sy7098bk0PKEVTvtOifq2KTz7YA"},"type":{"$oid":"60b4f154e077bf0f9da94376"},"user_id":{"$oid":"61033a6dfc10f6bf6383faa4"},"status_id":{"$oid":"60ba7de4c57cffaa5598811a"},"result_simulation":{"montant_prime":831277},"moyen_paiement":{"$oid":"610abe695c637c75446fa551"},"selected_pack":{"montant_prime":831277},"date_created":{"$date":"2021-08-05T13:36:06.353Z"},"updated_date":{"$date":"2021-08-27T11:25:36.183Z"},"numero_police":"TEST02"}]
status_id field is a reference on another document and this is how it look like :
[{"_id":{"$oid":"60b7cd6878817148201f98e3"},"libelle":"Devis généré"}, {"_id":{"$oid":"60ba7de4c57cffaa5598811a"},"libelle":"Saisie dans ORCYS"}]
And i want to group and count the first document by status_id.libelle and get something like this
[{
libelle:"Saisie dans ORCYS",
total: 1
},{
libelle:"Devis généré",
total: 0
}]
I solved the problem using a community member answer. The solution can be tested using this link:
https://mongoplayground.net/p/7xNYvHO-WAX
Considering this two objects:
db={
"simulation": [
{
"_id": {
"$oid": "610acbb68c1149ae85333753"
},
"status_id": {
"$oid": "60ba7de4c57cffaa5598811a"
}
},
{
"_id": {
"$oid": "610bfb41f76fcb2599219f17"
},
"status_id": {
"$oid": "60ba7de4c57cffaa5598811a"
}
}
],
"status": [
{
"_id": {
"$oid": "60b7cd6878817148201f98e3"
},
"libelle": "Devis généré"
},
{
"_id": {
"$oid": "60ba7de4c57cffaa5598811a"
},
"libelle": "Saisie dans ORCYS"
}
]
}
If i want to have all simulation group and count by status, I should do this :
db.status.aggregate([
{
"$lookup": {
"from": "simulation",
"localField": "_id",
"foreignField": "status_id",
"as": "sourceLookup"
}
},
{
"$project": {
"_id": 1,
"libelle": 1,
"total": {
$size: "$sourceLookup"
}
}
}
])
And the result will look like this :
[
{
"_id": ObjectId("60b7cd6878817148201f98e3"),
"libelle": "Devis généré",
"total": 0
},
{
"_id": ObjectId("60ba7de4c57cffaa5598811a"),
"libelle": "Saisie dans ORCYS",
"total": 2
}
]
I have 3 collections.
User:
{
"_id":ObjectId("60a495cdd4ba8b122899d415"),
"email":"br9#gmail.com",
"username":"borhan"
}
Panel:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415")
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
Team:
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
I want to receive a output from querying Panel colllection just like this:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415"),
"teams":[
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
]
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
I mean i want to add teams field (which is array of teams that user is existed on it) to each user in Panel collection
Here is my match query in mongoose to select specific panel:
panel_model.aggregate([
{
$match: {
users: {
$elemMatch: {user: ObjectId("60a495cdd4ba8b122899d415"), role:"admin"}
}
}
},
])
Is it possible to get my output with $lookup or $addFields aggregations?
You need to join all three collections,
$unwind to deconstruct the array
$lookup there are two kind of lookups which help to join collections. First I used Multiple-join-conditions-with--lookup, and I used standrad lookup to join Users and Teams collections.
$match to match the user's id
$expr - when you use $match inside lookup, u must use it.
$set to add new fields
$group to we already destructed using $unwind. No we need to restructure it
here is the code
db.Panel.aggregate([
{ $unwind: "$users" },
{
"$lookup": {
"from": "User",
"let": { uId: "$users.user" },
"pipeline": [
{
$match: {
$expr: {
$eq: [ "$_id", "$$uId" ]
}
}
},
{
"$lookup": {
"from": "Team",
"localField": "_id",
"foreignField": "users",
"as": "teams"
}
}
],
"as": "users.join"
}
},
{
"$set": {
"users.getFirstElem": {
"$arrayElemAt": [ "$users.join", 0 ]
}
}
},
{
$set: {
"users.teams": "$users.getFirstElem.teams",
"users.join": "$$REMOVE",
"users.getFirstElem": "$$REMOVE"
}
},
{
"$group": {
"_id": "$_id",
"name": { "$first": "name" },
"users": { $push: "$users" }
}
}
])
Working Mongo playground
Note : Hope the panel and user collections are in 1-1 relationship. Otherwise let me know
I have two collections name listings and moods.
listings sample:
{
"_id": ObjectId("5349b4ddd2781d08c09890f3"),
"name": "Hotel Radisson Blu",
"moods": [
ObjectId("507f1f77bcf86cd799439010"),
ObjectId("507f1f77bcf86cd799439011")
]
}
moods sample:
{
"_id": ObjectId("507f1f77bcf86cd799439011"),
"name": "Sports"
},
{
"_id": ObjectId("507f1f77bcf86cd799439010"),
"name": "Spanish Food"
},
{
"_id": ObjectId("507f1f77bcf86cd799439009"),
"name": "Action"
}
I need this record.
{
"_id": ObjectId("507f1f77bcf86cd799439011"),
"name": "Sports",
"count": 1
},
{
"_id": ObjectId("507f1f77bcf86cd799439010"),
"name": "Spanish Food",
"count": 1
},
{
"_id": ObjectId("507f1f77bcf86cd799439009"),
"name": "Action",
"count": 0
}
I need this type of record. I have no idea about aggregate.
You can do it using aggregate(),
$lookup to join collection listings
$match pipeline to check moods _id in listings field moods array
db.moods.aggregate([
{
"$lookup": {
"from": "listings",
"as": "count",
let: { id: "$_id" },
pipeline: [
{
"$match": {
"$expr": { "$in": ["$$id", "$moods"] }
}
}
]
}
},
$addFields to add count on the base of $size of array count that we got from above lookup
{
$addFields: {
count: { $size: "$count" }
}
}
])
Playground
did this work:
db.collection.aggrate().count()
Try to combine the functions, it might work.
I have a database with some users, who belong to teams. Each team has a leader. Each user has a subject.
I want to collate teams by the leader's subject.
My data looks like this:
db={
"teams": [
{
_id: "t1",
members: [
{
"_id": "u1",
"leader": true
},
{
"_id": "u2"
},
{
"_id": "u3"
}
],
},
{
_id: "t2",
members: [
{
"_id": "u2",
"leader": true
},
{
"_id": "u4"
}
],
},
{
_id: "t3",
members: [
{
"_id": "u1",
"leader": true
},
{
"_id": "u4"
}
],
},
{
_id: "t4",
members: [
{
"_id": "u2",
"leader": true
}
],
},
],
"users": [
{
"_id": "u1",
"subject": "history"
},
{
"_id": "u2",
"subject": "maths"
},
{
"_id": "u3",
"subject": "geography"
},
{
"_id": "u4",
"subject": "french"
}
]
}
The result I want is:
{
"history": ["t1", "t3"],
"maths": ["t2", "t4"]
}
I have an aggregation that gets me the _id of every leader, and from there I can get the result I want in stages, by first finding the subject of every leader, then going back through the projects and assigning a subject to each project based on the identify of the leader. It works but it is inelegant and I think it will be slow. It seems to me there should be some better way to do this, maybe something like a join?
Is there a nifty way to get the result I want from a single MongoDB operation?
Here is a Mongo Playground with my data:
https://mongoplayground.net/p/SIJv9-hVNzJ
Many thanks for any help.
Edit: my test data are confusing because '_id' is used in both collections, making it hard to unpack the answer. Here is an updated Mongo Playground that uses different key names for each collection and helped me to understand the perfect answer.
Yes, you should join your collections on users._id with a $lookup, and then transform value to key with $arrayToObject (introduced in Mongodb 3.4.4)
Here is a possible way to do this :
db.teams.aggregate([
{
"$unwind": "$members"
},
{
"$match": {
"members.leader": true
}
},
{
"$lookup": {
"from": "users",
"localField": "members._id",
"foreignField": "_id",
"as": "users"
}
},
{
"$unwind": "$users"
},
{
"$group": {
"_id": "$users.subject",
"team": {
"$push": "$_id"
}
}
},
{
"$replaceRoot": {
"newRoot": {
"$arrayToObject": [
[
{
k: "$_id",
v: "$team"
}
]
]
}
}
}
])
try it online: mongoplayground.net/p/TuEpMzHkI-0