$match in aggregate don't return data in mongodb - mongodb

I have three tables below is the structure like below
I'm looking to get a result like below
"type1": [ -- type from Accounts collection
{
"_id": "5e97e9a224f62f93d5x3zz46", -- _id from Accounts collection
"locs": "sampleLocks 1", -- field from Accounts collection
"solutions": "sample solutions 1", -- field from Accounts collection
"Clause": "clause 1" -- field from AccountsDesc collection
},
{
"_id": "5e97e9a884f62f93d5x3zz46",
"locs": "sampleLocks2",
"solutions": "sample solutions2",
"Clause": "clause2"
}
],
"type2": [
// same data construction as of type1 above
]
_id, locks, solution to be coming from Accounts collection
Clause field to be coming from AccountsDesc collection
accounts_id is kind of a foreign key in AccountsDesc coming from Account
competitor_id is kind of a foreign key in AccountsDesc coming from Competitor
Below is what my query looks like
db.accountDesc.aggregate([
{
$match : {accounts_Id : "123456"}, active: true}
},
{
$lookup: {
from: 'accounts',
pipeline: [{ $match: { type: { $in: ["type1, type2, type3"] } } }],
as: 'accountsData'
}
},
{
$group: {
_id: "$accountsData.type",
data: {
$push: {_id: "$accountsData._id", clause: "$clause", locs: "$type.locs", solutions: "$type.solutions"}
}
}
},
{
$group: {
_id: null,
data: {
$push: {
k: {
$toString: '$_id'
},
v: '$data'
}
}
}
},
{
$replaceRoot: {
newRoot: {
$arrayToObject: '$data'
}
}
}
])
Issues related with the query -
$match : {accountId : "123456"}, active: true} -- No data is returned if i use match on AccountsDesc collection
cant set localField, foriegnField if im using pipeline, then how the mapping will happen like a LEFT join.
clause: "$clause" don't get the value of this field in the response

As we discussed in chat, you want RIGHT OUTER JOIN for your aggregation.
Try the query below:
db.User_Promo_Map.aggregate([
{
$match: {
user_Id: ObjectId("5e8c1180d59de1704ce68112")
}
},
{
$lookup: {
from: "promo",
pipeline: [
{
$match: {
active: true,
platform: {
$in: [
"twitch",
"youtube",
"facebook"
]
}
}
}
],
as: "accountsData"
}
},
{
$unwind: "$accountsData"
},
{
$group: {
_id: "$accountsData.platform",
data2: {
$addToSet: {
amount: "$amount",
promo_Id: "$promo_Id"
}
},
data: {
$addToSet: {
_id: "$accountsData._id",
format: "$accountsData.format",
description: "$accountsData.description"
}
}
}
},
{
$addFields: {
data: {
$map: {
input: "$data",
as: "data",
in: {
"_id": "$$data._id",
"description": "$$data.description",
"format": "$$data.format",
amount: {
$reduce: {
input: "$data2",
initialValue: "$$REMOVE",
in: {
$cond: [
{
$eq: [
"$$this.promo_Id",
"$$data._id"
]
},
"$$this.amount",
"$$value"
]
}
}
}
}
}
}
}
},
{
$group: {
_id: null,
data: {
$push: {
k: {
$toString: "$_id"
},
v: "$data"
}
}
}
},
{
$replaceRoot: {
newRoot: {
$arrayToObject: "$data"
}
}
}
])
MongoPlayground

Related

MongoDB aggregation, use value from one document as key in another

So I’m trying to aggregate two documents matched on an id and based on the value of the first.
Document 1
{
“id”:3
“Whats for dinner”: “dinner”,
“What is for dinner tonight”: “dinner”,
“Whats for lunch”:“lunch”
}
Document 2
{
“Id”:3
“dinner” : “We are having roast!”,
“lunch” : “We are having sandwiches”
}
I’d like to start by matching the id and test if the question exists in doc1.
then return the question from doc1 and the answer from doc 2 . Like
{“Whats for dinner”:“We are having roast!”}
I’ve tried:
{ “$match”: { “id”: 3, “Whats for dinner”:{"$exists":True}} },
{
"$lookup": {
"from": "doc 2",
"localField": "id",
"foreignField": "id",
"as": "qa"
}
}
But from here I can’t figure out how to use the value from doc1 as key in doc2
It might be simple! but I’m a new to this, and just can’t get it to work!?
Crazy data model! This would be a solution:
db.doc1.aggregate([
{ $project: { data: { $objectToArray: "$$ROOT" } } },
{ $unwind: "$data" },
{
$lookup: {
from: "doc2",
pipeline: [
{ $project: { data: { $objectToArray: "$$ROOT" } } }
],
as: "answers"
}
},
{
$set: {
answers: {
$first: {
$filter: {
input: { $first: "$answers.data" },
cond: { $eq: [ "$$this.k", "$data.v" ] }
}
}
}
}
},
{ $match: { answers: { $exists: true } } },
{
$project: {
data: [
{
k: "$data.k",
v: "$answers.v"
}
]
}
},
{ $replaceWith: { $arrayToObject: "$data" } }
])
Mongo Playground
Better don't use any user data as key names, you will always have to juggle with $objectToArray and $arrayToObject
Maybe consider this:
questions: {
guildid: 3,
text: [
"Whats for dinner",
"What is for dinner tonight",
"Whats for lunch"
],
"nospace": 1
}

Referencing root _id in aggregate lookup match expression not working

This is my first experience using aggregate pipeline. I'm not able to get a "$match" expression to work inside the pipeline. If I remove the "_id" match, I get every document in the collection past the start date, but once I add the $eq expression, it returns empty.
I read a lot of other examples and tried many different ways, and this seems like it is correct. But the result is empty.
Any suggestions?
let now = new Date()
let doc = await Team.aggregate([
{ $match: { created_by: mongoose.Types.ObjectId(req.params.user_oid)} },
{ $sort: { create_date: 1 } },
{ $lookup: {
from: 'events',
let: { "team_oid": "$team_oid" },
pipeline: [
{ $addFields: { "team_oid" : { "$toObjectId": "$team_oid" }}},
{ $match: {
$expr: {
$and: [
{ $gt: [ "$start", now ] },
{ $eq: [ "$_id", "$$team_oid" ] }
]
},
}
},
{
$sort: { start: 1 }
},
{
$limit: 1
}
],
as: 'events',
}},
{
$group: {
_id: "$_id",
team_name: { $first: "$team_name" },
status: { $first: "$status" },
invited: { $first: "$invited" },
uninvited: { $first: "$uninvited" },
events: { $first: "$events.action" },
dates: { $first: "$events.start" } ,
team_oid: { $first: "$events.team_oid" }
}
}])
Example Docs (added by request)
Events:
_id:ObjectId("60350837c57b3a15a414d265")
invitees:null
accepted:null
sequence:7
team_oid:ObjectId("60350837c57b3a15a414d263")
type:"Calendar Invite"
action:"Huddle"
status:"Questions Issued"
title:"Huddle"
body:"This is a Huddle; you should receive new questions 5 days befor..."
creator_oid:ObjectId("5ff9e50a206b1924dccd691e")
start:2021-02-26T07:00:59.999+00:00
end:2021-02-26T07:30:59.999+00:00
__v:0
Team:
_id:ObjectId("60350837c57b3a15a414d263")
weekly_schedule:1
status:"Live"
huddle_number:2
reminders:2
active:true
created_by:ObjectId("5ff9e50a206b1924dccd691e")
team_name:"tESTI"
create_date:2021-02-23T13:50:47.172+00:00
__v:0
This is just a guess since you don't have schema in your question. But it looks like your have some of your _ids mixed up. Where you are currently trying to $match events whose _id is equal to a team_oid. Rather than the event's team_oid field being equal to the current 'team' _id.
I'm pretty confident this will produce the correct output. If you post any schema or sample docs I will edit it.
https://mongoplayground.net/p/5i1w2Ii7KCR
let now = new Date()
let doc = await Team.aggregate([
{ $match: { created_by: mongoose.Types.ObjectId(req.params.user_oid)} },
{ $sort: { create_date: 1 } },
{ $lookup: {
from: 'events',
// Set tea_oid as the current team _id
let: { "team_oid": "$_id" },
pipeline: [
{ $match: {
$expr: {
$and: [
{ $gt: [ "$start", now ] },
// Match events whose 'team_oid' field matches the 'team' _id set above
{ $eq: [ "$team_oid", "$$team_oid" ] }
]
},
}
},
{
$sort: { start: 1 }
},
{
$limit: 1
}
],
as: 'events',
}},
{
$group: {
_id: "$_id",
team_name: { $first: "$team_name" },
status: { $first: "$status" },
invited: { $first: "$invited" },
uninvited: { $first: "$uninvited" },
events: { $first: "$events.action" },
dates: { $first: "$events.start" } ,
team_oid: { $first: "$events.team_oid" }
}
}])

Dynamic key in MongoDB

Im trying to create a dynamic group by (with sum agg) in MongoDB. But don't know how to right syntax that.
Lets imaging 2 documents:
{
"_id": {"$oid":"5f69f6a360c8479d0908a649"},
"key":"key1",
"data":{
"key1":"value1",
"key2":"value2",
"key3":"value3",
"key4":"value4"
},
"count":10
}
{
"_id": {"$oid":"5f69f6a360c8479d0908a649"},
"key":"key2",
"data":{
"key1":"value5",
"key2":"value6",
"key3":"value7",
"key4":"value8"
},
"count":15
}
With the key attribute, I want to control, which is the groupby attribute.
A pseudo query could look like:
[{
$group: {
_id: {
'$key': data[$key]
},
sum: {
'$sum': '$count'
}
}
}]
Output should look like:
value1 : 10
value6 : 15
Somebody knows how to do that?
I don't understand the purpose of $sum and $group, there are no arrays in your documents.
This aggregation pipeline give desired result:
db.collection.aggregate([
{ $set: { data: { $objectToArray: "$data" } } },
{ $set: { data: { $filter: { input: "$data", cond: { $eq: ["$$this.k", "$key"] } } } } },
{ $set: { data: { k: { $arrayElemAt: ["$data.v", 0] }, v: "$count" } } },
{ $set: { data: { $arrayToObject: "$data" } } },
{ $replaceRoot: { newRoot: { $mergeObjects: ["$$ROOT", "$data"] } } },
{ $unset: ["key", "count", "data"] }
])
You can try,
$reduce input data as array using $objectToArray, check condition if key matches with data key then return key as value and value as count field
convert that returned key and value object array to exact object using $arrayToObject
replace field using $replaceWith
db.collection.aggregate([
{
$replaceWith: {
$arrayToObject: [
[
{
$reduce: {
input: { $objectToArray: "$data" },
initialValue: {},
in: {
$cond: [
{ $eq: ["$$this.k", "$key"] },
{
k: "$$this.v",
v: "$count"
},
"$$value"
]
}
}
}
]
]
}
}
])
Playground

Join multiple collections after parallel aggregation in Mongodb

I have a collection called "Reel" which has embedded Objects.
{
"_id":"reel_1",
"category":[
{
"_id" : "cat_1",
"videos": [ {"_id":"vid_1"},{"_id":"vid_2"} ] //_id is reference of Video collection
},
{
"_id" : "cat_2",
"videos": [ {"_id":"vid_3"},{"_id":"vid_4"} ]
}
]
}
Video is another collection whose _id is referred inside reel-> category -> videos -> _id
{
"_id":"vid_1",
"title":"title 1",
"groups":[{"_id":"group_1"},{"_id":"group_2"}]
},
{
"_id":"vid_2",
"title":"title 2",
"groups":[{"_id":"group_1"},{"_id":"group_4"}]
},
{
"_id":"vid_3",
"title":"title 3",
"groups":[{"_id":"group_1"},{"_id":"group_2"}]
},
{
"_id":"vid_4",
"title":"title 4",
"groups":[{"_id":"group_3"},{"_id":"group_4"}]
}
The Document collection which holds _id of Reel and _id of Category
{
"_id":"doc_1",
"title":"document title",
"assessments":[
{
"reel":"reel_1", // reel reference _id
"category":"cat_1", // category reference _id
"groups":[{"_id":"group_1"},{"_id":"group_2"}
]
}
]
}
I need to join and find all related embedded Objects which has group_1.
I have done joining between Reel collection and Video collection and working fine,
{ $unwind: { path: '$category', preserveNullAndEmptyArrays: true }},
{ $unwind: { path: '$category.videos', preserveNullAndEmptyArrays: true }},
{
$lookup: {
from: 'video',
localField: 'category.videos._id',
foreignField: '_id',
as: 'joinVideo'
}
},
{ $unwind: { path: "$joinVideo", preserveNullAndEmptyArrays: true }},
{ $unwind: { path: "$joinVideo.groups", preserveNullAndEmptyArrays: true }},
{ $match: { "joinVideo.groups._id": "group_1" }},
{ $addFields: { "category.videos": "$joinVideo" }},
{
$group: {
_id: {
_id: "$_id",
category: "$category._id"
},
videos: {
$addToSet: "$category.videos"
}
}
}, {
$group: {
_id: "$_id._id",
category: {
$addToSet: {
"_id": "$_id.category",
"videos": "$videos"
}
}
}
}
The document collection should be embedded inside the category object based on reel _id and and category _id filtered by group_1. My expected result is
{
"_id":"reel_1",
"category":[
{
"_id" : "cat_1",
"videos": [
{
"_id":"vid_1",
"title":"title 1",
"groups":[ {"_id":"group_1"},{"_id":"group_2"}]
},
{
"_id":"vid_2",
"title":"title 2",
"groups":[{"_id":"group_1"},{"_id":"group_4"}]
}
],
"documents":[
{ // this document comes by reel="reel_1", category="cat_1", filtered by "group_1"
"_id":"doc_1",
"title":"document title",
}
]
},
{
"_id" : "cat_2",
"videos": [
{
"_id":"vid_3",
"title":"title 3",
"groups":[{"_id":"group_1"},{"_id":"group_2"}]
}
]
}
]
}
I tried in many ways. Since I'm new to Mongodb, I couldn't sort this out.
Since MongoDB v3.6, $lookup allows perform uncorrelated sub-queries. This allows us perform non-standard queries to join two or more collections.
Note: Explanation why we need to use $expr inside $lookup pipeline
Explanation
We apply $unwind to flatten $category
We perform $lookup with 2 conditions:
video.groups._id == 'group_1' and video._id in reel.category.videos._id
Since $reel.category.videos._id returns an array, we need to use $in operator
Again we perform $lookup with 2 conditions. It creates documents field for every document
To remove fields dynamically, we need to use Aggregation expressions called $$REMOVE which allows us exclude conditionally a field from document
We perform $group stage to transform into desired result
db.reel.aggregate([
{
$unwind: {
path: "$category",
preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "video",
let: {
videos: "$category.videos._id"
},
pipeline: [
{
$match: {
"groups._id": "group_1",
$expr: {
$in: [
"$_id",
"$$videos"
]
}
}
}
],
as: "category.videos"
}
},
{
$lookup: {
from: "document",
let: {
reel_id: "$_id",
category_id: "$category._id"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$in: [
"$$reel_id",
"$assessments.reel"
]
},
{
$in: [
"$$category_id",
"$assessments.category"
]
}
]
}
}
},
{
$project: {
_id: 1,
title: 1
}
}
],
as: "category.documents"
}
},
{
$addFields: {
"category.documents": {
$cond: [
{
$eq: [
{
$size: "$category.documents"
},
0
]
},
"$$REMOVE",
"$category.documents"
]
}
}
},
{
$group: {
_id: "$_id",
category: {
$push: "$category"
}
}
}
])
MongoPlayground

mognodb aggregation group by actor

I have the following film collection structure:
{
"_id" : ObjectId,
"title" : "movie-1",
"actors" : [
"actor-1",
"actor-2",
"actor-3",
],
"categories" : [
"category-1",
"category-2"
]
}
I want to display result of all actors with associate movies and categories as like as given below:
{
"actor": "actor-1",
"result": {
"category-1": [ "movie-1", "movie-2" ],
"category-2": [ "movie-1", "movie-4" ]
}
}
I have tried aggregation as like as given below:
db.film.aggregate([
{ $unwind: "$actors" },
{ $group: {
_id: "$actors",
data: { $push: { movie: "$title", categories: "$categories" } }
}
},
{
$project: {
_id: 0,
actor: "$_id",
result: {
$reduce: {
input: "$data",
initialValue: {},
in: {
$let: {
vars: { movie: "$$this.movie", categories: "$$this.categories" },
in: {
$arrayToObject: {
$map: {
input: "$$categories",
in: { k: "$$this", v: "$$movie" }
}
}
}
}
}
}
}
}
}
])
But I get all actors list with only one movie with category as like as given below:
{
"actor" : "actor-1",
"result" : {
"category-1" : "movie-1",
"category-2" : "movie-2",
"category-3" : "movie-3"
}
}
How can I solve this problem? Thanks in advance.
You may need to do another $unwind on the categories array after flattening the actors array then group all the flattened docs by the two fields i.e. actor and category fields to create the movie titles list.
Another group to shape the result field is required.
The following pipeline should give you the desired result:
db.film.aggregate([
{ "$unwind": "$actors" },
{ "$unwind": "$categories" },
{ "$group": {
"_id": { "actor": "$actors", "category": "$categories" },
"movies": { "$push": "$title" }
} },
{ "$group": {
"_id": "$_id.actor",
"result": {
"$push": {
"k": "$_id.category",
"v": "$movies"
}
}
} },
{ "$addFields": {
"result": { "$arrayToObject": "$result" }
} }
])
I've used a sledgehammer to crack a nut (c)
Some stages could be replaced by $reduce, done inside $project stage (criticism and suggestions will be welcome)
db.film.aggregate([
{
$unwind: "$actors"
},
{
$group: {
_id: "$actors",
data: {
$push: {
movie: "$title",
categories: "$categories"
}
}
}
},
{
$unwind: "$data"
},
{
$unwind: "$data.categories"
},
{
$group: {
_id: {
actors: "$_id",
categories: "$data.categories"
},
movies: {
$push: "$data.movie"
}
}
},
{
$project: {
_id: 0,
actor: "$_id.actors",
result: {
k: "$_id.categories",
v: "$movies"
}
}
},
{
$group: {
_id: "$actor",
result: {
$push: "$result"
}
}
},
{
$project: {
_id: 0,
actor: "$_id",
result: {
$arrayToObject: "$result"
}
}
},
{
$sort: {
actor: 1
}
}
])
MongoPlayground