How to use current field in second $match? - mongodb

Let's say i have 2 collections
// Post collection:
[
{
"_id": "somepost1",
"author": "firstuser",
"title": "First post"
},
{
"_id": "somepost2",
"author": "firstuser",
"title": "Second post"
},
{
"_id": "somepost3",
"author": "firstuser",
"title": "Third post"
}
]
// User collection:
[
{
"_id": "firstuser",
"nickname": "John",
"posts": {
"voted": []
}
},
{
"_id": "seconduser",
"nickname": "Bob",
"posts": {
"voted": [
{
"_id": "somepost1",
"vote": "1"
},
{
"_id": "somepost3",
"vote": "-1"
}
]
}
}
]
And i need to get this result:
[
{
"_id": "somepost1",
"author": {
"_id": "firstuser",
"nickname": "John"
},
"title": "First post",
"myvote": "1"
},
{
"_id": "somepost2",
"author": {
"_id": "firstuser",
"nickname": "John"
},
"title": "Second post",
"voted": "0"
},
{
"_id": "somepost3",
"author": {
"_id": "firstuser",
"nickname": "John"
},
"title": "Third post",
"myvote": "-1"
}
]
How can i make a request with aggregation, which will display this output with dynamic _id of elements?
I have problem with using current _id of post in second $match and setting "myvote" to 0 if there are no element in "posts.voted" associated with current post.
Here what i've tried: https://mongoplayground.net/p/v70ZUioVSpQ
db.post.aggregate([
{
$match: {
author: "firstuser"
}
},
{
$lookup: {
from: "user",
localField: "author",
foreignField: "_id",
as: "author"
}
},
{
$addFields: {
author: {
$arrayElemAt: [
"$author",
0
]
}
}
},
{
$lookup: {
from: "user",
localField: "_id",
foreignField: "posts.voted._id",
as: "Results"
}
},
{
$unwind: "$Results"
},
{
$unwind: "$Results.posts.voted"
},
{
$match: {
"Results.posts.voted._id": "ID OF CURRENT POST"
}
},
{
$project: {
_id: 1,
author: {
_id: 1,
nickname: 1
},
title: 1,
myvote: "$Results.posts.voted.vote"
}
}
])

From the $match docs:
The query syntax is identical to the read operation query syntax
The query syntax does not allow usage of document values. which is what you're trying to do.
What we can do is use $expr within the $match stage, this allows us to use aggregation oprerators, thus also giving access to the document values. like so:
{
$match: {
$expr: {
$eq: ['$Results.posts.voted._id', '$_id'],
}
},
},

Related

How to aggregate nested lookup array in mongoose?

I have a problem with how to lookup nested array, for example i have 4 collections.
User Collection
"user": [
{
"_id": "1234",
"name": "Tony",
"language": [
{
"_id": "111",
"language_id": "919",
"level": "Expert"
},
{
"_id": "111",
"language_id": "920",
"level": "Basic"
}
]
}
]
Language Collection
"language": [
{
"_id": "919",
"name": "English"
},
{
"_id": "920",
"name": "Chinese"
}
]
Job
"job": [
{
"_id": "10",
"title": "Programmer",
"location": "New York"
}
],
CvSubmit Collection
"cvsubmit": [
{
"_id": "11",
"id_user": "1234",
"id_job": "11"
}
]
And my query aggregation is:
db.cvsubmit.aggregate([
{
$lookup: {
from: "user",
localField: "id_user",
foreignField: "_id",
as: "id_user"
}
},
{
$lookup: {
from: "language",
localField: "id_user.language.language_id",
foreignField: "_id",
as: "id_user.language.language_id"
}
},
])
But the result is:
[
{
"_id": "11",
"id_job": "11",
"id_user": {
"language": {
"language_id": [
{
"_id": "919",
"name": "English"
},
{
"_id": "920",
"name": "Chinese"
}
]
}
}
}
]
I want the result like this, also showing all user data detail like name:
[
{
"_id": "11",
"id_job": "11",
"id_user": {
"_id": "1234",
"name": "Tony"
"language": [
{
"_id": "919",
"name": "English",
"Level": "Expert"
},
{
"_id": "920",
"name": "Chinese",
"level": "Basic"
}
]
}
}
]
Mongo Playground link https://mongoplayground.net/p/i0yCucjruey
Thanks before.
$lookup with user collection
$unwind deconstruct id_user array
$lookup with language collection and return in languages field
$map to iterate look of id_user.language array
$reduce to iterate loop of languages array returned from collection, check condition if language_id match then return name
db.cvsubmit.aggregate([
{
$lookup: {
from: "user",
localField: "id_user",
foreignField: "_id",
as: "id_user"
}
},
{ $unwind: "$id_user" },
{
$lookup: {
from: "language",
localField: "id_user.language.language_id",
foreignField: "_id",
as: "languages"
}
},
{
$addFields: {
languages: "$$REMOVE",
"id_user.language": {
$map: {
input: "$id_user.language",
as: "l",
in: {
_id: "$$l._id",
level: "$$l.level",
name: {
$reduce: {
input: "$languages",
initialValue: "",
in: {
$cond: [
{ $eq: ["$$this._id", "$$l.language_id"] },
"$$this.name",
"$$value"
]
}
}
}
}
}
}
}
}
])
Playground
You database structure is not accurate as per NoSQL, there should be max 2 collections, loot of join using $lookup and $unwind will cause performance issues.

Select data from two MongoDB tables and update the results

We have the following problem
Given are the tables and fields
Offer
OfferId
State
Article
OfferId
ArticleId
NetPrice
GrossPrice
VatRate
Example-data:
Offer-Collection
{
"_id": "1",
"State": "INITIAL",
"_class": "com.example.dto.OfferData"
}
{
"_id": "2",
"State": "COMPLETED",
"_class": "com.example.dto.OfferData"
}
Article-Collection
{
"_id": {
"$oid": "a"
},
"Description": "asdf",
"NetPrice": "100",
"GrossPrice": "116",
"VatRate": "16",
"OfferId": "1",
"_class": "com.example.dto.Article"
}
{
"_id": {
"$oid": "b"
},
"Description": "my description",
"NetPrice": "100",
"GrossPrice": "119",
"VatRate": "19",
"OfferId": "1",
"_class": "com.example.dto.Article"
}
{
"_id": {
"$oid": "c"
},
"Description": "my description",
"NetPrice": "100",
"GrossPrice": "116",
"VatRate": "16",
"OfferId": "2",
"_class": "com.example.dto.Article"
}
Now we have to update all articles belonging to an offer with the state "initial" in the following way: if the VatRate is equal to 16 than it must be updated to 19 AND the GrossPrice must be recalculated from the existing NetPrice.
The result should be: the article with _id = "a" and VatRate = 16 for OfferId = 1 (State = INITIAL) should have VatRate = 19 and GrossPrice = 119. The fields should be updated and persisted in the original MongoDB collection.
Can we do this only with Mongo-shell? Our Version is 3.6.
Our tries:
We have played around with .aggregate, $lookup, $match and $project but without much luck. It's the first time we are using the Mongo-shell.
db.getCollection("Offers").aggregate([{
$lookup:{
from:"Articles",
localField:"OfferId",
foreignField:"OfferId",
as:"selected-articles"
}
},
{
$match: { "state": { "$eq": "INITIAL" } }
},
{
$project: { "articles": 1 }
}
]).forEach(...?)
$match your State condition
$lookup with Articles collection
$map to iterate loop of selected-articles array, check condition using $cond if VatRate is "16" then updated to 19 and recalculate GrossPrice as per NetPrice using $multiply before it convert NetPrice to integer because its in string type, back to merge objects with current objects using $mergeObjects
db.getCollection("Offers").aggregate([
{ $match: { State: { $eq: "INITIAL" } } },
{
$lookup: {
from: "Articles",
localField: "_id",
foreignField: "OfferId",
as: "selected-articles"
}
},
{
$addFields: {
"selected-articles": {
$map: {
input: "$selected-articles",
in: {
$mergeObjects: [
"$$this",
{
$cond: [
{ $eq: ["$$this.VatRate", "16"] },
{
VatRate: 19,
GrossPrice: {
$multiply: [{ $toInt: "$$this.NetPrice" }, 19]
}
},
{}
]
}
]
}
}
}
}
}
])
Playground

How can i display count of related documents on parent level?

I'm trying to build a voting system where you can have X num of options to vote for on an entry.
The query I'm building now is when retrieving an entry, I would like to get the numbers of votes per option on an entry.
I have a very clear understanding of how I would do this in SQL but grasping to understand the concepts of aggregation, lookup, and group in MongoDB
The model looks like this:
Entries
{
"_id": "5fc2765938401a2308e18ac5",
"options": [
{
"name": "First Option"
"_id": "5fc2765938401a2308e18ac6",
},
{
"name": "Second Option"
"_id": "5fc2765938401a2308e18are",
},
{
"name": "Third Option"
"_id": "5fc2765938401a2308e18aef",
},
],
},
{
"_id": "5fc2766438401a2308e18ac8",
"options": [
{
"name": "Some other option"
"_id": "5fc2766438401a2308e18ac9",
},
{
"_id": "5fc2766438401a2308e18aca",
"name": "This is also an option"
}
],
}
Votes
{
"_id": "5fc2765938401a2308e18ac5",
"entryId": "5fc2765938401a2308e18ac6"
},
{
"_id": "5fc2765938401a2308e18aer",
"entryId": "5fc2765938401a2308e18are"
},
{
"_id": "5fc2765938401a2308e18ek",
"entryId": "5fc2765938401a2308e18ac6"
}
...
And I want the results of Entry to look like this.
{
"_id": "5fc2765938401a2308e18ac5",
"options": [
{
"name": "First Option"
"_id": "5fc2765938401a2308e18ac6",
"votes": 1,
},
{
"name": "Second Option"
"_id": "5fc2765938401a2308e18are",
"votes": 0,
},
{
"name": "Third Option"
"_id": "5fc2765938401a2308e18aef",
"votes": 5,
},
],
},
{
"_id": "5fc2766438401a2308e18ac8",
"options": [
{
"name": "Some other option"
"_id": "5fc2766438401a2308e18ac9",
"votes": 3,
},
{
"_id": "5fc2766438401a2308e18aca",
"name": "This is also an option"
"votes": 10,
}
],
}
$lookup to join votes collection, pass local field optoins._id and foreign field entryId
$project get options votes, $map to iterate loop of options array, $filter to get matching entryId records and $size to get count of element in return array, merge votes field and current object using $mergeObjects
db.entries.aggregate([
{
$lookup: {
from: "votes",
localField: "options._id",
foreignField: "entryId",
as: "votes"
}
},
{
$project: {
options: {
$map: {
input: "$options",
as: "a",
in: {
$mergeObjects: [
"$$a",
{
votes: {
$size: {
$filter: {
input: "$votes",
cond: { $eq: ["$$this.entryId", "$$a._id"] }
}
}
}
}
]
}
}
}
}
}
])
Playground

MongoDB - $lookup in complex nested array

I'm building a new LMS in mongodb and I have the following collections:
Courses
{
"_id": ObjectId("5f6a6b159de1304fb885b194"),
"title": "Course test",
"sections": [
{
"_id": ObjectId("5f6a6b159de1304fb885b195"),
"title": "Section 1 - introduction",
"order": 1,
"modules": [
{
"_id": ObjectId("5f6a6b159de1304fb885b196"),
"module_FK_id": ObjectId("5f6a6b149de1304fb885b135"),
"title": "Module 1",
"order": 1
},
{
"_id": ObjectId("5f6a6b159de1304fb885b198"),
"module_FK_id": ObjectId("5f6a6b149de1304fb885b14a"),
"title": "Module 2",
"order": 2
},
]
},
{
"_id": ObjectId("5f6a6b149de1304fb885b175"),
"title": "Section 2 - How to do something",
"order": 2,
"modules": [
{
"_id": ObjectId("5f6a6b149de1304fb885b141"),
"module_FK_id": ObjectId("5f6a6b149de1304fb885b150"),
"title": "Module 1",
"order": 1
},
{
"_id": ObjectId("5f6a6b149de1304fb885b15f"),
"module_FK_id": ObjectId("5f6a6b149de1304fb885b18e"),
"title": "Module 2",
"order": 2
},
]
},
]
}
Modules (only one as example)
{
"_id": ObjectId("5f6a6b149de1304fb885b135"),
"text": "Lorem ipsum...",
"mediaUrl": "urllinkhere"
}
As shown, I choose to have embedded documents for sections and modules titles but I need also a second collection, modules, because each module contains a large amount of text and my course document may get too big quickly.
Now I need to rebuild the entire document as if it was completely embedded.
Here's an example:
{
"_id": ObjectId("5f6a6b159de1304fb885b194"),
"title": "Course test",
"sections": [
{
"_id": ObjectId("5f6a6b159de1304fb885b195"),
"title": "Section 1 - introduction",
"order": 1,
"modules": [
{
"_id": ObjectId("5f6a6b159de1304fb885b196"),
"module_FK_id": ObjectId("5f6a6b149de1304fb885b135"),
"title": "Module 1",
"order": 1
"text": "Lorem ipsum...",
"mediaUrl": "urllinkhere"
},
// last two fields from collection "modules"
I'm trying different combination of aggregation and lookups but I can't obtain the desired result.
Can anybody help me?
You can construct the aggregation pipeline like this below, just remember to $group in the reverse order of $unwind operations
db.courses.aggregate([
{
$unwind: "$sections"
},
{
$unwind: "$sections.modules"
},
{
"$lookup": {
"from": "modules",
"localField": "sections.modules.module_FK_id",
"foreignField": "_id",
"as": "module_details"
}
},
{
$unwind: {
path: "$module_details",
preserveNullAndEmptyArrays: true
}
},
{
"$project": {
title: 1,
sections: {
_id: "$sections._id",
modules: {
_id: "$sections.modules._id",
module_FK_id: "$sections.modules.module_FK_id",
order: "$sections.modules.order",
title: "$sections.modules.title",
mediaUrl: "$module_details.mediaUrl",
text: "$module_details.text"
},
order: "$sections.order",
title: "$sections.title"
}
}
},
{
"$group": {
"_id": "$sections._id",
main_id: {
$first: "$_id"
},
main_title: {
$first: "$title"
},
order: {
$first: "$sections.order"
},
title: {
$first: "$sections.title"
},
modules: {
$push: "$sections.modules"
}
}
},
{
"$project": {
"_id": "$main_id",
"title": "$main_title",
section: {
_id: "$_id",
modules: "$modules",
title: "$title",
order: "$order",
}
}
},
{
$group: {
_id: "$_id",
title: {
$first: "$title"
},
sections: {
$push: "$section"
}
}
}
])
Working Example

How can I reduce the duplicate data from my aggregation pipeline?

I have a pipeline that works great for what I need... but I think there is some redundant data that can be removed from the pipeline.
Expected output
This is what I want the output to look like
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": [
{
"_id": "5ecb263c166b8500047c1411",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1422",
"what": "Log OUT"
}
]
}
Current output
This is what the output currently looks like
{
"docs": [
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": {
"_id": "5ecb263c166b8500047c1411",
"what": "Log IN"
},
"historyIndex": 0
},
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": {
"_id": "5ecb263c166b8500047c1422",
"what": "Log OUT"
},
"historyIndex": 1
}
]
}
User doc
In real life there will be more users than this... of course...
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
}
History docs
again, to make it simple, I am keeping data short
[
{
"_id": "5ecb263c166b8500047c1411",
"userId": "5ecee2189fdd1b0004056936",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1422",
"userId": "5ecee2189fdd1b0004056999",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1433",
"userId": "5ecee2189fdd1b0004056936",
"what": "Log OUT"
},
{
"_id": "5ecb263c166b8500047c1444",
"userId": "5ecee2189fdd1b0004056999",
"what": "Log OUT"
}
]
mongoose-aggregate-paginate-v2 middleware
I am also using mongoose-aggregate-paginate-v2, but I don't think that is my issue, but it definitely comes into play when the results are returned. it needs to have the docs flattened so it can count and paginate them:
"totalDocs": 941,
"limit": 500,
"page": 1,
"totalPages": 2,
"pagingCounter": 1,
"hasPrevPage": false,
"hasNextPage": true,
"prevPage": null,
"nextPage": 2
Pipeline
Here is my pipeline
var agg_match = {
$match:
{
_id: mongoose.Types.ObjectId(userId)
}
};
var agg_lookup = {
$lookup: {
from: 'it_userhistories',
localField: '_id',
foreignField: 'userId',
as: 'history'
}
}
var agg_unwind = {
$unwind: {
path: "$history",
preserveNullAndEmptyArrays: true,
includeArrayIndex: 'historyIndex',
}
}
var agg = [
agg_match,
agg_lookup,
agg_unwind,
agg_project,
];
var pageAndLimit = {
page:page,
limit:limit
}
User.aggregatePaginate(myAggregate, pageAndLimit)
You can use $map operator to do this. Following query will be helpful (I have not included the match stage in the pipeline, you can easily include it):
db.user.aggregate([
{
$lookup: {
from: "history",
localField: "_id",
foreignField: "userId",
as: "history"
}
},
{
$project: {
name: 1,
history: {
$map: {
input: "$history",
as: "h",
in: {
_id: "$$h._id",
what: "$$h.what"
}
}
}
}
}
])
MongoPLayGroundLink