How to find documents from multiple collection with similar field value - mongodb

I have two collections:
Product
{ id: "1", model: "Light1", category: "Light"},
{ id: "2", model: "Light3", category: "Light"},
{ id: "3", model: "Lock1", category: "Lock"},
Item
{ id: "1", model: "Light1", category: "Light", color: "Blue"},
{ id: "2", model: "Light2", category: "Light", color: "Blue"},
{ id: "3", model: "Lock1", category: "Lock", color: "Blue"},
{ id: "4", model: "Light3", category: "Light", color: "Blue"}
{ id: "5", model: "Lock2", category: "Lock", color: "Blue"},
I want to find documents from the Item collection containing both model and category from the product collection.
From the example above, I want to get this so called new collection:
{ id: "1", model: "Light1", category: "Light", color: "Blue"},
{ id: "3", model: "Lock1", category: "Lock", color: "Blue"},
{ id: "4", model: "Light3", category: "Light", color: "Blue"}

You can try this aggregation query:
First $lookup from Item collection to join collections. This lookup uses a pipeline where you match the desired values: Local model is equal to foreign model and local category is equal to foreign category. This produces an array as output: if there is not any match the array will be empty.
So you can $match to not shown empty result array.
And use $project to output the values you want.
db.Item.aggregate([
{
"$lookup": {
"from": "Product",
"let": {
"model": "$model",
"category": "$category"
},
"pipeline": [
{
"$match": {
"$and": [
{
"$expr": {
"$eq": [
"$model",
"$$model"
]
}
},
{
"$expr": {
"$eq": [
"$category",
"$$category"
]
}
}
]
}
}
],
"as": "result"
}
},
{
"$match": {
"result": {
"$ne": []
}
}
},
{
"$project": {
"_id": 0,
"result": 0
}
}
])
Example here

Related

How to $lookup entries from array in MongoDB?

I have three collections, Posts, Languages and Translations. And the structure looks like this:
Post: {
id: ObjectID
title: String,
translated_versions: ObjectID
}
TranslatedVersion: {
id: ObjectID
translations: Translation[]
}
// Just an object structure in TranslatedVersion.translations, not collection
Translation: {
post: ObjectID
language: ObjectID
}
Language {
id: ObjectID
name: String
}
What I want to do is to get entry from translated versions collections, put it into my post and then for every entry in TranslatedVersion.translation I want to get post and language objects, so it will look like this:
Post {
title: "Test title",
translated_versions: {
id: 'some-id',
tranlations: [
Translation {
language: {
id: 'some-id',
name: 'Turkish'
},
post: {
id: 'some-id'
title: 'Turkish post'
}
},
Translation {
language: {
id: 'some-id',
name: 'English'
},
post: {
id: 'some-id',
title: 'English post'
}
}
]
}
}
I am using Golang and I tried to do it this way (sorry lots of lines of code):
lookup_translations_group := bson.D{
{
Key: "$lookup", Value: bson.D{
{
Key: "from", Value: Models.Translation,
},
{
Key: "localField", Value: "translations_group_id",
},
{
Key: "foreignField", Value: "_id",
},
{
Key: "as", Value: "translations",
},
},
},
}
unwind_translations_group := bson.D{
{
Key: "$unwind", Value: bson.D{
{
Key: "path", Value: "$translations",
},
{
Key: "preserveNullAndEmptyArrays", Value: true,
},
},
},
}
lookup_translations_posts := bson.D{
{
Key: "$lookup", Value: bson.D{
{
Key: "from", Value: Models.Post,
},
{
Key: "localField", Value: "translations.translations.post",
},
{
Key: "foreignField", Value: "_id",
},
{
Key: "as", Value: "translations.translations.post",
},
},
},
}
Then I run mongo.Pipeline and what I get is an object with "post" field which is an array with my translated posts:
So obviously I am doing something wrong here :( I assume I need to add another pipeline in lookup_translations_group stage, but I have no idea how to it correctly.
upd2:
Even more complex examples:
db.posts.insertMany([
{ "_id" : 1, "title": "English post", language: 1, category: 1, "translations": 1, },
{ "_id" : 2, "title": "Turkish post", langauge: 2, category: 1, "translations": 1, },
{ "_id" : 2, "title": "Finnish post", language: 3, category: 1, "translations": 1, },
])
db.languages.insertMany([
{ "_id" : 1, "name": "English", },
{ "_id" : 2, "name": "Turkish", },
{ "_id" : 3, "name": "Finnish", },
])
db.translations.insert({
"_id" : 1,
translations: [
{ language: 1, post: 1 },
{ language: 2, post: 2 },
{ language: 3, post: 3 },
]
})
db.categories.insert({
"_id" : 1,
"name": "Random category"
})
Result that I want:
{
"_id": 1,
"title": "English post",
"language": {
"_id": 1,
"name": "English"
},
"category": {
"_id": 1,
"name": "Random category"
},
"translations": {
"_id": 1,
translations: [
{
"language": {
"_id": 1,
"name": "English"
},
"post": {
"_id": 1,
"title": "English post",
"category": {
"_id": 1,
"name": "Random category"
}
}
},
{
"language": {
"_id": 2,
"name": "Turkish"
},
"post": {
"_id": 2,
"title": "Turkish post",
"category": {
"_id": 1,
"name": "Random category"
}
}
},
{
"language": {
"_id": 3,
"name": "Finnish"
},
"post": {
"_id": 3,
"title": "Finnish post",
"category": {
"_id": 1,
"name": "Random category"
}
}
},
]
}
}
If I understand correctly, you want something like:
db.translations.aggregate([
{$match: {_id: 1}},
{$lookup: {
from: "posts",
localField: "translations.post",
foreignField: "_id",
as: "posts"
}},
{$lookup: {
from: "languages",
localField: "posts.language",
foreignField: "_id",
as: "translations"
}},
{$project: {
data: {$map: {
input: "$posts",
in: {
post: {title: "$$this.title", _id: "$$this._id"},
language: {$arrayElemAt: [
"$translations",
{$indexOfArray: ["$translations._id", "$$this._id"]}
]}
}
}}
}},
{$replaceRoot: {
newRoot: {$mergeObjects: [
{$first: {$filter: {input: "$data", cond: {$eq: ["$$this.post._id", "$_id"]}}}},
{translations: {
_id: "$_id",
translations: {
$filter: {input: "$data", cond: {$ne: ["$$this.post._id", "$_id"]}}
}
}}
]}
}},
{$project: {_id: "$post._id", title: "$post.title", language: 1, translations: 1}}
])
See how it works on the playground example

Filter on collection and merge results in another collection in MongoDB

I am using MongoDB 4.2.9 and have the following requirements:
Collection 'A' has multiple documents with a string field 'status' that I need to filter on
Collection 'B' has multiple documents
Collection A
{ _id: "1",
status: "Report",
type: "Academy",
rating: "Excellent",
ReportNo: "A1"
},
{ _id: "2",
status: "Open",
type: "Academy",
rating: "",
ReportNo: ""
},
{ _id: "3",
status: "Draft",
type: "Academy",
rating: "",
ReportNo: ""
},
{ _id: "4",
status: "Report",
type: "Academy",
rating: "Great",
ReportNo: "A4"
}
Collection B
{ _id: "98",
status: "Archived",
type: "Academy",
rating: "So So",
ReportNo: "X2"
},
{ _id: "99",
status: "Archived",
type: "Academy",
rating: "Great",
ReportNo: "X1"
}
Resulting View
{ _id: "1",
status: "Report",
type: "Academy",
rating: "Excellent",
ReportNo: "A1"
},
{ _id: "4",
status: "Report",
type: "Academy",
rating: "Great",
ReportNo: "A4"
},
{ _id: "98",
status: "Archived",
type: "Academy",
rating: "So So",
ReportNo: "X2"
},
{ _id: "99",
status: "Archived",
type: "Academy",
rating: "Great",
ReportNo: "X1"
}
My goal is to create an aggregation view so that I can filter on a status value in Collection 'A' and then merge those results with Collection 'B' and show in the view ?
I can filter on Collection 'A' using the match call, just can't see how to merge resulting documents into Collection 'B'
From my understandings, your "merge" behaviour is actually a union view of filtered view of collection A and collection B.
With MongoDB v4.2, you can use $facet to handle collection A and collection B separately.
simply perform filtering on A
perform uncorrelated $lookup on B
wrangle the result and merge them together to get the union view that you are looking for.
db.createCollection(
"unionView",
{
"viewOn" : "A",
"pipeline" : [
{
"$facet": {
"A": [
{
"$match": {
status: "Report"
}
}
],
"B": [
{
$limit: 1
},
{
"$lookup": {
"from": "B",
"pipeline": [],
"as": "B"
}
},
{
$unwind: "$B"
},
{
"$replaceRoot": {
"newRoot": "$B"
}
}
]
}
},
{
$project: {
all: {
"$setUnion": [
"$A",
"$B"
]
}
}
},
{
$unwind: "$all"
},
{
"$replaceRoot": {
"newRoot": "$all"
}
}
]
}
)
Here is the Mongo Playground for your reference.
With MongoDB v4.4+, you can create a view with $unionWith
db.createCollection(
"unionView",
{
"viewOn" : "A",
"pipeline" : [
{
"$match": {
status: "Report"
}
},
{
"$unionWith": {
"coll": "B"
}
}
]
}
)
Here is the Mongo playground for your reference.

Unable to populate and structure document with aggregate

So I have this schema which have foreign keys to other collections in the database. The document has around 60k posts and each post can have multiple categories and there are around 200 categories. So I'm trying to fetch and structure data based on the category's foreign key and populate the category details and count.
Here's how the main schema and category schema looks like:
const postSchema = new mongoose.Schema( {
post: {
type: mongoose.Schema.Types.ObjectId,
ref: 'PostDetails'
},
categories: [ {
category: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Category'
},
subCategories: [ {
subCategory: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Subcategory'
}
} ]
} ]
} );
const categorySchema = new mongoose.Schema( {
category: {
type: String,
},
categorySlug: {
type: String,
}
} );
I was successful in making the count but the returned data is not what I expected. The returned data shows the id of the categories and the count but no name and slug. Here's how it looks like:
[
{
"_id": [
"617acfd232c766589c23a90c"
],
"count": 876,
"category": []
}
]
I got the above output with the following query:
const aggregateStages = [
{
$group: {
_id: '$categories.category',
count: { $sum: 1 }
}
},
{
$lookup: {
from: "Category",
localField: "categories.category",
foreignField: "_id",
as: "category"
}
}
];
const categories = await Post.aggregate( aggregateStages ).exec();
I'm hoping to get the data as follows:
[
{
"_id": "617acfd232c766589c23a90c",
"count": 876,
"category": 'SomeCategory',
"categorySlug": 'some-category'
}
]
Where am I going wrong and how can I fix it?
SAMPLE DATA FROM DB AS REQUESTED BY MATT OESTREICH
POST DATA
{
"_id": "617adad39054bae2c983c34f",
"post": "617ad1c80597c78ed4cc151e",
"author": "617acc689b309fdbbbdfdfe0",
"categories": [{
"category": "617acfd232c766589c23a8d1",
"subCategories":[]
}]
}
CATEGORY DATA
{
"_id": "617acfd232c766589c23a8d1",
"category": "Lorem Ipsum",
"categorySlug": "lorem-ipsum"
}
Ok so, it looks like you can resolve this by using the $size operator. The $size operator will give you the length (or count) of elements in an array.
Live demo here
Database
db={
"post": [
{
"_id": ObjectId("617adad39054bae2c983c34f"),
"post": ObjectId("617ad1c80597c78ed4cc151e"),
"author": ObjectId("617acc689b309fdbbbdfdfe0"),
"categories": [
{
"category": ObjectId("617acfd232c766589c23a8d1"),
"subCategories": []
}
]
}
],
"categories": [
{
"_id": ObjectId("617acfd232c766589c23a8d1"),
"category": "Lorem Ipsum",
"categorySlug": "lorem-ipsum"
}
]
}
Query
db.post.aggregate([
{
"$lookup": {
"from": "categories",
"localField": "categories.category",
"foreignField": "_id",
"as": "found_categories"
}
},
{
"$project": {
_id: "$_id",
count: {
"$size": "$found_categories"
},
"category": {
"$first": "$found_categories.category"
},
"categorySlug": {
"$first": "$found_categories.categorySlug"
}
}
}
])
Result
[
{
"_id": ObjectId("617adad39054bae2c983c34f"),
"category": "Lorem Ipsum",
"categorySlug": "lorem-ipsum",
"count": 1
}
]
Although, I do not believe this will give you what you are looking for if more than one category is found. Please let me know if it doesn't work and I will try to help fix it.

Comparing 2 fields from $project in a mongoDB pipeline

In a previous post I created a mongodb query projecting the number of elements matching a condition in an array. Now I need to filter this number of elements depending on another field.
This is my db :
db={
"fridges": [
{
_id: 1,
items: [
{
itemId: 1,
name: "beer"
},
{
itemId: 2,
name: "chicken"
}
],
brand: "Bosch",
size: 195,
cooler: true,
color: "grey",
nbMax: 2
},
{
_id: 2,
items: [
{
itemId: 1,
name: "beer"
},
{
itemId: 2,
name: "chicken"
},
{
itemId: 3,
name: "lettuce"
}
],
brand: "Electrolux",
size: 200,
cooler: true,
color: "white",
nbMax: 2
},
]
}
This is my query :
db.fridges.aggregate([
{
$match: {
$and: [
{
"brand": {
$in: [
"Bosch",
"Electrolux"
]
}
},
{
"color": {
$in: [
"grey",
"white"
]
}
}
]
}
},
{
$project: {
"itemsNumber": {
$size: {
"$filter": {
"input": "$items",
"as": "item",
"cond": {
$in: [
"$$item.name",
[
"beer",
"lettuce"
]
]
}
}
}
},
brand: 1,
cooler: 1,
color: 1,
nbMax: 1
}
}
])
The runnable example.
Which gives me this :
[
{
"_id": 1,
"brand": "Bosch",
"color": "grey",
"cooler": true,
"itemsNumber": 1,
"nbMax": 2
},
{
"_id": 2,
"brand": "Electrolux",
"color": "white",
"cooler": true,
"itemsNumber": 2,
"nbMax": 2
}
]
What I expect is to keep only the results having a itemsNumber different from nbMax. In this instance, the second fridge with _id:2 would not match the condition and should not be in returned. How can I modify my query to get this :
[
{
"_id": 1,
"brand": "Bosch",
"color": "grey",
"cooler": true,
"itemsNumber": 1,
"nbMax": 2
}
]
You can put a $match stage with expression condition at the end of your query,
$ne to check both fields should not same
{
$match: {
$expr: { $ne: ["$nbMax", "$itemsNumber"] }
}
}
Playground

How to get distinct combinations of two fields from a collection when one of the fields is in an array of subdocuments

From a collection consisting of documents representing products similar to the following:
[
{
code: "0WE3A5CMY",
name: "lorem",
category: "voluptas",
variants: [
{
color: "PapayaWhip",
stock: 17,
barcode: 4937310396997
},
{
color: "RoyalBlue",
stock: 13,
barcode: 9787252504890
},
{
color: "DodgerBlue",
stock: 110,
barcode: 97194456959791
}
]
},
{
code: "0WE3A5CMX",
name: "ipsum",
category: "temporibus",
variants: [
{
color: "RoyalBlue",
stock: 113,
barcode: 23425202111840
},
{
color: "DodgerBlue",
stock: 10,
barcode: 2342520211841
}
]
},
{
code: "0WE3A5CMZ",
name: "dolor",
category: "temporibus",
variants: [
{
color: "MaroonRed",
stock: 17,
barcode: 3376911253701
},
{
color: "RoyalBlue",
stock: 12,
barcode: 3376911253702
},
{
color: "DodgerBlue",
stock: 4,
barcode: 3376911253703
}
]
}
]
I would like to retrieve distinct combinations of variants.color and category. So the result should be:
[
{
category: 'voluptas',
color: 'PapayaWhip',
},
{
category: 'voluptas',
color: 'RoyalBlue',
},
{
category: 'voluptas',
color: 'DodgerBlue',
},
{
category: 'temporibus',
color: 'RoyalBlue',
},
{
category: 'temporibus',
color: 'DodgerBlue',
}
]
Based on some cursory research I think I will have to use an aggregate but I've never worked with those and could use some help. I've tried the solution at How to efficiently perform "distinct" with multiple keys?
I've tried the method mentioned by jcarter in the comments but it doesn't solve my problem. If I do:
db.products.aggregate([
{
$group: {
_id: {
"category": "$category",
"color": "$variants.color"
}
}
}
])
I get the result:
[
{
"_id": {
"category": "temporibus",
"color": [
"MaroonRed",
"RoyalBlue",
"DodgerBlue"
]
}
},
{
"_id": {
"category": "temporibus",
"color": [
"RoyalBlue",
"DodgerBlue"
]
}
},
{
"_id": {
"category": "voluptas",
"color": [
"PapayaWhip",
"RoyalBlue",
"DodgerBlue"
]
}
}
]
Which isn't what I need.
Since variants is an array you need to unwind it & group on two fields to get unique docs based on category + 'variants.color' combo.
As group stage results something like :
[
{
"_id": {
"category": "voluptas",
"color": "DodgerBlue"
}
},
{
"_id": {
"category": "voluptas",
"color": "PapayaWhip"
}
}
]
then using $replaceRoot stage you can make _id object field as root for each document to get desired result.
Query :
db.collection.aggregate([
{
$unwind: "$variants"
},
{
$group: { _id: { "category": "$category", "color": "$variants.color" } }
},
{
$replaceRoot: { newRoot: "$_id" }
}
])
Test : mongoplayground