MongoDB Query - query on values of any key in a sub-object: $match combined with $elemMatch - mongodb

How can I filter on all userIDs that have color blue and size 50 in the same element of the list? Only user 1347 should be output.
{
"userId": "12347",
"settings": [
{ name: "SettingA", color: "blue", size: 10 },
{ name: "SettingB", color: "blue", size: 20 },
{ name: "SettingC", color: "green", size: 50 }
],
}
{
"userId": "1347",
"settings": [
{ name: "SettingA", color: "blue", size: 10 },
{ name: "SettingB", color: "blue", size: 50 },
{ name: "SettingC", color: "green", size: 20 }
]
}
If this can be done with $elemMatch, how can I include it in the following query, assuming the following two elements needs to be in the same list: { "rounds.round_values.decision" : "Fold"},
{ "rounds.round_values.gameStage" : "PreFlop"}
I tried this query but it doesn't yield any results. I've read that because elemMatch deosnt' work in projections. But how can I tell $filter to only return objects that have the $elemmMatch conditions met?
db.games.aggregate([
{ $match: { $and: [
{ Template: "PPStrategy4016" },
{ FinalOutcome: "Lost" }]
}},
{ $elemMatch: {
{ "rounds.round_values.decision" : "Fold"},
{ "rounds.round_values.gameStage" : "PreFlop"}
} },
{
$group: {
_id: null,
total: {
$sum: "$FinalFundsChange"
}
}
} ] )

Following the given documents, the query is something such as follows:
db.games.aggregate(
{$unwind : "$settings"},
{$match: {"settings.color" : "blue", "settings.size" : 50}} ,
{$group: {_id: null, total: {$sum: "$settings.size"}}} )
If you have difficulties in transforming it into your own domain, pleas supply some example documents from your domain.

Related

How to get distinct combinations of two fields from a collection when one of the fields is in an array of subdocuments

From a collection consisting of documents representing products similar to the following:
[
{
code: "0WE3A5CMY",
name: "lorem",
category: "voluptas",
variants: [
{
color: "PapayaWhip",
stock: 17,
barcode: 4937310396997
},
{
color: "RoyalBlue",
stock: 13,
barcode: 9787252504890
},
{
color: "DodgerBlue",
stock: 110,
barcode: 97194456959791
}
]
},
{
code: "0WE3A5CMX",
name: "ipsum",
category: "temporibus",
variants: [
{
color: "RoyalBlue",
stock: 113,
barcode: 23425202111840
},
{
color: "DodgerBlue",
stock: 10,
barcode: 2342520211841
}
]
},
{
code: "0WE3A5CMZ",
name: "dolor",
category: "temporibus",
variants: [
{
color: "MaroonRed",
stock: 17,
barcode: 3376911253701
},
{
color: "RoyalBlue",
stock: 12,
barcode: 3376911253702
},
{
color: "DodgerBlue",
stock: 4,
barcode: 3376911253703
}
]
}
]
I would like to retrieve distinct combinations of variants.color and category. So the result should be:
[
{
category: 'voluptas',
color: 'PapayaWhip',
},
{
category: 'voluptas',
color: 'RoyalBlue',
},
{
category: 'voluptas',
color: 'DodgerBlue',
},
{
category: 'temporibus',
color: 'RoyalBlue',
},
{
category: 'temporibus',
color: 'DodgerBlue',
}
]
Based on some cursory research I think I will have to use an aggregate but I've never worked with those and could use some help. I've tried the solution at How to efficiently perform "distinct" with multiple keys?
I've tried the method mentioned by jcarter in the comments but it doesn't solve my problem. If I do:
db.products.aggregate([
{
$group: {
_id: {
"category": "$category",
"color": "$variants.color"
}
}
}
])
I get the result:
[
{
"_id": {
"category": "temporibus",
"color": [
"MaroonRed",
"RoyalBlue",
"DodgerBlue"
]
}
},
{
"_id": {
"category": "temporibus",
"color": [
"RoyalBlue",
"DodgerBlue"
]
}
},
{
"_id": {
"category": "voluptas",
"color": [
"PapayaWhip",
"RoyalBlue",
"DodgerBlue"
]
}
}
]
Which isn't what I need.
Since variants is an array you need to unwind it & group on two fields to get unique docs based on category + 'variants.color' combo.
As group stage results something like :
[
{
"_id": {
"category": "voluptas",
"color": "DodgerBlue"
}
},
{
"_id": {
"category": "voluptas",
"color": "PapayaWhip"
}
}
]
then using $replaceRoot stage you can make _id object field as root for each document to get desired result.
Query :
db.collection.aggregate([
{
$unwind: "$variants"
},
{
$group: { _id: { "category": "$category", "color": "$variants.color" } }
},
{
$replaceRoot: { newRoot: "$_id" }
}
])
Test : mongoplayground

Can I avoid using the same $match criteria twice when using $unwind?

Take the following data as an example:
{
_id: 1,
item: "abc",
stock: [
{ size: "S", color: "red", quantity: 25 },
{ size: "S", color: "blue", quantity: 10 },
{ size: "M", color: "blue", quantity: 50 }
]
}
{
_id: 2,
item: "def",
stock: [
{ size: "S", color: "blue", quantity: 20 },
{ size: "M", color: "blue", quantity: 5 },
{ size: "M", color: "black", quantity: 10 },
{ size: "L", color: "red", quantity: 2 }
]
}
{
_id: 3,
item: "ijk",
stock: [
{ size: "M", color: "blue", quantity: 15 },
{ size: "L", color: "blue", quantity: 100 },
{ size: "L", color: "red", quantity: 25 }
]
}
Say I'm going to filter out the stocks that matches the criteria size = 'L'. I already have a multikey index on the stock.size field.
In the aggregation pipeline, if I use the following two operations:
[{$unwind: {path: "$stock"}},
{$match: {"stock.size": "L"}}]
I will get the desired results, but when the db gets very large, the $unwind step will have to scan the whole collection, without utilizing the existing index, which is very inefficient.
If I reverse the order of the $unwind and $match operations, the $match will utilize the index to apply an early filtering, but the final result will not be as desired: it will fetch the extra stocks that are not of size L, but have sibling L-sized stocks that belong to the same item.
Would I have to use the same $match operation twice, i.e. both before and after the $unwind, to make it both utilizing the index and return the correct results?
Yes you can use $match stage twice in the aggregation pipeline but here only the first $match stage will use the index second one will do the collscan.
[
{ "$match": { "stock.size": "L" }},
{ "$unwind": { "path": "$stock" }},
{ "$match": { "stock.size": "L" }}
]
If you want to avoid the $match twice then use $filter aggregation
[
{ "$match": { "stock.size": "L" } },
{ "$addFields": {
"stock": {
"$filter": {
"input": "$stock",
"as": "st",
"cond": { "$eq": ["$stock.size", "L"] }
}
}
}}
]

Faceted filters in Mongo DB with for products with multiple options

we are trying to create call to MangoDB to receive all possible filters for products.
I will try to create example of our products
First product is Adidas Shoes which have two options to select - colour and size. But for different colours you have different sizes.
{
id: 1
name: "Adidas Shoes",
filters: [
[
{
code: "brand",
value: "Adidas"
},
{
code: "colour",
value: "white"
},
{
code: "size",
value: 41
}
],
[
{
code: "brand",
value: "Adidas"
},
{
code: "colour",
value: "white"
},
{
code: "size",
value: 42
}
],
[
{
code: "brand",
value: "Adidas"
},
{
code: "colour",
value: "white"
},
{
code: "size",
value: 43
}
],
[
{
code: "brand",
value: "Adidas"
},
{
code: "colour",
value: "blue"
},
{
code: "size",
value: 41
}
],
[
{
code: "brand",
value: "Adidas"
},
{
code: "colour",
value: "blue"
},
{
code: "size",
value: 44
}
]
]
}
Second product is Nike Shoes.
{
id: 2
name: "Nike Shoes",
filters: [
[
{
code: "brand",
value: "Nike",
},
{
code: "colour",
value: "white",
},
{
code: "size",
value: 41,
}
],
[
{
code: "brand",
value: "Nike",
},
{
code: "colour",
value: "white",
},
{
code: "size",
value: 42,
}
],
[
{
code: "brand",
value: "Nike",
},
{
code: "colour",
value: "green",
},
{
code: "size",
value: 41,
}
]
]
}
And Reebook shoes
{
id: 3
name: "Reebook Shoes",
filters: [
[
{
code: "brand",
value: "Reebook",
},
{
code: "colour",
value: "black",
},
{
code: "size",
value: 41,
}
]
]
}
as you can see option size is dependent on colour and colour is dependent on size.
How we can create MongoDb.aggregate to have all possible filters?
Brand: Adidas (1), Nike (1), Reebook (1)
Size: 41 (3), 42 (2), 43 (1), 44 (1)
Colour: White (2), Blue (1), Green (1), Black (1)
And call should be independent on how many and which filters we have (products with one option, products with more options and different filters). Can you explain how to use $group, $unwind in this situation? And how we can improve it later with $facet?
Thanks a lot!!!
EDIT 02/10/2017
Sample response
facets: [
{
code: "brand",
values: [
{
name: "Adidas",
count: 1
},
{
name: "Nike",
count: 1
},
{
name: "Reebook",
count: 1
}
]
},
{
code: "size",
values: [
{
name: 41,
count: 3
},
{
name: 42,
count: 2
},
{
name: 43,
count: 1
},
{
name: 44,
count: 1
}
]
},
{
code: "colour",
values: [
{
name: "White",
count: 2
},
{
name: "Blue",
count: 1
},
{
name: "Green",
count: 1
},
{
name: "Black",
count: 1
}
]
}
]
or
facets: {
"brand": {
"Adidas": 1,
"Nike":1,
"Reebook":1
},
"size": {
"41": 3,
"42":2,
"43":1,
"44":1
},
"colour": {
"White": 2,
"Blue":1,
"Green":1,
"Black":1
}
}
This is our first stage. Next step will be how to search possible filters when I have selected Size: 41 and Colour: White.
Thanks
Here is an aggregation that might work for you.
db.getCollection('test').aggregate([
{$unwind: '$filters'},
{$unwind: '$filters'},
{
$group: {
_id: {code: '$filters.code', value: '$filters.value'},
products: {$addToSet: '$_id'}
}
},
{
$project: {
'filter.value': '$_id.value',
'filter.count': {$size: '$products'}
}
},
{
$group: {
_id: '$_id.code',
filters: {$push: '$filter'}
}
}
]);
The data you need comes in a slightly different format because there's no easy way to convert array of grouped values to object properties.
If some filters are already selected you need another $match stage after the first $unwind.
It also supports multi selects. Say I want white/black shoes made by Reebook/Adidas.
db.getCollection('test').aggregate([
{$unwind: '$filters'},
{
$match: {
$and: [
//Add objects here fo everything that is selected already
{'filters': {$elemMatch: {code: 'colour', value: {$in: ['black', 'white']}}}},
{'filters': {$elemMatch: {code: 'brand', value: {$in: ['Adidas', 'Reebook']}}}}
]
}
},
{$unwind: '$filters'},
{
$group: {
_id: {code: '$filters.code', value: '$filters.value'},
products: {$addToSet: '$_id'}
}
},
{
$project: {
'filter.value': '$_id.value',
'filter.count': {$size: '$products'}
}
},
{
$group: {
_id: '$_id.code',
filters: {$push: '$filter'}
}
}
]);
The last thing is the dependent behavior like this:
Select Nike => Size and Color are filtered by brand but you are still able to select all brands.
Select Nike + 42 Size => You can select only brands having size 42, colors and brands for which there are shoes of 42 size.
And so on.
You can leverage $facet for it. In fact when the idea is next.
If we're calculating brands - we should filter records by what's selected in size and color dropdowns.
If we're calculating size - applying color and brand. Same logic for color.
Here is the code that worked in mongo shell:
//This hash is going to by dynamic
//Add and remove properties, change $in arrays
//Depending on what user does
var conditionsForUserSelect = {
'colour': {'filters': {$elemMatch: {code: 'colour', value: {$in: ['green']}}}},
'brand': {'filters': {$elemMatch: {code: 'brand', value: {$in: ['Nike']}}}},
'size': {'filters': {$elemMatch: {code: 'size', value: {$in: [41]}}}}
};
var getFacetStage = function (code) {
//Empty object, accept all filters if nothing is selected
var matchStageCondition = {};
var selectedFilters = Object.keys(conditionsForUserSelect);
if (selectedFilters && selectedFilters.length) {
//Take all queries EXCEPT for the passed
//E.g. if we are counting brand filters then we should apply color and size.
//Because for example if no size/colour selected we should
//allow all brands even if Reebok is selected
var conditionsToApply = selectedFilters
.filter(function (key) {
return key !== code
})
.map(function (key) {
return conditionsForUserSelect[key]
});
if (conditionsToApply && conditionsToApply.length) {
matchStageCondition = {
$and: conditionsToApply
};
}
}
return [
{$unwind: '$filters'},
{
$match: matchStageCondition
},
{$unwind: '$filters'},
{
$group: {
_id: {code: '$filters.code', value: '$filters.value'},
products: {$addToSet: '$_id'}
}
},
{
$project: {
'filter.value': '$_id.value',
'filter.count': {$size: '$products'}
}
},
{
$group: {
_id: '$_id.code',
filters: {$push: '$filter'}
}
},
{
$match: {_id: code}
}
];
};
db.getCollection('test').aggregate([
{
$facet: {
colour: getFacetStage('colour'),
size: getFacetStage('size'),
brand: getFacetStage('brand')
}
}
]);

MongoDB grouping based on intervals

I would like to group my data based on number intervals in measurements. Can I do this with the aggregation framework, or with some map-reduce function?
I would like to group by color and whether the size is larger or smaller than 5. I would also want to add e.g. "medium" for sizes between 3 and 5.
I can group by size and color, but then each different size will have its own object.
I know this can be done by checking each different object's size by db.collection.find(), and then adding them according to my specifications, but that would be very slow.
Example:
Objects:
{
color: "red",
size: 2
}
{
color: "red",
size: 4
}
{
color: "blue",
size: 2
}
{
color: "blue",
size: 1
}
{
color: "blue",
size: 7
}
Output:
{
_id: {
color: "red",
size: "small"
}
total size: 6
}
{
_id: {
color: "red",
size: "large"
}
total size: 0
}
{
_id: {
color: "blue",
size: small
}
total size: 3
}
{
_id: {
color: "blue",
size: "large"
}
total size: 7
}
This is easy using $cond:
db.collection.aggregate([
{ "$group": {
"_id": {
"color": "$color",
"size": {
"$cond": [
{ "$lt": [ "$size", 3 ] },
"small",
{ "$cond": [
{ "$lt": [ "$size", 6 ] },
"medium",
"large"
]}
]
}
},
"total_size": { "$sum": "$size" }
}}
])
So just conditionally select the value in the grouping key based on the current value in the document and count.

MongoDB aggregate object map to distinct values

I got three documents:
[
{ _id: 1, article: 1, details: { color: "red" } },
{ _id: 2, article: 1, details: { color: "blue", size: 44 },
{ _id: 3, article: 2, details: { color: "blue", size: 44 }
]
which I want to transform in a query to:
[
{ article: 1, details: { color: ["red", "blue"], size: [44] } },
{ article: 2, details: { color: ["blue"], size: [44] }
]
at the moment this is achieved by mapReduce:
db.varieties.mapReduce(map, reduce, { out: { inline: 1 } });
function map() {
for (var key in this.details) {
this.details[key] = [this.details[key]];
}
emit(this.article, this.details);
}
function reduce(article, details) {
var result = {};
details.forEach(function(detail) {
for (var key in detail) {
if (!Array.isArray(result[key])) result[key] = [];
if (~result[key].indexOf(detail[key])) result[key].concat(detail[key]);
}
});
return result;
}
However I would like to do this through the mongodb aggregation frame work as the map reduce implementation in my environment is very "difficult".
Regarding the aggregation I am so far:
var pipeline = [];
pipeline.push({ $project: { article: 1, details: 1 } });
pipeline.push({ $group: { _id: "$article", details: { $push: '$details' } });
db.varieties.aggregate(pipeline);
However this will just return:
[
{ article: 1, details: [{ color: "red", size: 44 }, { color: "blue", size: 44 }] },
{ article: 2, details: [{ color: "blue", size: 44 }]
]
I read somewhere that this a use case for $unwind unfortunately this will not work on objects.
So lets get to my questions:
Is it possible to somehow convert the details object to an array with { key: "color", value: "red" } and if yes how to achieve this?
If the above is not possible and I would restructure my documents to the be stored in the above format (details as array) how would I need to complete my aggregation to get the same result from my origin mapReduce?
I cannot hardcode the keys of details. The aggregation must work on details of unknown keys.
You are better of using the aggregation framework:
db.colors.aggregate([
{ "$group": {
"_id": "$article",
"color": {"$addToSet": "$details.color" },
"size": { "$addToSet": "$details.size" }
}},
{ "$project": {
"details": {
"color": "$color",
"size": "$size"
}
}}
])
Produces:
{ "_id" : 2, "details" : { "color" : [ "blue" ], "size" : [ 44 ] } }
{ "_id" : 1, "details" : { "color" : [ "blue", "red" ], "size" : [ 44 ] } }
So you cannot have those keys under "details" when you $group but you can always $project to the form you want in the result.
The aggregation framework is a native code implementation and runs much faster than the JavaScript interpreter driven mapReduce.
But if you really need the flexibility the concept is similar, it simply takes a bit longer but will work with different keys under details:
db.colors.mapReduce(
function () {
emit( this.article, this.details );
},
function (key,values) {
var reduced = {
};
values.forEach(function(value) {
for ( var k in value ) {
if ( !reduced.hasOwnProperty(k) )
reduced[k] = [];
if ( reduced[k].indexOf( value[k] ) == -1 )
reduced[k].push( value[k] );
}
});
return reduced;
},
{
"finalize": function(key,value) {
for (var k in value) {
if ( Object.prototype.toString.call( value[k] ) !== '[object Array]') {
var replace = [];
replace.push( value[k] );
value[k] = replace;
}
}
return value;
},
"out": { "inline": 1 }
}
)
But that is all in a very "mapReduce" way, so the values of the main fields are going to be different.
{ "_id" : 1, "value" : { "color" : [ "blue", "red" ], "size" : [ 44 ] } }
{ "_id" : 2, "value" : { "color" : [ "blue" ], "size" : [ 44 ] } }