In mongo db, I have the input collection
Collection name - dirPermission and sample record
[
{
dirId: "1",
dirName: "firstDir",
usersRead: [
"user1",
"user2"
],
userWrite: [
"user2",
"user3"
]
}
]
and I want to create a materialized view like the below
[{
dirId:'1',
dirName:'firstDir',
userId:'user1',
canRead:'Y',
canWrite:'N'
},
{
dirId:'1',
dirName:'firstDir',
userId:'user2',
canRead:'Y',
canWrite:'Y'
},
{
dirId:'1',
dirName:'firstDir',
userId:'user3',
canRead:'N',
canWrite:'Y'
}]
Again since my back ground is more of SQL + Java, I am struggling to find an answer using mongodb and any pointers will be helpful.
You can use $setUnion to create a distinct set of users. $unwind it to create documents and use $cond to set canRead and canWrite.
db.collection.aggregate([
{
"$addFields": {
"allUsers": {
$setUnion: [
"$usersRead",
"$userWrite"
]
}
}
},
{
"$unwind": "$allUsers"
},
{
"$project": {
dirId: 1,
dirName: 1,
userId: "$allUsers",
canRead: {
"$cond": {
"if": {
"$in": [
"$allUsers",
"$usersRead"
]
},
"then": "Y",
"else": "N"
}
},
canWrite: {
"$cond": {
"if": {
"$in": [
"$allUsers",
"$userWrite"
]
},
"then": "Y",
"else": "N"
}
}
}
}
])
Mongo Playground
Related
Trying to figure out the syntax to make a mongoDB group that would contain the documents that haven't been grouped because it didn't pass the conditional.
Document looks in part like
{
"_id" : ObjectId("value"),
"updatedAt" : ISODate("value"),
"zip" : "11209",
"state" : "NY",
"city" : "New York",
}
I created mutually exclusive string arrays for matching the other groups, like
east_ny_zipcodes = ["11209", "11210", "11211", ...]
lower_ny_zipcodes = ["11212", "11213", "11214", ...]
This is an example query from #Neal Lunn with some defined groups and values
db.zips.aggregate([
{ "$group": {
"_id": null,
"lower_ny": {
"$sum": {
"$cond": [{ "$in": [ "$zip", lower_ny_zipcodes ] },1,0]
}
},
"east_ny": {
"$sum": {
"$cond": [{ "$in": [ "$zip", east_ny_zipcodes ] },1,0]
}
},
"upper_ny": {
"$sum": {
"$cond": [{ "$in": [ "$zip", upper_ny_zipcodes ] },1,0]
}
}
}}
])
I would like to create another catch all group that would catch everything that these defined groups didn't, but also give me the documents and not just count so I can drill down into it, something like
"other_zips":{
"$push":{
"$cond": [{ "$nin": [ "$zip", lower_ny, east_ny, upper_ny,...]}]
}
You can try below code.
all - variable holding all zip codes
$$ROOT - complete document
The below code will push the whole document when not found else it will insert null element.
You can filter the null document in the subsequent stage if you chose.
$addToSet will keep atmost one null doc.
Documents:
"other_zips_doc":{
"$addToSet":{
"$cond": [{ $not: [ { $in: [ "$zip", all ] } ] }, "$$ROOT", null]
}
}
Count:
"other_zips_count":{
"$sum":{
"$cond": [{ $not: [ { $in: [ "$zip", all ] } ] }, 1, 0]
}
}
I have a collection set with documents like :
{
"_id": ObjectId("57065ee93f0762541749574e"),
"name": "myName",
"results" : [
{
"_id" : ObjectId("570e3e43628ba58c1735009b"),
"color" : "GREEN",
"week" : 17,
"year" : 2016
},
{
"_id" : ObjectId("570e3e43628ba58c1735009d"),
"color" : "RED",
"week" : 19,
"year" : 2016
}
]
}
I am trying to build a query witch alow me to return all documents of my collection but only select the field 'results' with subdocuments if week > X and year > Y.
I can select the documents where week > X and year > Y with the aggregate function and a $match but I miss documents with no match.
So far, here is my function :
query = ModelUser.aggregate(
{$unwind:{path:'$results', preserveNullAndEmptyArrays:true}},
{$match:{
$or: [
{$and:[
{'results.week':{$gte:parseInt(week)}},
{'results.year':{$eq:parseInt(year)}}
]},
{'results.year':{$gt:parseInt(year)}},
{'results.week':{$exists: false}}
{$group:{
_id: {
_id:'$_id',
name: '$name'
},
results: {$push:{
_id:'$results._id',
color: '$results.color',
numSemaine: '$results.numSemaine',
year: '$results.year'
}}
}},
{$project: {
_id: '$_id._id',
name: '$_id.name',
results: '$results'
);
The only thing I miss is : I have to get all 'name' even if there is no result to display.
Any idea how to do this without 2 queries ?
It looks like you actually have MongoDB 3.2, so use $filter on the array. This will just return an "empty" array [] where the conditions supplied did not match anything:
db.collection.aggregate([
{ "$project": {
"name": 1,
"user": 1,
"results": {
"$filter": {
"input": "$results",
"as": "result",
"cond": {
"$and": [
{ "$eq": [ "$$result.year", year ] },
{ "$or": [
{ "$gt": [ "$$result.week", week ] },
{ "$not": { "$ifNull": [ "$$result.week", false ] } }
]}
]
}
}
}
}}
])
Where the $ifNull test in place of $exists as a logical form can actually "compact" the condition since it returns an alternate value where the property is not present, to:
db.collection.aggregate([
{ "$project": {
"name": 1,
"user": 1,
"results": {
"$filter": {
"input": "$results",
"as": "result",
"cond": {
"$and": [
{ "$eq": [ "$$result.year", year ] },
{ "$gt": [
{ "$ifNull": [ "$$result.week", week+1 ] },
week
]}
]
}
}
}
}}
])
In MongoDB 2.6 releases, you can probably get away with using $redact and $$DESCEND, but of course need to fake the match in the top level document. This has similar usage of the $ifNull operator:
db.collection.aggregate([
{ "$redact": {
"$cond": {
"if": {
"$and": [
{ "$eq": [{ "$ifNull": [ "$year", year ] }, year ] },
{ "$gt": [
{ "$ifNull": [ "$week", week+1 ] }
week
]}
]
},
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}}
])
If you actually have MongoDB 2.4, then you are probably better off filtering the array content in client code instead. Every language has methods for filtering array content, but as a JavaScript example reproducible in the shell:
db.collection.find().forEach(function(doc) {
doc.results = doc.results.filter(function(result) {
return (
result.year == year &&
( result.hasOwnProperty('week') ? result.week > week : true )
)
]);
printjson(doc);
})
The reason being is that prior to MongoDB 2.6 you need to use $unwind and $group, and various stages in-between. This is a "very costly" operation on the server, considering that all you want to do is remove items from the arrays of documents and not actually "aggregate" from items within the array.
MongoDB releases have gone to great lengths to provide array processing that does not use $unwind, since it's usage for that purpose alone is not a performant option. It should only ever be used in the case where you are removing a "significant" amount of data from arrays as a result.
The whole point is that otherwise the "cost" of the aggregation operation is likely greater than the "cost" of transferring the data over the network to be filtered on the client instead. Use with caution:
db.collection.aggregate([
// Create an array if one does not exist or is already empty
{ "$project": {
"name": 1,
"user": 1,
"results": {
"$cond": [
{ "$ifNull": [ "$results.0", false ] },
"$results",
[false]
]
}
}},
// Unwind the array
{ "$unwind": "$results" },
// Conditionally $push based on match expression and conditionally count
{ "$group": {
"_id": "_id",
"name": { "$first": "$name" },
"user": { "$first": "$user" },
"results": {
"$push": {
"$cond": [
{ "$or": [
{ "$not": "$results" },
{ "$and": [
{ "$eq": [ "$results.year", year ] },
{ "$gt": [
{ "$ifNull": [ "$results.week", week+1 ] },
week
]}
]}
] },
"$results",
false
]
}
},
"count": {
"$sum": {
"$cond": [
{ "$and": [
{ "$eq": [ "$results.year", year ] },
{ "$gt": [
{ "$ifNull": [ "$results.week", week+1 ] },
week
]}
] }
1,
0
]
}
}
}},
// $unwind again
{ "$unwind": "$results" }
// Filter out false items unless count is 0
{ "$match": {
"$or": [
"$results",
{ "count": 0 }
]
}},
// Group again
{ "$group": {
"_id": "_id",
"name": { "$first": "$name" },
"user": { "$first": "$user" },
"results": { "$push": "$results" }
}},
// Now swap [false] for []
{ "$project": {
"name": 1,
"user": 1,
"results": {
"$cond": [
{ "$ne": [ "$results", [false] ] },
"$results",
[]
]
}
}}
])
Now that is a lot of operations and shuffling just to "filter" content from an array compared to all of the other approaches which are really quite simple. And aside from the complexity, it really does "cost" a lot more to execute on the server.
So if your server version actually supports the newer operators that can do this optimally, then it's okay to do so. But if you are stuck with that last process, then you probably should not be doing it and instead do your array filtering in the client.
I have these three MongoDB documents:
{
"_id" : ObjectId("571094afc2bcfe430ddd0815"),
"name" : "Barry",
"surname" : "Allen",
"address" : [
{
"street" : "Red",
"number" : NumberInt(66),
"city" : "Central City"
},
{
"street" : "Yellow",
"number" : NumberInt(7),
"city" : "Gotham City"
}
]
}
{
"_id" : ObjectId("57109504c2bcfe430ddd0816"),
"name" : "Oliver",
"surname" : "Queen",
"address" : {
"street" : "Green",
"number" : NumberInt(66),
"city" : "Star City"
}
}
{
"_id" : ObjectId("5710953ac2bcfe430ddd0817"),
"name" : "Tudof",
"surname" : "Unknown",
"address" : "homeless"
}
The address field is an Array of Objects in the first document, an Object in the second and a String in the third.
My target is to find how many documents of my collection containinig the field address.street. In this case the right count is 1 but with my query I get two:
db.coll.find({"address.street":{"$exists":1}}).count()
I also tried map/reduce. It works but it is slower; so if it is possible, I would avoid it.
The distinction here is that the .count() operation is actually "correct" in returning the "document" count where the field is present. So the general considerations break down to:
If you just want to exlude the documents with the array field
Then the most effective way of excluding those documents where the "street" was a property of the "address" as an "array", then just use the dot-notation property of looking for the 0 index to not exist in the exlcusion:
db.coll.find({
"address.street": { "$exists": true },
"address.0": { "$exists": false }
}).count()
As a natively coded operator test in both cases $exists does the correct job and efficiently.
If you intended to count field occurences
If what you are actually asking is the "field count", where some "documents" contain array entries where that "field" may be present several times.
For that you need the aggregation framework or mapReduce like you mention. MapReduce uses JavaScript based processing and is therefore going to be considerably slower than the .count() operation. The aggregation framework also needs to calculate and "will" be slower than .count(), but not by as much as mapReduce.
In MongoDB 3.2 you get some help here by the expanded ability of $sum to work on an array of values as well as being an grouping accumulator. The other helper here is $isArray which allows a different processing method via $map when the data is in fact "an array":
db.coll.aggregate([
{ "$group": {
"_id": null,
"count": {
"$sum": {
"$sum": {
"$cond": {
"if": { "$isArray": "$address" },
"then": {
"$map": {
"input": "$address",
"as": "el",
"in": {
"$cond": {
"if": { "$ifNull": [ "$$el.street", false ] },
"then": 1,
"else": 0
}
}
}
},
"else": {
"$cond": {
"if": { "$ifNull": [ "$address.street", false ] },
"then": 1,
"else": 0
}
}
}
}
}
}
}}
])
Earlier versions hinge on a bit more conditional processing in order to treat the array and non-array data differently, and generally require $unwind to process array entries.
Either transposing the array via $map with MongoDB 2.6:
db.coll.aggregate([
{ "$project": {
"address": {
"$cond": {
"if": { "$ifNull": [ "$address.0", false ] },
"then": "$address",
"else": {
"$map": {
"input": ["A"],
"as": "el",
"in": "$address"
}
}
}
}
}},
{ "$unwind": "$address" },
{ "$group": {
"_id": null,
"count": {
"$sum": {
"$cond": {
"if": { "$ifNull": [ "$address.street", false ] },
"then": 1,
"else": 0
}
}
}
}}
])
Or providing conditional selection with MongoDB 2.2 or 2.4:
db.coll.aggregate([
{ "$group": {
"_id": "$_id",
"address": {
"$first": {
"$cond": [
{ "$ifNull": [ "$address.0", false ] },
"$address",
{ "$const": [null] }
]
}
},
"other": {
"$push": {
"$cond": [
{ "$ifNull": [ "$address.0", false ] },
null,
"$address"
]
}
},
"has": {
"$first": {
"$cond": [
{ "$ifNull": [ "$address.0", false ] },
1,
0
]
}
}
}},
{ "$unwind": "$address" },
{ "$unwind": "$other" },
{ "$group": {
"_id": null,
"count": {
"$sum": {
"$cond": [
{ "$eq": [ "$has", 1 ] },
{ "$cond": [
{ "$ifNull": [ "$address.street", false ] },
1,
0
]},
{ "$cond": [
{ "$ifNull": [ "$other.street", false ] },
1,
0
]}
]
}
}
}}
])
So the latter form "should" perform a bit better than mapReduce, but probably not by much.
In all cases the logic falls to using $ifNull as the "logical" form of $exists for the aggregation framework. Paired with $cond, a "truthfull" result is obtained when the property actually exsists, and a false value is returned when it is not. This determines whether 1 or 0 is returned respectively to the overall accumulation via $sum.
Ideally you have the modern version that can do this in a single $group pipeline stage, but otherwise you need the longer path.
Can you try this:
db.getCollection('collection_name').find({
"address.street":{"$exists":1},
"$where": "Array.isArray(this.address) == false && typeof this.address === 'object'"
});
In where clause, we are excluding if address is array and
Including address if it's type is object.
I have a little weird database structure it is as follows:
I have a document with normal properties, then I have a metadata property which is an array of objects.
metadata: {[
{
key: [key],
value: [value]
},
...
]}
Edit: There will never be a metadata sub-document which has a duplicate key
It was done this way to retain the order of the metadata objects
Now I want to get distinct values of a metadata object with a given key.
I want to find every distinct [value] where [key] = "x" using MongoDB. And have the distinct values returned in an array (not the document)
I guess this is not possible using the distinct command, but is this possible using an aggregation pipeline or do I have to use Map-Reduce?
Any suggestions?
Thanks in advance! :)
I presume you mean this:
{
"metadata": [
{ "key": "abc", "value": "borf" },
{ "key": "cdc", "value": "biff" }
]
},
{
"metadata": [
{ "key": "bbc", "value": "barf" },
{ "key": "abc", "value": "borf" },
{ "key": "abc", "value": "barf" }
]
}
Where if you filter for "abc" and get the distinct "value" entries like this:
db.collection.aggregate([
{ "$match": { "metadata.key": "abc" } },
{ "$unwind": "$metadata" },
{ "$match": { "metadata.key": "abc" } },
{ "$group": {
"_id": "$metadata.value"
}}
])
Or even better:
db.collection.aggregate([
{ "$match": { "metadata.key": "abc" } },
{ "$redact": {
"$cond": {
"if": { "$eq": [ { "$ifNull": [ "$key", "abc" ] }, "abc" ] },
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}},
{ "$unwind": "$metadata" },
{ "$group": {
"_id": "$metadata.value",
"count": { "$sum": 1 }
}}
])
Which would basically give:
{ "_id": "barf", "count": 1 },
{ "_id": "borf", "count": 2 }
But it is not possible for this to just be an array of "barf" and "borf". The distinct() method does an array of keys only, but it is also very limited. Therefore it can only do this:
db.collection.distinct("metadata.value",{ "metadata.key": "abc" })
[ "biff", "borf", "barf" ]
Which is incorrect as a result. So just take the "document" results from above and apply some "post processing":
db.collection.aggregate([
{ "$match": { "metadata.key": "abc" } },
{ "$redact": {
"$cond": {
"if": { "$eq": [ { "$ifNull": [ "$key", "abc" ] }, "abc" ] },
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}},
{ "$unwind": "$metadata" },
{ "$group": {
"_id": "$metadata.value"
}}
]).map(function(doc) {
return doc._id;
})
And that result is a plain array of just the distinct values:
[ "borf", "barf" ]
I have collection of products. Each product contains array of items.
> db.products.find().pretty()
{
"_id" : ObjectId("54023e8bcef998273f36041d"),
"shop" : "shop1",
"name" : "product1",
"items" : [
{
"date" : "01.02.2100",
"purchasePrice" : 1,
"sellingPrice" : 10,
"count" : 15
},
{
"date" : "31.08.2014",
"purchasePrice" : 10,
"sellingPrice" : 1,
"count" : 5
}
]
}
So, can you please give me an advice, how I can query MongoDB to retrieve all products with only single item which date is equals to the date I pass to query as parameter.
The result for "31.08.2014" must be:
{
"_id" : ObjectId("54023e8bcef998273f36041d"),
"shop" : "shop1",
"name" : "product1",
"items" : [
{
"date" : "31.08.2014",
"purchasePrice" : 10,
"sellingPrice" : 1,
"count" : 5
}
]
}
What you are looking for is the positional $ operator and "projection". For a single field you need to match the required array element using "dot notation", for more than one field use $elemMatch:
db.products.find(
{ "items.date": "31.08.2014" },
{ "shop": 1, "name":1, "items.$": 1 }
)
Or the $elemMatch for more than one matching field:
db.products.find(
{ "items": {
"$elemMatch": { "date": "31.08.2014", "purchasePrice": 1 }
}},
{ "shop": 1, "name":1, "items.$": 1 }
)
These work for a single array element only though and only one will be returned. If you want more than one array element to be returned from your conditions then you need more advanced handling with the aggregation framework.
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$unwind": "$items" },
{ "$match": { "items.date": "31.08.2014" } },
{ "$group": {
"_id": "$_id",
"shop": { "$first": "$shop" },
"name": { "$first": "$name" },
"items": { "$push": "$items" }
}}
])
Or possibly in shorter/faster form since MongoDB 2.6 where your array of items contains unique entries:
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$project": {
"shop": 1,
"name": 1,
"items": {
"$setDifference": [
{ "$map": {
"input": "$items",
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el.date", "31.08.2014" ] },
"$$el",
false
]
}
}},
[false]
]
}
}}
])
Or possibly with $redact, but a little contrived:
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$redact": {
"$cond": [
{ "$eq": [ { "$ifNull": [ "$date", "31.08.2014" ] }, "31.08.2014" ] },
"$$DESCEND",
"$$PRUNE"
]
}}
])
More modern, you would use $filter:
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$addFields": {
"items": {
"input": "$items",
"cond": { "$eq": [ "$$this.date", "31.08.2014" ] }
}
}}
])
And with multiple conditions, the $elemMatch and $and within the $filter:
db.products.aggregate([
{ "$match": {
"$elemMatch": { "date": "31.08.2014", "purchasePrice": 1 }
}},
{ "$addFields": {
"items": {
"input": "$items",
"cond": {
"$and": [
{ "$eq": [ "$$this.date", "31.08.2014" ] },
{ "$eq": [ "$$this.purchasePrice", 1 ] }
]
}
}
}}
])
So it just depends on whether you always expect a single element to match or multiple elements, and then which approach is better. But where possible the .find() method will generally be faster since it lacks the overhead of the other operations, which in those last to forms does not lag that far behind at all.
As a side note, your "dates" are represented as strings which is not a very good idea going forward. Consider changing these to proper Date object types, which will greatly help you in the future.
Based on Neil Lunn's code I work with this solution, it includes automatically all first level keys (but you could also exclude keys if you want):
db.products.find(
{ "items.date": "31.08.2014" },
{ "shop": 1, "name":1, "items.$": 1 }
{ items: { $elemMatch: { date: "31.08.2014" } } },
)
With multiple requirements:
db.products.find(
{ "items": {
"$elemMatch": { "date": "31.08.2014", "purchasePrice": 1 }
}},
{ items: { $elemMatch: { "date": "31.08.2014", "purchasePrice": 1 } } },
)
Mongo supports dot notation for sub-queries.
See: http://docs.mongodb.org/manual/reference/glossary/#term-dot-notation
Depending on your driver, you want something like:
db.products.find({"items.date":"31.08.2014"});
Note that the attribute is in quotes for dot notation, even if usually your driver doesn't require this.