Match Documents based on Nested Array Values and Count Unique - mongodb

I have a MongoDB Collection which has Documents in Given format,
{
"_id" : ObjectId("595f5661f34ae7b2adee31bc"),
"app_userUpdatedOn" : "2017-03-09T12:01:07.615Z",
"appId" : 31625,
"app_lastCommunicatedAt" : "2017-03-09T12:18:53.067Z",
"currentDate" : "2017-03-09T12:19:28.626Z",
"objectId" : "58c14850e4b0b2406992b29e",
"name" : "APPSESSION",
"action" : "START",
"installationId" : "98088f6641a0fa79",
"userName" : "98088f6641a0fa79",
"properties" : [
[
"userid",
"98088f6641a0fa79"
],
[
"app_os_version",
"6.0.1"
],
[
"app_installAt",
"2017-03-09T12:01:01.307Z"
],
[
"app_model",
"SM-J210F"
],
[
"app_lastCommunicatedAt",
"2017-03-09T12:18:53.067Z"
],
[
"app_carrier",
"Jio 4G"
],
[
"app_counter",
1
],
[
"app_brand",
"samsung"
],
[
"app_lib_version",
"1.0"
],
[
"app_app_version",
"3.0.2"
],
[
"app_os",
"Android"
]
],
"date" : "2017-03-09"
}
{
"_id" : ObjectId("595f5661f34ae7b2adee31bd"),
"app_userUpdatedOn" : "2017-02-05T07:38:32.866Z",
"appId" : 31625,
"app_lastCommunicatedAt" : "2017-03-09T08:09:05.342Z",
"currentDate" : "2017-03-09T12:19:28.806Z",
"objectId" : "58c14850e4b06ec88ecaa9c6",
"name" : "APPINSTALL",
"action" : "START",
"installationId" : "eef436554fbdf4ac",
"userName" : "eef436554fbdf4ac",
"properties" : [
[
"userid",
"eef436554fbdf4ac"
],
[
"app_os_version",
"5.1"
],
[
"app_installAt",
"2017-02-05T11:20:49.809Z"
],
[
"app_model",
"Micromax Q465"
],
[
"app_lastCommunicatedAt",
"2017-03-09T08:09:05.342Z"
],
[
"app_carrier",
"JIO 4G"
],
[
"app_counter",
1
],
[
"app_brand",
"Micromax"
],
[
"app_lib_version",
"1.0"
],
[
"app_app_version",
"3.0.2"
],
[
"app_os",
"Android"
]
],
"date" : "2017-03-09"
}
I want to Fetch the Count and Unique Count of the Documents where currentDate lies in between, startDate and endDate, name is x (eg. APPSESSION), Containing multiple Properties Nested Array (like ["app_installAt","This can be any value instead of null"] ,["app_model","This can be any value instead of null"], and so on... ), Group By userName
Previously i have created a Query in which Nested Array Both Element are Known, and it is as follows
db.testing.aggregate(
[
{$match: {currentDate: {$gte:"2017-03-01T00:00:00.000Z", $lt:"2017-03-02T00:00:00.000Z"},name:"INSTALL"}},
{$match: {properties: ["app_os_version","4.4.2"]}},
{$match: {properties: ["app_carrier","telenor"]}},
{$match: {properties: ["app_brand","Micromax"]}},
{$group: {_id: "$userName"}},
{$count: "uniqueCount"}
]
);
But i am unable to find the Data where i know only 0th index of Property Data Nested Array.
Please do Help.
Thanks in Advance.... :)

The query for this is essentially the use of $all for the multiple conditions to match in the array and then use $elemMatch and $eq to match the individual array elements.
For example to match and count the first document supplied in your question "only" the parameters would be:
db.testing.find({
"currentDate": {
"$gte": "2017-03-09T00:00:00.000Z",
"$lt": "2017-03-10T00:00:00.000Z"
},
"properties": {
"$all": [
{ "$elemMatch": { "$eq": ["app_os_version","6.0.1"] } },
{ "$elemMatch": { "$eq": ["app_carrier", "Jio 4G"] } },
{ "$elemMatch": { "$eq": ["app_brand", "samsung"] } }
]
}
})
With .aggregate() then you put the whole query into a single $match stage as in:
db.testing.aggregate([
{ "$match": {
"currentDate": {
"$gte": "2017-03-09T00:00:00.000Z",
"$lt": "2017-03-10T00:00:00.000Z"
},
"properties": {
"$all": [
{ "$elemMatch": { "$eq": ["app_os_version","6.0.1"] } },
{ "$elemMatch": { "$eq": ["app_carrier", "Jio 4G"] } },
{ "$elemMatch": { "$eq": ["app_brand", "samsung"] } }
]
}
}},
{ "$group": { "_id": "$userName" }
{ "$count": "unique_count"
])
So $elemMatch in this context is going to examine each "inner" array and see if it matches the supplied conditions, which we give in argument as an "array" to the $eq operator.
The wrapping $all means that "all" the provided $elemMatch conditions "must" be met in order to fulfill the query conditions. And that is how the selection gets made with this type of structure.
If you needed to adjust one of those then the "inner" match is using the element of the array. So on the key it would use the "0" for the index position. i.e:
{ "$elemMatch": { "0": "app_os_version" } },

Related

Retrieve only queried element from a single document in mongoDB

I have a collection like below :
`{
"topics" : [
{
"id" : "2",
"name" : "Test1",
"owner" : [
"123"
]
},
{
"id" : "3",
"name" : "Test2",
"owner" : [
"123",
"456"
]
}
]
}`
As, this data is in single document, and I want only matching elements based on their owner, I am using below query ( using filter in aggregation ), but I am getting 0 matching elements.
Query :
Thanks in advance...!!
db.getCollection('topics').aggregate([
{"$match":{"topics.owner":{"$in":["123","456"]}}},
{"$project":{
"topics":{
"$filter":{
"input":"$topics",
"as":"topic",
"cond": {"$in": ["$$topic.owner",["123","456"]]}
}},
"_id":0
}}
])
This query should produce below output :
{
"topics" : [
{
"id" : "1",
"name" : "Test1",
"owner" : ["123"]
},
{
"id" : "2",
"name" : "Test2",
"owner" : ["123","456"]
}
]
}
As the topic.owner is an array, you can't use $in directly as this compares whether the array is within in an array.
Instead, you should do as below:
$filter - Filter the document in the topics array.
1.1. $gt - Compare the result from 1.1.1 is greater than 0.
1.1.1. $size - Get the size of the array from the result 1.1.1.1.
1.1.1.1. $setIntersection - Intersect the topic.owner array with the input array.
{
"$project": {
"topics": {
"$filter": {
"input": "$topics",
"as": "topic",
"cond": {
$gt: [
{
$size: {
$setIntersection: [
"$$topic.owner",
[
"123",
"456"
]
]
}
},
0
]
}
}
},
"_id": 0
}
}
Demo # Mongo Playground
db.getCollection('topics').aggregate([
{"$unwind":"$topics"},
{"$addFields":{
"rest":{"$or":[{"$in":["12z3","$topics.owner"]},{"$in":["456","$topics.owner"]}]}
}},
{"$match":{
"rest":true
}},
{"$group":{
"_id":"$_id",
"topics":{"$push":"$topics"}
}}
])

$merge whenMatched pipeline

I have been looking at $merge and Variables in Aggregation Expressions but I am struggling to understand. What I would like to do in a very general sense is take two collections, match them on the unique "Role ID" field and see if they are exactly the same or not. If they are the same I want to update the "Status" field to "Updated".
Where I am struggling is on the whenMatched pipeline. I am not sure how to target the "new" and "old" document for the $cmp expression. I am also not tied to this approach. I feel like $mergeObjects could be used as well. I appreciate the help.
const mergePipeline = [
{'$unset': "_id"},
{'$addFields' : {"Status" : "New"}},
{'$merge' : {
into: "previous",
on: "Role ID",
whenMatched: [
// compare the documents with $cmp <-- is it possible to only compare a few fields without unsetting them?
// if different replace root with "new" document
// change status to "updated"
],
whenNotMatched: "insert"
}}
];
db.current.aggregate(mergePipeline);
Coll1 (we aggregate on this one)
[
{
"role_id": 1,
"a": 2
},
{
"role_id": 2,
"a": 3
},
{
"role_id": 3,
"a": 20
}
]
Coll2 (the one in the disk)
first should be updated (= roots)
second should be replaced from the pipeline
and rold_id :3 has no match => inserted
[
{
"role_id": 1,
"a": 2
},
{
"role_id": 2,
"a": 10
}
]
Query
removes the :_id (else error, we cant update that)
merge on role_id
if 2 roots(from pipeline and from disk) equals => status updated
else replace with the root of the pipeline
*let is used to have the ROOT. of the pipeline, as variable "$$p-root"
coll1.aggregate(
[
{
"$unset": [
"_id"
]
},
{
"$merge": {
"into": {
"db": "testdb",
"coll": "coll2"
},
"on": [
"role_id"
],
"let": {
"p_root": "$$ROOT"
},
"whenMatched": [
{
"$unset": [
"_id"
]
},
{
"$replaceRoot": {
"newRoot": {
"$cond": [
{
"$eq": [
"$$p_root",
"$$ROOT"
]
},
{
"$mergeObjects": [
"$$ROOT",
{
"status": "updated"
}
]
},
"$$p_root"
]
}
}
}
],
"whenNotMatched": "insert"
}
}
])
Results that i got
(i used simple $eq , you can use $cmp but i dont think we need it, because we care only for the equality not the > <)
[
{
"role_id": 1,
"a": 2,
"status": "updated" // roots were equal (pipeline root,disk root)
},
{
"role_id": 2, // root not equal i kept the pipelines
"a": 3
},
{
"role_id": 3, // no match happened => insert
"a": 20
}
]
const mergePipeline = [
{'$unset': "_id"},
{'$merge' : {
into: "currentStatusSample",
on: "Role ID",
whenMatched: [
{$unset: ["_id", "Status"]},
{$addFields : {compare: {$cmp: ["$$new","$$ROOT"]}}},
{$set: {"Status" : {$cond: [ {$ne : ["$compare", 0]}, "Updated", "Unchanged"]}}},
{$unset: "compare"}
],
whenNotMatched: "insert"
}}
];
I wish there was a way to only $cmp on specified fields but for now this seemed to have worked. I plan on moving forward with either this or the answer posted by Takis above.

Mongodb- Delete some of the array elements within embedded document

There are 2 kinds of documents.
Type 1. Documents contain the either MCN-ONE, MCN-TWO, MCN-THREE(or all 3) along with other values
2. Another type of documents do not contain any among these values.
First, I would like to get the documents having those array elements(either 1 or 2 or all 3). Then I want to keep MCN-ONE,MCN-TWO,MCN-THREE and delete all others (CCC-ALARM..etc) in bulk. Could you help to write the query? The below mentioned document falls in type 1.
{
"_id" : ObjectId("5d721f5296eaaafd1df263e8"),
"assetId" : "ALL",
"createdTime" : ISODate("2019-09-06T08:56:50.065Z"),
"default" : false,
"lastUpdatedTime" : ISODate("2019-09-06T09:11:35.463Z"),
"preferences" : {
"MCN-TWO" : [
"TEST"
],
"MCN-ONE" : [
"TEST",
"TEST",
"TEST"
],
"MCN-THREE" : [
"TEST"
],
"CCC-ALARM" : [
"TEST"
],
"SSD-ALARM" : [
"TEST"
],
"TFT-ALARM" : [
"TEST",
"TEST"
],
"REC-WARN" : []
}
}
The generic approach would be to transform preferences subdocument with $objectToArray, then filter desired elements with $filter, $map or $reduce and transform back with $arrayToObject. However, your requirement is "get elements MCN-ONE,MCN-TWO,MCN-THREE". The simple way is to update element preferences and replace just with conten of MCN-ONE,MCN-TWO,MCN-THREE. It can be done by this aggregation:
In order to filter documents, set the $match stage:
db.collection.aggregate(
[
{
$match: {
$expr: {
$or: [
{ $ne: ["$preferences.MCN-ONE", null] },
{ $ne: ["$preferences.MCN-TWO", null] },
{ $ne: ["$preferences.MCN-THREE", null] }
]
}
}
},
{
$set: {
preferences: {
$mergeObjects: [
{ "MCN-ONE": "$preferences.MCN-ONE" },
{ "MCN-TWO": "$preferences.MCN-TWO" },
{ "MCN-THREE": "$preferences.MCN-THREE" }
]
}
}
}
]
)

Compare Size of Arrays Inside an Array of Objects

I want to find all documents where sCompetitions.length is greater than competitions.length.
Here's some sample documents document:
{
"_id" : ObjectId("59b28f432b4353d3f311dd1b"),
"name" : "Ford Focus RS 2008",
"requirements" : [
{
"rankType" : "D1",
"competitions" : [
ObjectId("59b151fd2b4353d3f3116827"),
ObjectId("59b151fd2b4353d3f3116829")
],
"sCompetitions" : [
"Rallye Monte-Carlo",
"Rally Sweden"
]
},
{
"rankType" : "A3",
"competitions" : [
ObjectId("59b151fd2b4353d3f3116f6b")
],
"sCompetitions" : [
"Rally Italia Sardegna",
"Neste Rally Finland"
]
}
]
},
{
"_id" : ObjectId("0000b28f432b4353f311dd1b"),
"name" : "Ford Focus RS 2012",
"requirements" : [
{
"rankType" : "D1",
"competitions" : [
ObjectId("59b151fd2b4353d3f3116827"),
ObjectId("59b151fd2b4353d3f3116829")
],
"sCompetitions" : [
"Rallye Monte-Carlo",
"Rally Sweden"
]
},
{
"rankType" : "A3",
"competitions" : [
ObjectId("59b151fd2b4353d3f3116f6b"),
ObjectId("59b151fd2b4353d3f3116f6b")
],
"sCompetitions" : [
"Rally Italia Sardegna",
"Neste Rally Finland"
]
}
]
}
So looking at the samples it would only return ObjectId("59b28f432b4353d3f311dd1b")
My problem is that requirements is an array by itself, so I would need to somehow iterate it
No need to "iterate". All you really need is an $anyElementTrue check after returning results from $map. And you can do this all inside a $redact action:
Model.aggregate([
{ "$redact": {
"$cond": {
"if": {
"$anyElementTrue": {
"$map": {
"input": "$requirements",
"as": "r",
"in": {
"$gt": [
{ "$size": "$$r.sCompetitions" },
{ "$size": "$$r.competitions" }
]
}
}
}
},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
So it's a simple comparison by $size for each array element, and then if "any" of those elements is true, the document is "kept" or otherwise "pruned" from the results.

Find a single field from a nested Array in MongoDB

Here I have a sample Nested Array. I have a problem with writing proper queries on this collection which is deeply nested.
{
"productUUID" : "craft001",
"providers": [
{
"providerUUID": "prov001",
"orgs": [
{
"orgUUID": "org001",
"location": {
"buildings": [
{
"buildingUUID": "sit001",
"floors": [
{
"floorUUID": "GrndFlr",
"assets": [ ],
"agents": [ ],
"users": [ ]
},
{
"floorUUID": "1stFlr",
"assets": [ ],
"agents": [ ],
"users": [ ]
}
]
},
{
"buildingUUID": "ist001",
"floors": [ ]
}
]
}
},
{
"orgUUID": "org002",
"location": {
"buildings": [ ]
}
}
]
},
{
"providerUUID": "prov002",
"orgs": [ ]
}
]
}
Question in simple words, "1. Get all orgUUIDs which fall under providerUUID: "prov001"".
Similarly, "2. Get all floorUUIDs where "buildingUUID": "sit001"".
If someone can help me with the 1st ques, I hope I can solve the 2nd ques myself.
Mongo aggregation use to finding to nested documents. First unwind all providers array then use match to match providerUUID as given prov001 then used project to get all orgUUID and aggregation query as :
db.collectionName.aggregate({"$unwind":"$providers"},
{"$match":{"providers.providerUUID":"prov001"}},
{"$project":{"orgUUID":"$providers.orgs.orgUUID"}},
{"$unwind":"$orgUUID"},
{"$project":{"_id":0,"orgUUID":1}}
).pretty()
this will returns all orgUUID in an array.
If you use $elemMacth then this operator having it's own limitation as
The $elemMatch operator matches documents that contain an array field with at least one element that matches all the specified query criteria.
elemMatch query as :
db.collectionName.find({"providers":{"$elemMatch":{"providerUUID":"prov001"}}},
{"providers.$.providerUUID.orgs.orgUUID":1}).pretty()
it returns whole matching providers array.
I hope you will find out "2" question query yourself, If you having any trouble with finding with "2" query I will post "2" query also. Try to yourself to find out second query answer yourself :)
For some reason, I had to change the data in collection as following.
{
"productUUID": "prod001",
"providers": [
{
"providerUUID": "prov001",
"orgs": [
{
"orgUUID": "org001",
"floors": [
{ "floorUUID": "SIT_GrndFlr" },
{ "floorUUID": "SIT_1stFlr" }
],
"assets": [{},{}],
"agents": [{},{}],
"users": [{},{}]
},
{
"orgUUID": "org002",
"floors": [
{ "floorUUID": "IST_1stFlr" },
{ "floorUUID": "IST_2ndFlr" }
],
"assets": [{},{}],
"agents": [{},{}],
"users": [{},{}]
}
]
},
{
"providerUUID": "prov002",
"orgs": [
{
"orgUUID": "org001",
"floors": [{},{}],
"assets": [{},{}],
"agents": [{},{}],
"users": [{},{}]
},
{
"orgUUID": "org002",
"floors": [{},{}],
"assets": [{},{}],
"agents": [{},{}],
"users": [{},{}]
}
]
}
]
}
so, now with the help of #yogesh, I was introduced to aggregate and was able to write queries for my questions.
1. Get all `orgUUID`s under `providerUUID: "prov001"`.
db.collectionName.aggregate({"$unwind":"$providers"},
{"$match":{"providers.providerUUID":"prov001"}},
{"$project":{"orgUUID":"$providers.orgs.orgUUID"}},
{"$unwind":"$orgUUID"},
{"$project":{"_id":0,"orgUUID":1}}
)
2. Get all `floorUUID`s under `orgUUID : "org001"`.
db.collectionName.aggregate(
{ "$unwind" : "$providers" },
{ "$match" : { "providers.providerUUID" : "prov001" } },
{ "$unwind" : "$providers.orgs" },
{ "$match" : { "providers.orgs.orgUUID" : "org001" } },
{ "$project" : { "floorUUID" : "$providers.orgs.floors.floorUUID" } },
{ "$unwind" : "$floorUUID" },
{ "$project" : { "_id":0 , "floorUUID" : 1 } }
)