I have a document with a nested array array_field:
{
"_id": {
"$oid": "1"
},
"id": "1",
"array_field": [
{
"data": [
{
"regions": [
{
"result": {
"item": [
"4",
"5",
"3"
]
}
},
{
"result": {
"item": [
"5"
]
}
},
{
"result": {
"item": [
"1"
]
}
}
]
}
]
}
]
}
I need add new field, new_added_field for example, with each array element from array_field.data.regions.result.item and remove array_field from document.
For example:
{
"_id": {
"$oid": "1"
},
"id": "1",
"new_added_field": [4,5,3,5,1]
}
I think i can do this with help of $unwind or $map but have difficulties and need dome hint, how i can do it with help op aggregation?
As you said,
db.collection.aggregate([
{
"$project": {
newField: {
"$map": {
"input": "$array_field",
"as": "m",
"in": "$$m.data.regions.result.item"
}
}
},
},
{ "$unwind": "$newField" },
{ "$unwind": "$newField" },
{ "$unwind": "$newField" },
{ "$unwind": "$newField" },
{
"$group": {
"_id": "$_id",
"newField": { "$push": "$newField" }
}
}
])
Working Mongo playground
Related
I want to sum of values inside array of objects which have another array of objects.
In my case; how can I count 'url' values in all documents inside 'urls' array under 'iocs' array;
Mongo playground: open
Here is document example;
[
{
"_id": {
"$oid": "63b4993d0625ebe8b6f5b06e"
},
"iocs": [
{
"urls": [
{
"url": "7.1.5.2",
}
],
},
{
"urls": [
{
"url": "https://l-ink.me/GeheimeBegierde",
},
{
"url": "GeheimeBegierde.ch",
}
],
},
{
"urls": [
{
"url": "https://l-ink.me/GeheimeBegierde",
}
],
}
],
type: "2"
},
{
"_id": {
"$oid": "63b4993d0624ebe8b6f5b06e"
},
"iocs": [
{
"urls": [
{
"url": "7.1.5.2",
}
],
},
{
"urls": [
{
"url": "https://l-ink.me/GeheimeBegierde",
},
{
"url": "GeheimeBegierde.ch",
}
],
},
{
"urls": [
{
"url": "https://l-ink.me/GeheimeBegierde",
}
],
}
],
type: "3"
},
{
"_id": {
"$oid": "63b4993d0615ebe8b6f5b06e"
},
"iocs": [
{
"urls": [
{
"url": "www.google.com",
}
],
},
{
"urls": [
{
"url": "abc.xyz",
},
{
"url": "GeheimeBegierde.ch",
}
],
},
{
"urls": [
{
"url": "https://123.12",
}
],
}
],
type: "1"
}
]
expected output be like;
url: "7.1.5.2",
count:2,
types:[2,3]
url: "https://l-ink.me/GeheimeBegierde",
count:4,
types:[2,3],
url: "abc.xyz",
count:1,
types:[1],
I tried unwind iocs then project urls but can't figure out how to get this output. I think i must use group but how ? Newbie in mongodb.
Any help would be appreciated. Thanks all.
NOTE: All the answers are working. Thank you all for the contributing.
You could do something like this !
db.collection.aggregate([
{
"$unwind": "$iocs"
},
{
"$unwind": "$iocs.urls"
},
{
"$group": {
"_id": "$iocs.urls.url",
"count": {
"$sum": 1
},
"types": {
"$addToSet": "$type"
}
}
},
{
"$project": {
url: "$_id",
_id: 0,
types: 1,
count: 1
}
},
])
https://mongoplayground.net/p/hhMqh2zI_SX
Here's one way you could do it.
db.collection.aggregate([
{"$unwind": "$iocs"},
{"$unwind": "$iocs.urls"},
{
"$group": {
"_id": "$iocs.urls.url",
"count": {"$count": {}},
"types": {"$addToSet": {"$toInt": "$type"}}
}
},
{
"$set": {
"url": "$_id",
"_id": "$$REMOVE"
}
}
])
Try it on mongoplayground.net.
You can try this query:
Double $unwind to deconstruct the nested array.
Then group by url get the count using $sum nad add the types into a set (to avoid duplicates, otherwise you can use simply $push)
db.collection.aggregate([
{
"$unwind": "$iocs"
},
{
"$unwind": "$iocs.urls"
},
{
"$group": {
"_id": "$iocs.urls.url",
"count": {
"$sum": 1
},
"types": {
"$addToSet": "$type"
}
}
}
])
Example here
Since $unwind is considered an inefficient operation, another option is to use $reduce and only $unwind once:
db.collection.aggregate([
{$project: {
type: 1,
urls: {
$reduce: {
input: "$iocs",
initialValue: [],
in: {$concatArrays: ["$$value", "$$this.urls.url"]}
}
}
}},
{$unwind: "$urls"},
{$group: {
_id: "$urls",
type: {$addToSet: "$type"},
count: {$sum: 1}
}},
{$project: {url: "$_id", count: 1, type: 1}}
])
See how it works on the playground example
Employee has multiple employeeActions, the employeeActions data looks like this:
[
{
"email": "one#gmail.com",
"companyRegNo": 105,
"event": {
"created": ISODate("2022-09-16T06:42:04.387Z"),
"desc": "COMPLETED_APPLICATIONS",
"note": "Direct apply"
}
},
{
"email": "one#gmail.com",
"companyRegNo": 105,
"event": {
"created": ISODate("2022-09-20T06:42:42.761Z"),
"desc": "ASKED_TO_REVIEW",
}
},
{
"email": "two#gmail.com",
"companyRegNo": 227,
"event": {
"created": ISODate("2022-09-16T06:42:04.387Z"),
"desc": "COMPLETED_APPLICATIONS",
"note": "Direct apply",
}
},
{
"email": "two#gmail.com",
"companyRegNo": 227,
"event": {
"created": ISODate("2022-09-28T06:42:42.761Z"),
"desc": "ASKED_TO_REVIEW",
}
},
{
"email": "three#gmail.com",
"companyRegNo": 157,
"event": {
"created": ISODate("2022-09-16T06:42:04.387Z"),
"desc": "COMPLETED_APPLICATIONS",
"note": "Direct apply",
}
},
{
"email": "four#gmail.com",
"companyRegNo": 201,
"deleted": true,
"event": {
"created": ISODate("2022-09-15T06:42:42.761Z"),
"desc": "COMPLETED_APPLICATIONS",
}
},
]
I need to write an aggregation query to get all email ids where the employee action of the user
- Does not have an ASKED_TO_REVIEW event created before '2022-09-25'
- deleted is either false or does not exist
The out put should have only
{"email": "one#gmail.com"}
{"email": "three#gmail.com"}
The below match and project query did not work
db.collection.aggregate([
{
"$match": {
"$and": [
{
"deleted": {
"$ne": true
}
},
{
"$or": [
{
"$and": [
{
"event.name": {
"$eq": "ASKED_TO_REVIEW"
}
},
{
"event.created": {
"$lt": ISODate("2022-09-25")
}
}
]
},
{
"event.name": {
"$ne": "ASKED_TO_REVIEW"
}
}
]
}
]
}
},
{
"$project": {
"email": 1,
"_id": 0
}
}
])
How do i go about this?
You need to group the events by email and then apply your filtering logic to those groups, something like this:
db.collection.aggregate([
{
"$group": {
"_id": "$email",
"field": {
"$push": "$$ROOT"
}
}
},
{
"$match": {
$expr: {
"$eq": [
0,
{
"$size": {
"$filter": {
"input": "$field",
"as": "item",
"cond": {
"$or": [
{
"$and": [
{
"$eq": [
{
"$getField": {
"field": "desc",
"input": "$$item.event"
}
},
"ASKED_TO_REVIEW"
]
},
{
"$lt": [
{
"$getField": {
"field": "created",
"input": "$$item.event"
}
},
ISODate("2022-09-25")
]
}
]
},
{
"$eq": [
{
"$getField": {
"field": "deleted",
"input": "$$item"
}
},
true
]
}
]
}
}
}
}
]
}
}
},
{
"$project": {
email: "$_id",
"_id": 0
}
}
])
Playground link.
Figured out the working query. After grouping by email, $elemMatch needs to be used for the and condition between "event.desc" and "event.created"
db.collection.aggregate([
{
"$group": {
"_id": "$email",
"field": {
"$push": "$$ROOT"
}
}
},
{
"$match": {
"$and": [
{
"field.deleted": {
"$ne": true
}
},
{
"$or": [
{
"field": {
"$elemMatch": {
"event.desc": "ASKED_TO_REVIEW",
"event.created": {
"$lt": ISODate("2022-09-25")
}
}
}
},
{
"field.event.desc": {
"$ne": "ASKED_TO_REVIEW"
}
}
]
}
]
}
},
{
"$project": {
email: "$_id",
"_id": 0
}
}
])
Playground Link
So I have two collection. collectionA and collectionB
collection A has following documents
db={
"collectiona": [
{
"_id": "6173ddf33ed09368a094e68a",
"title": "a"
},
{
"_id": "61wefdf33ed09368a094e6dc",
"title": "b"
},
{
"_id": "61wefdfewf09368a094ezzz",
"title": "c"
},
],
"collectionb": [
{
"_id": "6173ddf33ed0wef368a094zq",
"collectionaID": "6173ddf33ed09368a094e68a",
"data": [
{
"userID": "123",
"visibility": false,
"response": false
},
{
"userID": "2345",
"visibility": true,
"response": true
}
]
},
{
"_id": "6173ddf33ed09368awef4e68g",
"collectionaID": "61wefdf33ed09368a094e6dc",
"data": [
{
"userID": "5678",
"visibility": false,
"response": false
},
{
"userID": "674",
"visibility": true,
"response": false
}
]
}
]
}
So What I need is documents from collection A which has response false in collection B
and document should be sorted by first the ones that have visibility false and then the ones that have visibility true
for eg. userID : 123 should get 3 documents
{
"_id": "6173ddf33ed09368a094e68a",
"title": "a"
},
{
"_id": "61wefdf33ed09368a094e6dc",
"title": "b"
},
{
"_id": "61wefdfewf09368a094ezzz",
"title": "c"
},
whereas userID 2345 should get two
{
"_id": "61wefdf33ed09368a094e6dc",
"title": "b"
},
{
"_id": "61wefdfewf09368a094ezzz",
"title": "c"
},
User 674 will receive 3 objects from collection A but second would be in the last as it has visibility true for that document
{
"_id": "6173ddf33ed09368a094e68a",
"title": "a"
},
{
"_id": "61wefdfewf09368a094ezzz",
"title": "c"
},
{
"_id": "61wefdf33ed09368a094e6dc",
"title": "b"
},
MongoDB Playground link : https://mongoplayground.net/p/3rLry0FPlw-
Really appreciate the help. Thanks
You can start from collectionA:
$lookup the collectionB for the record related to the user specified
filter out collectionB documents according to response
assign a helper sortrank field based on the visibility and whether collectionaID is a match
$sort according to sortrank
wrangle back to the raw collection A
db.collectiona.aggregate([
{
"$lookup": {
"from": "collectionb",
let: {
aid: "$_id"
},
"pipeline": [
{
$unwind: "$data"
},
{
$match: {
$expr: {
$and: [
{
$eq: [
"$data.userID",
"2345"
]
},
{
$eq: [
"$collectionaID",
"$$aid"
]
}
]
}
}
}
],
"as": "collB"
}
},
{
$match: {
"collB.data.response": {
$ne: true
}
}
},
{
"$unwind": {
path: "$collB",
preserveNullAndEmptyArrays: true
}
},
{
"$addFields": {
"sortrank": {
"$cond": {
"if": {
$eq: [
"$collB.data.visibility",
false
]
},
"then": 1,
"else": {
"$cond": {
"if": {
$eq: [
"$collB.collectionaID",
"$_id"
]
},
"then": 3,
"else": 2
}
}
}
}
}
},
{
$sort: {
sortrank: 1
}
},
{
$project: {
collB: false,
sortrank: false
}
}
])
Here is the Mongo playground for your reference.
Playground
Lets say I have this collection:
[
{ "Topics": [ "a", "b" ] },
{ "Topics": [ "x", "a" ] },
{ "Topics": [ "k", "c", "z" ] }
]
I want to transform this string array to a single string with the itens of it in alphabetical order. The result would be:
[
{ Topic: "a/b"},
{ Topic: "a/x"},
{ Topic: "c/k/z"}
]
How can I project this result? Using Map? Reduce?
I have Mongo 5.0
Playground
cheers
just found the solution after some tries...
Just A Unwind, Sort, Group, Project with Reduce made the job...
Data
[
{
"Topics": [
"a",
"b"
]
},
{
"Topics": [
"x",
"a"
]
},
{
"Topics": [
"k",
"c",
"z"
]
}
]
Query
db.collection.aggregate([
{
"$unwind": "$Topics"
},
{
"$sort": {
"Topics": 1
}
},
{
"$group": {
"_id": "$_id",
Topics: {
"$push": "$Topics"
}
}
},
{
"$project": {
Topic: {
$reduce: {
input: "$Topics",
initialValue: "1T1",
in: {
$concat: [
"$$value",
"/",
"$$this"
]
}
}
}
}
}
])
Result:
[
{
"Topic": "1T1/a/x",
"_id": ObjectId("5a934e000102030405000001")
},
{
"Topic": "1T1/c/k/z",
"_id": ObjectId("5a934e000102030405000002")
},
{
"Topic": "1T1/a/b",
"_id": ObjectId("5a934e000102030405000000")
}
]
The common way to do this is
unwind
sort
group by id
reduce to 1 string
Bellow is a way to not unwind all collection but do a "local unwind".
Query
lookup with a dummy collection of 1 empty document [{}]
(this is "trick" that allows us to use stage operators like sort inside 1 document array) you need that collection in your database
unwind topics, sort them, group in 1 array, reduce them and create 1 string
we will have only 1 joined document (the transformed root document),
we replace the root with that
remove the "/" from start (it could be done on the reduce stage also)
added one extra case where topics are empty array to return ""
Test code here
db.topics.aggregate([
{
"$lookup": {
"from": "dummy",
"let": {
"topics": "$Topics"
},
"pipeline": [
{
"$set": {
"Topics": "$$topics"
}
},
{
"$unwind": {
"path": "$Topics"
}
},
{
"$sort": {
"Topics": 1
}
},
{
"$group": {
"_id": null,
"Topics": {
"$push": "$Topics"
}
}
},
{
"$project": {
"_id": 0
}
},
{
"$set": {
"Topics": {
"$reduce": {
"input": "$Topics",
"initialValue": "",
"in": {
"$let": {
"vars": {
"s": "$$value",
"t": "$$this"
},
"in": {
"$concat": [
"$$s",
"/",
"$$t"
]
}
}
}
}
}
}
}
],
"as": "joined"
}
},
{
"$replaceRoot": {
"newRoot": {
"$cond": [
{
"$eq": [
"$joined",
[]
]
},
{
"Topics": ""
},
{
"$arrayElemAt": [
"$joined",
0
]
}
]
}
}
},
{
"$set": {
"Topics": {
"$cond": [
{
"$gt": [
{
"$strLenCP": "$Topics"
},
0
]
},
{
"$substrCP": [
"$Topics",
1,
{
"$strLenCP": "$Topics"
}
]
},
""
]
}
}
}
])
I have a document which contains an array of array as given below.
This is the first document.
{
"_id": "5d932a2178fdfc4dc41d75da",
"data": [
{
"nestedData": [
{
"_id": "5d932a2178fdfc4dc41d75e1",
"name": "Special 1"
},
{
"_id": "5d932a2178fdfc4dc41d75e0",
"name": "Special 2"
}
]
}
]
}
I need to lookup(join) to another collection with the _id in the nestedData array in the aggregation framework.
The 2nd document from which I need to lookup is
{
"_id": "5d8b1ac3b15bc72d154408e1",
"status": "COMPLETED",
"rating": 4
}
I know I need to $unwind it twice to convert nestedData array into object.
But how do I group back again to form the same object like given below
{
"_id": "5d932a2178fdfc4dc41d75da",
"data": [
{
"array": [
{
"_id": "5d932a2178fdfc4dc41d75e1",
"name": "Special 1",
"data": {
"_id": "5d8b1ac3b15bc72d154408e1",
"status": "COMPLETED",
"rating": 4
},
{
"_id": "5d932a2178fdfc4dc41d75e0",
"name": "Special 2",
"data": {
"_id": "5d8b1ac3b15bc72d154408e0",
"status": "COMPLETED",
"rating": 4
},
}
]
}
]
}
Try this query
db.testers.aggregate([
{$lookup: {
from: 'demo2',
pipeline: [
{ $sort: {'_id': 1}},
],
as: 'pointValue',
}},
{
$addFields:{
"data":{
$map:{
"input":"$data",
"as":"doc",
"in":{
$mergeObjects:[
"$$doc",
{
"nestedData":{
$map:{
"input":"$$doc.nestedData",
"as":"nestedData",
"in":{
$mergeObjects:[
{ $arrayElemAt: [ {
"$map": {
"input": {
"$filter": {
"input": "$pointValue",
"as": "sn",
"cond": {
"$and": [
{ "$eq": [ "$$sn._id", "$$nestedData._id" ] },
]
}
}
},"as": "data",
"in": {
"name": "$$nestedData.name",
"data":"$$data",
}}
}, 0 ] },'$$nestedData'
],
}
}
}
}
]
}
}
}
}
},
{$project: { pointValue: 0 } }
]).pretty()