I have this type of data
{
"_id" : 6444,
"name" : [
{
"name" : "John",
"sourcesID" : [
1,
2
]
},
{
"name" : "Jack",
"sourcesID" : [
3,
4
]
}
],
"address" : [
{
"city" : "Chicago",
"sourcesID" : [
3,
4
]
},
{
"city" : "Boston",
"sourcesID" : [
5,
6
]
}
]
}
I want to aggregate the data so that I will be able to match a certain sourcesID and find all the information types that came from this source.
This is what I am looking to achieve
{"type" : "name", "sourceID" : 1}
{"type" : "name", "sourceID" : 2}
{"type" : "name", "sourceID" : 3}
{"type" : "name", "sourceID" : 4}
{"type" : "address", "sourceID" : 3}
{"type" : "address", "sourceID" : 4}
{"type" : "address", "sourceID" : 5}
{"type" : "address", "sourceID" : 6}
Thanks for your help.
Assuming that there's always sourceID field you can run $objectToArray to transform be able to read object keys dynamically and then run $unwind three times to get single document per sourceID:
db.collection.aggregate([
{
$project: {
data: {
$filter: {
input: { $objectToArray: "$$ROOT" },
cond: {
$ne: [ "$$this.k", "_id" ]
}
}
}
}
},
{ $unwind: "$data" },
{ $unwind: "$data.v" },
{ $unwind: "$data.v.sourcesID" },
{
$project: {
_id: 0,
type: "$data.k",
sourceID: "$data.v.sourcesID"
}
}
])
Mongo Playground
Related
i have mongo db collection the follwing structure
{
{
"_id" : ObjectId("63e37afe7a3453d5014c011b"),
"schemaVersion" : NumberInt(1),
"Id" : "ObjectId("63e37afe7a3453d5014c0112")",
"Id1" : "ObjectId("63e37afe7a3453d5014c0113")",
"Id2" : "ObjectId("63e37afe7a3453d5014c0114")",
"collectionName" : "Country",
"List" : [
{
"countryId" : NumberInt(1),
"name" : "Afghanistan",
},{
"countryId" : NumberInt(1),
"name" : "India",
},
{
"countryId" : NumberInt(1),
"name" : "USA",
}
}
i need to match the value with id, id1, id2, collectionName and name in the list to get country id for example if match the below value
"Id" : "ObjectId("63e37afe7a3453d5014c0112")",
"Id1" : "ObjectId("63e37afe7a3453d5014c0113")",
"Id2" : "ObjectId("63e37afe7a3453d5014c0114")",
"collectionName" : "Country",
"name" : "Afghanistan",
i need result
{
"countryId" : 1,
"name" : "Afghanistan",
}
i tried like below
db.country_admin.aggregate([
{ $match: { collectionName: "Country" } },
{ $unwind : '$countryList' },
{ $project : { _id : 0, 'countryList.name' : 1, 'countryList.countryId' : 1 } }
]).pretty()
and i have following output
[
{
"List" : {
"countryId" : 1.0,
"name" : "Afghanistan"
}
},
{
"List" : {
"countryId" : 2.0,
"name" : "india"
}
},
{
"List" : {
"countryId" : 3.0,
"name" : "USA"
}
}]```
You can try using $filter to avoid $unwind like this example:
First $match by your desired condition(s).
Then $filter and get the first element (as "List.name": "Afghanistan" is used into $match stage there will be at least one result).
And output only values you want using $project.
db.collection.aggregate([
{
"$match": {
"Id": ObjectId("63e37afe7a3453d5014c0112"),
"Id1": ObjectId("63e37afe7a3453d5014c0113"),
"Id2": ObjectId("63e37afe7a3453d5014c0114"),
"collectionName": "Country",
"List.name": "Afghanistan",
}
},
{
"$project": {
"country": {
"$arrayElemAt": [
{
"$filter": {
"input": "$List",
"cond": {
"$eq": [
"$$this.name",
"Afghanistan"
]
}
}
},
0
]
}
}
},
{
"$project": {
"_id": 0,
"countryId": "$country.countryId",
"name": "$country.name"
}
}
])
Example here
By the way, using $unwind is also possible and you can check this example
Helo everyone!
I have a products collection like this:
db.products.insertMany( [
{ "_id" : 1, "name" : "Apple", "variants" : [ { "_id" : 1, "name" : "Red Apple" }, { "_id" : 2, "name" : "Green Apple" }] },
{ "_id" : 2, "name" : "Banana", "variants" : [ { "_id" : 3, "name" : "Yellow Banana" }, { "_id" : 4, "name" : "Green Banana" }] },
] )
and a orders collection
db.orders.insertMany( [
{ "_id" : 1, "price" : 123, "itemId": 2},
] )
How to join products collection to orders collection by itemId (itemId == variants._id) with aggregate?
I try with this way but it's not working
db.orders.aggregate([
{
$lookup: {
from: 'products',
as: 'product',
let: { variantId: '$_id' },
pipeline: [
{
$match: {
$expr: { $eq: ['$$variantId', '$variants._id'] },
},
}
],
},
},
])
maybe issues from $expr { $eq: ['$$variantId', '$variants._id'] } but i cannot resolve it. anybody can help?
Thanks for help!
Report table sample data
{
"_id" : ObjectId("614415f4a6566a001623b622"),
"record" : [
{
"dateTime" : ISODate("2021-09-17T04:13:39.465Z"),
"status" : "time-in",
"month" : 9,
"day" : 17,
"year" : 2021,
"time" : 1631852019465.0,
"date" : ISODate("2021-09-17T00:00:00.000Z"),
},
{
"dateTime" : ISODate("2021-09-17T04:14:01.182Z"),
"status" : "time-out",
"month" : 9,
"day" : 17,
"year" : 2021,
"time" : 1631852041182.0,
"date" : ISODate("2021-09-17T00:00:00.000Z"),
}
],
"uid" : ObjectId("614415b0a6566a001623b80b"),
"date" : ISODate("2021-09-17T00:00:00.000Z"),
"status" : "time-out",
"createdAt" : ISODate("2021-09-17T04:13:40.102Z"),
"updatedAt" : ISODate("2021-09-17T04:14:01.831Z"),
"__v" : 0
}
Users table sample data
{
"_id" : ObjectId("615c0f6db30aff375cd05ac1"),
"displayName" : "test test",
"firstName" : "test",
"lastName" : "test",
"email" : "test#gmail.com",
"brand" : "Jollibee",
"phone" : "+632312312312312",
"role" : 1,
"isVerified" : true,
"isArchived" : false,
"createdAt" : ISODate("2021-10-05T08:40:13.208Z"),
"updatedAt" : ISODate("2021-10-05T08:40:13.208Z"),
"__v" : 0
}
I have a data like this
db.getCollection('users').aggregate([
{
"$match": { brand: "Jollibee" }
},
{
$lookup: {
from: "orders",
let: { id: 'id' },
pipeline: [
{
$match: {
date: { $gte: ISODate("2020-11-01"), $lt: ISODate("2021-11-31") },
}
}
],
as: "orders",
},
},
{
$project: {
"_id": 1,
"name": 1,
"orders": 1
}
}
])
when I'm using this aggregation I'm getting all the data inserted per user.
What I want to happen is that. I will only get the data that belong to the user and not all the data of all users.
Added the sample documents for each collection
You are not comparing the userIds of both collections. You should add that on your $match. Playground
db.users.aggregate([
{
"$match": {
brand: "Jollibee"
}
},
{
$lookup: {
from: "orders",
let: {
id: "$_id"
},
pipeline: [
{
$match: {
date: {
$gte: ISODate("2020-11-01"),
$lt: ISODate("2021-11-30")
},
$expr: {
$eq: [
"$uid",
"$$id"
]
}
}
}
],
as: "orders",
},
},
{
$project: {
"_id": 1,
"name": 1,
"orders": 1
}
}
])
This example is following https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/#use-lookup-with-mergeobjects
db.orders.insert([
{ "_id" : 1, "item" : "almonds", "price" : 12, "quantity" : 2 },
{ "_id" : 2, "item" : "pecans", "price" : 20, "quantity" : 1 }
])
db.items.insert([
{ "_id" : 1, "item" : "almonds", description: "almond clusters", "instock" : 120 },
{ "_id" : 2, "item" : "bread", description: "raisin and nut bread", "instock" : 80 },
{ "_id" : 3, "item" : "pecans", description: "candied pecans", "instock" : 60 }
])
Aggregation:
db.orders.aggregate([
{
$lookup: {
from: "items",
localField: "item", // field in the orders collection
foreignField: "item", // field in the items collection
as: "fromItems"
}
},
{
$replaceRoot: { newRoot: { $mergeObjects: [ { $arrayElemAt: [ "$fromItems", 0 ] }, "$$ROOT" ] } }
},
{ $project: { fromItems: 0 } }
])
Result:
{ "_id" : 1, "item" : "almonds", "description" : "almond clusters", "instock" : 120, "price" : 12, "quantity" : 2 }
{ "_id" : 2, "item" : "pecans", "description" : "candied pecans", "instock" : 60, "price" : 20, "quantity" : 1 }
Question: How to modify the aggregation to project the specific fields? e.g. project "_id", "item" and "description" only:
{ "_id" : 1, "item" : "almonds", "description" : "almond clusters" }
{ "_id" : 2, "item" : "pecans", "description" : "candied pecans" }
You're getting an empty array, because the $lookup catching anything.
match the types
$addFields to convert
PLAYGROUND
This should be the first stage:
{
$addFields: {
itemId: {
$convert: {
input: "$itemId",
to: "int"
}
}
}
},
If you prefer, there is no need to add a stage
You could also remove addFields and use $lookup+let.
Modify the lookup this way:
{
$lookup: {
from: "items",
let: {
itemId: {
$convert: {
input: "$itemId",
to: "int"
}
}
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$_id",
"$$itemId"
]
}
}
}
],
/** field in the items collection*/
as: "fromItems"
}
}
PLAYGROUND2
I have collection like this:
collection name is - account.
and it has sub-documents like account > buildings > gateways > devices.
{
"_id" : ObjectId("5e1fe45cd05bfb0cc549297d"),
"apiCallCount" : 0,
"email" : "info#data.com",
"password" : "dummy",
"userName" : "AAAAA",
"companyName" : "The AAAAAA",
"apiKey" : "5e1fe45cd05bfb0cc549297c",
"solutionType" : "VVVVVV",
"parentCompany" : "",
"buildings" : [
{
"_id" : ObjectId("5e1fe5e3d05bfb0cc5494146"),
"buildingName" : "xxxxxx",
"address" : "xxx",
"suite" : "101",
"floor" : "22",
"timeZone" : "us/eastern",
"gateways" : [
{
"_id" : ObjectId("5e1fe624d05bfb0cc549453a"),
"gatewayName" : "CC-GW-THF-001",
"gatewayKey" : "gk_5e1fe624d05bfb0cc549453a",
"suite" : "area1",
"devices" : [
{
"_id" : ObjectId("5e1fe751d05bfb0cc549578d"),
"serialNumber" : "129300000013",
"area" : "area1",
"connectionStatus" : 1,
"gatewayKey" : "gk_5e1fe624d05bfb0cc549453a",
"applicationNumber" : 30,
"firmwareVersion" : "1.0",
"needsAttention" : false,
"verificationCode" : "GAAS",
"createdAt" : ISODate("2020-01-16T04:32:17.899Z"),
"updatedAt" : ISODate("2020-01-16T08:53:54.460Z")
}
],
"createdAt" : ISODate("2020-01-16T04:27:16.678Z"),
"updatedAt" : ISODate("2020-01-16T08:53:54.460Z")
},
{
"_id" : ObjectId("5e1fe651d05bfb0cc54947f0"),
"gatewayName" : "AA-GW-THF-002",
"gatewayKey" : "gk_5e1fe651d05bfb0cc54947f0",
"suite" : "area2",
"devices" : [
{
"_id" : ObjectId("5e1fe7a9d05bfb0cc5495cdf"),
"serialNumber" : "129300000012",
"area" : "area2",
"connectionStatus" : 0,
"gatewayKey" : "gk_5e1fe651d05bfb0cc54947f0",
"applicationNumber" : 30,
"firmwareVersion" : "1.0",
"needsAttention" : false,
"verificationCode" : "VG3K",
"createdAt" : ISODate("2020-01-16T04:33:45.698Z"),
"updatedAt" : ISODate("2020-01-16T08:54:17.604Z")
}
],
"createdAt" : ISODate("2020-01-16T04:28:01.532Z"),
"updatedAt" : ISODate("2020-01-16T08:54:17.604Z")
},
],
"createdAt" : ISODate("2020-01-16T04:26:11.941Z"),
"updatedAt" : ISODate("2020-01-16T08:56:32.657Z")
}
],
"createdAt" : ISODate("2020-01-16T04:19:40.310Z"),
"updatedAt" : ISODate("2020-04-06T18:18:39.628Z"),
"__v" : 1,
}
I have accountId, buildingId, gatewayId, deviceId.
I am trying to find matched device object using $lookup operator.
I think I have to fist find the building object by using buildingId and then filter gateway under that building using gatewayId and then find device object using deviceId that I have.
I basically need to have access to device object fields to project in final output.
Having difficulty in coming up with correct pipleline for usingg lookup operator.
I have this so far:
db.getCollection('test').aggregate([
{
$lookup: {
from: 'account',
let: {
accountId: "$accountId"
},
pipeline: [
{
"$match": {
"$expr": {
"$eq": ["$_id", "$$accountId"]
}
}
},
],
as: "accountDetails"
}
}, {
$unwind: "$accountDetails"
}, {
$lookup: {
from: 'account',
let: {
accountId: "$accountId",
buildingId: "$buildingId",
buildings: "$accountDetails"
},
pipeline: [
{
"$match": {
"$expr": {
"$eq": ["$buildings._id", "$$buildingId"] // how to dig through nested document to get to devices ?
}
}
},
],
as: "buildingDetails"
}
}
{
$project: { ... ...
}
])
if I do this:
{
$lookup: {
from: 'account',
localField: "accountId",
foreignField: "_id",
as: "accountDetails"
}
},
accountDetails gives me access the account document based on accountId. but I need to get to buildings > gateways > devices and find the matching device.
UPDATE:
I forgot to mention, I am working with 2 collections here.
sensingresults and accounts.
Main purpose is to aggregate data from sensingresults, but also find deviceId from account collection and return with the result.
That's why lookup is needed to join 2 collections?
UPDATE2:
Current output:
{
"accountId": ObjectId("5e1fe45cd05bfb0cc549297d"),
"avgZoneCountNumber": 0,
"avgZoneCountNumberInstant": 0,
"buildingId": ObjectId("5e1fe5e3d05bfb0cc5494146"),
"companyName": "The AAAAAA",
"createdAt": ISODate("1970-01-01T00:00:00Z"),
"dateHour": "2020-03-19T18",
"deviceId": ObjectId("5e1fe81ed05bfb0cc5496406"),
"gatewayId": ObjectId("5e1fe6a6d05bfb0cc5494d25"),
"minuteBucket": 1
}
Expected result:
{
"accountId": ObjectId("5e1fe45cd05bfb0cc549297d"),
"avgZoneCountNumber": 0,
"avgZoneCountNumberInstant": 0,
"buildingId": ObjectId("5e1fe5e3d05bfb0cc5494146"),
"createdAt": ISODate("1970-01-01T00:00:00Z"),
"dateHour": "2020-03-19T18",
"deviceId": ObjectId("5e1fe81ed05bfb0cc5496406"),
"gatewayId": ObjectId("5e1fe6a6d05bfb0cc5494d25"),
"minuteBucket": 1,
"serialNumber: 1, // this value should come from device object
"area": 1 // this value should come from device object
}
You can find nested device using $filter, $arrayElemAt and $let:
device: {
$let: {
vars: {
building: {
$arrayElemAt: [ { $filter: { input: "$company_name.buildings", cond: { $eq: [ "$$this._id", "$buildingId" ] }} }, 0 ]
}
},
in: {
$let: {
vars: {
gateway: {
$arrayElemAt: [ { $filter: { input: "$$building.gateways", cond: { $eq: [ "$$this._id", "$gatewayId" ] }} }, 0 ]
}
},
in: { $arrayElemAt: [ { $filter: { input: "$$gateway.devices", cond: { $eq: [ "$$this._id", "$deviceId" ] }} }, 0 ] }
}
}
}
}
Full Solution