I've following schema
{
"_id" : ObjectId("xxxxx"),
"updatedAt" : ISODate("2022-06-29T13:10:36.659+0000"),
"createdAt" : ISODate("2022-06-29T08:06:51.264+0000"),
"payments" : [
{
"paymentId" : "xxxxx",
"paymentType" : "charge",
"paymentCurrency" : "PKR",
"paymentMode" : "cash",
"paymentTotal" : 13501.88,
"penalties" : 100
},
{
"paymentId" : "ccccc",
"paymentType" : "refund",
"paymentCurrency" : "PKR",
"paymentMode" : "",
"paymentTotal" : 13061.879999999997,
"penalties" : 430.0
}
]
}
I want to get all paymentTotal sum if paymentType is 'charge' else subtract the paymentTotal from the sum if paymentType is other than charge, i.e refund also subtract penalties from total sum
I've tried following query which is not working giving me syntax error like,
A syntax error was detected at the runtime. Please consider using a higher shell version or use the syntax supported by your current shell.
xxx
Blockquote
db.getCollection("booking").aggregate([
{
$match: {
createdAt : {
"$gte":ISODate("2022-06-28"),
"$lte":ISODate("2022-06-30"),
}
}
},
{$unwind: '$payments'},
{
"$group":{
"_id" : "$_id",
"total" : {
$sum: "$payments.paymentTotal"
}
},
},
{
$project :
{
"grandTotal":{
$cond:{
if:{$eq:["$payments.paymentType", "charge"]},
then:{$add : {"$total,$payments.paymentTotal"}},
else:{ $subtract: {"$total,$payments.paymentTotal"}}
}
}
}
}
]);
I've tried, Condition and Switch statements but both are not working, or maybe I'm using them wrong.
You can use $reduce for it:
db.collection.aggregate([
{
$match: {
createdAt: {
$gte: ISODate("2022-06-28T00:00:00.000Z"),
$lte: ISODate("2022-06-30T00:00:00.000Z")
}
}
},
{
$project: {
grandTotal: {
$reduce: {
input: "$payments",
initialValue: 0,
in: {
$cond: [
{$eq: ["$$this.paymentType", "charge"]},
{$add: ["$$this.paymentTotal", "$$value"]},
{$subtract: ["$$value", "$$this.paymentTotal"]}
]
}
}
}
}
}
])
See how it works on the playground example
You can do simple math:
db.collection.aggregate([
{
$set: {
grandTotal: {
$map: {
input: "$payments",
in: {
$multiply: [
"$$this.paymentTotal",
{ $cond: [{ $eq: ["$$this.paymentType", "charge"] }, 1, -1] }
]
}
}
}
}
},
{ $set: { grandTotal: { $sum: "$grandTotal" } } }
])
Related
I have a collection called countries:
{
"_id" : "123456",
"enabled" : true,
"entity" : {
"name" : [
{
"locale" : "en",
"value" : "Lithuania"
},
{
"locale" : "de",
"value" : "Litauen"
}
]
}
}
I like to return only the ObjectId and the value when the locale is "en".
{"_id":"123456", "value":"Lithuania"}
Ideally renaming value to country for:
{"_id":"123456", "country":"Lithuania"}
Using a projection like:
db.countries.aggregate([
{$project:
{country: {$arrayElemAt:["$entity.name",0]}}
}
])
returns almost the desired results:
{"_id" : "1234565", "country" : { "locale" : "en", "value" : "Lithuania" } }
This this one:
db.collection.aggregate([
{
$set: {
country: {
$filter: {
input: "$entity.name",
cond: { $eq: [ "$this.locale", "en" ] }
}
}
}
},
{ $project: { country: { $first: "$country.value" } } },
])
See Mongo playground
You can try,
$reduce to iterate loop of entity.name array, $cond check locale is "en" then return value
db.collection.aggregate([
{
$project: {
country: {
$reduce: {
input: "$entity.name",
initialValue: "",
in: {
$cond: [
{ $eq: ["$$this.locale", "en"] },
"$$this.value",
"$$value"
]
}
}
}
}
}
])
Playground
I should have specified the MongoDB Version. Server is running 3.6.20. For that the following solution works:
db.countries.aggregate([
{
$addFields: {
country: {
$filter: {
input: "$entity.name",
cond: { $eq: [ "$$this.locale", "en" ] }
}
}
}
},
{ $project: { country: { $arrayElemAt: [["$country.value"],0] } } },
])
I have documents storing IoT data.
Following MongoDB schema design best practices for IoT, I came to documents with the following structure:
"_id" : "AQ106_2020-09-12T09",
"date" : "2020-09-12T09:00:00.000Z",
"station" : {
"name" : "AQ106",
"loc" : {
"type" : "Point",
"coordinates" : [
14.339263,
40.814224
]
},
"properties" : {
}
},
"samples" : [
{
"t" : ISODate("2020-09-12T11:02:00.000+02:00"),
"data" : {
"pm1_mg_m3" : 2.7,
"pm2_5_mg_m3" : 4.6,
"pm10_mg_m3" : 12,
"P0" : 152,
"P1" : 16,
"P2" : 4.7,
"P3" : 0.8,
"P4" : 0.86,
"P5" : 0.6,
"P6" : 0.28,
"P7" : 0.152,
"P8" : 0.094,
"P9" : 0.092,
"P10" : 0.019,
"P11" : 0,
"P12" : 0,
"P13" : 0.0188,
"P14" : 0,
"P15" : 0,
"P16" : 0,
"P17" : 0,
"P18" : 0,
"P19" : 0,
"P20" : 0,
"P21" : 0,
"P22" : 0,
"P23" : 0,
"temp_celsius" : 32.59,
"humRelPercent" : 34,
"press_mBar" : 1010.79,
"CO2mA" : 4,
"NO2_WE_mV" : 226.419,
"NO2_AE_mV" : 229.553,
"OX_WE_mV" : 252.287,
"OX_AE_mV" : 220.419,
"CO_WE_mV" : 509.077,
"AE_WE_mV" : 348.51,
"batt_V" : 13.5,
"source_V" : 17.6
}
},
.... additional arrays
}
Now I want to compute hourly or daily averages (or another metric) to populate a new collection with only summarised data.
I coded the following solution for hourly means:
db.collection.aggregate([{$match: {
'station.name':'AQ104'
}}, {$unwind: {
path: "$samples"
}}, {$group: {
_id: "$date",
P0: {
$avg : "$samples.data.P0"
},
temp:{
$avg:"$samples.data.temp_celsius"
}
}}])
This works but I need to manually create a field for each property in samples.data in the original document and that's tedious.
Moreover, how to group both by date and station.name?
You can find a working example here.
Thanks.
Let's start with the easy question, how to group on multiple fields? With a simple syntax change:
{
$group: {
_id: {
date: "$date",
station: "$station.name"
}
}
Now for the second question this will be a bit more tedious. Mongo does not support "merging" objects by their keys with custom logic (in this case $avg). So we will have to convert the object to an array. unwind it, calculate the average per field and eventually group to restore the required structure like so:
db.collection.aggregate([
{
$match: {
"station.name": "AQ106"
}
},
{
$unwind: {
path: "$samples"
}
},
{
$addFields: {
objArr: {
"$objectToArray": "$samples.data"
}
}
},
{
$unwind: "$objArr"
},
{
$group: {
_id: {
date: "$date",
station: "$station.name",
objKey: "$objArr.k"
},
value: {
$avg: "$objArr.v"
}
}
},
{
$addFields: {
data: {
"$arrayToObject": [
[
{
k: "$_id.objKey",
v: "$value"
}
]
]
}
}
},
{
$group: {
_id: {
date: "$_id.date",
station: "$_id.station"
},
data: {
"$mergeObjects": "$data"
}
}
},
{
$replaceRoot: {
newRoot: {
"$mergeObjects": [
"$data",
"$_id"
]
}
}
}
])
MongoPlayground
------- EDIT ---------
For Mongo v4.4+ you can use $accumulator which allows you to execute custom javascript code in your pipeline. I am unsure how this will fare against the native Mongo pipeline in terms of performance in scale.
One thing to note is that I added the initial $addFields stage under the assumption that different samples may have different keys. if this is not the case it is not needed.
db.collection.aggregate([
{
$addFields: {
sampleKeys: {
$reduce: {
input: {
$map: {
input: "$samples",
as: "sample",
in: {
$map: {
input: {
"$objectToArray": "$$sample.data"
},
as: "sampleArrItem",
in: "$$sampleArrItem.k"
}
}
}
},
initialValue: [],
in: {
"$setUnion": [
"$$this",
"$$value"
]
}
}
}
}
},
{
$addFields: {
samples: {
$accumulator: {
init: function(keys){
return keys.map(k => {return {k: {v: 0, c: 0}}});
},
initArgs: ["$sampleKeys"],
accumulateArgs: ["$samples"],
accumulate: function(state, sample){
Object.keys(state).forEach((key) => {
if (key in sample.data) {
state[key].v += sample.data[key];
state[key].c++;
};
});
return state;
},
merge: function(state1, state2){
Object.keys(state1).forEach((key) => {
state1[key].v += state2[key].v;
state1[key].c += state2[key].c;
});
return state1;
},
lang: "js"
}
}
}
},
{
$replaceRoot: {
newRoot: {
$mergeObject: [
"$samples",
{station: "$station.name", date: "$date"},
]
}
}
}
])
I partially resolved my question in terms of grouping by multiple fields (MongoDB documentation was not so clear at this regard, in my opinion)
db.collection.aggregate([
{
$unwind: {
path: "$samples"
}
},
{
$group: {
_id: {
date: "$date",
station: "$station.name"
},
P0: {
$avg: "$samples.data.P0"
},
temp: {
$avg: "$samples.data.temp_celsius"
}
}
}
])
Here the updated working example.
Thanks to Tom Slabbaert, I solved my question with the following query:
db.collection.aggregate([
{
$unwind: {
path: "$samples"
}
},
{
$addFields: {
objArr: {
"$objectToArray": "$samples.data"
}
}
},
{
$unwind: "$objArr"
},
{
$group: {
_id: {
date: "$date",
station: "$station",
objKey: "$objArr.k"
},
value: {
$avg: "$objArr.v"
}
}
},
{
$addFields: {
data: {
"$arrayToObject": [
[
{
k: "$_id.objKey",
v: "$value"
}
]
]
}
}
},
{
$group: {
_id: {
date: "$_id.date",
station: "$_id.station"
},
data: {
"$mergeObjects": "$data"
}
}
},
{
"$project": {
_id: "$_id.date",
station: "$_id.station",
data: 1
}
}
])
Here
I wonder if it is possible to simplify the above solution using the new $function operator.
Thanks.
I've set up a fairly long mongo aggregate query to join several mongo collections together and shape up them into output of set of string fields. The query works fine as long as all the required values (ie : ids) exists but it breaks when it encounters null or empty values when doing the $lookup.
Following is the patientFile collection thats being queried :
{
"no" : "2020921008981",
"startDateTime" : ISODate("2020-04-01T05:19:02.263+0000")
"saleId" : "5e8424464475140d19c6941b",
"patientId" : "5e8424464475140d1955941b"
}
sale collection :
{
"_id" : ObjectId("5e8424464475140d19c6941b"),
"invoices" : [
{
"billNumber" : "2020921053467",
"type" : "CREDIT",
"insurancePlanId" : "160"
},
{
"billNumber" : "2020921053469",
"type" : "DEBIT",
"insurancePlanId" : "161"
}
],
"status" : "COMPLETE"
}
insurance collection :
{
"_id" : ObjectId("5b55aca20550de00210a6d25"),
"name" : "HIJKL"
"plans" : [
{
"_id" : "160",
"name" : "UVWZ",
},
{
"_id" : "161",
"name" : "LMNO",
}
]
}
patient collection :
{
"_id" : ObjectId("5b55cc5c0550de00217ae0f3"),
"name" : "TAN NAI",
"userId" : {
"number" : "787333128H"
}
}
Heres the aggregate query :
db.getCollection("patientFile").aggregate([
{ $match: { "startDateTime": { $gte: ISODate("2020-01-01T00:00:00.000Z"),
$lt: ISODate("2020-05-01T00:00:00.000Z") } } },
{
$lookup:
{
from: "patient",
let: { pid: "$patientId" },
pipeline: [
{
$match: {
$expr: {
$eq: ["$_id", { $toObjectId: "$$pid" }]
}
}
},
{ "$project": { "name": 1, "userId.number": 1, "_id": 0 } }
],
as: "patient"
}
},
{
$lookup:
{
from: "sale",
let: { sid: "$saleId" },
pipeline: [
{
$match: {
$expr: {
$eq: ["$_id", { $toObjectId: "$$sid" }]
}
}
}
],
as: "sale"
}
},
{ $unwind: "$sale" },
{ $unwind: "$patient" },
{
$lookup: {
from: "insurance",
let: { pid: {$ifNull:["$sale.bill.insurancePlanId", [] ]} },
pipeline: [
{
$unwind: "$plans"
},
{
$match: { $expr: { $in: ["$plans._id", "$$pid"] } }
},
{
$project: { _id: 0, name: 1 }
}
],
as: "insurances"
}
},
{ $match: { "insurances.name": { $exists: true, $ne: null } } },
{
$addFields: {
invoice: {
$reduce: {
input: {$ifNull:["$sale.bill.billNumber", [] ]},
initialValue: "",
in: {
$cond: [{ "$eq": ["$$value", ""] }, "$$this", { $concat: ["$$value", "\n", "$$this"] }]
}
}
},
insurances: {
$reduce: {
input: {$ifNull:["$insurances.name", [] ]},
initialValue: "",
in: {
$cond: [{ "$eq": ["$$value", ""] }, "$$this", { $concat: ["$$value", "\n", "$$this"] }]
}
}
}
}
},
{
"$project": {
"startDateTime": 1,
"patientName": "$patient.name",
"invoice": 1,
"insurances": 1
}
}
],
{ allowDiskUse: true }
)
Error :
Unable to execute the selected commands
Mongo Server error (MongoCommandException): Command failed with error 241 (ConversionFailure): 'Failed to parse objectId '' in $convert with no onError value: Invalid string length for parsing to OID, expected 24 but found 0' on server localhost:27017.
The full response is:
{
"ok" : 0.0,
"errmsg" : "Failed to parse objectId '' in $convert with no onError value: Invalid string length for parsing to OID, expected 24 but found 0",
"code" : NumberInt(241),
"codeName" : "ConversionFailure"
}
As a solution i have found, used $ifNull but this error keeps coming. What would be the best step to take for this scenario?
I see a couple of ways:
Instead of converting the string value to an ObjectId to test, convert the ObjectId to a string
$match: {
$expr: {
$eq: [{$toString: "$_id"}, "$$pid" ]
}
}
Instead of the $toObjectId helper, use $convert and provide onError and/or onNull values:
$match: {
$expr: {
$eq: ["$_id", { $convert: {
input: "$$pid",
to: "objectId",
onError: {error:true},
onNull: {isnull:true}
}}]
}
}
please check this query
db.billsummaryofthedays.aggregate([
{
'$match': {
'userId': ObjectId('5e43de778b57693cd46859eb'),
'adminId': ObjectId('5e43e5cdc11f750864f46820'),
'date': ISODate("2020-02-11T16:30:00Z"),
}
},
{
$lookup:
{
from: "paymentreceivables",
let: { userId: '$userId', adminId: '$adminId' },
pipeline: [
{
$match:
{
paymentReceivedOnDate:ISODate("2020-02-11T16:30:00Z"),
$expr:
{
$and:
[
{ $eq: ["$userId", "$$userId"] },
{ $eq: ["$adminId", "$$adminId"] }
]
}
}
},
{ $project: { amount: 1, _id: 0 } }
],
as: "totalPayment"
}
}, {'$unwind':'$totalPayment'},
{ $group:
{ _id:
{ date: '$date',
userId: '$userId',
adminId: '$adminId' },
totalBill:
{
$sum: '$billOfTheDay'
},
totalPayment:
{
$sum: '$totalPayment.amount'
}
}
},
}
}])
this is the result i am getting in the shell
{
"_id" : {
"date" : ISODate("2020-02-11T18:30:00Z"),
"userId" : ObjectId("5e43de778b57693cd46859eb"),
"adminId" : ObjectId("5e43e5cdc11f750864f46820")
},
"totalBill" : 1595.6799999999998,
"totalPayments" : 100
}
now this is not what i expected,i assume due to {'$unwind':'$totalPayment'} it takes out all the values from the array and because of which every document is getting counted 2 times. When i remove {'$unwind':'$totalPayment'} then totalBill sum turns out to be correct but totalPayment is 0.
I have tried several other ways but not able to achieve the desired result
Below are my collections:
// collection:billsummaryofthedays//
{
"_id" : ObjectId("5e54f784f4032c1694535c0e"),
"userId" : ObjectId("5e43de778b57693cd46859eb"),
"adminId" : ObjectId("5e43e5cdc11f750864f46820"),
"date" : ISODate("2020-02-11T16:30:00Z"),
"UID":"acex01"
"billOfTheDay" : 468,
}
{
"_id" : ObjectId("5e54f784f4032c1694535c0f"),
"UID":"bdex02"
"userId" : ObjectId("5e43de778b57693cd46859eb"),
"adminId" : ObjectId("5e43e5cdc11f750864f46820"),
"date" : ISODate("2020-02-11T16:30:00Z"),
"billOfTheDay" : 329.84,
}
// collection:paymentreceivables//
{
"_id" : ObjectId("5e43e73169fe1e3fc07eb7c5"),
"paymentReceivedOnDate" : ISODate("2020-02-11T16:30:00Z"),
"adminId" : ObjectId("5e43e5cdc11f750864f46820"),
"userId" : ObjectId("5e43de778b57693cd46859eb"),
"amount" : 20,
}
{
"_id" : ObjectId("5e43e73b69fe1e3fc07eb7c6"),
"paymentReceivedOnDate" : ISODate("2020-02-11T16:30:00Z"),
"adminId" : ObjectId("5e43e5cdc11f750864f46820"),
"userId" : ObjectId("5e43de778b57693cd46859eb"),
"amount" : 30,
}
desired result should be totalBill:797.83 i.e[468+329.84,] and totalPayment:50 i.e[30+20,] but i am getting double the expected result and even if i am able to calculate one of the value correctly the other one result 0.How to tackle this??
Since you've multiple documents with same data in billsummaryofthedays collection then you can group first & then do $lookup - that way JOIN between two collections would be 1-Vs-many rather than many-Vs-many as like it's currently written, So you can try below query for desired o/p & performance gains :
db.billsummaryofthedays.aggregate([
{
"$match": {
"userId": ObjectId("5e43de778b57693cd46859eb"),
"adminId": ObjectId("5e43e5cdc11f750864f46820"),
"date": ISODate("2020-02-11T16:30:00Z"),
}
},
{
$group: {
_id: {
date: "$date",
userId: "$userId",
adminId: "$adminId"
},
totalBill: {
$sum: "$billOfTheDay"
}
}
},
{
$lookup: {
from: "paymentreceivables",
let: {
userId: "$_id.userId",
adminId: "$_id.adminId"
},
pipeline: [
{
$match: {
paymentReceivedOnDate: ISODate("2020-02-11T16:30:00Z"),
$expr: {
$and: [
{
$eq: [
"$userId",
"$$userId"
]
},
{
$eq: [
"$adminId",
"$$adminId"
]
}
]
}
}
},
{
$project: {
amount: 1,
_id: 0
}
}
],
as: "totalPayment"
}
},
{
$addFields: {
totalPayment: {
$reduce: {
input: "$totalPayment",
initialValue: 0,
in: {
$add: [
"$$value",
"$$this.amount"
]
}
}
}
}
}
])
Test : MongoDB-Playground
Am trying to find a way to get the minimum number of orders between
2019-03-17 and 2019-03-19 excluding 2019-03-15 from the results ..
{
"_id" : ObjectId("5c8ffdadde62bf097d54ec47"),
"productId" : "32886845998",
"orders" : [
{
"date" : ISODate("2019-03-15T00:00:00.000+0000"),
"orders" : NumberInt(9)
},
{
"date" : ISODate("2019-03-17T00:00:00.000+0000"),
"orders" : NumberInt(21)
},
{
"date" : ISODate("2019-03-18T00:00:00.000+0000"),
"orders" : NumberInt(20)
},
{
"date" : ISODate("2019-03-19T00:00:00.000+0000"),
"orders" : NumberInt(30)
}
]
}
I tried using $min and $max operator but that didn't help because it iterated through the full array to find maximum & minimum
db.products.aggregate([
{
$project: {
maximum: {
$reduce: {
input: "$orders",
initialValue: 0,
in: {
$max: [
"$$value",
{
$cond: [
{ $gte: [ "$$this.date", ISODate("2019-03-17T00:00:00.000+0000") ] },
"$$this.orders",
0
]
}
]
}
}
}
}
}
])
You can use $filter to apply filtering by orders.date and then you can apply $min and $max on filtered set:
db.col.aggregate([
{
$project: {
filteredOrders: {
$filter: {
input: "$orders",
cond: {
$and: [
{ $gte: [ "$$this.date", ISODate("2019-03-17T00:00:00.000+0000") ] },
{ $lte: [ "$$this.date", ISODate("2019-03-19T00:00:00.000+0000") ] },
]
}
}
}
}
},
{
$project: {
min: { $min: "$filteredOrders.orders" },
max: { $max: "$filteredOrders.orders" },
}
}
])