weird behaviour mongo aggregation framework - mongodb

I have one Document called account holding and it has below records,
{ "_id" : ObjectId("57cfbb09e4b024be2f1bce57"),
"_class" : "com.commercestudio.domain.AccountHolding",
"accountId" : "5732933ae4b0b709443b0d1e",
"companyId" : "57223d6de4b06c4ef00415b5",
"brokerageAccountId" : "5KC05007",
"symbol" : "AGG",
"quantity" : 1.0,
"pricePaid" : 112.55,
"processDate" : ISODate("2016-09-06T00:00:00.000Z"),
"recordDate" : ISODate("2016-09-06T00:00:00.000Z"),
"createdOn" : ISODate("2016-09-07T07:00:25.479Z")
}
{ "_id" : ObjectId("57cfbb09e4b024be2f1bce5b"),
"_class" : "com.commercestudio.domain.AccountHolding",
"accountId" : "5732933ae4b0b709443b0d1e",
"companyId" : "57223d6de4b06c4ef00415b5",
"brokerageAccountId" : "5KC05007",
"symbol" : "LQD",
"quantity" : 4.0,
"pricePaid" : 123.78,
"processDate" : ISODate("2016-09-06T00:00:00.000Z"),
"recordDate" : ISODate("2016-09-06T00:00:00.000Z"),
"createdOn" : ISODate("2016-09-07T07:00:25.498Z")
}
.....
now I apply aggrigration framework for finding out latest record date data for perticulat accountId,
db.accountHolding.aggregate(
[
{
"$match": {
"accountId": "5834caf32ae7bacc527ef2f3",
"symbol": {
"$in": [
"IUSG",
"VEA",
"IEMG",
"SCHX",
"VBR",
"IUSV",
"VOE"
]
}
}
},
{
"$group": {
"_id": "$symbol",
"recordDate": {
"$last": "$recordDate"
},
"quantity": {
"$last": "$quantity"
},
"pricePaid": {
"$last": "$pricePaid"
}
}
}
])
and it returns two different results in two different environments,
On my development env. it shows,
{
"_id" : "VEA",
"recordDate" : ISODate("2018-03-02T00:00:00.000Z"),
"quantity" : 22.79609, "pricePaid" : 44.14
}
{ "_id" : "IUSG",
"recordDate" : ISODate("2018-03-02T00:00:00.000Z"),
"quantity" : 8.87831,
"pricePaid" : 55.79
}
something like this and from production env. it shows,
{
"_id" : "VEA",
"recordDate" : ISODate("2018-02-26T00:00:00Z"),
"quantity" : 22.79609,
"pricePaid" : 45.76
}
{
"_id" : "IUSG",
"recordDate" : ISODate("2018-02-26T00:00:00Z"),
"quantity" : 8.87831,
"pricePaid" : 57.47
}
actually, I am unable to find out the solution why this weird behaviour is taken place, as both env has same data.
My database server is deployed on AWS instance.
Can someone help me out with finding out the root cause and solution for the same?

This is expected behavior.
From the docs,
Returns the value that results from applying an expression to the last
document in a group of documents that share the same group by a field.
Only meaningful when documents are in a defined order.
Add $sort before $group stage.
{$sort:{recordDate:1}}

Related

Tried limiting the Group By data on mongo db 3.0

I have tried limiting the Group by data on Mongo DB 3.0 but seems like there is no proper option to do this.
I want to display 5 latest records for each day based on this date dateCreated which is in array and corresponding records count/day
db.entries.find().limit(1).pretty();
{
"_id" : "bd348fb4dd38dd7a2",
"className" : "com.model.Entry",
"name" : "yQLs3T5NCJocPlOPuLgyEkQ9",
"description" : "4Z09BNPQNFhFiMbjqL RWC5SMs0d0XzogqdNmjk5dx1mw9roHgRrl8ljbHo16p1WTlNYU",
"account" : DBRef("accounts", "248a3-448b-a912-6573f23d34a5"),
"iconUrl" : "gA9QTuqYv9wZq1xKM37jdL",
"userCreatedBy" : DBRef("users", "8044-45d2-8567-a6cb808ce164"),
"timezone" : "Atlantic/Faroe",
"globalAccess" : false,
"tags" : [
{
"_id" : "8926079483331",
"category" : "PPq5k",
"value" : "NdKFQq",
"description" : "uDQVnhJ2tu5XWHinb",
"origin" : "User",
"dateCreated" : ISODate("2021-07-16T18:20:41.731Z"),
"dateModified" : ISODate("2021-07-16T18:20:53.319Z"),
"externalId" : "xkblzrwE"
}
{
"_id" : "89389483331",
"category" : "PPe5k",
"value" : "NdKFQq",
"description" : "uDQVnhJ2tu5XWHinb",
"origin" : "User",
"dateCreated" : ISODate("2021-07-16T18:20:41.731Z"),
"dateModified" : ISODate("2021-07-16T18:20:53.319Z"),
"externalId" : "xkblzrwE"
}
]}
Output I'm expecting this :
[2021-07-16 (Date)-> (5 Latest Entries) , 2 (total records for that day) ]
I have tried using below solution
How to get lastest N records of each group in mongodb?
and slice is not available on mongo 3.0 i'm kind of stuck here
I'm not sure you can use aggregation but below aggregation can give you the result you wanted.
db.collection.aggregate([
{
"$unwind": "$tags"
},
{
$match: {
"tags.dateCreated": {
"$gte": ISODate("2021-07-16")
}
}
},
{
"$replaceRoot": {
"newRoot": "$tags"
}
},
{
"$limit": 5
}
])
Playground
If you have any question feel free to ask me, please.

Mongo aggregation groups and subgroup

Hi I have a Mongo aggregation:
[
{
"$match" : {
"dateTime" : {
"$gte" : ISODate("2017-01-01T00:00:00.000+0000"),
"$lt" : ISODate("2018-01-01T00:00:00.000+0000")
}
}
},
{
"$group" : {
"_id" : "dateTime",
"totals" : {
"$sum" : "$payment.totalAmount"
},
"count" : {
"$sum" : 1.0
}
}
}
],
{
"allowDiskUse" : false
}
);
This works fine. It aggregates, and sums by date range I supplied and I get an output as follows.
{
"_id" : "dateTime",
"totals" : 2625293.825017198,
"count" : 12038.0
}
However, I also want to further refine the groupings.
I have a field called 'companyId' and I want to calculate the sum and count by each company Id for the given time range.
I would like to get an output similar to this, where I get a sum and count for each company ID in the date range I queried, not just a sum/count of all the data:
[
{
"companyId" : "Acme Co",
"totals" : 2625293.825017198,
"count" : 12038.0
},
{
"companyId" : "Beta Co",
"totals" : 162593.82198,
"count" : 138.0
},
{
"companyId" : "Cel Co",
"totals" : 593.82,
"count" : 38.0
}
]
How do I do this? I have not been able to find a good example online.
Thanks

MongoDB, Grouping by Multiple Fields

pretty new to Mongo and am finding some simple things that i would do in SQL frustratingly difficult in Mongo.
I have an object similar to this below
[{
"_id" : ObjectId("5870fb29a1fe030e1a2909db"),
"updatedAt" : ISODate("2017-01-07T14:28:57.224Z"),
"createdAt" : ISODate("2017-01-07T14:28:57.224Z"),
"state" : "Available",
},
{
"_id" : ObjectId("5870fb29a1fe030e1a2909dc"),
"updatedAt" : ISODate("2017-01-07T14:28:57.224Z"),
"createdAt" : ISODate("2017-01-07T14:28:57.224Z"),
"state" : "notReady",
},
{
"_id" : ObjectId("5870fb29a1fe030e1a2909d9"),
"updatedAt" : ISODate("2017-01-07T14:28:57.224Z"),
"createdAt" : ISODate("2017-01-07T14:28:57.224Z"),
"state" : "Disconnected",
}]
What i'm looking to do it group the data by the Maximum date and the state.
Ideally the result i would be looking for would be something like the following.
{
latestDate: "2017-01-07T14:28:57",
states : {
available : 10,
disconnected : 5,
notReady : 2
}}
Basically i'm looking for the SQL equivalent of this:
SELECT createdAt, state, COUNT(rowid)
FROM db
WHERE date = (SELECT MAX(createdAt) FROM db)
GROUP BY 1,2
I've searched around here and have found some good info but am probably missing something straight forward. Ive only managed to get here so far
db.collection.aggregate([
{$project: {"_id" : 0,"state": 1, "date" : "$createdAt"}},
{$group : {"_id" : {"date":"$date", "state": "actual"}, "count":{"$sum":1}}}
])
Any help would be appreciated :)
db.collection.aggregate([
{
$group : {
_id : {
date : "$createdAt",
state : "$state"
},
count : {$sum : 1}
}
},
{
$group : {
_id : "$_id.date",
states : {
$addToSet : {
state : "$_id.state",
count : "$count"
}
}
}
},
{
$sort : {_id : -1}
},
{
$limit : 1
},
{
$project : {
_id : 0,
latestDate : "$_id",
states : "$states"
}
}
])
output :
{
"latestDate" : ISODate("2017-01-07T14:28:57.224Z"),
"states" : [
{
"state" : "Available",
"count" : 1
},
{
"state" : "notReady",
"count" : 1
},
{
"state" : "Disconnected",
"count" : 1
}
]
}

Get document based on multiple criteria of embedded collection

I have the following document, I need to search for multiple items from the embedded collection"items".
Here's an example of a single SKU
db.sku.findOne()
{
"_id" : NumberLong(1192),
"description" : "Uploaded via CSV",
"items" : [
{
"_id" : NumberLong(2),
"category" : DBRef("category", NumberLong(1)),
"description" : "840 tag visual",
"name" : "840 Visual Mini Round",
"version" : NumberLong(0)
},
{
"_id" : NumberLong(7),
"category" : DBRef("category", NumberLong(2)),
"description" : "Maxi",
"name" : "Maxi",
"version" : NumberLong(0)
},
{
"_id" : NumberLong(11),
"category" : DBRef("category", NumberLong(3)),
"description" : "Button",
"name" : "Button",
"version" : NumberLong(0)
},
{
"_id" : NumberLong(16),
"category" : DBRef("category", NumberLong(4)),
"customizationFields" : [
{
"_class" : "CustomizationField",
"_id" : NumberLong(1),
"displayText" : "Custom Print 1",
"fieldName" : "customPrint1",
"listOrder" : 1,
"maxInputLength" : 12,
"required" : false,
"version" : NumberLong(0)
},
{
"_class" : "CustomizationField",
"_id" : NumberLong(2),
"displayText" : "Custom Print 2",
"fieldName" : "customPrint2",
"listOrder" : 2,
"maxInputLength" : 17,
"required" : false,
"version" : NumberLong(0)
}
],
"description" : "2 custom lines of farm print",
"name" : "Custom 2",
"version" : NumberLong(2)
},
{
"_id" : NumberLong(20),
"category" : DBRef("category", NumberLong(5)),
"description" : "Color Red",
"name" : "Red",
"version" : NumberLong(0)
}
],
"skuCode" : "NF-USDA-XC2/SM-BC-R",
"version" : 0,
"webCowOptions" : "840miniwithcust2"
}
There are repeat items.id throughout the embedded collection. Each Sku is made up of multiple items, all combinations are unique, but one item will be part of many Skus.
I'm struggling with the query structure to get what I'm looking for.
Here are a few things I have tried:
db.sku.find({'items._id':2},{'items._id':7})
That one only returns items with the id of 7
db.sku.find({items:{$all:[{_id:5}]}})
That one doesn't return anything, but it came up when looking for solutions. I found about it in the MongoDB manual
Here's an example of a expected result:
sku:{ "_id" : NumberLong(1013),
"items" : [ { "_id" : NumberLong(5) },
{ "_id" : NumberLong(7) },
{ "_id" : NumberLong(12) },
{ "_id" : NumberLong(16) },
{ "_id" :NumberLong(2) } ] },
sku:
{ "_id" : NumberLong(1014),
"items" : [ { "_id" : NumberLong(5) },
{ "_id" : NumberLong(7) },
{ "_id" : NumberLong(2) },
{ "_id" : NumberLong(16) },
{ "_id" :NumberLong(24) } ] },
sku:
{ "_id" : NumberLong(1015),
"items" : [ { "_id" : NumberLong(5) },
{ "_id" : NumberLong(7) },
{ "_id" : NumberLong(12) },
{ "_id" : NumberLong(2) },
{ "_id" :NumberLong(5) } ] }
Each Sku that comes back has both a item of id:7, and id:2, with any other items they have.
To further clarify, my purpose is to determine how many remaining combinations exist after entering the first couple of items.
Basically a customer will start specifying items, and we'll weed it down to the remaining valid combinations. So Sku.items[0].id=5 can only be combined with items[1].id=7 or items[1].id=10 …. Then items[1].id=7 can only be combined with items[2].id=20 … and so forth
The goal was to simplify my rules for purchase, and drive it all from the Sku codes. I don't know if I dug a deeper hole instead.
Thank you,
On the part of extracting the sku with item IDs 2 and 7, when I recall correctly, you have to use $elemMatch:
db.sku.find({'items' :{ '$all' :[{ '$elemMatch':{ '_id' : 2 }},{'$elemMatch': { '_id' : 7 }}]}} )
which selects all sku where there is each an item with _id 2 and 7.
You can use aggregation pipelines
db.sku.aggregate([
{"$unwind": "$sku.items"},
{"$group": {"_id": "$_id", "items": {"$addToSet":{"_id": "$items._id"}}}},
{"$match": {"items._id": {$all:[2,7]}}}
])

mongodb Embedded document search on parent and child field

I have a nested embedded document CompanyProduct below is structure
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca3"),
"CompanyID" : 90449,
"Name" : Company1,
"CompanyDepartment" : [
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca4")
"DepartmentID" : 287,
"DepartmentName" : "Stores",
"DepartmentInventory" : [
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 1,
"ProductName" : "abc",
"Quantity" : 100
},
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 2,
"ProductName" : "xyz",
"Quantity" : 1
}
],
}
],
}
There can be N no of companies and each company can have N number of departments and each department can have N number of products.
I want to do a search to find out a particular product quantity under a particular company
I tried below query but it does not work. It returns all the products for the specific company, the less than 20 condition doesn't work.
db.CompanyProduct.find({$and : [{"CompanyDepartment.DepartmentInventory.Quantity":{$lt :20}},{"CompanyID":90449}]})
How should the query be?
You are searching from companyProduct's sub documents. So it will return you companyProduct whole document, it is NoSQL database , some how we do not need to normalize the collection , but your case it has to be normalize , like if you want to EDIT/DELETE any sub document and if there are thousand or millions of sub document then what will you do ... You need to make other collection with the name on CompanyDepartment and companyProduct collection should be
productCompany
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca3"),
"CompanyID" : 90449,
"Name" : Company1,
"CompanyDepartment" : ['53d213c5ddbb1912343a8ca4'],
}
and other collection companyDepartment
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca4")
"DepartmentID" : 287,
"DepartmentName" : "Stores",
"DepartmentInventory" : [
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 1,
"ProductName" : "abc",
"Quantity" : 100
},
{
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 2,
"ProductName" : "xyz",
"Quantity" : 1
}
],
}
after this you got array of companyDeparment' ID and only push and pull query will be used on productCompany
A Solution can be
db.YourCollection.aggregate([
{
$project:{
"CompanyDepartment.DepartmentInventory":1,
"CompanyID" : 1
}
},{
$unwind: "$CompanyDepartment"
},{
$unwind: "$CompanyDepartment.DepartmentInventory"
},{
$match:{$and : [{"CompanyDepartment.DepartmentInventory.Quantity":{$lt :20}},{"CompanyID":90449}]}
}
])
the result is
{
"result" : [
{
"_id" : ObjectId("53d213c5ddbb1912343a8ca3"),
"CompanyID" : 90449,
"CompanyDepartment" : {
"DepartmentInventory" : {
"_id" : ObjectId("53b7b92eecdd765430d763bd"),
"ProductID" : 2,
"ProductName" : "xyz",
"Quantity" : 1
}
}
}
],
"ok" : 1
}