What's the best way to using nosql in this situation? - mongodb

I want to find the most up-to-date data by user in mongodb.
Assuming the data exists as follows:
{
_id: "1",
user_id: "userA",
date: "2022-10-20 11:00:00.000000000"
},
{
_id: "2",
user_id: "userA",
date: "2022-10-25 18:00:00.000000000"
},
{
_id: "3",
user_id: "userB",
date: "2022-10-25 18:00:00.000000000"
},
{
_id: "4",
user_id: "userC",
date: "2022-10-25 18:00:00.000000000"
}
So I want find result like this:
{
_id: "2",
user_id: "userA",
date: "2022-10-25 18:00:00.000000000"
},
{
_id: "3",
user_id: "userB",
date: "2022-10-25 18:00:00.000000000"
},
{
_id: "4",
user_id: "userC",
date: "2022-10-25 18:00:00.000000000"
}
What's the best way to this?

If you're using Mongo version 5.2+ then you can use $group with the new $bottom operator, like so:
db.collection.aggregate([
{
$group: {
_id: "$user_id",
root: {
"$bottom": {
"sortBy": {
"date": 1
},
"output": "$$ROOT"
}
}
}
},
{
$replaceRoot: {
newRoot: "$root"
}
}
])
MongoPlayground
Otherwise there is no real good way of doing this, you'll have to sort the entire collection by date (assuming all dates are saved in a proper format) then group by user id and get the "latest" instance.
db.collection.aggregate([
{
$sort: {
date: -1
}
},
{
$group: {
_id: "$user_id",
root: {
$first: "$$ROOT"
}
}
},
{
$replaceRoot: {
newRoot: "$root"
}
}
])
Mongo Playground

Related

How to use $match (multiple conditions) and $group in Mongodb

have list of records with the following fields - postBalance, agentId, createdAt, type. I want to filter by “type” and date. After this is done I want to get the $last postBalance for each agent based on the filter and sum up the postBalance. I have been struggling with this using this query
db.transaction.aggregate(
[{ $match: {
$and: [ {
createdAt: { $gte: ISODate('2022-09-15'), $lt:
('2022-09-16') } },
{ type: "CASH_OUT"}]}},
{
$group:
{
_id: {createdAt: {$last: "$createdAt"}},
totalAmount: { $sum: "$postBalance" },
}
}
]
)
An empty array is returned with this query and there are data in the collection.
Below are samples of the documents
{
"_id": {
"$oid": "6334cefd0048787d5535ff16"
},
"type": "CASH_OUT",
"postBalance": {
"$numberDecimal": "23287.625"
},
"createdAt": {
"$date": {
"$numberLong": "1664405245000"
}
},
}
{
"_id": {
"$oid": "6334d438c1ab8a577677cbf3"
},
"userID": {
"$oid": "62f27bc29f51747015fdb941"
},
"aggregatorID": "0000116",
"transactionFee": {
"$numberDecimal": "0.0"
},
"type": "AIRTIME_VTU",
"postBalance": {
"$numberDecimal": "2114.675"
},
"walletHistoryID": 613266,
"walletID": 1720,
"walletActionAt": {
"$date": {
"$numberLong": "1664406584000"
}
},
{
"type": "FUNDS_TRANSFER",
"postBalance": {
"$numberDecimal": "36566.39"
},
"createdAt": {
"$date": {
"$numberLong": "1664407090000"
}
}
}
This is the output I am expecting
{
"date" : 2022-10-09,
"CASHOUT ": 897663,088,
"FUNDS_TRANSFER": 8900877,
"AIRTIME_VTU": 8890000
}
How can my query be aggregated to get this? Thanks
It look like you want something like:
db.collection.aggregate([
{$match: {
createdAt: {
$gte: ISODate("2022-09-15T00:00:00.000Z"),
$lt: ISODate("2022-09-30T00:00:00.000Z")
}
}
},
{$group: {
_id: "$type",
createdAt: {$first: "$createdAt"},
totalAmount: {$sum: "$postBalance"}
}
},
{$group: {
_id: 0,
createdAt: {$first: "$createdAt"},
data: {$push: {k: "$_id", v: "$totalAmount"}}
}
},
{$project: {
data: {$arrayToObject: "$data"},
createdAt: 1,
_id: 0
}
},
{$set: {"data.date": "$createdAt"}},
{$replaceRoot: {newRoot: "$data"}}
])
See how it works on the playground example

How to group by two fields mongoDB aggregation

I have the following documents in mongoDB:
{
_id: "YYYY-MM-DD",
workedTimes: [
{..., driver: {username: string} }
]
}
every workedTime has a driver with a userName prop, and i want to group them by date and driver.username with a aggregation but i can't get it
I've tried something like this
$group: {
"_id": "$workedTimes.driver.username",
"date": {
$push: "$_id"
},
"clocks": {
$push: {
"date": "$_id",
"workedTimes": "$workedTimes"
}
}
}
but i cant get the output i want
I want a output something like this
{
date: 2022-08-24
username: driver1
workedTimes: []
},
{
date: 2022-08-24
username: driver2
workedTimes: []
},
{
date: 2022-08-25
username: driver1
workedTimes: []
}
Maybe this is what you are looking for:
db.collection.aggregate([
{ // Your other stages here },
{
"$unwind": "$workedTimes" // your $unwind stage
},
{
"$group": {
"_id": {
_idField: "$_id",
_driver: "$workedTimes.driver.username"
},
"_workedTimes": {
"$push": "$workedTimes"
}
}
},
{
"$project": {
date: "_id.$_idField",
driver: "$_id._driver",
workedTimes: "$_workedTimes",
_id: 0
}
}
])
Maybe the fields syntaxis is a little weird, but I think this works.

Nested output by group in mongoDB

I have a collection like this.
let Movies= [
{year: '2000', language: 'English', genre: 'Romance' , name: 'A beautiful day'},
{year: '2000', language: 'English', genre: 'Action' , name: 'A Dangerous day'},
{year: '2000', language: 'French', genre: 'Romance' , name: 'someromancename'},
{year: '2000', language: 'French', genre: 'Action' , name: 'someactionname'},]
I need to get a output in group format which may look something like this:
{
"2000" : {
"English": {
"Romance": [{
"name":"A beautiful day"
//other fields
}]
"Action": [{
"name":"A dangerous day"
}]
},
"French": {
"Romance": [{
"name":"Some Romance Name"
}]
"Action": [{
"name":"Some Action Name"
}]
},
}
}
I have tried using aggregation but not able to get the exact query as I am new to MongoDB. This is what I tried to do but the expected result is not achieved.
db.getCollection('Movies').aggregate([
{$group: {_id: {"year" : "$year",
"language": "$language" ,
"genre": "$genre" ,
},
"movies": {"$push": "$$ROOT"}
}
},
{$group: {_id: "$_id.year",
"movies": {"$push": {
"language": "$_id.language",
"genre": "$_id.genre" ,
"movies": "$movies"
}}
}
},
])
You can try,
$group by year, language and genre, and make array of movies name in movies
$group by year and language and make array of with genre in k and v format
$group by year and make array of language after converting above genre array to object using $arrayToObject
convert language array to object using $arrayToObject, convert year with languages object to object using $arrayToObject and replace that object to root using $replaceRoot
db.getCollection('Movies').aggregate([
{
$group: {
_id: {
"year": "$year",
"language": "$language",
"genre": "$genre"
},
"movies": { "$push": { name: "$name" } }
}
},
{
$group: {
_id: {
year: "$_id.year",
language: "$_id.language"
},
"movies": {
"$push": {
k: "$_id.genre",
v: "$movies"
}
}
}
},
{
$group: {
_id: "$_id.year",
movies: {
$push: {
k: "$_id.language",
v: { $arrayToObject: "$movies" }
}
}
}
},
{
$replaceRoot: {
newRoot: {
$arrayToObject: [[
{
k: "$_id",
v: { $arrayToObject: "$movies" }
}
]]
}
}
}
])
Playground

MongoDB - Help needed to make some aggregation

I am having a bad time trying to do an aggregation in MongoDB.
I need to cross some infos from each user and as a final result I want a list of users (where there is only one object for each user) and for each object there is some lists with distinct information.
1 - The createdAtList array must be ordered from the oldest to the newest date. The sumOfTotal means the current position total summed up with the previous sumOfTotal (Exemplified in the code below), not just the sum of the total's
2 - The categotyList must be ordered like: category1, category2, category3 ...
3 - The desired final result must be ordered like: user1, user2, user3 ...
Basically I need some help to do the following:
//List of docs from my collection:
[
{
_id: "doc1",
user: "user1",
category: "category1",
createdAt: "2018-01-01T00:00:00.000Z"
},
{
_id: "doc2",
user: "user1",
category: "category2",
createdAt: "2017-12-12T00:00:00.000Z",
},
{
_id: "doc3",
user: "user1",
category: "category1",
createdAt: "2017-12-12T00:00:00.000Z",
},
{
_id: "doc4",
user: "user1",
category: "category2",
createdAt: "2018-01-01T00:00:00.000Z"
},
{
_id: "doc5",
user: "user1",
category: "category3",
createdAt: "2017-11-11T00:00:00.000Z"
}
]
//Desired result:
{
user: "user1",
createdAtList: [ //list ordered by createdAt
{
createdAt: "2017-11-11T00:00:00.000Z",
total: 1,
sumOfTotal: 0
}
{
createdAt: "2017-12-12T00:00:00.000Z",
total: 2,
sumOfTotal: 3 //summed up with the previous
}
{
createdAt: "2018-01-01T00:00:00.000Z",
total: 2,
sumOfTotal: 5 //summed up with the previous
}
],
categotyList: [ //list ordered by category
{
category: "category1",
total: 2
},
{
category: "category2",
total: 2
},
{
category: "category3",
total: 1
}
]
},
...
Is possible to do this in the same aggregate?
I do not think it really makes sense to have the createdAtList.sumOfTotal field. I do not think the fields in an array should be dependent upon a particular order of the elements. If you want some field to contain the sum of the createdAtList.total field, I think there should only be one field (outside of the array). That being said, here is the query I came up with to give you the desired results (using "users" as the name of the collection):
db.users.aggregate([
{
$group: {
_id: {
user: "$user",
createdAt: "$createdAt"
},
total: { $sum: 1 },
category: { $push: "$category" }
}
},
{
$project: {
_id: 0,
user: "$_id.user",
createdAt: "$_id.createdAt",
total: "$total",
category: 1
}
},
{ $unwind: "$category" },
{
$group: {
_id: {
user: "$user",
category: "$category"
},
catTotal: { $sum: 1 },
createdAtList: {
$push: {
createdAt: "$createdAt",
total: "$total"
}
}
}
},
{
$project: {
_id: 0,
user: "$_id.user",
createdAtList: 1,
category: "$_id.category",
catTotal: 1
}
},
{ $unwind: "$createdAtList" },
{
$group: {
_id: "$user",
createdAtList: {
$addToSet: "$createdAtList"
},
categoryList: {
$addToSet: {
category: "$category",
total: "$catTotal"
}
}
}
},
{ $unwind: "$createdAtList" },
{ $sort: { "createdAtList.createdAt": 1 } },
{
$group: {
_id: "$_id",
createdAtList: {
$push: "$createdAtList"
},
categoryList: {
$first: "$categoryList"
}
}
},
{ $unwind: "$categoryList" },
{ $sort: { "categoryList.category": 1 } },
{
$group: {
_id: "$_id",
createdAtList: {
$first: "$createdAtList"
},
categoryList: {
$push: "$categoryList"
}
}
},
{
$project: {
_id: 0,
user: "$_id",
createdAtList: 1,
sumOfTotal: { $sum: "$createdAtList.total" },
categoryList: 1
}
},
{ $sort: { user: 1 } },
]).pretty()

Mongodb Aggregation count array/set size

Here's my problem:
Model:
{ application: "abc", date: Time.now, status: "1" user_id: [ id1, id2,
id4] }
{ application: "abc", date: Time.yesterday, status: "1", user_id: [
id1, id3, id5] }
{ application: "abc", date: Time.yesterday-1, status: "1", user_id: [
id1, id3, id5] }
I need to count the unique number of user_ids in a period of time.
Expected result:
{ application: "abc", status: "1", unique_id_count: 5 }
I'm currently using the aggregation framework and counting the ids outside mongodb.
{ $match: { application: "abc" } }, { $unwind: "$users" }, { $group:
{ _id: { status: "$status"},
users: { $addToSet: "$users" } } }
My arrays of users ids are very large, so I have to iterate the dates or I'll get the maximum document limit (16mb).
I could also $group by
{ year: { $year: "$date" }, month: { $month: "$date" }, day: {
$dayOfMonth: "$date" }
but I also get the document size limitation.
Is it possible to count the set size in mongodb?
thanks
The following will return number of uniqueUsers per application. This will apply an group operation to a result of a group operation by using pipeline feature of mongodb.
{ $match: { application: "abc" } },
{ $unwind: "$users" },
{ $group: { _id: "$status", users: { $addToSet: "$users" } } },
{ $unwind:"$users" },
{ $group : {_id : "$_id", count : {$sum : 1} } }
Hopefully this will be done in an easier way in the following releases of mongo by a command which gives the size of an array under a projection. {$project: {id: "$_id", count: {$size: "$uniqueUsers"}}}
https://jira.mongodb.org/browse/SERVER-4899
Cheers
Sorry I'm a little late to the party. Simply grouping on the 'user_id' and counting the result with a trivial group works just fine and doesn't run into doc size limits.
[
{$match: {application: 'abc', date: {$gte: startDate, $lte: endDate}}},
{$unwind: '$user_id'},
{$group: {_id: '$user_id'}},
{$group: {_id: 'singleton', count: {$sum: 1}}}
];
Use $size to get the size of set.
[
{
$match: {"application": "abc"}
},
{
$unwind: "$user_id"
},
{
$group: {
"_id": "$status",
"application": "$application",
"unique_user_id": {$addToSet: "$user_id"}
}
},
{
$project:{
"_id": "$_id",
"application": "$application",
"count": {$size: "$unique_user_id"}
}
}
]