MongoDB group by date and fill zero counts - mongodb

I would like to group my search results by date. The results are then also counted.
var fromDate = new Date(req.query.fromDate);
var toDate = new Date(req.query.toDate);
global.databaseStatistic.collection(global.ARTICLEPRESS).aggregate([
{
$match: {
article: { $in: articleIdArray },
date: { $gte: fromDate, $lte: toDate }
}
},
{
$group: {
"_id": {
"date": {
$dateToString: { format: "%d.%m.%Y", date: "$date" }
}
},
"count": { $sum: 1 }
}
}
])
This works perfectly, but I would also like to show all days that have a count of zero. It should be so displayed every day within an interval, also having a count of zero. How can I do that?

as it looks very trivial - it is not.
There is no way to have a reference sequence to compare with, even $lookup cannot help as this a kind of inner join type.
The way you could have this done is a kind of post process of result-set returned to mongoose.
The steps I have in mind could be:
create array of dates from req.query.fromDate to req.query.toDate formated in the same way as in query
remove entries in array which we have in result set
merge our array with count:0 and date
sort results (if needed)
any comments welcome!

Related

How to write a single query to count elements above a certain value in MongoDB

I have the following sample collection of movies:
[
{
"title":"Boots and Saddles",
"year":1909,
"cast":[],
"genres":[]
},
{
"title":"The Wooden Leg",
"year":1909,
"cast":[],
"genres":[]
},
{
"title":"The Sanitarium",
"year":1910,
"cast":["Fatty Arbuckle"],
"genres":["Comedy"]
},
{
"title":"Snow White",
"year":1916,
"cast":["Marguerite Clark"],
"genres":["Fantasy"]
},
{
"title":"Haunted Spooks",
"year":1920,
"cast":["Harold Lloyd"],
"genres":["Comedy"]
},
{
"title":"Very Truly Yours",
"year":1922,
"cast":["Shirley Mason", "lan Forrest"],
"genres":["Romance"]
}
]
I want to count number of movies appeared in the last 20 years (from the last movie recorded in this collection).
I have following query to find which year is the most recent movie (result shows 2018):
db.movies.find({},{"_id":0, "year":1}).sort({year:-1}).limit(1)
So to find how many movies appeared in the last 20 years I wrote this:
db.movies.aggregate([{$match:{year:{$gte:1999}}},{$count:"title"}])
However, this is not very optimized, because if the database is modified or updated,I will have to modify that query every time.
Is there a more elegant way to find the result?
Thank you in advance!
You can use mongodb aggregate method.
db.movies.aggregate([
{ $sort: { year: -1 } },
{ $limit: 1 },
{
$project: {
currentYear: { $year: new Date() },
latestMovieYear: "$year",
last20Years: { $subtract: [ "$currentYear", 20 ] }
}
},
{
$match: {
year: { $gte: "$last20Years", $lte: "$latestMovieYear" }
}
},
{ $count: "movies" }
]);
Sort the documents by year in descending order, and limit the number of documents to 1. It will return latest movie present in the collection.
Use the $project operator to create a new field currentYear that returns the current year, latestMovieYear that returns the year of the latest movie, and last20Years that subtracts 20 from the current year.
Use $match operator to filter out the movies that have a year greater than or equal to last20Years and less than or equal to latestMovieYear.
Use the $count operator to count the number of documents that match the above criteria.

MongoDB, Panache, Quarkus: How to do aggregate, $sum and filter

I have a table in mongodb with sales transactions each containing a userId, a timestamp and a corresponding revenue value of the specific sales transaction.
Now, I would like to query these users and getting the minimum, maximum, sum and average of all transactions of all users. There should only be transactions between two given timestamps and it should only include users, whose sum of revenue is greater than a specified value.
I have composed the corresponding query in mongosh:
db.salestransactions.aggregate(
{
"$match": {
"timestamp": {
"$gte": new ISODate("2020-01-01T19:28:38.000Z"),
"$lte": new ISODate("2020-03-01T19:28:38.000Z")
}
}
},
{
$group: {
_id: { userId: "$userId" },
minimum: {$min: "$revenue"},
maximum: {$max: "$revenue"},
sum: {$sum: "$revenue"},
avg: {$avg: "$revenue"}
}
},
{
$match: { "sum": { $gt: 10 } }
}
]
)
This query works absolutely fine.
How do I implement this query in a PanacheMongoRepository using quarkus ?
Any ideas?
Thanks!
A bit late but you could do it something like this.
Define a repo
this code is in kotkin
class YourRepositoryReactive : ReactivePanacheMongoRepository<YourEntity>{
fun getDomainDocuments():List<YourView>{
val aggregationPipeline = mutableListOf<Bson>()
// create your each stage with Document.parse("stage_obj") and add to aggregates collections
return mongoCollection().aggregate(aggregationPipeline,YourView::class.java)
}
mongoCollection() automatically executes on your Entity
YourView, a call to map related properties part of your output. Make sure that this class has
#ProjectionFor(YourEntity.class)
annotation.
Hope this helps.

MONGODB return min value for a field if falls within last year otherwise return min value for collection

I have a collection that consists of documents in this format:
{"_id":{"date_played":{"$date":"1998-03-28T00:00:00.000Z"},"course_played":4,"player_id":11},"score":[5,6,4,4,5,9,6,6,5,7,6,6,5,7,5,3,9,4],"handicap":30,"cash_won":0,"sort_order":6,"gross_score":102,"gross_sfpts":34,"skins_group":1,"score_differential":28,"pcc_adjustment":0,"player_name":"Dave"}
By _id.player_id I am trying to return the min value of "score_differential" for records within the last year(_id.date_played), if no records in the last year then I want the min "score_differential" for the player in the collection.
I have tried lots of combinations but this is the closest I have got:-
Which returns the correct values but the problem is if a date is found within the year I get two records back, one with _id: false which has lowest in collection and one with _id: true which has lowest for year. My problem is that I only want one record back not two. Any help is much appreciated as I have spent days on this, relatively new to mongodb coming from mysql.
{
'$match': {
'_id.player_id': 11
}
}, {
'$group': {
'_id': {
'$min': [
{
'$gt': [
'$_id.date_played', {
'$dateFromParts': {
'year': {
'$subtract': [
{
'$year': new Date()
}, 1
]
},
'month': {
'$month': new Date()
},
'day': {
'$dayOfMonth': new Date()
}
}
}
]
}
]
},
'minWHI': {
'$min': '$score_differential'
}
}
}
] ```
Thx but I no longer need a solution to this, I actually use the above code in a $lookup stage in another query and as that returns an array I end up with just one document returned where I can test the matched field for the values I need, I was just confused as I was trying to test each part of the main query separately and in doing so getting two documents.

Mongodb group epoch to YYYY-MM-DD

So currently store start and end time that a user listens to our stations in epoch format, however now I need to do a query that groups listeners if they where listening with an hour block.
For example:
last 7 days Query would need to go back 7 days normally I would just do something like this
items.find({
history.$.starttime: {
$gte: ISODate("2020-07-10T00:00:00.000Z"),
$lt: ISODate("2020-009-17T24:59:59.000Z")
}})
However we don't store the date in this format we store it looking like this.
So how do a search between dates if the dates are in epoch format?
I have tried
{'history': {'$elemMatch':{'starttime': {'$gte':1592966066060}}}
UPDATE:
this works
{"history.starttime":{$gte:1593236606706}}
You can convert timestamp to ISO date using $toDate, like this,
db.collection.aggregate([
{
$addFields: {
startTimeDate: {
$toDate: "$startTime"
},
endTimeDate: {
$toDate: "$endTime"
}
}
},
{
$match: {
startTimeDate: {
$gte: ISODate("2020-01-20T21:20:00Z")
}
}
}
])
You can removed that added field using:
{
$project: {
startTimeDate: 0,
endTimeDate: 0
}
}
Working Playground: https://mongoplayground.net/p/XiEYVxvmfpv
Convert the times to Unix timestamps in your application, specify the timestamps in the query.

How to find data between two dates in mongdb

I want to find all ipAddress "192.168.0.52" between two dates,
any help please
varMatch = { $match : { "ipAddress": "192.168.0.52"}},
varGroup = { $group: {
"_id": {timestamp:{"$auditMessages.timestamp":
{$gt: ISODate("2020-05-14T13:08:30.748Z"),
$lt: ISODate("2020-04-08T13:00:34.567Z")
}}}}},
db.compte.aggregate([varMatch,varGroup])
I have a problem in date i can't find a correct syntax fo date, however in json file I don't have after timestamp.
This is example of my DB
Thank you
You can find all documents with IP address '192.168.0.52' and timestamps between two dates by adding an additional expression to your match stage.
const matchStage = {
$match: {
ipAddress: '192.168.0.52',
'auditMessages.timestamp': {
$gt: new Date('2020-05-14T13:08:30.748Z'),
$lt: new Date('2020-04-08T13:00:34.567Z')
}
}
};
Once that's done, you can perform your grouping operation as desired.