I have the following document:
{
id: "myId",
boundedPlan: {
plannedWeeks: [
0 : {
weekStartDate: date
weekEndDate: date
plannedDays: []
},
...
]
},
unboundedPlan: {
plannedWeeks: [
0 : {
weekStartDate: date
weekEndDate: date
plannedDays: []
},
...
]
}
}
This plan represent some number of weeks in the future. The plan has a bounded or unbounded plan .
(I have the same structure on two different fields, because in the code they correspond to two different classes with different behavior).
I now have to do the following query.
"Get the current plan week given a date"
I wrote the following pipeline:
[
{ "$match" : { "ownerId" : "defaultOwnerId"}},
{ "$project" : {
"boundedPlan" : 1,
"unboundedPlan" : 1,
"plannedWeeks" : {
"$cond" : {
"if" : { "$ne" : ["$boundedPlan", null]}, "then" : "$boundedPlan.plannedWeeks",
"else" : "$unboundedPlan.plannedWeeks"}
}
}
},
{ "$match" : {
"boundedPlan.plannedWeeks" : {
"$elemMatch" : { "weekStart" : { "$lte" : { "$date" : "2021-03-10T00:00:00Z"}}, "weekEnd" : { "$gte" : { "$date" : "2021-03-10T00:00:00Z"}}}},
"$or" : [{
"unboundedPlan.plannedWeeks" : {
"$elemMatch" : { "weekStart" : { "$lte" : { "$date" : "2021-03-10T00:00:00Z"}}, "weekEnd" : { "$gte" : { "$date" : "2021-03-10T00:00:00Z"}}}}
}]}
}
]
The problem is the following:
knowing that Im operating over a plan with an unbounded plan and explicitly setting the second match :
"$match" : {
"unboundedPlan.plannedWeeks" : {
"$elemMatch" : { "weekStart" : { "$lte" : { "$date" : "2021-03-10T00:00:00Z"}}, "weekEnd" : { "$gte" : { "$date" : "2021-03-10T00:00:00Z"}}}},
}
works.
of course I dont know if the plan is from the unbounded or bounded field, so I tried to add the or operator, which causes no selection at all.
Is there something Im missing?
(working with spring data mongo)
Thank you
Ok, found out... I was using the OrOperator class from spring data mongo wrongly:
new Criteria("field1").orOperator(new Criteria("field2"))
is not the same as
new Criteria().orOperator(new Criteria("field1"), new Criteria("field2")
Related
We have a collection in Mongodb which saves a value linked to a timestamp.
Our document looks as follows (I have pasted an actual one here):
{
"_id" : ObjectId("5a99596b0155fe271cfcf41d"),
"Timestamp" : ISODate("2018-03-02T16:00:00.000Z"),
"TagID" : ObjectId("59f8609eefbb4102f4c249e3"),
"Value" : 71.3,
"FileReferenceID" : ObjectId("000000000000000000000000"),
"WasValueInterpolated" : 0
}
What we then do is calculate the avg between two intervals for a given period, in more basic terms, work out an aggregated profile.
The aggregation code we use is:
{[{ "$match" :
{
"TagID" : ObjectId("59f8609eefbb4102f4c249e3") }
},
{
"$match" : { "Timestamp" : { "$gte" : ISODate("2018-03-12T00:00:00.001Z") } }
},
{
"$match" : { "Timestamp" : { "$lte" : ISODate("2018-03-13T00:00:00.001Z") } }
},
{
"$group" :
{
"_id" : { "GroupedMillisecond" :
{
"$let" :
{
"vars" :
{ "newMillisecondField" :
{
"$subtract" : ["$Timestamp", ISODate("2018-03-12T00:00:00.001Z")]
}
},
"in" : { "$subtract" : ["$$newMillisecondField", { "$mod" : ["$$newMillisecondField", NumberLong(1800000)] }] }
}
} }, "AverageValue" : { "$avg" : "$Value" }
}
}, { "$sort" : { "_id.GroupedMillisecond" : 1 } }
]}
The problem is this, the value it should give back is 71.3, but we get back 71.299999999999997
In this case, I posted above we are calculating the avg value, half hourly aggregated, for a day. And there is only one value per half hour logged (I checked this in the database). The value is also logged as a constant, as far back as I manually checked (a few months back) it is 71.3
So my question is why does the value differ?
I have the following aggregation which has a date comparison where I only want to grab documents that have the $$item.date less than the current time.
[
{ $match : { "_id" : "57b4c5f0291ebb13110b888e" } },
{ $project : {
"fulfillments" : {
$filter : {
"input" : "$fulfillments",
"as" : "item",
"cond" : { "$gte" : ["$$item.date","new Date()"]}
}
}
}
},
...
...
]
The important part that I have a question about above is the following:
"cond" : { "$gte" : ["$$item.date","new Date()"]}
This doesn't seem to be working as I can change the new Date() to 1 or 0 or pretty much any value and it still returns all documents. I need it to only return documents that have the date greater than or equal to the current date.
For example, given the following document
{
"_id": "57b4c5f0291ebb13110b888e",
"fulfillments": [
{
"_id": "582deb33bb117300010a2ae5",
"date": new Date("2016-11-23T17:00:00-0700"),
},
{
"_id": "582deb33bdf17300010a2ae5",
"date": new Date("2017-11-23T17:00:00-0700"),
}
}
Only the following fulfillment should be returned because it is 2017-11-23
{
"_id": "582deb33bdf17300010a2ae5",
"date": new Date("2017-11-23T17:00:00-0700"),
}
Update
There is question if I am giving an accurate document strucutre, so I included a screenshot below to validate this.
If you just want the current time on the machine just remove the " from the query, see below for my example
> db.test.find().pretty()
{
"_id" : "57b4c5f0291ebb13110b888e",
"fulfillments" : [
{
"_id" : "582deb33bb117300010a2ae5",
"date" : ISODate("2016-11-24T00:00:00Z")
},
{
"_id" : "582deb33bdf17300010a2ae5",
"date" : ISODate("2017-11-24T00:00:00Z")
}
]
}
>
> db.test.aggregate([
... { $match : { "_id" : "57b4c5f0291ebb13110b888e" } },
... { $project : {
... "fulfillments" : {
... $filter : {
... "input" : "$fulfillments",
... "as" : "item",
... "cond" : { "$gte" : ["$$item.date",new Date()]}
... }
... }
... }
... }
... ]).pretty()
{
"_id" : "57b4c5f0291ebb13110b888e",
"fulfillments" : [
{
"_id" : "582deb33bdf17300010a2ae5",
"date" : ISODate("2017-11-24T00:00:00Z")
}
]
}
>
I have collection (tvshow episodes) with more than 1,200,000 document,
here is my schema :
var episodeSchema = new Schema({
imdbId: { type : String },
showId: {type : String},
episodeId: { type : String },
episodeIdNumber:{ type : Number },
episodeTitle:{ type : String },
showTitle:{type : String},
seasonNumber:{type : Number},
episodeNumber:{type : Number},
airDate : {type : String},
summary:{type : String}
});
I created Index for episodeTitle episodeIdNumber seasonNumber episodeNumber episodeId and showId
Now i used mongodb aggregate group to get every tvshow episodes
here is the aggregate query i used :
episode.aggregate( [
{ $match : { showId : "scorpion" } },
{$sort:{"episodeNumber":-1}},
{ $group: {
_id: "$seasonNumber", count: { $sum: 1 } ,
episodes : { $push: { episodeId : "$episodeId" , episodeTitle: "$episodeTitle" , episodeNumber: "$episodeNumber" , seasonNumber: "$seasonNumber" , airDate: "$airDate" } }
} }
,
{ $sort : { _id : -1 } }
] )
Now when i am run this query its take more than 2605.907 ms , after some digging i found out why its slow , it was because of using {$sort:{"episodeNumber":-1}} , without using {$sort:{"episodeNumber":-1}} its take around 19.178 ms to run.
As i mentioned above i create an index for episodeNumber field and based on MongoDB Aggregation Pipeline Optimization
i used sort after match so basically everything was ok , and i didn't anything wrong.
So after this i thought something wrong with my indexes , so i removed episodeNumber index and reindexd , but i had same time nothing changed.
At end one time i tried run aggregate group query without episodeNumber indexed and surprisingly it was faster ! its take around 20.118 ms .
I wants know why this happened , isn't indexes to get faster query ?
Update
query explain output :
{
"waitedMS" : NumberLong(0),
"stages" : [
{
"$cursor" : {
"query" : {
"showId" : "scorpion"
},
"sort" : {
"episodeNumber" : -1
},
"fields" : {
"airDate" : 1,
"episodeId" : 1,
"episodeNumber" : 1,
"episodeTitle" : 1,
"seasonNumber" : 1,
"_id" : 0
},
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "test.episodes",
"indexFilterSet" : false,
"parsedQuery" : {
"showId" : {
"$eq" : "scorpion"
}
},
"winningPlan" : {
"stage" : "EOF"
},
"rejectedPlans" : [ ]
}
}
},
{
"$group" : {
"_id" : "$seasonNumber",
"count" : {
"$sum" : {
"$const" : 1
}
},
"episodes" : {
"$push" : {
"episodeId" : "$episodeId",
"episodeTitle" : "$episodeTitle",
"episodeNumber" : "$episodeNumber",
"seasonNumber" : "$seasonNumber",
"airDate" : "$airDate"
}
}
}
},
{
"$sort" : {
"sortKey" : {
"_id" : -1
}
}
}
],
"ok" : 1
}
Just can't get this seemingly simple query to work. All I want to do is match records that have a specific date range (the range could be 1 for the same date). Any insight is appreciated. I have verified the collection has documents with date = "2015-11-23T09:00:00.000Z".
db.getCollection('MyCollection').find(
{ "$and" :
[ { "name" : { "$in" : [ "Joe", "Jane"]}} ,
{ "date" : { "$gte" : { "$date" : "2015-11-23T09:00:00.000Z"} , "$lte" : { "$date" : "2015-11-23T09:00:00.000Z"}}}
]}
)
frostbite,
Just change your above query to:
db.getCollection('MyCollection').find(
{
"$and" :
[
{ "name" : { "$in" : [ "Joe", "Jane"]}} ,
{ "date" : {
"$gte" :ISODate("2015-11-23T09:00:00.000Z"),
"$lte":ISODate("2015-11-23T09:00:00.000Z")
}
]
})
I have the following mysql statement but I'd like to use it with spring mongodb driver for java. How to convert it? Have looked at aggregation but have no clue how to.
SELECT SUM(CASE WHEN CreatedTime BETWEEN ('7:00:00' AND '7:14:59') THEN 1 ELSE 0) as firstCount,
SUM(CASE WHEN CreatedTime BETWEEN ('7:15:00' AND '7:29:59') THEN 1 ELSE 0) as secondCount,
FROM MyTable
Where username='Jim'
Mongo document:
{ _id: ObjectId("5asd3ea3402984ca53"), username: "Jim", comment: "hi", CreatedTime: ISODate("2014-10-15T16:39:26.870Z") }
UPDATE on translating it to java using spring data:
When calling getTemplate().executeCommand(match); I get this:
{ "serverUsed" : "xxxxxxx" , "ok" : 0.0 , "errmsg" : "no such cmd: $match" , "bad cmd" : {
"$match" : { "username" : "Jim"} ,
"$group" : {
"firstCount" : {
"$sum" : {
"$cond" : {
"if" : {
"$and" : [ [ { "$gte" : { "$CreatedTime" : { "$date" : "2014-09-20T16:02:10.924Z"}}} , 1 , 0] ,
[ { "$lte" : { "$CreatedTime" : { "$date": "2014-10-20T15:48:19.744Z"}}} , 1 , 0]]} ,
"then" : { "$ifTrue" : 1} ,
"else" : { "$else" : 0}
}
}
}
}}}
Code that I use to get the JSON is here (its quite long).Query looks about the same that was suggested by #Wizard
What could be the trouble with $match? I read somewhere in Stackoverflow that old versions of mongodb do not support $match but I have Aug-2014 release so that can't be the case.
Like this:
db.MyTable.aggregate([{
$match : {
username : 'Jim'
}
}, {
$group : {
_id : 0,
firstCount : {
$sum : {
$cond : {
"if" : {
$and : [{
$gte : [ "$CreateTime", '7:00:00' ]
}, {
$lte : [ "$CreateTime", '7:14:59' ]
}]
},
"then" : 1,
"else" : 0
}
}
},
secondCount : {
$sum : {
$cond : {
"if" : {
$and : [{
$gte : [ "$CreateTime", '7:15:00' ]
}, {
$lte : [ "$CreateTime", '7:29:59' ]
}]
},
"then" : 1,
"else" : 0
}
}
},
}
}]);