Mongodb Aggregation Framework and timestamp - mongodb

I have a collection
{ "_id" : 1325376000, "value" : 13393}
{ "_id" : 1325462400, "value" : 13393}
ObjectIds are Unix Timestamp and are storing as Number manually.(at insert time).
now I'm searching for a solution that i could calculate sum of values for each month with Aggregation Framework.

Here is a way you can do it by generating the aggregation pipeline programmatically:
numberOfMonths=24; /* number of months you want to go back from today's */
now=new Date();
year=now.getFullYear();
mo=now.getMonth();
months=[];
for (i=0;i<numberOfMonths;i++) {
m1=mo-i+1; m2=m1-1;
d = new Date(year,m1,1);
d2=new Date(year,m2,1);
from= d2.getTime()/1000;
to= d.getTime()/1000;
dt={from:from, to:to, month:d2}; months.push(dt);
}
prev="$nothing";
cond={};
months.forEach(function(m) {
cond={$cond: [{$and :[ {$gte:["$_id",m.from]}, {$lt:["$_id",m.to]} ]}, m.month, prev]};
prev=cond;
} );
/* now you can use "cond" variable in your pipeline to generate month */
db.collection.aggregate( { $project: { month: cond , value:1 } },
{ $group: {_id:"$month", sum:{$sum:"$value"} } }
)

Related

Get record having highest date inside nested group in Mongodb

I am having a record set like below :
I need to write a query where foreach datatype of every parent I show the data type with highest date i.e
So far I am able to create two groups one on parent id & other on data type but i am unable to understand how to get record with max date.
Below is my query :
db.getCollection('Maintenance').aggregate( [{ $group :
{ _id :{ parentName: "$ParentID" , maintainancename : "$DataType" }}},
{ $group : {
_id : "$_id.parentName",
maintainancename: {
$push: {
term:"$_id.DataType"
}
}
}
}] )
You don't have to $group twice, try below aggregation query :
db.collection.aggregate([
/** group on two fields `ParentID` & `Datatype`,
* which will leave docs with unique `ParentID + Datatype`
* & use `$max` to get max value on `Date` field in unique set of docs */
{
$group: {
_id: {
parentName: "$ParentID",
maintainancename: "$Datatype"
},
"Date": { $max: "$Date" }
}
}
])
Test : mongoplayground
Note : After group stage you can use $project or $addFieldsstages to transform fields the way you want.

MongoDB: subtract months from date, with value from database

Is it possible in MongoDB to somehow subtract only months from a date, with a value from the same document. So for example i have to subtract date from field "valid_to" with months from field "offset", only months need to be changed.
So for example:
2019-11-25 - 3 months = 2019-08-25
2019-12-20 - 5 months = 2019-07-20
Documents in collection:
{
"_id" : ObjectId("5d96f027ad768a1532aa2698"),
"name" : "Q1",
"valid_to" : ISODate("2019-11-25T14:36:32.221+01:00"),
"offset" : 3,
},
{
"_id" : ObjectId("5d96f027ad768a1532aa2699"),
"name" : "Q2",
"valid_to" : ISODate("2019-12-20T14:36:32.221+01:00"),
"offset" : 5,
}
I tried just as an example using JavaScript date function and taking current date and subtracting months but it's not working like this
{
$project: {
"subtractedDate": new Date(new Date().setMonth({ $subtract: [new Date().getMonth(), "$offset"] }))
}
},
])
As a result I am getting ISODate("1970-01-01T01:00:00.000+01:00")
Javascript functions do not work with aggregation framework. The whole aggregation is parsed to BSON format on input, so you can't call any JS inside. The reason is in converting BSON to C++ implementation.
In your example you are using mix of operators and JS, when you try to $subtract inside .setMonth():
$project: {
"subtractedDate": new Date(new Date().setMonth({ $subtract: [new Date().getMonth(), "$offset"] }))
}
So JS doesn't know in advance how to handle Mongo operators and doesn't know about $offset value
Instead you have to use JS, that will be executed before input:
$project: {
"subtractedDate": new Date(new Date().setMonth(new Date().getMonth() - 5))
}
That's the only solution to get you example work. But it won't help in your case since you don't know the exact offset.
What you can do is implement your logic using aggregation operators, there are lots of functions to work with Date like $dateToParts and $dateFromParts where you can easily manipulate with months, years and so on separately.
Or you can add a callback to your aggregation where you can handle all your dates:
db.collection.aggregate([
{
// ...
}
], function( err, results ) {
if ( !err ) {
results.forEach( result => {
result.date = new Date(result.date)
.setMonth(new Date(result.date).getMonth() - result.offset);
});
}
callback(err, results);
});
}

MongoDB query for finding schemes with more than a year?

Given the following schema, I want to write a MongoDB scheme to find all schemes with duration of more than a year. ie (start - end) > 1 year.
I am not sure if I can specify such an expression in mongodb query (start - end) > 1 year.
{
"update" : ISODate("2017-09-26T15:22:13.172Z"),
"create" : ISODate("2017-09-26T15:22:13.172Z"),
"scheme" : {
"since" : ISODate("2017-09-26T15:22:13.172Z"),
"startDate": ISODate("2017-09-26T15:22:13.172Z"),
"endDate": ISODate("2018-09-26T15:22:13.172Z"),
},
}
You can use aggregation with subtract:
db.yourcollection.aggregate(
[{
$project : { dateDifference: { $subtract: [ "$scheme.endDate", "$scheme.endDate" ] }},
$match : { "$dateDifference" : { $gt : 31536000000 } }
}]);
(* 31536000000 = milliseconds per year)
You may use another $project to output any fields you need in the matching documents.

Select data where the range between two different fields contains a given number

I want to make a find query on my database for documents that have an input value between or equal to these 2 fields, LOC_CEP_INI and LOC_CEP_FIM
Example: user input a number to the system with value : 69923994, then I use this input to search my database for all documents that have this value between the range of the fields LOC_CEP_INI and LOC_CEP_FIM.
One of my documents (in this example this document is selected by the query because the input is inside the range):
{
"_id" : ObjectId("570d57de457405a61b183ac6"),
"LOC_CEP_FIM" : 69923999, //this field is number
"LOC_CEP_INI" : 69900001, // this field is number
"LOC_NO" : "RIO BRANCO",
"LOC_NU" : "00000016",
"MUN_NU" : "1200401",
"UFE_SG" : "AC",
"create_date" : ISODate("2016-04-12T20:17:34.397Z"),
"__v" : 0
}
db.collection.find( { field: { $gt: value1, $lt: value2 } } );
https://docs.mongodb.com/v3.2/reference/method/db.collection.find/
refer this mongo provide range facility with $gt and $lt .
You have to invert your field names and query value.
db.zipcodes.find({
LOC_CEP_INI: {$gte: 69923997},
LOC_CEP_FIM: {$lte: 69923997}
});
For your query example to work, you would need your documents to hold an array property, and that each item in this prop hold a 69923997 prop. Mongo would then check that this 69923997 prop has a value that is both between "LOC_CEP_INI" and "LOC_CEP_FIM" for each item in your array prop.
Also I'm not sure whether you want LOC_CEP_INI <= 69923997 <= LOC_CEP_FIM or the contrary, so you might need to switch the $gte and $lte conditions.
db.zipcodes.find( {
"LOC_CEP_INI": { "$lte": 69900002 },
"LOC_CEP_FIM": { "$gte": 69900002 } })
Here is the logic use it as per the need:
Userdb.aggregate([
{ "$match": { _id: ObjectId(session._id)}},
{ $project: {
checkout_list: {
$filter: {
input: "$checkout_list",
as: "checkout_list",
cond: {
$and: [
{ $gte: [ "$$checkout_list.createdAt", new Date(date1) ] },
{ $lt: [ "$$checkout_list.createdAt", new Date(date2) ] }
]
}
}
}
}
}
Here i use filter, because of some reason data query on nested data is not gets succeed in mongodb

retrieving multiple (transformed) values in mongodb group matching a criterion from the group itself

I am finding my way through mongodb and have a collection that contains some documents of this shape:
{
"_id" : ObjectId("547a13b70dc5d228db81c475"),
"INSTRUMENT" : "InstrumentA",
"BID" : 5287,
"ASK" : 5290,
"TIMESTAMP" : ISODate("2014-10-01T23:57:27.137Z")
}
{
"_id" : ObjectId("547a0da20dc5d228db2f034d"),
"INSTRUMENT" : "InstrumentB",
"BID" : 0.88078,
"ASK" : 0.88098,
"TIMESTAMP" : ISODate("2014-10-01T23:58:59.637Z")
}
What I am looking to get is the last known mid (BID + ASK)/2 before a given ISODate for each INSTRUMENT. I got as far as getting the time of the last information across instruments and the last value of that last instrument. Even though the following looks like it works, the lastOccurance is being polluted across instruments.
db.runCommand(
{
group:
{
ns: 'collectionTest',
key : { INSTRUMENT : 1} ,
cond: { TIMESTAMP: { $lte: ISODate("2014-10-01 08:30:00") } } ,
$reduce: function( curr, result ) {
if(curr.TIMESTAMP > result.lastOccurance)
{
result.lastOccurance = curr.TIMESTAMP;
result.MID = (curr.BID + curr.ASK)/2;
result.INSTRUMENT = curr.INSTRUMENT;
}else
{
result.lastOccurance = null;
result.MID = null;
result.INSTRUMENT = null;
}
},
initial: { lastOccurance : ISODate("1900-01-01 00:00:00") }
}
}
)
If anybody can see a fix for this code, please let me know.
It's better to use aggregate instead of group whenever possible because it provides better performance and supports sharding.
With aggregate you can do this as:
db.test.aggregate([
// Only include the docs prior to the given date
{$match: {TIMESTAMP: { $lte: ISODate("2014-10-01 08:30:00") }}},
// Sort them in descending TIMESTAMP order
{$sort: {TIMESTAMP: -1}},
// Group them by INSTRUMENT, taking the first one in each group (which will be
// the last one before the given date) and computing the MID value for it.
{$group: {
_id: '$INSTRUMENT',
MID: {$first: {$divide: [{$add: ['$BID', '$ASK']}, 2]}},
lastOccurance : {$first: '$TIMESTAMP'}
}}
])