sorry for my english ..
I need to compare the user'param(MonthYear / 6characters) with the field TxtDtVts (a date with 7 OR 8 characters)
If they match, it must return the fields "CodeTva" and "TauxTVA" by the desired Month
below is my collection "tickets":
{
"_id" : ObjectId("59e66bdda00472964e6a950b"),
"Pharma" : "HEA00001",
"TxtDtVts" : 2012016, // Or 22012016 (7 or 8 characts)
"TxtHrsVts" : 842,
"NumVts" : 845613,
"NumEmp" : 19,
"NumPoste" : 127,
"PVHT" : 1.0575,
"CodeTva" : 4,
"TauxTVA" : 2.1,
"PVTTC" : 1.08,
}
Here my end point and my aggregate function :
secureRoutes.route('/ticketTVA/month/:MonthYear') // Example 012016
.get(function(req, res){
var mois= req.params.month;
Ticket.aggregate([
{$project:{
TxtDtVts:1,
Correspondance: {
$let: {
vars: {
monthSubstring: { $substr: [ "$TxtDtVts", 0, -1 ] },
moisReq:{$substr: ["$mois",0,-1]},
},
in: { $cmp: [ "$$monthSubstring", "$$moisReq" ] }
}
}
}},
],function (err, result) {
if (err) {
console.log(err);
return;
}
console.log(result);
res.json(result);
});
})
I tried to use $substr to convert data into string, save them in vars and use $cmp to compare them.
If i do that (without match before), i have an error : errmsg: 'aggregation result exceeds maximum document size (16MB)
More, it's a bad way because TxtDtVts length change (1012016 OR 10012016)..
How can i compare this 2 data and if it match, return me "CodeTva" and "TauxTVA" ?
Thank you in advance
Try this on for size: Make your input an int, NOT a string. It is easy to take your input and do so. Then:
db.foo.aggregate([
{ $addFields: {
"rc": {$eq: [{$divide:[{ "$subtract": [ "$date", input ] },1000000]},
{$trunc: {$divide:[{ "$subtract": [ "$date", input ] },1000000]}} ]}
}}
,{ $match: {"rc":true}}
]);
The idea is to take a date like 8022013 (Feb 8, 2013) or 25121970 (Dec 25, 1970)
and subtract off the MMYYYY component. So an input of 22013 (the int version of the string "022013", note how leading zero drops off) yields 8000000. Division by 1000000 on a floating point and integer basis yields the same number (8). If the DB date was 8032013, the diff is 8010000. This yields 8 and 8.01 which are not equal.
I found a solution
jrsDuMois= [];
var i = 1;
for (; i <= 31; i++) {
jrsDuMois.push(parseInt(i+""+mois));
}
And i compare the field TxtDtVts with my array "jrsDuMois":
{ "$match": {
"TxtDtVts": { "$in": jrsDuMois },
}},
Related
Let's say I have a collection that looks like:
{
_id: 'aaaaaaaaaaaaaaaaaaaaaaaaa',
score: 10
hours: 50
},
{
_id: 'aaaaaaaaaaaaaaaaaaaaaaaab',
score: 5
hours: 55
},
{
_id: 'aaaaaaaaaaaaaaaaaaaaaaaac',
score: 15
hours: 60
}
I want to sort this list by a custom order, namely
value = (score - 1) / (T + 2) ^ G
score: score
T: current_hours - hours
G: some constant
How do I do this? I assume this is going to require writing a custom sorting function that compares the score and hours fields in addition to taking a current_hours as an input, performs that comparison and returns the sorted list. Note that hours and current_hours is simply the number of hours that have elapsed since some arbitrary starting point. So if I'm running this query 80 hours after the application started, current_hours takes the value of 80.
Creating an additional field value and keeping it constantly updated is probably too expensive for millions of documents.
I know that if this is possible, this is going to look something like
db.items.aggregate([
{ "$project" : {
"_id" : 1,
"score" : 1,
"hours" : 1,
"value" : { SOMETHING HERE, ALSO REQUIRES PASSING current_hours }
}
},
{ "$sort" : { "value" : 1 } }
])
but I don't know what goes into value
I think value will look something like this:
"value": {
$let: {
vars: {
score: "$score",
t: {
"$subtract": [
80,
"$hours"
]
},
g: 3
},
in: {
"$divide": [
{
"$subtract": [
"$$score",
1
]
},
{
"$pow": [
{
"$add": [
"$$t",
2
]
},
"$$g"
]
}
]
}
}
}
Playground example here
Although it's verbose, it should be reasonably straightforward to follow. It uses the arithmetic expression operators to build the calculation that you are requesting. A few specific notes:
We use $let here to set some vars for usage. This includes the "runtime" value for current_hours (80 in the example per the description) and 3 as an example for G. We also "reuse" score here which is not strictly necessary, but done for consistency of the next point.
$ refers to fields in the document where $$ refer to variables. That's why everything in the vars definition uses $ and everything for the actual calculation in in uses $$. The reference to score inside of in could have been done via just the field name ($), but I personally prefer the consistency of this approach.
Let's say I have a whole mess of data that yields a range of integer values for a particular field... I'd like to see those ranked by a grouping of intervals of occurrence, perhaps because I am clustering...like so:
[{
_id: {
response_time: "3-4"
},
count: 234,
countries: ['US', 'Canada', 'UK']
}, {
_id: {
response_time: "4-5"
},
count: 452,
countries: ['US', 'Canada', 'UK', 'Poland']
}, ...
}]
How can I write a quick and dirty way to A) group the collection data by equally spaced intervals over B) a minimum and maximum range using a MongoDB aggregator?
Well, in order to quickly formulate a conditional grouping syntax for MongoDB aggregators, we first adopt the pattern, per MongoDB syntax:
$cond: [
{ <conditional> }, // test the conditional
<truthy_value>, // assign if true
$cond: [ // evaluate if false
{ <conditional> },
<truthy_value>,
... // and so forth
]
]
In order to do that muy rapidamente, without having to write every last interval out in a deeply nested conditional, we can use this handy recursive algorithm (that you import in your shell script or node.js script of course):
$condIntervalBuilder = function (field, interval, min, max) {
if (min < max - 1) {
var cond = [
{ '$and': [{ $gt:[field, min] }, { $lte: [field, min + interval] }] },
[min, '-', (min + interval)].join('')
];
if ((min + interval) > max) {
cond.push(ag.$condIntervalBuilder(field, (max - min), min, max));
} else {
min += interval;
cond.push(ag.$condIntervalBuilder(field, interval, min, max));
}
} else if (min >= max - 1 ) {
var cond = [
{ $gt: [field, max] },
[ max, '<' ].join(''), // Accounts for all outside the range
[ min, '<' ].join('') // Lesser upper bound
];
}
return { $cond: cond };
};
Then, we can invoke it in-line or assign it to a variable that we use elsewhere in our analysis.
I want to make a find query on my database for documents that have an input value between or equal to these 2 fields, LOC_CEP_INI and LOC_CEP_FIM
Example: user input a number to the system with value : 69923994, then I use this input to search my database for all documents that have this value between the range of the fields LOC_CEP_INI and LOC_CEP_FIM.
One of my documents (in this example this document is selected by the query because the input is inside the range):
{
"_id" : ObjectId("570d57de457405a61b183ac6"),
"LOC_CEP_FIM" : 69923999, //this field is number
"LOC_CEP_INI" : 69900001, // this field is number
"LOC_NO" : "RIO BRANCO",
"LOC_NU" : "00000016",
"MUN_NU" : "1200401",
"UFE_SG" : "AC",
"create_date" : ISODate("2016-04-12T20:17:34.397Z"),
"__v" : 0
}
db.collection.find( { field: { $gt: value1, $lt: value2 } } );
https://docs.mongodb.com/v3.2/reference/method/db.collection.find/
refer this mongo provide range facility with $gt and $lt .
You have to invert your field names and query value.
db.zipcodes.find({
LOC_CEP_INI: {$gte: 69923997},
LOC_CEP_FIM: {$lte: 69923997}
});
For your query example to work, you would need your documents to hold an array property, and that each item in this prop hold a 69923997 prop. Mongo would then check that this 69923997 prop has a value that is both between "LOC_CEP_INI" and "LOC_CEP_FIM" for each item in your array prop.
Also I'm not sure whether you want LOC_CEP_INI <= 69923997 <= LOC_CEP_FIM or the contrary, so you might need to switch the $gte and $lte conditions.
db.zipcodes.find( {
"LOC_CEP_INI": { "$lte": 69900002 },
"LOC_CEP_FIM": { "$gte": 69900002 } })
Here is the logic use it as per the need:
Userdb.aggregate([
{ "$match": { _id: ObjectId(session._id)}},
{ $project: {
checkout_list: {
$filter: {
input: "$checkout_list",
as: "checkout_list",
cond: {
$and: [
{ $gte: [ "$$checkout_list.createdAt", new Date(date1) ] },
{ $lt: [ "$$checkout_list.createdAt", new Date(date2) ] }
]
}
}
}
}
}
Here i use filter, because of some reason data query on nested data is not gets succeed in mongodb
I am trying to process a number series ( collection ) get sum of odd / even numbers separately along with elements considered for calculations of each.
The numberseries document structure is as follows:
{
_id: <Autogenerated>,
number: <any number, it can repeat. Even if it repeats, it should be added each time. >
}
The output is something like below( not exact but in general )
{
..
{
"odd":<result>, elements:{n1,n3,n5}
},
{
"even":<result>, elements:{n2,n4,n6}
}
..
}
Map Function:
mapf = function(){
var value = { sum : 0, elements :[] };
value.sum = this.number;
value.elements.push(this.number);
print(tojson(value));
if( this.number % 2 != 0 ){
emit( "odd", value );
}
if( this.number % 2 == 0 ){
emit( "even", value );
}
}
Reduce Values argument:
Values is an array of JSON emitted from map:
[{
"sum": 1,
"elements": [1]
}, {
"sum": 3,
"elements": [3]
} ... ]
Reduce Function:
reducef = function(key, values){
var result = { sum : 0 , elements:[] };
print("K " + key +"Values array " + tojson(values) );
for(var i = 0; i<values.length;i++ ){
v = values[i];
print("Key "+key+"V.JSON"+tojson(v)+" V.SUM -> "+v.sum);
result.sum += v.sum;
result.elements.push(v.elements[0]);
print(tojson(result));
}
return result;
}
I am getting sum correctly, but the elements array is not properly getting populated. It is containing only some of the elements considered for calculations.
UPDATE
As per the answer given by Neil, I further verified my code. I found that my code, without any modification, works for small dataset, but does not work for large data-set.
Below are points which I have verified as pointed out, I found my code to be correct.
print("K " + key +"Values array " + tojson(values) );
Above line in reduce function results in following values object printed.
[{
"sum": 1,
"elements": [1]
}, {
"sum": 3,
"elements": [3]
}, {
"sum": 5,
"elements": [5]
}, {
"sum": 7,
"elements": [7]
}, {
"sum": 9,
"elements": [9]
}, {
"sum": 11,
"elements": [11]
}, {
"sum": 13,
"elements": [13]
}, {
"sum": 15,
"elements": [15]
}, {
"sum": 17,
"elements": [17]
}, {
"sum": 19,
"elements": [19]
}]
Hence the line to push elements to array in final results result.elements.push(v.elements[0]); should be correct.
In map function, before emitting, I am modifying value.sum as follows
value.sum = this.number;
This ensures that sum is not zero and numbers are properly getting added due to this.
When I test this code with 20 records, 40 records, 100 records, it works perfectly.
When I test this code with 20000 records, the sum value is correct but the element array
does not contain 10000 elements each( Odd and even numbers are equally distributed in collection ) .
In later case, I get below message:
query not recording (too large)
Okay, there is a clear reason and you do appear to have read some of the documentation and at least applied this rule:
"the type of the return object must be identical to the type of the value emitted by the map function ..."
And by that this means that both the map function and the reduce function essentially have the same output, which you did:
{ sum : 0, elements :[] };
But there was a piece of documentation that has not been understood:
"MongoDB can invoke the reduce function more than once for the same key. In this case, the previous output from the reduce function for that key will become one of the input values to the next reduce function invocation for that key."
So where the whole thing goes wrong is that you have assumed that since your "map" function only emits one element, that then there will be only one element in the "elements" array. A careful re-read of the above says that this is not true. And in fact the output from "reduce" will very likely be fed back into the "reduce" function again. This is indeed how mapReduce deals with a large number of values for the "values" array.
To fix it, change this in the "reduce" function:
result.elements.push(v.elements[0]);
To this:
v.elements.forEach(function(element) {
result.elements.push(element);
}
And in that way, when the "reduce" function returns a result that has summed up a few "elements" already and pushed them to the list, then that "input" will be processed correctly and merged with any other "values" that come in with it.
BTW. I Think you actually meant this in your mapper:
var value = { sum : 1, elements :[] };
Otherwise this code down here would just be summing 0's:
result.sum += v.sum;
But aggregate does this better
All of that said the following aggregation framework statement does the same thing but better and faster with an implementation in native code:
db.collection.aggregate([
{ "$project": {
"type": { "$cond": [
{ "$eq": [ { "$mod": [ "$number", 2 ] }, 0 ] },
"even",
"odd"
]},
"number": 1
}},
{ "$group": {
"_id": "$type",
"sum": { "$sum": 1 },
"elements": { "$push": "$number" }
}}
])
And also note that in both cases you are not really "summing the elements", but rather "counting" them. So if your want the sum then the mapReduce part becomes:
//result.sum += v.sum;
v.elements.forEach(function(element) {
result.sum += element;
result.elements.push(element);
}
And the aggregate part becomes:
{ "$group": {
"_id": "$type",
"sum": { "$sum": "$number" },
"elements": { "$push": "$number" }
}}
Which truly sums the "odd" or "even" numbers as found in your collection.
How do I get MongoDB to calculate the sum of array values when the array field may be missing completely (as is the case for month 10)?
For example:
> db.month.save({MonthNum: 10,
... NumWeekdays: 23});
> db.month.save({MonthNum: 11,
... NumWeekdays: 21,
... Holidays: [ {Description: "Thanksgiving", NumDays: 2} ] });
> db.month.save({MonthNum: 12,
... NumWeekdays: 22,
... Holidays: [ {Description: "Christmas", NumDays: 6},
... {Description: "New Year's Eve", NumDays: 1} ] });
> db.month.aggregate( { $unwind: "$Holidays" },
... { $group: { _id: "$MonthNum",
... total: { $sum: "$Holidays.NumDays" } } });
{
"result" : [
{
"_id" : 12,
"total" : 7
},
{
"_id" : 11,
"total" : 2
}
],
"ok" : 1
}
How do I get month 10 to show up in the above results (showing "total" as 0)?
Bonus: How do I get the above to show the available weekdays (the NumWeekdays minus the sum of the Holidays)?
I've tried $project to get the data into a canonical format first but without success so far... thanks!
$unwind isn't passing along your document with MonthNum 10 because your Holidays array is empty on that document (see the note at the bottom of the $unwind docs). Assuming that Holidays is always either an array containing at least one item or completely absent from a document, you can use the $ifNull operator inside of $project to add a "Holiday" document that just has NumDays = 0 to your Holidays is null:
db.month.aggregate([
// Make "Holidays" = [{NumDays:0}] if "Holidays" is null for this document (i.e. absent)
{$project:{NumWeekDays:1, MonthNum:1, Holidays:{$ifNull:["$Holidays", [{"NumDays":0}]]}}},
// Now you can unwind + group as normal
{$unwind:"$Holidays"},
{$group:{_id:"$MonthNum", NumWeekDays:{$first:"$NumWeekDays"}, "total":{$sum:"$Holidays.NumDays"}}},
// This should take care of "available weekdays"
{$project:{total:1, available:{$subtract:["$NumWeekDays", "$total"]}}}
]);
Note that $ifNull won't work if for some of your documents Holidays is an empty array; it has to be absent completely.