I have a Collection with the following fields:
TestTable
{
"ID"
"Name"
"Ver"
"Serial"
"DateTime"
"FeatureID"
"FeatureName"
}
I want to have a map reduce function to get the count of the records in a particular Year.
The map reduce function i wrote is:
map= function(){
year= Date.UTC(this.DateTime.getFullYear());
emit({year: year}, {count: 1});
}
reduce= function(key, values){
var count=0;
for(v in values){
count+= v['count'];
});
return {count: count};
}
Now the output should give the count of douments in each year. Is the map reduce function correct?
The result i got is:
> db.Years.find()
{ "_id" : { "year" : NaN }, "value" : { "count" : NaN } }
> db.Years.find().count()
1
Which is not what i expected.
edited
one of the TestTable document:
> db.TestTable.findOne()
{
"_id" : ObjectId("527c48e99000cf10bc2a1d82"),
"ID" : "LogID16587",
"Name" : "LogName15247",
"Ver" : "VersionID11",
"Serial" : "ProductID727",
"DateTime" : ISODate("1998-12-15T18:30:00Z"),
"FeatureID" : "FeatureID465",
"FeatureName" : "FeatureName 1460"
}
Thanks in advance.
Date.UTC requires both year and month parameters. So your map function should look like this instead:
map= function(){
year= Date.UTC(this.DateTime.getFullYear(), 0);
emit({year: year}, {count: 1});
}
Also, don't use in to iterate over the elements of an array in your reduce method as it doesn't work in the way you're using it. Use a traditional for loop instead:
reduce= function(key, values){
var count=0;
for(var i=0; i<values.length; i++) {
count += values[i]['count'];
};
return {count: count};
}
Related
I use the following map/reduce setup to collect some data into array:
map: function() { emit(this.key, [this.item]); },
reduce: function(key, values) {
var items = [];
values.forEach( function(value) {items.concat(value.item);} );
return items;
},
out: {reduce: "result_collection"}
I want to improve the code and detect if the resulting collection has been changed during the re-reduce stage (when mongo invokes reduce with the current content of the "result_collection").
In other words, how to know that any documents have been emitted by the Map contain "item" that does not exist in the "result_collection" yet (under the same key, of course)?
This information can help at some further processing stages e.g. query "result_collection" to get the documents that have been updated during the map/reduce stage.
If you must do this, use a finalize function to adjust the value after all reduction is finished. You'll have to add more logic to the reduce function to handle the modified output.
I'll show you an example with the simple map-reduce defined by the following map and reduce functions:
var map = function() { emit(this.k, this.v) }
var reduce = function(key, values) { return Array.sum(values) }
On documents that look like { "k" : 0, "v" : 1 }, the map-reduce defined by the above functions produces result documents that look like { "_id" : 0, "value" : 17 }. Define a finalize function to modify the final document:
var finalize = function (key, reducedValue) { return { "m" : true, "v" : reducedValue } }
Now modify reduce to handle an element of values that might be an object of the above form:
var reduce2 = function(key, values) {
var sum = 0;
for (var i = 0; i < values.length; i++) {
if (typeof values[i] == "object") { sum += values[i].v }
else { sum += values[i] }
}
return sum
}
Output looks like
{ "_id" : 0, "value" : { "m" : true, "v" : 14 } }
{ "_id" : 1, "value" : { "m" : true, "v" : 34 } }
{ "_id" : 2, "value" : { "m" : true, "v" : 8 } }
so you can tell what's been modified by value.m. Your further processing can set v.m to false so you'll see what hasn't been processed yet after each map-reduce.
I'm trying what I think should be a simple map reduce, but am having trouble because I can't find a reference of how to write the server side javascript.
Given two documents:
{
"_id" : ObjectId("530c8b58d95cd926144055d9"),
"atomic" : "p",
"doc" : {
"d1" : "t"
},
"array" : ["e"]
},
{
"_id" : ObjectId("530c8b71d95cd926144055da"),
"atomic" : "p",
"doc" : {
"d2" : "r"
},
"array" : ["f"]
}
I would like the result to be
{
"_id" : "p",
"value" : {
"doc" : {
"d1" : "t",
"d2" : "r"
},
"array" : ["e", "f"]
}
}
The map function is:
function () {
emit(
this.atomic,
{doc: this.doc, array: this.array}
);
}
The incorrect reduce function is:
function (key, values) {
var reduced = {doc:{}, array:[]};
values.forEach(function(val){
for(var i = 0; i < val.array.length; i++)
reduced.array.push(val.array[i]);
val.doc.forEach(function(kvp){reduced.doc.add(kvp.key, kvp.value);});
});
return reduced;
}
The part with the array is fine, it is trying to combine the documents that is messing up (i.e. not executing due to missing function). I've tried all permutations I can think off -- if I add the val.doc to an array then they all show up, it's just that I can't figure out how to merge it into a single document.
The fields in the doc will be dynamic so there is no way to reference it by name.
Any help would be appreciated.
Not sure the reduced.doc.add bit will work.
Maybe try:
function (key, values) {
var reduced = {doc:{}, array:[]};
values.forEach(function(val){
for(var i = 0; i < val.array.length; i++)
reduced.array.push(val.array[i]);
for (kvp in val.doc){
reduced.doc[kvp]=val.doc[kvp];
}
});
return reduced;
}
Problem
I have a document with a _id and a Collection of Answers I am trying to write a map-reduce function to sum the total score of answers for each id.
Document
/* 0 */
{
"_id" : ObjectId("527b6ba88d251d58a18f3f0a"),
"Answers" : [{
"Score" : 2
}, {
"Score" : 0
}, {
"Score" : 2
}, {
"Score" : 2
}]
}
Here is the Map-Reduce I though would be correct reading the documentation
Map
function() {
this.Answers.forEach(function(val)
{
emit(this._id, val.Score);
});
}
also tried this
function() {
for (var i = 0; i < this.Answers; i++)
{
emit(this._id, this.Answers[i].Score);
}
}
Reduce
function(key, values)
{
return Array.sum(values);
}
I am getting no information back with this, but it does appear to be processing it takes 2-5 seconds to return. I guess I am not understanding something about map-reduce.
Also I am using MongoVUE to access MongoDB.
EDIT
I just ran my map reduce through the console and got this output
{
"results" : [ ],
"timeMillis" : 2506,
"counts" : {
"input" : 1655,
"emit" : 0,
"reduce" : 0,
"output" : 0
},
"ok" : 1,
}
so it's my map function that's incorrect I guess as nothing was emitted.
EDIT 2
Updated document with output from mongovue
In JavaScript loops adding the length property allows you to iterate by the count of the items in the array, so you cna change your second attempt to:
function() {
for (var i = 0; i < this.Answers.length; i++)
{
emit(this._id, this.Answers[i].Score);
}
}
It should also be noted that your reduce can run multiple times per key, specifically it can repeat every 101 rows, technically this shouldn't matter since you are summing up the array values and the previous reduce value will be passed as an array element in the new reduce so it should work just fine; however, good to keep in mind.
I think the 'this' variable is not what you expect in the .forEach() function in your map method. Try this instead;
function() {
var row = this;
this.Answers.forEach(function(val)
{
emit(row._id, val.Score);
});
}
I am trying to aggregate the total sum of packets in this document.
{
"_id" : ObjectId("51a6cd102769c63e65061bda"),
"capture" : "1369885967",
"packets" : {
"0" : "595",
"1" : "596",
"2" : "595",
"3" : "595",
...
}
}
The closest I can get is about
db.collection.aggregate({ $match: { capture : "1369885967" } }, {$group: { _id:null, sum: {$sum:"$packets"}}});
However it returns sum 0, which is obviously wrong.
{ "result" : [ { "_id" : null, "sum" : 0 } ], "ok" : 1 }
How do I get the sum of all the packets?
Since you have the values in an object instead of an array, you'll need to use mapReduce.
// Emit the values as integers
var mapFunction =
function() {
for (key in this.packets) {
emit(null, parseInt(this.packets[key]));
}
}
// Reduce to a simple sum
var reduceFunction =
function(key, values) {
return Array.sum(values);
}
> db.collection.mapReduce(mapFunction, reduceFunction, {out: {inline:1}})
{
"results" : [
{
"_id" : null,
"value" : 2381
}
],
"ok" : 1,
}
If at all possible, you should emit the values as an array of a numeric type instead since that gives you more options (ie aggregation) and (unless the data set is large) probably performance benefits.
If you don't know how many keys are in the packet subdocument and since you also seem to be storing counts as strings (why???) you will have to use mapReduce.
Something like:
m=function() {
for (f in "this.packets") {
emit(null, +this.packets[f]);
};
r=function(k, vals) {
int sum=0;
vals.forEach(function(v) { sum+=v; } );
return sum;
}
db.collection.mapreduce(m, r, {out:{inline:1}, query:{your query condition here}});
I have a Collection with many columns: col1,col2, WebsiteCode, CreatedDate, col3....,coln.
I want to group by WebsiteCode in a range of CreatedDate.
So I do:
map :
function Map() {
var key={WebsiteCode:this.WebsiteCode};
val={Count: 1};
emit(key,val);
}
reduce :
function Reduce(key, values) {
var res = {Total:0};
values.forEach(function(value) {
res.Total += value.Count;
});
return res;
}
And query range DateTime:
{ "CreatedDate" : { "$gte" : dateFrom , "$lte" : dateTo } }
Finally, I run this mapreduce command.
The result returns not what I expected with many rows having Total = NaN
Ex: {_id:{WebsiteCode:"websitecode1"}}, {value:{Total:NaN}}
But when I run count command:
db.collect.find({ "WebsiteCode" : "websitecode1", "CreatedDate" : { "$gte" : dateFrom), "$lte" : dateTo } }).count();
Result return: 927
Could you explain to me what I did wrong?
Your reduce function must return a value that has the same shape as the emitted values. So you can't use Count in the map and Total in the reduce.
Try this instead:
function Reduce(key, values) {
var res = {Count:0};
values.forEach(function(value) {
res.Count += value.Count;
});
return res;
}