In SQL it is possible to add fields which are not in the table model and to render them during the query. Is MongoDB able to do the same in the model and / or in the query ?
For instance, is there "a way" to store a document in the example collection:
db.example.save(
{
"name":"randomValue",
"random": function(){ return Math.random() };
})
Where find.example.find(); Would result in an "evaluated" document result like:
{"name":"randomValue", "random":0.9879878, "_id" : { "$oid" : "4ef1d1…" }}
(the function being evaluated would be replaced by its returned value)
If this is only feasible in specifying the function in the query, how to do so ?
Short answer is: no, it is not possible.
Such functionality is a part of 'presentation' layer. When you displaying data on ui you usually prepare ui model. During this step you can call any function and extend your model from client language.
Or if you need to have some extra value (that should be evaluated by some function) in the document you can call this function before save document. For example, shell script that save random value evaluated by Math.Random in random field:
random = Math.random();
db.example.save(
{
"name":"randomValue",
"random": random
})
Hope this helpful.
There is no current way to dynamically evaluate or mix-in values to a recordset.
As you say you can store functions but you have to evaluate them by iterating through each document and calling doc.random = db.eval('doc.random') that would replace the original function with a value - which probably isn't what you want.
You can also store functions in db.system.js and call them to return a modified dataset - running db.eval does have some draw backs, map reduce may be a better fit.
Check out: http://www.mongodb.org/display/DOCS/Server-side+Code+Execution
Take a look at MongoDB.Dynamic:
http://mongodbdynamic.codeplex.com/documentation
Related
I want to make a publication with several additional fields, but I don't want to either use Collection.aggregate and lose my publication updates when the collection change (so I can't just use self.added in it either).
I plan to use Cursor.observeChanges in order to achieve that. I have two major constraints:
I don't want to publish all the documents fields
I want to use some of the unpublished fields to create new ones. For example, I have a field item where I store an array of item _id. I don't want to publish it, but I want to publish a item_count field with the length of my field array
Here comes the approach:
I plan to chain find queries. I never did that so I wonder if it possible. The general (simplified) query structure would be like this: http://jsfiddle.net/Billybobbonnet/1cgrqouj/ (I cant get the code properly displayed here)
Based on the count example in Meteor documentation, I store my query in a variable handle in order to stop the changes notification if a client unsubscribes:
self.onStop(function () {
handle.stop();
});
I add a flag initializing = true; before my query and I set it to true just before calling self.ready();. I use this flag to change my itemCount variable only if it is the publication is initialized. So basically, I change my switch like that:
switch (field) {
case "item"
if (!initializing)
itemCount = raw_document.item.length;
break;
default:
}
I wanted to check that this approach is good and possible before committing into big changes in my code. Can someone confirm me if this is the right way to go?
It's relatively easy to keep fields private even if they are part of the database query. The last argument to self.added is the object being passed to the client, so you can strip/modify/delete fields you are sending to the client.
Here's a modified version of your fiddle. This should do what you are asking for. (To be honest I'm not sure why you had anything chained after the observeChanges function in your fiddle, so maybe I'm misunderstanding you, but looking at the rest of your question this should be it. Sorry if I got it wrong.)
var self = this;
// Modify the document we are sending to the client.
function filter(doc) {
var length = doc.item.length;
// White list the fields you want to publish.
var docToPublish = _.pick(doc, [
'someOtherField'
]);
// Add your custom fields.
docToPublish.itemLength = length;
return docToPublish;
}
var handle = myCollection.find({}, {fields: {item:1, someOtherField:1}})
// Use observe since it gives us the the old and new document when something is changing.
// If this becomes a performance issue then consider using observeChanges,
// but its usually a lot simpler to use observe in cases like this.
.observe({
added: function(doc) {
self.added("myCollection", doc._id, filter(doc));
},
changed: function(newDocument, oldDocument)
// When the item count is changing, send update to client.
if (newDocument.item.length !== oldDocument.item.length)
self.changed("myCollection", newDocument._id, filter(newDocument));
},
removed: function(doc) {
self.removed("myCollection", doc._id);
});
self.ready();
self.onStop(function () {
handle.stop();
});
To solve your first problem, you need to tell MongoDB what fields it should return in the cursor. Leave out the fields you don't want:
MyCollection.find({}, {fields: {'a_field':1}});
Solving your second problem is also pretty easy, I would suggest using the collection helpers packages. You could accomplish this easily, like so:
// Add calculated fields to MyCollection.
MyCollection.helpers({
item_count: function() {
return this.items.length;
}
});
This will be run before an object is added to a cursor, and will create properties on the returned objects that are calculated dynamically, not stored in MongoDB.
In mongo I have a collections with records. These record are very complex. Now I would like to duplicate one of them.
I can easily select the one
mongo> var row = db.barfoo.find({"name":"bar"});
Now I actually don't know what to do. I don't know what is in row because I cannot find a way to print its content. How can I change specific properties and finally insert this modified row again
mongo> db.barfoo.insert(row);
thnx
You must change value _id - generate new:
var row = db.barfoo.findOne({"name":"bar"});
row._id = ObjectId();
db.barfoo.insert(row);
Good Luck!
I am going to assume that you're working directly inside the mongo shell.
Once you have your document (not a row :P ), you'd modify the properties in the same way you would a normal JavaScript object:
var doc = db.barfoo.findOne( { "name": "bar" } );
doc.name = "Mr Bar";
Note that the find() command returns a cursor, so if you're looking to extract a single document, you should use the findOne() function. This function returns a single document.
If you are interested in duplicating numerous documents, you can use the find() function and iterate over the cursor to retrieve each document:
db.barfoo.find( { "name": "bar" } ).forEach( function( doc ){
doc.name = "Mr Bar";
}
After you change the relevant properties, you can use the insert/save methods to persist the data back to mongo. Don't forget to change/delete the _id attribute so that you'll actually create a new document.
As a side note, in order to view the contents of an object in the mongo shell, you can use the print() function. If you want a more visually appealing output, you could use printjson().
So i'm new with mongodb and mapreduce in general and came across this "quirk" (or atleast in my mind a quirk)
Say I have objects in my collection like so:
{'key':5, 'value':5}
{'key':5, 'value':4}
{'key':5, 'value':1}
{'key':4, 'value':6}
{'key':4, 'value':4}
{'key':3, 'value':0}
My map function simply emits the key and the value
My reduce function simply adds the values AND before returning them adds 1 (I did this to check to see if the reduce function is even called)
My results follow:
{'_id': 3, 'value': 0}
{'_id':4, 'value': 11.0}
{'_id':5, 'value': 11.0}
As you can see, for the keys 4 & 5 I get the expected answer of 11 BUT for the key 3 (with only one entry in the collection with that key) I get the unexpected 0!
Is this natural behavior of mapreduce in general? For MongoDB? For pymongo (which I am using)?
The reduce function combines documents with the same key into one document. If the map function emits a single document for a particular key (as is the case with key 3), the reduce function will not be called.
I realize this is an older question, but I came to it and felt like I still didn't understand why this behavior exists and how to build map/reduce functionality so it's a non-issue.
The reason MongoDB doesn't call the reduce function if there is a single instance of a key is because it isn't necessary (I hope this will make more sense in a moment). The following are requirements for reduce functions:
The reduce function must return an object whose type must be identical to the type of the value emitted by the map function.
The order of the elements in the valuesArray should not affect the output of the reduce function
The reduce function must be idempotent.
The first requirement is very important and it seems a number of people are overlooking it because I've seen a number of people mapping in the reduce function then dealing with the single-key case in the finalize function. This is the wrong way to address the issue, however.
Think about it like this: If there's only a single instance of a key, a simple optimization is to skip the reducer entirely (there's nothing to reduce). Single-key values are still included in the output, but the intent of the reducer is to build an aggregate result of the multi-key documents in your collection. If the mapper and reducer are outputting the same type, you should be blissfully unaware by looking at the object structure of the output from your map/reduce functions. You shouldn't have to use a finalize function to correct the structure of your objects that didn't run through the reducer.
In short, do your mapping in your map function and reduce multi-key values into a single, aggregate result in your reduce functions.
Solution:
added new field in map: single: 0
in reduce change this field to: single: 1
in finalize make checking for this field and make required actions
$map = new MongoCode("function() {
var value = {
time: this.time,
email_id: this.email_id,
single: 0
};
emit(this.email, value);
}");
$reduce = new MongoCode("function(k, vals) {
// make some need actions here
return {
time: vals[0].time,
email_id: vals[0].email_id,
single: 1
};
}");
$finalize = new MongoCode("function(key, reducedVal) {
if (reducedVal.single == 0) {
reducedVal.time = 11111;
}
return reducedVal;
};");
"MongoDB will not call the reduce function for a key that has only a single value. The values argument is an array whose elements are the value objects that are “mapped” to the key."
http://docs.mongodb.org/manual/reference/command/mapReduce/#mapreduce-reduce-cmd
Is this natural behavior of mapreduce in general?
Yes.
I am using mongoose with nodejs. I am using mapReduce to fetch data grouped by a field.So all it gives me as a collection is the key with the grouping field only from every row of database.
I need to fetch all the fields from the database grouped by a field and sorted on the basis of another field.e.g.: i have a database having details of places and fare for travelling to those places and a few other fields also.Now i need to fetch the data in such a way that i get the data grouped on the basis of places sorted by the fare for them. MapReduce helps me to get that, but i cannot get the other fields.
Is there a way to get all the fields using map reduce, rather than just getting the two fields as mentioned in the above example??
I must admit I'm not sure I understand completely what you're asking.
But maybe one of the following thoughts helps you:
either) when you iterate over your mapReduce results, you could fetch complete documents from mongodb for each result. That would give you access to all fields in each document for the cost of some network traffic.
or) The value that you send into emit(key, value) can be an object. So you could construct a value object that contains all your desired fields. Just be sure to use the exactly same object structure for your reduce method's return value.
I try to illustrate with an (untested) example.
map = function() {
emit(this.place,
{
'field1': this.field1,
'field2': this.field2,
'count' : 1
});
}
reduce = function(key, values) {
var result = {
'field1': values[0].field1,
'field2': values[0].field2,
'count' : 0 };
for (v in values) {
result.count += values[v].count;
}
return obj;
}
First, the background. I used to have a collection logs and used map/reduce to generate various reports. Most of these reports were based on data from within a single day, so I always had a condition d: SOME_DATE. When the logs collection grew extremely big, inserting became extremely slow (slower than the app we were monitoring was generating logs), even after dropping lots of indexes. So we decided to have each day's data in a separate collection - logs_YYYY-mm-dd - that way indexes are smaller, and we don't even need an index on date. This is cool since most reports (thus map/reduce) are on daily data. However, we have a report where we need to cover multiple days.
And now the question. Is there a way to run a map/reduce (or more precisely, the map) over multiple collections as if it were only one?
A reduce function may be called once, with a key and all corresponding values (but only if there are multiple values for the key - it won't be called at all if there's only 1 value for the key).
It may also be called multiple times, each time with a key and only a subset of the corresponding values, and the previous reduce results for that key. This scenario is called a re-reduce. In order to support re-reduces, your reduce function should be idempotent.
There are two key features in a idempotent reduce function:
The return value of the reduce function should be in the same format as the values it takes in. So, if your reduce function accepts an array of strings, the function should return a string. If it accepts objects with several properties, it should return an object containing those same properties. This ensures that the function doesn't break when it is called with the result of a previous reduce.
Don't make assumptions based on the number of values it takes in. It isn't guaranteed that the values parameter contains all the values for the given key. So using values.length in calculations is very risky and should be avoided.
Update: The two steps below aren't required (or even possible, I haven't checked) on the more recent MongoDB releases. It can now handle these steps for you, if you specify an output collection in the map-reduce options:
{ out: { reduce: "tempResult" } }
If your reduce function is idempotent, you shouldn't have any problems map-reducing multiple collections. Just re-reduce the results of each collection:
Step 1
Run the map-reduce on each required collection and save the results in a single, temporary collection. You can store the results using a finalize function:
finalize = function (key, value) {
db.tempResult.save({ _id: key, value: value });
}
db.someCollection.mapReduce(map, reduce, { finalize: finalize })
db.anotherCollection.mapReduce(map, reduce, { finalize: finalize })
Step 2
Run another map-reduce on the temporary collection, using the same reduce function. The map function is a simple function that selects the keys and values from the temporary collection:
map = function () {
emit(this._id, this.value);
}
db.tempResult.mapReduce(map, reduce)
This second map-reduce is basically a re-reduce and should give you the results you need.
I used map-reduce method. here is an example.
var mapemployee = function () {
emit(this.jobid,this.Name);};
var mapdesignation = function () {
emit(this.jobid, this.Designation);};
var reduceF = function(key, values) {
var outs = {Name:null,Designation: null};
values.forEach(function(v){
if(outs.Name ==null){
outs.Name = v.Name }
if(outs.Name ==null){
outs.Nesignation = v.Designation}
});
return outs;
};
result = db.employee.mapReduce(mapemployee, reduceF, {out: {reduce: 'output'}});
result = db.designation.mapReduce(mapdesignation,reduceF, {out: {reduce: 'output'}});
Refference : http://www.itgo.me/a/x3559868501286872152/mongodb-join-two-collections