I want to use mongoid to implement the query funcation likes 'GROUP BY',but i caught an exception:
failed with error 16052: "exception: could not create cursor over th_test.messages for query : { sr_flag: /.*541260c5aee1a93f70000001.*/ } sort : { created_at: -1 }"
My code is here:
def messages
map = %Q{
function() {
emit(this.sr_flag, { count: 1 });
}
}
reduce = %Q{
function(key, values) {
var result = { count: 0 };
values.forEach(function(value) {
result.count += value.count;
});
return result;
}
}
result = Message.where(sr_flag: /.*#{self.id}.*/).map_reduce(map, reduce).out(inline: true).to_a
result
end
Can someone help me explain why? I had searched a blog.Does the mongoid set the created_at column as primary key?
I had fixed my problem.The reason was someone writed a default_scope for my Message model,but the column sorted on was not the key column of the map method.Just Using unscoped method to make program work.
result = Message.unscoped.where(sr_flag: /.*#{self.id}.*/).map_reduce(map, reduce).out(inline: true).to_a
Related
I'm trying to implement a "range query" in MongoDB using Mongoose, ordered by a 'criteria' and then by '_id'.
And I would like to return to the client a string containing both cursors.
I was trying to implement something like the code below, with the commented block 2. However, I'm getting an error. Not even the log messages are being printed.
In my test, the query is empty, because the collection is empty.
I suspected that I was not getting the cursor, so I've tested with 'block 1' instead of block 2, and it worked.
But since I need the last cursor, I guess what I really need to use is the .toArray method, right?
What am I doing wrong?
Feed.find({
"criteria": {$lt: cursorCriteria},
"_id": {$lt: cursorId}
})
.sort({
criteria: -1,
_id: -1
})
.limit( 50 )
// block 1: just to test if I'm getting the cursor
.then( items => {
items.forEach( function(item) {
console.log('an item');
})
})
/* block 2: if I try this block instead of block 1, I get an error
.toArray( items => {
if (items.length > 0) {
console.log('not empty);
} else {
console.log('empty');
}
var nextCursor = '${item.criteria}_${item._id}';
res.status(200).json({item, nextCursor});
})
*/
Mongoose doesn't a have toArray() method
This worked fine:
.then( items => {
if (items.length > 0) {
console.log('not empty');
} else {
console.log('empty');
}
var nextCursor;
if (items.length > 0) {
nextCursor = '' + items[items.length-1].criteria + "_" + items[items.length-1]._id;
} else {
nextCursor = '';
}
res.status(200).json({items, nextCursor});
})
I'm trying to generate a statistical mode by using a simple count. The error I'm getting is
{
"errmsg" : "exception: reduce -> multiple not supported yet",
"code" : 10075,
"ok" : 0
}
Here's my code.
var mapFunction = function(){
emit(this.mode, 1);
};
var reduceFunction = function(key, value){
Array.sum(value)
return value;
};
db.runCommand(
{
mapReduce : 'total_contractor_earnings_MR',
map: mapFunction,
reduce: reduceFunction,
out: { replace: 'mapReduceContractorMode', db: 'large'}
}
);
Here you count a sum and do nothing with it.
Array.sum(value)
return value;
The thing you ment to write was:
return Array.sum(value);
The error occured because mongodb currently doesn't support returning an array from a reduce function
I have an aggregation query on a students collection that is returning two sets of results
for each student like this
{ _id: 1543,
name: 'Bill Jackson',
scores: { type: 'homework', score: 38.86823689842918 } }
{ _id: 1543,
name: 'Bill Jackson',
scores: { type: 'homework', score: 15.861613903793295 } }
That's working fine. Now in the callback I want to remove one of the scores for each student. I use ugly nested conditionals below to isolate which of the two records I want to remove, and, once that's achieved I create a find and Modify query to remove the doc but there's no evidence of it getting run. Neither the error or success callback to the findAndModify are getting run, however I am able to log that I'm inside the area where the findAndModify is getting called.
Is it possible to query the db in the callback to an aggregation? If not, how should I perform an operation that persists in the db?
//aggregation query ommitted
, function(err, result) { //callbackstarts here with result of aggregation query that returns two records for each student
for (var i=0; i<result.length; i++) {
var id = result[i]['_id'];
if (id === result[i]['_id']){
if (foo && foo === result[i]['_id']){
//if we're in here, we know we need to remove score associated with this result[i]['_id']
//create findAndModify to remove the record
var query = { '_id' : result[i]['_id']}
var sort = []
var operation = { '$pull' : { 'scores.score' : result[i]['scores']['score'] } };
var options = []
console.log('this code is getting called but findAndModify not')
db.collection('students').findAndModify(query, sort, operation, options,function(err, doc) {
if(err) throw err;
if (!doc) {
console.log("record not found");
}
else {
console.log("changed doc" + doc);
}
});
}else {
var foo = result[i]['_id'] //part of logic to isolate which of two records to remove
}
this is my collection structure :
coll{
id:...,
fieldA:{
fieldA1:[
{
...
}
],
fieldA2:[
{
text: "ciao",
},
{
text: "hello",
},
]
}
}
i want to extract all fieldA2 in my collection but if the fieldA2 is in two or more times i want show only one.
i try this
Db.runCommand({distinct:’coll’,key:’fieldA.fieldA2.text’})
but nothing. this return all filedA1 in the collection.
so i try
db.coll.group( {
key: { 'fieldA.fieldA2.text': 1 },
cond: { } },
reduce: function ( curr, result ) { },
initial: { }
} )
but this return an empty array...
How i can do this and see the execution time?? thank u very match...
Since you are running 2.0.4 (I recommend upgrading), you must run this through MR (I think, maybe there is a better way). Something like:
map = function(){
for(i in this.fieldA.fieldA2){
emit(this.fieldA.fieldA2[i].text, 1);
// emit per text value so that this will group unique text values
}
}
reduce = function(values){
// Now lets just do a simple count of how many times that text value was seen
var count = 0;
for (index in values) {
count += values[index];
}
return count;
}
Will then give you a collection of documents whereby _id is the unique text value from fieldA2 and the value field is of the amount of times is appeared i the collection.
Again this is a draft and is not tested.
I think the answer is simpler than a Map/Reduce .. if you just want distinct values plus execution time, the following should work:
var startTime = new Date()
var values = db.coll.distinct('fieldA.fieldA2.text');
var endTime = new Date();
print("Took " + (endTime - startTime) + " ms");
That would result in a values array with a list of distinct fieldA.fieldA2.text values:
[ "ciao", "hello", "yo", "sayonara" ]
And a reported execution time:
Took 2 ms
Here's a specific query I'm having trouble with. I'm using Lift-mongo-
records so that i can use Rogue. I'm happy to use Rogue specific
syntax , or whatever works.
While there are good examples for using javascript strings via java noted below, I'd like to know what the best practices might be.
Imagine here that there is a table like
comments {
_id
topic
title
text
created
}
The desired output is a list of topics and their count, for example
cats (24)
dogs (12)
mice (5)
So a user can see an list, ordered by count, of a distinct/group by
Here's some psuedo SQL:
SELECT [DISTINCT] topic, count(topic) as topic_count
FROM comments
GROUP BY topic
ORDER BY topic_count DESC
LIMIT 10
OFFSET 10
One approach is using some DBObject DSL like
val cursor = coll.group( MongoDBObject(
"key" -> MongoDBObject( "topic" -> true ) ,
//
"initial" -> MongoDBObject( "count" -> 0 ) ,
"reduce" -> "function( obj , prev) { prev.count += obj.c; }"
"out" -> "topic_list_result"
))
[...].sort( MongoDBObject( "created" ->
-1 )).skip( offset ).limit( limit );
Variations of the above do not compile.
I could just ask "what am I doing wrong" but I thought I could make my
confusion more acute:
can I chain the results directly or do I need "out"?
what kind of output can I expect - I mean, do I iterate over a
cursor, or the "out" param
is "cond" required?
should I be using count() or distinct()
some examples contain a "map" param...
A recent post I found which covers the java driver implies I should
use strings instead of a DSL :
http://blog.evilmonkeylabs.com/2011/02/28/MongoDB-1_8-MR-Java/
Would this be the preferred method in either casbah or Rogue?
Update: 9/23
This fails in Scala/Casbah (compiles but produces error {MapReduceError 'None'} )
val map = "function (){ emit({ this.topic }, { count: 1 }); }"
val reduce = "function(key, values) { var count = 0; values.forEach(function(v) { count += v['count']; }); return {count: count}; }"
val out = coll.mapReduce( map , reduce , MapReduceInlineOutput )
ConfiggyObject.log.debug( out.toString() )
I settled on the above after seeing
https://github.com/mongodb/casbah/blob/master/casbah-core/src/test/scala/MapReduceSpec.scala
Guesses:
I am misunderstanding the toString method and what the out.object is?
missing finalize?
missing output specification?
https://jira.mongodb.org/browse/SCALA-43 ?
This works as desired from command line:
map = function (){
emit({ this.topic }, { count: 1 });
}
reduce = function(key, values) { var count = 0; values.forEach(function(v) { count += v['count']; }); return {count: count}; };
db.tweets.mapReduce( map, reduce, { out: "results" } ); //
db.results.ensureIndex( {count : 1});
db.results.find().sort( {count : 1});
Update
The issue has not been filed as a bug at Mongo.
https://jira.mongodb.org/browse/SCALA-55
The following worked for me:
val coll = MongoConnection()("comments")
val reduce = """function(obj,prev) { prev.csum += 1; }"""
val res = coll.group( MongoDBObject("topic"->true),
MongoDBObject(), MongoDBObject( "csum" -> 0 ), reduce)
res was an ArrayBuffer full of coll.T which can be handled in the usual ways.
Appears to be a bug - somewhere.
For now, I have a less-than-ideal workaround working now, using eval() (slower, less safe) ...
db.eval( "map = function (){ emit( { topic: this.topic } , { count: 1 }); } ; ");
db.eval( "reduce = function(key, values) { var count = 0; values.forEach(function(v) { count += v['count']; }); return {count: count}; }; ");
db.eval( " db.tweets.mapReduce( map, reduce, { out: \"tweetresults\" } ); ");
db.eval( " db.tweetresults.ensureIndex( {count : 1}); ");
Then I query the output table normally via casbah.