MapReduce gives odd result when count == 1 - mongodb

Stack: MongoDB 2.6.5, Mongoid 3.1.6, Ruby 2.1.1
I'm using Mongoid to do some MongoDB Map/Reduce stuff.
Here's the setup:
class User
include Mongoid::Document
has_many :given_bonuses, class_name: 'Bonus', inverse_of: :giver
has_many :received_bonuses, class_name: 'Bonus', inverse_of: :receiver
end
class Bonus
include Mongoid::Document
belongs_to :giver, class_name: 'User', inverse_of: :given_bonuses
belongs_to :receiver, class_name: 'User', inverse_of: :received_bonuses
end
I'm using the following Map/Reduce code:
map = %Q{
function() {
emit(this.receiver_id, this.giver_id)
}
}
reduce = %Q{
function(key, values) {
var result = {};
values.forEach(function(value) {
if(! result[value]) {
result[value] = 0;
}
result[value] += 1;
});
return result;
}
}
Bonus.all.map_reduce(map, reduce).out(inline: 1)
Let's say I have two or more bonus documents:
> Bonus.count
=> 2
> Bonus.all.to_a
=> [#<Bonus _id: 547612a21dbe8b7859000071, giver_id: "547612441dbe8bf35b000005", receiver_id: "547612531dbe8b4a7200006a">, #<Bonus _id: 547612a21dbe8b78590000f9, giver_id: "547612441dbe8bf35b000005", receiver_id: "547612531dbe8b4a7200006a">]
Then the Map/Reduce result is:
=> [{"_id"=>"547612531dbe8b4a7200006a", "value"=>{"ObjectId(\"547612441dbe8bf35b000005\")"=>2.0}}]
Notice that the value key points to a hash of the form {"ObjectID" => Number}. This is as it should be.
Now let's say I have only one bonus document:
> Bonus.count
=> 1
> Bonus.first
=> #<Bonus _id: 547612a21dbe8b7859000071, giver_id: "547612441dbe8bf35b000005", receiver_id: "547612531dbe8b4a7200006a">
Then the Map/Reduce result is:
=> [{"_id"=>"547612531dbe8b4a7200006a", "value"=>"547612441dbe8bf35b000005"}]
Notice that the schema of the result has changed. It should be:
=> [{"_id"=>"547612531dbe8b4a7200006a", "value"=>{"ObjectId(\"547612441dbe8bf35b000005\")"=>1.0}}]
What's going on here?

From the docs:
MongoDB will not call the reduce function for a key that has only a
single value. The values argument is an array whose elements are the
value objects that are “mapped” to the key.
In the first case the key this.receiver_id, has two documents in its group and hence the reduce function is invoked for that key.
In the second case when your emitted key has only one record in its group, the reduce function won't be called at all.
So the value(this.giver_id) that you emit for the key, is being displayed as emitted without being reduced, there is no need for it to be reduced further.

Related

Different conditions for different fields in thinking-sphinx

I have:
ThinkingSphinx::Index.define :new_post, with: :real_time do
indexes title, sortable: true
indexes text
has state, type: :string
has forum_hidden, type: :boolean
has created_at, type: :timestamp
has publish_at, type: :timestamp
has reminde_at, type: :timestamp
has deleted_at, type: :timestamp
has content_category_ids, type: :integer, multi: true
end
And let's i need to get all the records where #title=query with any value publish_at or #text=query with publish_at = 1.month.ago..Time.current
That is, I need to combine these two requests:
NewPost.search(conditions: { title: query })
NewPost.search(conditions: { text: query }, with: { publish_at: 1.month.ago..Time.current })
The result is needed with excerpts
UPDATE
published_at interval for the #title and #text fields is always different and depends on the user's rights. For example, there can be such a situation:
NewPost.search(conditions: { title: query }, with: { publish_at: 1.year.ago..Time.current })
NewPost.search(conditions: { text: query }, with: { publish_at: 6.month.ago..Time.current })
and all results that do not fall under these conditions should not be displayed at all
The main thing when wanting to 'combine' multiple criteria is deciding on a 'calculation' how to compute a weight that gives that effect. Sphinx computes a weight and orders by that. Rather than unions of multiple distinct queries
For example
https://freelancing-gods.com/thinking-sphinx/searching.html#sorting
ThinkingSphinx.search(
:select => '*, WEIGHT() + IF(publish_at>NOW()-2592000,1000,0) AS custom_weight',
:order => 'custom_weight DESC'
)
would use a the 'full-text' search weight, but add 1000 if in last month.
Uses the sphinx NOW() function
http://sphinxsearch.com/docs/current.html#expr-func-now
You might also want field weights
https://freelancing-gods.com/thinking-sphinx/searching.html#fieldweights
to boost title matches over 'text' matches
:field_weights => { :title=>10 }
Bringing all togehter (if got the ruby syntax right... )
NewPost.search( query ,
:select => '*, weight() + IF(publish_at>NOW()-2592000,1000,0) as custom_weight',
:order => 'custom_weight DESC',
:field_weights => { :title=>10 }
)
... in theory title matches will be first. And recent ones towards start too. Not EXACTLY what you asked for, but close.

Meteor/Mongo - add/update element in sub array dynamically

So I have found quite few related posts on SO on how to update a field in a sub array, such as this one here
What I want to achieve is basically the same thing, but updating a field in a subarray dynamically, instead of just calling the field name in the query.
Now I also found how to do that straight in the main object, but cant seem to do it in the sub array.
Code to insert dynamically in sub-object:
_.each(data.data, function(val, key) {
var obj = {};
obj['general.'+key] = val;
insert = 0 || (Documents.update(
{ _id: data._id },
{ $set: obj}
));
});
Here is the tree of what I am trying to do:
Documents: {
_id: '123123'
...
smallRoom:
[
_id: '456456'
name: 'name1'
description: 'description1'
],
[
...
]
}
Here is my code:
// insert a new object in smallRoom, with only the _id so far
var newID = new Mongo.ObjectID;
var createId = {_id: newID._str};
Documents.update({_id: data._id},{$push:{smallRooms: createId}})
And the part to insert the other fields:
_.each(data.data, function(val, key) {
var obj = {};
obj['simpleRoom.$'+key] = val;
console.log(Documents.update(
{
_id: data._id, <<== the document id that I want to update
smallRoom: {
$elemMatch:{
_id : newID._str, <<== the smallRoom id that I want to update
}
}
},
{
$set: obj
}
));
});
Ok, having said that, I understand I can insert the whole object straight away, not having to push every single field.
But I guess this question is more like, how does it work if smallRoom had 50 fields, and I want to update 3 random fields ? (So I would NEED to use the _each loop as I wouldnt know in advance which field to update, and would not want to replace the whole object)
I'm not sure I 100% understand your question, but I think the answer to what you are asking is to use the $ symbol.
Example:
Documents.update(
{
_id: data._id, smallRoom._id: newID._str
},
{
$set: { smallroom.$.name: 'new name' }
}
);
You are finding the document that matches the _id: data._id, then finding the object in the array smallRoom that has an _id equal to newId._str. Then you are using the $ sign to tell Mongo to update that object's name key.
Hope that helps

adding multiple same documents using addtoset command in mongoose

My schema is
var UserQuizSchema = mongoose.Schema({
uid:{type:ObjectId,required: true,index:true},
answer:[{content:String, qid:ObjectId ,time:Date }],
});
In this schema, 'uid' represents user identifier, while the 'answer' array stores the answers the student had answered. in each answer, qid relates to the question ID, and 'content' is the student's real answer, 'time' is the modified time stamp for the answer.
Here I use mongoose to upsert the new answers into the array
function updateAnswer(uid,question_id,answer,callback){
var options = { new: false };
var quiz_id = mongoose.Types.ObjectId(quiz_id);
var qid = mongoose.Types.ObjectId(question_id);
UserQuizModel.findOneAndUpdate({'uid':uid},{'$addToSet':{'answer':{'qid':qid, 'content':answer} } },options,function(err,ref){
if(err) {
console.log('update '.red,err);
callback(err, null);
}else{
console.log('update '.green+ref);
callback(null,ref);
}
})
}
In the common sense, by using addToSet command, the element in the answer array should be unique, but in my example, the answer array could have multiple same embedded documents only except each embedded document has one unique OjbectId _id
such as
answer:
[ { qid: 5175aecf0e5b061414000001, _id: 518a5e5895fc9ddc1e000003 },
{ qid: 5175aecf0e5b061414000001, _id: 518a5e5f95fc9ddc1e000004 } ] }
you see the qid of two embedded documents are the same, but _id are different.
Why there is a additional _id, I don't put it the schema design ??
You can disable the _id in your embedded objects by explicitly defining a schema for the elements with the _id option set to false:
var UserQuizSchema = mongoose.Schema({
uid:{type:ObjectId,required: true,index:true},
answer:[new Schema({content:String, qid:ObjectId, time:Date}, {_id:false})]
});

How to remove ranking of query results

I have the following pg_search scope on my stories.rb model:
pg_search_scope :with_text,
:against => :title,
:using => { :tsearch => { :dictionary => "english" }},
:associated_against => { :posts => :contents }
I want the query to return the results ignoring any ranking (I care only about the date the story was last updated order DESC). I know that this is an easy question for most of the people who view it, but how do I turn off the rank ordering in pg_search?
I'm the author of pg_search.
You could do something like this, which uses ActiveRecord::QueryMethods#reorder
MyModel.with_text("foo").reorder("updated_at DESC")

Mongodb mongoid model attributes sorted by alphabetical order not insertion order

I have a model User where I make heavy use of dynamic attributes.
When I am displaying the user show, I skip the first set of attributes to skip the id etc.
Problem is that the attributes are sorted by alphabetical order, not by order of creation
so if for example I create a users as:
MONGODB startuplab_co_development['users'].insert([{"provider"=>"google", "uid"=>"https://www.google.com/accounts/o8/id?id=AItOawl_oas_", "_id"=>BSON::ObjectId('4dc5ad606acb26049e000002'), "email"=>"dan#gmail.com", "first_name"=>"Daniel", "last_name"=>"Palacio", "name"=>"Daniel Palacio"}])
Even though the insertion second attribute is uid, when I retrieve the keys they are sorted alphabetically.
%h1 User
%ul
-keys = #user.attributes.keys[3..-1]
- keys.each do |key|
%li
%span
%strong= "#{key.capitalize()}:"
%span= "#{#user[key]}"
So for instance this will print UID as the last attribute, since they were sorted.
First_name: Daniel
Last_name: Palacio
Name: Daniel Palacio
Provider: google
Uid: https://www.google.com/accounts/o8/id?id=AItOawl_oas_8jcY1VSTQchsc
Is there anyway that I can make sure the attributes position stay in the order of insertion ?
Here is the chain of events where attributes get sorted
After creation uid is the 3rd attribute
ruby-1.9.2-p0 > user = User.first
=> _id: 4dc5c5946acb26049e000005, _type: nil, _id: BSON::ObjectId('4dc5c5946acb26049e000005'), provider: "google", uid: "https://www.google.com/accounts/o8/id?id=AItOawl_oas_", admin: nil, email: "danpal#gmail.com", first_name: "Daniel", last_name: "Palacio", name: "Daniel Palacio">
ruby-1.9.2-p0 > user.attributes
=> {"_id"=>BSON::ObjectId('4dc5c5946acb26049e000005'), "provider"=>"google", "uid"=>"https://www.google.com/accounts/o8/id?id=AItOawl_oas_", "email"=>"danpal#gmail.com", "first_name"=>"Daniel", "last_name"=>"Palacio", "name"=>"Daniel Palacio"}
Now we update the admin attribute and save it, uid is still the 3rd attribute
ruby-1.9.2-p0 > user.update_attributes(:admin => true)
=> true
ruby-1.9.2-p0 > user.attributes
=> {"_id"=>BSON::ObjectId('4dc5c5946acb26049e000005'), "provider"=>"google", "uid"=>"https://www.google.com/accounts/o8/id?id=AItOawl_oas_", "email"=>"danpal#gmail.com", "first_name"=>"Daniel", "last_name"=>"Palacio", "name"=>"Daniel Palacio", "admin"=>true}
ruby-1.9.2-p0 > user.save
=> true
ruby-1.9.2-p0 > user.attributes
=> {"_id"=>BSON::ObjectId('4dc5c5946acb26049e000005'), "provider"=>"google", "uid"=>"https://www.google.com/accounts/o8/id?id=AItOawl_oas_", "email"=>"danpal#gmail.com", "first_name"=>"Daniel", "last_name"=>"Palacio", "name"=>"Daniel Palacio", "admin"=>true}
Know we retrieve the object from the database again, uid is now the last attribute, and they have been sorted alphabetically.
ruby-1.9.2-p0 > user = User.first
=> #
ruby-1.9.2-p0 > user.attributes
=> {"_id"=>BSON::ObjectId('4dc5c5946acb26049e000005'), "admin"=>true, "email"=>"danpal#gmail.com", "first_name"=>"Daniel", "last_name"=>"Palacio", "name"=>"Daniel Palacio", "provider"=>"google", "uid"=>"https://www.google.com/accounts/o8/id?id=AItOawl_oas_"}
Ok here is the answer:
Basically during an update if the Document allocated space is not sufficient( Eg. becouse the update add's a new field or grows an existing field), the document will be moved and the fields are reordered(alphanumerically).
From the MOngoDB docs:
http://www.mongodb.org/display/DOCS/Updating
Field (re)order
During an update the field order may
be changed. There is no guarantee that
the field order will be consistent, or
the same, after an update. At the
moment, if the update can be applied
in place then the order will be the
same (with additions applied at the
end), but if a move is required for
the document (if the currently
allocated space is not sufficient for
the update) then the fields will be
reordered (alphanumerically).