I have two collections in mongodb, one for posts and one for comments. What would be the best approach to get one most recent comment for each post? I'm looking for a similar solution but for mongodb: http://www.xaprb.com/blog/2006/12/07/how-to-select-the-firstleastmax-row-per-group-in-sql/
You should be able to do this with the aggregation framework by combining $group with $max.
I would like to give you an exact solution, but I can't do so unless you give an example of your data.
By the way: The proper way to structure this data in MongoDB would be to put the comments into a sub-Array of the posts.
Just in case anyone else have a similar problem, I solved mine using the Map-Reduce:
First I create a map function like this:
$map = "function() { emit(this.post_id, this); }";
and reduce function:
$reduce = "function(k, vals) {".
"var newest = null;".
"for ( var i in vals ) {".
"if ( newest === null ) {".
"newest = vals[i];".
"}".
"else {".
"if ( vals[i]['_id'] > newest['_id'])".
"newest = vals[i]".
"}".
"}".
"return newest;".
"}";
and a new collection with the necessary data is ready...
$commentsAggregated = $db->command(array(
"mapreduce" => "comments",
"map" => $map,
"reduce" => $reduce,
"query" => $query,
"out" => array("merge" => "commentsCollectionNew")
));
$getComments = $db->selectCollection($commentsAggregated['result'])->find();
Related
I have tried to do this a number of ways from updateOne, findOneAndUpdate to insert and even tried bulkWrite with no success.
What I have done is I have two collections the users collection and the image_upload collection. I store the users profile image inside image_upload along side all the other images that the user uploads.
What I then store in the users collection is the ObjectID of the image_upload collection that matched the image the user uploaded while creating their account (they can upload a new profile image anytime via edit profile).
So what I would like is the ability to update a ObjectId as I get.
The field personal.profile_id must be an array but is of type objectId in document. Here is the code. I ideally want it to have the ObjectID and not just a string.
$db = static::db()->image_upload;
try {
$newdata = [
"data"=>
[
"url" => $publlic_url,
"type"=> $mimetype,
"date"=>new MongoDB\BSON\UTCDateTime(),
"profile_pic" => true
],
"uid"=>New MongoDB\BSON\ObjectId($uid)
];
$oauth_update = $db->insertOne($newdata);
$view['newdata'] = $newdata;
} catch(MongoResultException $e) {
return $response->withStatus(200)
->withHeader('Content-Type', 'application/json')
->write($e->getDocument());
}
$ids = $oauth_update->getInsertedId();
$latest = $db->findOne(array("uid"=>New MongoDB\BSON\ObjectId($uid)));
// Check first, last and other personal details.
$db = static::db()->users;
try {
$newdata = ['$set' =>["personal.profile_id"=>New MongoDB\BSON\ObjectId($ids)]];
$member_profile = $db->findOneAndUpdate(
['kst'=>New MongoDB\BSON\ObjectId($uid)],
['$push' =>["personal.profile_id"=>['$oid'=>New MongoDB\BSON\ObjectId($ids)]]],
[
'projection' =>
[ 'personal' => 1 ],
"returnDocument" => MongoDB\Operation\FindOneAndUpdate::RETURN_DOCUMENT_AFTER
]);
} catch(MongoResultException $e) {
echo $e->getCode(), " : ", $e->getMessage(), "\n";
var_dump($e->getDocument());
return $response->withStatus(200)
->withHeader('Content-Type', 'application/json')
->write(array('code'=>$e->getCode(), 'message'=>$e->getMessage));
}
return $response->withStatus(200)
->withHeader('Content-Type', 'application/json')
->write(json_encode($member_profile));
There is no work-around for the $push operator requiring an array type. Unfortunately, this is going to require migrating documents in the users collection. Some approaches for doing so are discussed in the following threads:
Converting some fields in Mongo from String to Array
mongodb type change to array
Alternatively, if you'd rather not migrate all documents in users at once, you can have the code execute two findOneAndUpdate() operations in sequence. The first operation could use $type to only match a document with { "personal.profile_id": { $type: "string" }} and then use $set to assign the most recent image ID. A second operation could then match a document with an array type and use the $push strategy (note that $type cannot be used for detecting an array field). The calling code would then expect exactly one of these operations to actually find and update a document (consider logging an error if that is not the case). This approach would then allow you to start collecting old image IDs for migrated documents, but continue overwriting the field for non-migrated documents.
One other observation based on the code you provided:
['$push' =>["personal.profile_id"=>['$oid'=>New MongoDB\BSON\ObjectId($ids)]]]
$oid looks like extended JSON syntax for an ObjectID. That is neither an update operator nor a valid key to use in a BSON document. Attempting to execute this update via the mongo shell yields the following server-side error:
The dollar ($) prefixed field '$oid' 'personal.profile_id..$oid' is not valid for storage."
If you are only looking to push the ObjectID onto the array, the following should be sufficient:
['$push' => ['personal.profile_id' => new MongoDB\BSON\ObjectId($ids)]]
So I have the following PHP code which runs a map-reduce command on a MongoDB database collection:
$map = new MongoCode("function() { emit(this.app, this.bytes); }");
$reduce = new MongoCode("function(k, vals) { ".
"var sum = 0;".
"for (var i in vals) {".
"sum += vals[i];".
"}".
"return sum; }");
$dateAdded = mktime(0,0,0,5,1,2015);
//echo $dateAdded." = ".date("r",$dateAdded)."<br>\n";
$request = $db->command(array(
"mapreduce" => "log",
"map" => $map,
"reduce" => $reduce,
"query" => array("event" => "destroy", "systimelong" => array('$gt' => $dateAdded)),
"out" => array("inline" => 1)));
var_dump($request);
This actually works really great when the data is stored in the database as an integer. But sometimes the data gets stored as a string. Why? Thats another story that cant be changed right now. Ultimately it will be an integer, but I'd really like to know if and how I can modify this to handle the cases that the data is a string just in case it ever happens.
Since Mongo uses Javscript I feel like I should be able to use the parseInt() function inside the $map and/or$reduce functions, but it doesn't seem to be working.
Also, how would I handle the query? The systimelong field is just unixtime, and I am using the PHP mktime() function to generate a integer value for the beginning of the month. Again, it works great then comparing integers, but I need to first convert the string value.
Any ideas?
I am trying to make a simple map/reduce function on one of my MongoDB database collections.
I get data but it looks wrong. I am unsure about the Map part. Can I use IF/ELSE in this way?
UPDATE
I want to get the amount of authors that ownes the files. In other words how many of the authors own the uploaded files and thus, how many authors has no files.
The objects in the collection looks like this:
{
"_id": {
"$id": "4fa8efe33a34a40e52800083d"
},
"file": {
"author": "john",
"type": "mobile",
"status": "ready"
}
}
The map / reduce looks like this:
$map = new MongoCode ("function() {
if (this.file.type != 'mobile' && this.file.status == 'ready') {
if (!this.file.author) {
return;
}
emit (this.file.author, 1);
}
}");
$reduce = new MongoCode ("function( key , values) {
var count = 0;
for (index in values) {
count += values[index];
}
return count;
}");
$this->cimongo->command (array (
"mapreduce" => "files",
"map" => $map,
"reduce" => $reduce,
"out" => "statistics.photographer_count"
)
);
The map part looks ok to me. I would slightly change the reduce part.
values.forEach(function(v) {
count += v;
}
You should not use for in loop to iterate an array, it was not meant to do this. It is for enumerating object's properties. Here is more detailed explanation.
Why do you think your data is wrong? What's your source data? What do you get? What do you expect to get?
I just tried your map and reduce in mongo shell and got correct (reasonable looking) results.
The other way you can do what you are doing is get rid of the inner "if" condition in the map but call your mapreduce function with appropriate query clause, for example:
db.files.mapreduce(map,reduce,{out:'outcollection', query:{"file.author":{$exists:true}}})
or if you happen to have indexes to make the query efficient, just get rid of all ifs and run mapreduce with query:{"file.author":{$exists:true},"file.type":"mobile","file.status":"ready"} clause. Change the conditions to match the actual cases you want to sum up over.
In 2.2 (upcoming version available today as rc0) you can use the aggregation framework for this type of query rather than writing map/reduce functions, hopefully that will simplify things somewhat.
I'm new to both MongoDB and Lithium and I can't really find the "good way" of working with nested documents. I noticed that when I try
$user = Users::find('first' ... );
$user->somenewfield = array('key' => 'val');
what I get for "somenewfield" is a Document object. But there is also a DocumentArray class - what is the difference between them?
When I call
$user->save();
this results in Mongo (as expected):
"somenewfield" : {
"key": "value"
}
OK, but when I later want to add a new key-value to the array and try
$user->somenewfield['newkey'] = 'newval';
var_dump($user->somenewfield->to('array')); // shows the old and the new key-value pairs
$user->save(); // does not work - the new pair is not added
What is the correct way to adding a new array to a document using lithium? What is the correct way of updating the array/adding new values to the array? Shall I alywas give a key for the array value?
Thanks for the help in advance. I'm kinda stuck ... reading the documentation, reading the code ... but at some points it gets difficult to find out everything alone :)
Edit:
What I found at the end was that the way I shall use nested arrays is with $push and $pull:
Users::update(array('$push' => array('games' => (string) $game->_id)),
array(
'_id' => $this->user()->_id,
'games' => array('$ne' => (string) $game->_id)),
array('atomic' => false));
I think there are some quirks in handling subdocuments, you can try:
$somenewfield = $user->somenewfield;
$somenewfield->newkey = newvalue;
$user->somenewfield = $somenewfield;
$user->save();
Or the alternative syntax:
$user->{'somenewfield.newkey'} = $newvalue;
$user->save();
You should be able to find more examples in the tests (look in tests/data at any tests for Document).
I have a collection with a subdocument tags like :
Collection News :
title (string)
tags: [tag1, tag2...]
I want to select all the tags who start with a pattern, but returning only the matching tags.
I already use a regex but it returns all the news containing the matching tag, here is the query :
db.news.find( {"tags":/^proga/i}, ["tags"] ).sort( {"tags":1} ).
limit( 0 ).skip( 0 )
My question is : How can I retrieve all the tags (only) who match the pattern ?
(The final goal is to make an autocomplete field)
I also tried using distinct, but I didn't find a way to make a distinct with a find, it always returning me all the tags :(
Thanks for your time
A bit late to the party, but hopefully will help others who are hunting for a solution. I've found a way to do this using the aggregation framework and combining $project and $unwind with the $match, by chaining them together. I've done it using PHP but you should get the gist:
$ops = array(
array('$match' => array(
'collectionColumn' => 'value',
)
),
array('$project' => array(
'collection.subcollection' => 1
)
),
array('$unwind' => '$subCollection'),
array('$match' => array(
subCollection.subColumn => 'subColumnValue'
)
)
);
The first match and project are just use to filter out to make it faster, then the unwind on subcollection spits out each subcollection item by item which can then be filtered using the final match.
Hope that helps.
UPDATE (from Ryan Wheale):
You can then $group the data back into its original structure. It's like having an $elemMatch which returns more than one subdocument:
array('$group' => array(
'_id' => '$_id',
'subcollection' => array(
'$push' => '$subcollection'
)
)
);
I translated this from Node to PHP, so I haven't tested in PHP. If anybody wants the Node version, leave a comment below and I will oblige.
Embedded documents are not collections. Look at your query: db.news.find will return documents from the news collection. tags is not a collection, and cannot be filtered.
There is a feature request for this "virtual collection feature" (SERVER-142), but don't expect to see this too soon, because it's "planned but not scheduled".
You can do the filtering client-side, or move the tags to a separate collection. By retrieving only a subset of fields - only the tags field - this should be reasonably fast.
Hint: Your regex uses the /i flag, which makes it impossible to use indexation. Your db strings should be case-normalized (e.g. all upper case)