Laravel MongoDB - aggregation, ordering query - mongodb

I am wondering how could I achieve a specific result.
Starting of
I am using https://github.com/jenssegers/laravel-mongodb
The code sample below is used to get an array of documents that contains my specific slug in the rewards node. And till that point, everything works as intended.
$array = [
'rewards.slug' => ['$eq' => 'example_slug'],
'expired' => ['$gte' => \Carbon\Carbon::now()->toDateTimeString()]
];
$models = Master::raw(function ($collection) use (&$array) {
return $collection->find(
$array, ["typeMap" => ['root' => 'array', 'document' => 'array']])
->toArray();
});
My example document
{
"_id": {
"$oid": "5be4464eafad20007245543f"
},
"some_int_value": 100,
"some_string_value": "String",
"rewards": [
{
"slug": "example_slug",
"name": "Example slug",
"quantity": 4,
"estimated": {
"value": 18750
}
},
{
"slug": "example_slug",
"name": "Example slug",
"quantity": 1,
"estimated": {
"value": 100
}
},
{
"slug": "other_example",
"name": "Other slug example",
"quantity": 1,
"estimated": {
"value": 100
}
}
],
"expires": "2018-11-08 20:20:45",
}
Desired result
I would like to implement some more complex query, which would do the following.
Retrieve documents that : pseudo select all documents that contain reward "slug": "example_slug", sum the quantity of them, return greater than X quantity documents, order by sum quantity desc
and a very similar one to the above select all documents that contain reward "slug": "example_slug", sum estimated.value, return greater than X estimated.value documents, order by sum of estimated.value desc
If you do need more explanation feel free to ask, I feel like I don't even know where to start with this one.
All help is greatly appreciated

You can use below aggregation in 3.6.
$addFields to create an extra slugcount field to hold the result.
$filter rewards with slug matching example_slug followed by $sum to sum the quantity field.
$match with $gt > X - aggregation expression to filter documents where the sum of all matching quantities is greater than X
$sort slugcount desc and $project with exclusion to remove the slugcount from the final response.
db.colname.aggregate([
{"$addFields":{
"slugcount":
{"$let":{
"vars":{
"mslug":{
"$filter":{
"input":"$rewards",
"cond":{"$eq":["$$this.slug","example_slug"]}
}
}
},
"in":{"$sum":"$$mslug.quantity"}
}}
}},
{"$match":{"slugcount":{"$gt":X}}},
{"$sort":{"slugcount":-1}},
{"$project":{"slugcount":0}}
])
Something like
ModelName::raw(function ($collection) {
return $collection->aggregate([
['$match' => ['expired' => ['$gte' => \Carbon\Carbon::now()->toDateTimeString()]]],
['$addFields' => [
'slugcount'
['$let' => [
'vars' => [
'mslug' => [
'$filter' => [
'input' => '$rewards',
'cond' => ['$eq' => ['$$this.slug','example_slug']]
]
]
],
'in' => ['$sum' => '$$mslug.quantity']
]]
]],
['$match' => ['slugcount'=> ['$gt' => X]]],
['$sort' => ['slugcount' => -1]],
['$project' => ['slugcount' => 0]]]);
});
You can replace quantity with estimated.value for second aggregation.

Related

MongoDB Group By '$group' issue getting records in aggregate function on date we have datetime format in tabel DB

I having issue on mongo db with codeignitor3 to getting records using group by '$group' in aggregate function on date as 'created' but we have datetime format in table USER_DATA mongoDb database.
when I run the mongo query I am getting error "can't convert from BSON type string to Date"
$date = date('Y-m-d',strtotime(-1 month));
$users = $this->common_model->aggregate($this->database['USER_DATA'], array(array('$match' => array('created' => array('$gte' => $date))),
array('$sort' => array('_id.created' => -1)),
array('$group' => array('_id' => array('year' => array('$year' => '$created'), 'month' => array('$month' => '$created'), 'day' => array('$dayOfMonth' => '$created')), 'count' => array('$sum' => 1))),
));
before we are using
array('$group' => array('_id' => array('created' => '$created')), 'count' => array('$sum' => 1)),
Its was working fine but records not as required per group by date, because in DB table we have datetime format.
Is their any way on mongo query with codeignitor library function aggrigation?
Kindly go through Mongodb documentation for Date conversions.
var project = {
"$project":{
"_id": 0,
"y": {
"$year": ISODate("$created").getFullYear()
},
"m": {
"$month": ISODate("$created").getMonth()+1 // months start from 0
},
"d": {
"$dayOfMonth": ISODate("$created").getDate()
}
}
},
group = {
"$group": {
"_id": {
"year": "$y",
"month": "$m",
"day": "$d"
},
"count" : { "$sum" : "$users" } //Users count
}
};
For now I did alternate option, that is I created another field in table name "created_date": "2019-01-10" and I did the group by with "created_date" and its working fine.
$date = date('Y-m-d H:i:s',strtotime(-1 month));
$users = $this->common_model->aggregate($this->database['USER_DATA'], array(array('$match' => array('created' => array('$gte' => $date))),
array('$sort' => array('_id.created' => -1)),
array('$group' => array('_id' => array('created_date' => '$created_date')), 'count' => array('$sum' => 1)),
));
But I want to the way I was asking in question, because sometimes we can't create new extra field just for do it, that's why I try the option I asked in question, I hope in future I will get that solution too. Thank you guys:)

Updating multiple embedded document not working [duplicate]

This question already has answers here:
How to Update Multiple Array Elements in mongodb
(16 answers)
Closed 4 years ago.
I have a table with data like
{
_id: ....
Name
...
"RoomStatusDetails": [
{
"StatusEntryId": ObjectId("5bd6ea81d2ccda0a780054da"),
"RoomId": "78163a07-76db-83c1-5c22-0749fab73251",
"CurrentStatus": ObjectId("5bd17295d2ccda11f0007765"),
"StartDate": ISODate("2018-10-09T22:00:00.0Z"),
"Notes": "Notes for in service",
"Discrepancy": "Discrepency",
"Waiver": "Waiver",
"TFlight": "T Flight",
"IsActive": "Inactive"
},
{
"StatusEntryId": ObjectId("5bd6ecf3d2ccda0a780054db"),
"RoomId": "78163a07-76db-83c1-5c22-0749fab73251",
"CurrentStatus": ObjectId("5bd17295d2ccda11f0007766"),
"StartDate": ISODate("2018-10-16T22:00:00.0Z"),
"Notes": "Out of service",
"Discrepancy": "",
"Waiver": "",
"TFlight": "",
"IsActive": "Active"
},
...
}
I have written below lines of code for updating IsActive field to "Inactive" on the basis of RoomId
$this->collection->updateOne(array('_id' => new MongoDB\BSON\ObjectID($this->id), "RoomStatusDetails" =>
array('$elemMatch' => array("RoomId" => $this->RoomId))),
array('$set' => array("RoomStatusDetails.$.IsActive" => 'Inactive')), array("multi" => true, "upsert" => false));
The above code is not updating all the IsActive field. Please help!!!
You have to use arrayFilters to pass in the room id filter.
Without arrayFilters it will update all [] the "RoomStatusDetails" array elements without taking into account the room id query filter.
Something like
db.col.update(
{"RoomStatusDetails.RoomId" :"78163a07-76db-83c1-5c22-0749fab73251"},
{"$set":{"RoomStatusDetails.$[room].IsActive" : "Inactive"}},
{"arrayFilters":{"room.RoomId":"78163a07-76db-83c1-5c22-0749fab73251"}}
);
In php
$bulkbatchStatus->update(
array('_id' => new MongoDB\BSON\ObjectID($this->id), "RoomStatusDetails" => array('$elemMatch' => array("RoomId" => $this->RoomId))),
array('$set' => array("RoomStatusDetails.$[room].IsActive" => 'Inactive')),
array("multi" => true, "upsert" => false, "arrayFilters" => array("room.RoomId" => $this->RoomId))
);
All matched embedded document:
For updating all matched embedded documents you should use $[] because $ refer the first position of matched embedded document.
the positional $ operator acts as a placeholder for the first element
that matches the query document

projection in php mongodb

I have a collection with document like
{
"_id": ObjectId("5b4dd622d2ccda10c00000f0"),
"Code": "Route-001",
"Name": "Karan Nagar - Jawahar Nagar",
"StopsDetails": [
{
"StopId": "cb038446-bbad-5460-79f7-4b138024968b",
"Code": "Stop- 001",
"Name": "Lane market",
"Fee": "600"
},
{
"StopId": "2502ce2a-900e-e686-79ea-33a2305abf91",
"Code": "Stop-002",
"Name": "City center",
"Fee": "644"
}
],
"StopsTiming" :
[
....
]
}
I want to fetch all embedded documents "StopDetails" only. I am trying to tweak below lines of codes but I am not able get what to write in $query "StopsDetails" in order to fetch only StopsDetails embedded document.
Please help !!!
$query = ['_id' => new MongoDB\BSON\ObjectID($this->id),
'StopsDetails' => ];
try
{
$cursor = $this->collection->find($query);
}
catch (Exception $e)
{
}
return $cursor->toArray();
You need to use projection in the second parameter of the query
$query = ['_id' => new MongoDB\BSON\ObjectID($this->id)]
$projection = ['StopsDetails' => true]
$cursor = $this->collection->find($query, $projection);
which is similar to the javascript query
db.collection.find({ '_id': ObjectId }, { 'StopDetails': 1 })
YourModel.findOne({_id:yourId},function(err,result){
console.log(result); //will return all data
console.log(result.StopsDetails); //will retrun your stop details
}
If you want to fetch specific embedded fields in MongoDB you can use projection
Projection
Try it mongodb playground
db.col.find({"_id" : ObjectId("5b4dd622d2ccda10c00000f0")},{"StopsDetails" : 1});

Generating a Structure for Aggregation

So here's a question. What I want to do is generate a data structure given a set of input values.
Since this is a multiple language submission, let's consider the input list to be an array of key/value pairs. And therefore an array of Hash, Map, Dictionary or whatever term that floats your boat. I'll keep all the notation here as JSON, hoping that's universal enough to translate / decode.
So for input, let's say we have this:
[ { "4": 10 }, { "7": 9 }, { "90": 7 }, { "1": 8 } ]
Maybe a little redundant, but lets stick with that.
So from that input, I want to get to this structure. I'm giving a whole structure, but the important part is what gets returned for the value under "weight":
[
{ "$project": {
"user_id": 1,
"content": 1,
"date": 1,
"weight": { "$cond": [
{ "$eq": ["$user_id": 4] },
10,
{ "$cond": [
{ "$eq": ["$user_id": 7] },
9,
{ "$cond": [
{ "$eq": ["$user_id": 90] },
7,
{ "$cond": [
{ "$eq": ["$user_id": 1] },
8,
0
]}
]}
]}
]}
}}
]
So the solution I'm looking for populates the structure content for "weight" as shown in the structure by using the input as shown.
Yes the values that look like numbers in the structure must be numbers and not strings, so whatever the language implementation, the JSON encoded version must look exactly the same.
Alternately, give me a better approach to get to the same result of assigning the weight values based on the matching user_id.
Does anyone have an approach to this?
Would be happy with any language implementation as I think it is fair to just see how the structure can be created.
I'll try to add myself, but kudos goes to the good implementations.
Happy coding.
When I had a moment to think about this, I ran back home to perl and worked this out:
use Modern::Perl;
use Moose::Autobox;
use JSON;
my $encoder = JSON->new->pretty;
my $input = [ { 4 => 10 }, { 7 => 9 }, { 90 => 7 }, { 1 => 8 } ];
my $stack = [];
foreach my $item ( reverse #{$input} ) {
while ( my ( $key, $value ) = each %{$item} ) {
my $rec = {
'$cond' => [
{ '$eq' => [ '$user_id', int($key) ] },
$value
]
};
if ( $stack->length == 0 ) {
$rec->{'$cond'}->push( 0 );
} else {
my $last = $stack->pop;
$rec->{'$cond'}->push( $last );
}
$stack->push( $rec );
}
}
say $encoder->encode( $stack->[0] );
So the process was blindingly simple.
Go through each item in the array and get the key and value for the entry
Create a new "document" that has in array argument to the "$cond" key just two of required three entries. These are the values assigned to test the "$user_id" and the returned "weight" value.
Test the length of the outside variable for stack, and if it was empty (first time through) then push the value of 0 as seen in the last nested element to the end of the "$cond" key in the document.
If there was something already there (length > 0) then take that value and push it as the third value in the "$cond" key for the document.
Put that document back as the value of stack and repeat for the next item
So there are a few things in the listing such as reversing the order of the input, which isn't required but produces a natural order in the nested output. Also, my choice for that outside "stack" was an array because the test operators seemed simple. But it really is just a singular value that keeps getting re-used, augmented and replaced.
Also the JSON printing is just there to show the output. All that is really wanted is the resulting value of stack to be merged into the structure.
Then I converted the logic to ruby, as was the language used by the OP from where I got the inspiration for how to generate this nested structure:
require 'json'
input = [ { 4 => 10 }, { 7 => 9 }, { 90 => 7 }, { 1 => 8 } ]
stack = []
input.reverse_each {|item|
item.each {|key,value|
rec = {
'$cond' => [
{ '$eq' => [ '$user_id', key ] },
value
]
}
if ( stack.length == 0 )
rec['$cond'].push( 0 )
else
last = stack.pop
rec['$cond'].push( last )
end
stack.push( rec )
}
}
puts JSON.pretty_generate(stack[0])
And then eventually into the final form to generate the pipeline that the OP wanted:
require 'json'
userWeights = [ { 4 => 10 }, { 7 => 9 }, { 90 => 7}, { 1 => 8 } ]
stack = []
userWeights.reverse_each {|item|
item.each {|key,value|
rec = {
'$cond' => [
{ '$eq' => [ '$user_id', key ] },
value
]
}
if ( stack.length == 0 )
rec['$cond'].push( 0 )
else
last = stack.pop
rec['$cond'].push( last )
end
stack.push( rec )
}
}
pipeline = [
{ '$project' => {
'user_id' => 1,
'content' => 1,
'date' => 1,
'weight' => stack[0]
}},
{ '$sort' => { 'weight' => -1, 'date' => -1 } }
]
puts JSON.pretty_generate( pipeline )
So that was a way to generate a structure to be passed into aggregate in order to apply "weights" that are specific to a user_id and sort the results in the collection.
First thank you Neil for your help with this here, this workout great for me and it's really fast. For those who use mongoid, this is what I used to create the weight parameter where recommended_user_ids is an array:
def self.project_recommended_weight recommended_user_ids
return {} unless recommended_user_ids.present?
{:weight => create_weight_statement(recommended_user_ids.reverse)}
end
def self.create_weight_statement recommended_user_ids, index=0
return 0 if index == recommended_user_ids.count
{"$cond" => [{ "$eq" => ["$user_id", recommended_user_ids[index]] },index+1,create_weight_statement(recommended_user_ids,index+1)]}
end
So to add this to the pipeline simply merge the hash like this:
{"$project" => {:id => 1,:posted_at => 1}.merge(project_recommended_weight(options[:recommended_user_ids]))}

MongoDB query on same collection with count

I have only one collection "details". It is used in the query twice with different alias. As Mongo does not have alias, I think so mapreduce will give the results.
I also tried aggregation with unwind, but it will unwind on a field and not on the collection.
Any help with aggregation or mapreduce.
Collection:
"details"
{
"user_id":1,
"lft":2
"rgt":5
},
{
"user_id":2,
"lft":1
"rgt":6
},
{
"user_id":3,
"lft":3
"rgt":4
}
SQL query:
SELECT CONCAT( REPEAT('-', COUNT(parent.user_id) - 1), node.user_id)
AS user_id
FROM details AS node,
details AS parent
WHERE node.lft BETWEEN parent.lft AND parent.rgt
GROUP BY node.user_id
ORDER BY node.lft;
It should output:
1
-2
--3
I have tried:
$mongodb = Connections::get('default')->connection;
$details = Details::connection()->connection->command(array(
'aggregate' => 'details',
'pipeline' => array(
array('$project' => array(
'_id' => array(
'parent'=>array(
'puser_id'=>'$user_id',
'pleft'=>'$left',
'pright'=>'$right',
),
'node'=>array(
'nuser_id'=>'$user_id',
'nleft'=>'$left',
'nright'=>'$right',
)
),
),
'$group'=>array('_id'=>'$_id.parent.puser_id'),
'$match' => array(
'$_id.node.nleft'=>array('$gt'=>'$_id.parent.pleft'),
'$_id.node.nright'=>array('$gt'=>'$_id.parent.pright')
)
),
)
));
I am stuck at $group and $match!
I found the answer by changing the schema:
{
"_id": ObjectId("5114a7eb9d5d0c640900001e"),
"user_id": "5114a7eb9d5d0c640900001d",
"username": "user8",
"refer_username": "user7",
"refer_id": "5114a7c59d5d0c6409000018",
"ancestors": {
"0": null,
"1": "Initial",
"2": "user6",
"3": "user7"
},
}
This has helped me to find all ancestors by using the query:
user_id = '5114a7eb9d5d0c640900001d'
db.details.find({'user_id':user_id});
and all descendents by using the query:
username = 'user8'
db.details.find('ancestors':username)
Without using Map/Reduce, and using aggregation framework, with $unwind function I am able to count all the ancestors and descendents of the user.
You can check this page: http://docs.mongodb.org/manual/tutorial/model-tree-structures/ it will really improve your understanding of Model Tree structure for MongoDB. It is fast and better than MySQL.