Proper indexing and sorting with MongoDB - mongodb

I have a relatively complex query that I'm running. I've included it below, but essentially I have a location query combined with an $or and $in query all to be sorted by date/time. Whenever I run the full query though it returns in ascending order rather than descending.
If I pull out the location part of the query, it successfully returns the results in the proper order, but once I put the location component of the query back in, it continues to fail.
Here's the query as written in php:
$query = array(
'$or' => array(
array( ISPUBLIC => true ),
array( GROUP_INVITES.".".TO.".".INVITE_VAL => array( '$in' => $user_groups ) )
),
LOC => array( '$near' => array( (float)$lon , (float)$lat ), '$maxDistance' => 1 ) //Using 1 degree for the max distance for now
);
$res = $this->db->find( $query )->sort(array('ts'=>-1))->limit($limit)->skip($start);
The fields used are constants that map to fields in the database. My first assumption was that the indexes are not configured properly, but how should I have this indexed when combining geospatial queries with others?
UPDATE
I've been able to replicate the problem by simplifying the query down to:
$query = array(
HOLLER_GROUP_INVITES.".".HOLLER_TO.".".INVITE_VAL => array( '$in' => $user_groups ),
HOLLER_LOC => array( '$near' => array( (float)$lon , (float)$lat ), '$maxDistance' => 1 )
);
UPDATE 2
I've run additional tests and it appears that when limit is removed it's sorted in the correct direction. No idea what's going on here.

This question appears to have the answer: Sorting MongoDB GeoNear results by something other than distance? .... apparently $near was already performing a sort. If you simply want to limit within a given boundary but sort by something else, you should use $within instead of $near.

Related

TYPO3 TCA select with own items

I use TYPO3 version 9.
I am trying to configure TCA selectMultipleSideBySide. I want to use without foreign table, I want to pass my own items. It displays correctly, but when I try to add more then 1 item, I get error:
These fields of record 3 in table "tx_scout24_domain_model_vehicle" have not been saved correctly: equipment! The values might have changed due to type casting of the database.
It because TYPO3 try to save data to main table, but not in mm table.
My current TCA:
'equipment' => array(
'label' => 'LLL:EXT:scout24/Resources/Private/Language/locallang_db.xlf:equipment',
'config' => array(
'type' => 'select',
'renderType' => 'selectMultipleSideBySide',
'items' => \Istar\Scout24\Service\FieldService::getFields('equipment'),
'MM' => 'tx_scout24_vehicle_equipment_mm',
),
),
According to the documentation the values are stored as comma-separated values. It is not possible to use a MM relation table. To store the values the columns must be of type varchar. Because you can store one value it seems that the type of your columns is an int, which can be stored without a problem.
So you have to do:
Remove the MM relation in your configuration
Change your table column to varchar

MongoDB-PHP: How to sort the result of find query by timestamp in ascending/descending order?

I want to sort the result of find query in ascending/descending order w.r.t timestamp(time_created).
My query is:
$mongoResult = $mongoDb->find(array('organization_id' => new MongoId($_SESSION['user_id'])));
Try:
$mongoResult = $mongoDb->find(
array(
'organization_id' => new MongoId($_SESSION['user_id'])
))->sort(array("time_created" => -1)
);
You can use the _id field to sort by timestamp.
The following query can print the latest 100 records.
$mongoResult = $mongoDb->find()->sort( array("_id" => -1 ))->limit(100);

How to optimize this mongo query?

I have a simple mongo query here which uses aggregate function. It is taking 1 min and 38 sec to load 10 rows in my php script. How can I optimize this query as it takes lesser time.
$data = $mongo->command(
array(
'aggregate' =>"my_collection",
'pipeline' => array(
array('$match' => $filter_query),
array('$group' => array('_id'=>'$email')),
array('$skip'=>$offset),
array('$limit'=>10)
)
),
array( 'timeout' => "-1")
);
This is my query and the filter_query is another array which is
Array ( [event] => spamreport [timestamp] => Array ( [$gt] => 1384236000 [$lt] => 1384840800 ) )
Also I have one more query to find the distinct values like this
$distinct_notifications = $my_collection->distinct('email',$filter_query);
It is taking forever to load the distinct values. My collection is little larger and has around 20700144 rows. It is increasing day by day. Also it will be helpful if someone can point out a good map/reduce tutorial as I am new to this.

how to do multiple where with same field?

How to convert this script for mongoDB?
select * from table where id > -1 and id in (4,5)
I need to execute this query with two where conditions on the same field.
This doesn't seem to work in PHP:
$db->$col->find(
array(
'id' => array( '$gt' => -1 ),
'id' => array( '$in' => array( 4, 5 ) )
)
);
Basically the problem is that you have a duplicate key in your array so the latter overwrites the prior so your query is only doing the $in.
Most poeople look to $and to solve this problem however, very few know that actually:
'id'=>array('$gt'=>-1, '$in'=>array(4,5))
Will also solve it. You can just chain your constraints for that field.

Select unique fields from array using MongoDB

I have a MongoDB structure which currently looks like this:
[campaigns] => Array (
[0] => Array (
[campaign_id] => 4e8cba7a0b7aabea08000006
[short_code] => IHEQnP
[users] => Array (
)
)
[1] => Array (
[campaign_id] => 4e8ccf7c0b7aabe508000007
[short_code] => QLU_IY
[users] => Array (
)
)
)
What I would like to be able to do, is search for the short code, and just have the relevant array returned. I initially tried:
db.users.find({'campaigns.short_code':'IHEQnP'}, {'campaigns.campaign_id':1})
However that returns all the arrays, as opposed to just the one (or field) that I want.
Is there a way in Mongo to get the correct array (or even field associated with the array)? Or is that something I would have to do on the server? Am using the lithium framework to retrieve the results (in case it helps).
Thanks in advance :)
Dan
When you use a criteria like campaigns.short_code you are still searching the collection, the campaigns is just a property of a document, your find returns documents.
So given this structure you can not achieve what you want directly by a query.
Arrays in MongoDb can be sliced, but not sorted:
db.users.find({}, {campaigns: { $slice : 1}})
This would give you the first campaign, but as you cant sort it so IHEQnP is at top its of no help in this situation.
Read more here.
You could however filter this quite simple in Lithium after retrieving the full document:
$id = 'id to match against';
$result = $user->campaigns->find(function($model) use ($id) {
return $model->campaign_id === $id
});
See docs for Entity::find here
My solution would be to keep it in User if the amount of campaigns is low (fast to sort and filter in PHP as long as document size isnt too big).
If this is expected to grow then look at moving it to its own model/collection or re-think how you modelled your data.