how to sort by proximity and date in mongoDB?
I tried this. But they just sort by date:
coll.find({'date':{$gte:date},'location':{$nearSphere:[lat,lng]}}).sort({'date':1}).execFind(function (err, docs) {})
I appreciate the help.
There's no direct way to use $near or $nearSphere and sort by another field, because both of these operators already sort the results of doing a find(). When you sort again by 'date', you're re-sorting the results. What you can do, however, is grab results from the $nearSphere incrementally, and sort each set of results. For example:
function sortByDate(a, b) { return a.date - b.date; }
// how many results to grab at a time
var itersize = 10;
// this will hold your final, two-way sorted results
var sorted_results = new Array();
for (var i=0, last=db.coll.count(); i<last-itersize; i+=itersize) {
var results = db.coll.find( {"date":{$gte:date},
// longitude, then latitude
"location":[lng, lat]} ).skip(i).limit(itersize).toArray();
// do date sorting app-side for each group of nearSphere-sorted results
sorted_results = sorted_results.concat( results.sort(sortByDate) );
}
You should also be aware of the order you specify geospatial coordinates in mongodb queries. MongoDB uses the geojson spec, which does coordinates in X, Y, Z order (i.e., longitude, latitude).
It's recommended to use $geoNear in an aggregate :
https://docs.mongodb.com/manual/reference/operator/aggregation/geoNear/
You can sort on your date and distance in the aggregate :
coll.aggregate([
{
$geoNear: {
near: { type: "Point", coordinates: [ lng, lat ] },
key: "location",
spherical: true,
distanceField: "dist.calculated",
query: { "date": {"$gte": date} }
}
},
{$sort: {"dist.calculated":1, "date": 1}}
])
Related
Say I have a collection with entries like so:
{_id: "foo" lng: -98.21 lat: 77.131 }
{_id: "foo" lng: -100.12 lat: 770.313 }
{_id: "foo" lng: -98.32 lat: 772.123 }
where lng and lat correspond to the longitude and latitude.
I want to return only the entries that are within a certain radius of the the input longitude and latitude. So that would mean that for radius R, sqrt((inputLatiude - docLatitude)^2 + (inputLongitude - docLonguitude)^2) < R.
I don't want to return all the documents with an added field that tells me whether it's within R or not. I want to return only the fields that match the condition.
How do I query this?
You'll want to use $near here to do so. However $near requires a 2dsphere index built which currently seems like you don't have one as the document structure does not match the required structure.
So what do you have to do?
Restructure your document so the lng and lat fields could be 2dsphere indexed:
{
_id: "foo",
loc : { type: "Point", coordinates: [ -98.21, 77.131 ] },
}
Build such index on the field loc.
Use $near to query:
db.collection.find(
{
loc:
{ $near :
{
$geometry: { type: "Point", coordinates: [ inputLng, inputLat ] },
$maxDistance: R
}
}
}
)
I have a route that does a $near search on my mongo database. It returns documents that have a geo tag within 100 miles. This request searches each document for "bandLocation". I have a second field, that is indexed, called bandTour - It holds several other geo locations in the same format. I want the request to also include these locations but have been unsuccessful - How do I add the second query?
Here is my route - If "bandLocation" is the only request, it works... How would I add "bandTour"?
router.get('/allbands/:lng/:lat', (req, res) => {
quoteGenerator.find(
{
"bandLocation.geometry":
{ $near :
{
$geometry: {
type: "Point",
coordinates: [parseFloat(req.params.lng), parseFloat(req.params.lat)]
},
$maxDistance: 160934,
}
},
"bandTour.geometry":
{ $near :
{
$geometry: {
type: "Point",
coordinates: [parseFloat(req.params.lng), parseFloat(req.params.lat)]
},
$maxDistance: 160934,
}
}
})
.then(
function(bands){
res.send(bands)
}
)
});
MongoDB only supports a single geo-expressions such as $near in a query. While aggregation does support geo queries, it is only supported as the first stage of the pipeline, so aggregation is not a solution in this case.
You would need to implement this as 2 queries, then combine the results. Since your sample query is requiring both locations to match, you could use a find with $near on bandLocation, projecting just the _id field. Then use the returned _id values to build a second query that tests for _id:{$in:[_array of ids_]} along with the $near on bandTour
I have a search controller where I want to exclude some documents by an array of ids. Next is my aggregation:
var aggregation = [
{
$geoNear:{
query : {
//_id:{$nin:user.friends}, <---- user.friends is an array of id's that I want to exclude
"profile.age":{$gte: minAge, $lte: maxAge},
"approved":true
},
near: { type: "Point", coordinates: [ user.location[0], user.location[1]] },
limit:100,
maxDistance:radius*1000,
distanceField: "dist.calculated",
includeLocs: "dist.location",
distanceMultiplier:1/1000,
spherical: true
}
},
{
$project:{
online:1,
promoted:1,
location:1,
profile:1,
dist:1
}
},
{ $sort : { online:-1, promoted:-1} }
];
How can I achieve that the result would exclude some doc's having an array 'user.friends' populated with the id's that needs to be excluded.
_id:{$nin:user.friends} is not working
I would assume your array user.friends is an array of strings, So since it checks against the _id which is an Object, not a String, the query cannot work as you expect it to work.
So what you have to is to convert your array user.fridns into an array of Objects. Something like:
user.friends = user.friends.map(x => ObjectId(x) );
I wanted to do this:
model.User.aggregate([
//step 1 match criteria
{
$match: criteria
},
//step 2 skip
{
$skip: offset
},
//step 3 limit
{
$limit: limit
},
//step 4 sort by computed distance
{
$geoNear : {
near: {type: 'Point', coordinates: coords },
distanceField: 'currentCity.computed_distance',
includeLocs: 'currentCity.loc',
spherical: true,
uniqueDocs: true,
distanceMultiplier: 3963.2, //convert to miles (this number is the radius of the earth in miles)
}
}
],function(err,users){
if (err) return res.error(err);
if (!users.length) return res.error('no matched criteria');
res.apiResponse(users);
});
but the documentation of $geoNear states:
You can only use $geoNear as the first stage of a pipeline.
Reading the documentation, i see that i can simply move $match inside of $geoNear via the query option. likewise, $limit may be placed inside of $geoNear via the limit option. the one problem is, there is no equivalent for the $skip option, so it looks as if facilitating pagination would not be possible? i'm really confused here, why $geoNear can't be the 4th step in the pipeline. The goal of the query is simply to find the best n matches, where n = limit, then sort by nearest in proximity. is this even possible? I am having trouble finding an answer for this specific use case.
I suppose one solution might be to perform a query to select only ids matching documents, convert to list of ids, then do the aggregates with an $in query like so:
model.User.find(criteria).skip(offset).limit(limit).select('_id').exec(function (err, userIds) {
var ids = [];
userIds.forEach(function(u){
ids.push(u._id);
});
model.User.aggregate([
{
$geoNear : {
query: { _id: {$in: $ids } },
near: {type: 'Point', coordinates: coords },
distanceField: 'currentCity.computed_distance',
includeLocs: 'currentCity.loc',
spherical: true,
uniqueDocs: true,
distanceMultiplier: 3963.2, //convert to miles (this number is the radius of the earth in miles)
}
}
],function(err,users){
if (err) return res.error(err);
if (!users.length) return res.error('no matched criteria');
res.apiResponse(users);
});
});
This would work, but ideally i could do it in 1 query if possible. any ideas greatly appreciated.
One solution is this one:
result = db.cafes.aggregate([{
'$geoNear': {
'near': {
'type': 'Point',
'coordinates': [
-73.991084,
40.735863]},
'spherical': True,
'distanceField': 'dist',
'num': 20}
}, {
'$skip': 10
}])
There is also a better solution with this approach:
ids = [42]
result = db.command(
'geoNear', 'cafes',
near={
'type': 'Point',
'coordinates': [
-73.991084,
40.735863]},
spherical=True,
minDistance=268,
query={
'_id': {
'$nin': ids}},
num=10)
And a really nice explanation on speed and issues over here:
https://emptysqua.re/blog/paging-geo-mongodb/
I'm using MongoDB 2.6.3 to query against a large collection of geospatial data. Specifically, I'm looking at querying the dataset for all pings within a few kilometers of a central location, and then collapsing them by user identifier to get a count of how many pings each user has.
Naturally, I'm using MongoDB aggregation for this, and specifically the $geoNear pipeline stage. However, it looks like, even though aggregation returns a cursor in 2.6.0, the $geoNear still has restrictions on the size of the result set tied to when aggregation returned a document. Namely, aggregation with $geoNear is returning only 65,000 records, while an equivalent (cursored) query is returning 200,000+.
Does anyone have any insight as to how I can perform large-scale aggregation then with geoNear?
edit:
Sample document:
{
"initial_epoch_time" : 1370062800,
"location" : [
-72.3458073902,
41.8241332683
],
"_id" : ObjectId("540a34050dc2520000912286"),
"__v" : 0
}
The following cursored query returns a count of ~200,000 documents, which I suspect is the correct number:
var cursor = db.pings.find( { location : { $near: { $geometry: { type: 'Point', coordinates: [-71.10560939999999, 42.3465666] }, $maxDistance: 10*1000 } } } )
var ctr = 0;
while(cursor.hasNext())
{ ctr++;
var ping = cursor.next()
}
print(ctr)
while the following aggregation-based query:
var cursor = db.pings.aggregate ( [ {$geoNear: { near: {type: "Point", coordinates: [-71.10560939999999, 42.3465666]},limit: 100000000, spherical: true, maxDistance: 10*1000, distanceField: "distance"} } ] )
var ctr = 0;
while(cursor.hasNext())
{ ctr++;
var ping = cursor.next()
}
print(ctr)
returns ~65,000 documents, regardless of maxDistance.
$geoNear as an command has a 16MB document limit to the output. I've found out that you won't retrieve any errors but the document will be automatically cut to the aggregation's document size limit. You can test by reducing size of your documents in the collection. You will get more results when you make your collection's documents smaller.
Line 235:
https://github.com/mongodb/mongo/blob/master/src/mongo/db/commands/geo_near_cmd.cpp