I am using a single MongoDB 3.0.1 instance (without sharded cluster, replicas, etc) with one database containing a collection with 15324247 points. Of course, points are indexed with a 2dsphere index. Queries are done through a Node.js app.
Looking for points near a concrete lon&lat, it requires 11710ms to return 59925 points.
The same query including a restricted project (only geometry), it still requires 4351ms to return 59925 points.
find({
"geometry": {
"$near": {
"$geometry": {
"type": "Point",
"coordinates": [110.30838012695312,
-20.86522808076763]
},
"$maxDistance": 1000
}
}
},
{
geometry: 1
})
Changing the query and using and aggregation instead of find it requires: 5799ms but returns 8882 points, and projecting only the geometry 4606ms returns 8882 points.
aggregate([{
"$geoNear": {
"near": {
"type": "Point",
"coordinates": [110.30838012695312,
-20.86522808076763]
},
"distanceField": "dist.calculated",
"maxDistance": 964,
"spherical": true,
"num": 70000
}
}])
Although all the elements are indexed, is this a normal performance? How could be improved? I have tried $geoWithin instead of $geoNear, or adding more keys to geoindex, using cursor.get instead of cursor.on in Node.js side, increasing/decreasing batchSize for aggregation... but performances are quite similar in all cases.
And the second question is why the aggregation is returning less results than the find?
Related
I am creating a matching system based on user proximity. The basic idea is I have user A at point A and user B at another point B. Both users have search filters that determine how far a user can be to make a match. This distance is in Kilometers.
Anyways would it be possible to do this in a single vanilla query (without using an aggregate query)? Here is the query I have so far
var loc = userA.location;
var rad = userA.radius;
{$near : {
$geometry : {
type : "Point",
coordinates : loc
},
$maxDistance : rad
}
}
How do I account for user's B preference though?
I was planning to store a geometry that was basically their circle of preference and then see if userA was within that circle, but this just doesn't seem right.
You already have the radius property stored on each user, so that is a good thing, and not only for a $maxDistance option, but also as a check to see if the distance between users is "less than" the radius attribute.
What you now basically need is a way to "see" that distance and "compare". For this you use the aggregation framework and $geoNear:
db.users.aggregate([
{ "$geoNear": {
"near": {
"type": "Point",
"coordinates": userA.location
},
"spherical": true,
"maxDistance": userA.radius,
"distanceField": "distance"
}},
{ "$redact": {
"$cond": {
"if": { "$lt": [ "$distance", "$radius" ] },
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
So what this does it executes the same sort of $near result along with the "maxDistance" constraint, but actually returns an additional property in the named "distanceField" representing the actual distance between found 'users'. The "spherical" option is required when the index is "2dsphere".
As an aggregation "pipeline", this allows another operation to act on the output. The $redact stage makes a "logical comparison" of the existing "radius" data and the generated "distance" field, to see if that distance is "less than" the stored radius data.
Only when that condition is true is the document returned via $$KEEP, otherwise the distance is larger than the users radius as the document is removed from results via $$PRUNE.
That's how you make a comparison that respects the radius on the user data for returning results.
I have a list of shops they have a useCount and a geolocation.
How would I search and order by useCount but also have a property on each object returned signifying how close they are to me.
schema:
{
name: String,
useCount: { type: Number, index: true },
location: { 'type': {type: String, enum: "Point", default: "Point"}, coordinates: { type: [Number], default: [0,0]} }
}
e.g results
shop1 usecount-12 closest-3 geo-1333.222,222.222
shop2 usecount-3 closest-1 geo-1333.222,222.222
shop3 usecount-1 closest-2 geo-1333.222,222.222
Presuming your data is actually properly arranged for MongoDB and looks something like this:
{
"shop": 1,
"usecount": 12,
"closest": 3,
"geo": {
"type": "Point",
"coordinates": [1333.222,222.222]
}
}
And your coordinates are in fact in "longitude/latitude" order as is requireed from GeoJSON and MongoDB and that you have a geospatial index that is "2dsphere", then your best option for "composite sort" is using the $geoNear aggregate command pipeline, along with aggregation $sort:
Model.aggregate(
[
{ "$geoNear": {
"near": {
"type": "Point",
"coordinates": [1333.222,222.222]
},
"distanceField": "dist",
"spherical": true
}},
{ "$sort": { "dist": 1, "usecount": -1 } }
],
function(err,results) {
}
)
Where the $geoNear projects the "distance" as the nominated field here in "dist", and then you use that in the $sort along with the other field "usecount" as shown here in descending order for the "largest" value if "usecount" first, and within each "dist" already sorted.
The aggregation framework though .aggregate() does more than just "aggregate" documents. It is your "main tool" for projecting new values into a document, useful for such things as sorting results by values that "calculate" by one means or the other.
Unlike $near ( or $nearSphere ) the distance is returned as a true field in the document rather than just a "default" sort order. This allows that key to be used in the sorted results, along with any other field value present or projected into the document at the $sort stage.
Also noting that your data here does not appear to be valid spherical coordinates, which is going to cause problems with GeoJSON storage and also a "2dsphere" index. If not real global coordinates but coordinates on a "plane, then just use a plane legacy array for "geo" as [1333.222,222.222] and a "2d" index only. As well the argument to "near" within $geoNear is simply an array as well, and the "spherical" option would then not be required.
But possibly a problem with typing in your question as well.
I'm having a series of points which forms a poly-line (path). I need to store it in the mongodb and query it nearby point.
How to store a poly-line in the mongodb?
Can i query it with a $near?
After saving the poly-line to mongodb, i will have a point and distance from the point and need to query the db.
EDIT :
i'm gone track the user's location using the GPS and i need to save the path in mongodb, how to do this?
Then the user can able to search path nearby his place with certain distance so i need to search the mongodb with the $near function.
So what you want is actually a LineString GeoJSON type, which is supported by MongoDB:
{
"loc": {
"type": "LineString",
"coordinates": [ [ 40, 5 ], [ 41, 6 ] ]
}
}
This allows you record a set of "coordinates" along a "path" that you wish to contain as a singular object in your data store. The main beauty of this is that you can do geospatial queries against such an object ( rather than a distinct "Point" and retrieve the whole "set" of connecting "Points" in a single document as "nearest". Better yet "multiple shapes" like this can be matched with ease.
So then you can just query with $near or other operators as appropriate:
db.collection.find({
"loc": {
"$near": {
"$geometry": {
"type": "Point",
"coordinates": [ 41,5 ]
},
"$maxDistance": 10000
}
}
})
A $near operator in a query will return the results ordered by the "nearest" to the queried GeoJSON object or legacy coordinate point provided to the query.
That should basically work for you.
I am trying to create a geospacial query in MongoDB that finds all circles (with varying radius) that overlap a single point.
My data looks something like this:
{
name: "Pizza Hut",
lat: <latitude>
lon: <longitude>
radius: 20
...
}
Basically, I am trying to do exactly what is described in this SO post but with MongoDB - Get all points(circles with radius), that overlap given point
geoIntersects (http://docs.mongodb.org/manual/reference/operator/query/geoIntersects/) looks like what I need. But in my case, the lat, lon, and radius is stored with each mongodb document and is not a fixed radius that is part of the query. Can this be done?
A different approach would be to find all documents whose distance from my query point is less than the value of their radius field (ie - 20km in the example above). How do you structure a MongoDB query where the calculated distance is part of the query filter criteria?
Thanks!
Well it would be nicer if you could use a GeoJSON object to represent the location but as of present the supported types are actually limited so a "Circle" type which would be ideal is not supported.
The closest you could do is a "Polygon" approximating a circle, but this is probably a little too much work to construct just for this query purpose. The other gotcha with doing this and then applying $geoIntersects is that the results will not be "sorted" by the distance from the query point. That seems to be the opposite of the purpose of finding the "nearest pizza" to the point of origin.
Fortunately there is a $geoNear operation added to the aggregation framework as of MongoDB 2.4 and greater. The good thing here is it allows the "projection" of a distance field in the results. This then allows you to do the logical filtering on the server to those points that are "within the radius" constraint to the distance from the point of origin. It also allows sorting on the server as well.
But you are still going to need to change your schema to support the index
db.places.insert({
"name": "Pizza Hut",
"location": {
"type": "Point",
"coordinates": [
151.00211262702942,
-33.81696995135973
]
},
"radius": 20
})
db.places.ensureIndex({ "location": "2dsphere" })
And for the aggregation query:
db.places.aggregate([
// Query and project distance
{ "$geoNear": {
"near": {
"type": "Point",
"coordinates": [
150.92094898223877,
-33.77654333272719
]
},
"distanceField": "distance",
"distanceMultiplier": 0.001,
"maxDistance": 100000,
"spherical": true
}},
// Calculate if distance is within delivery sphere
{ "$project": {
"name": 1,
"location": 1,
"radius": 1,
"distance": 1,
"within": { "$gt": [ "$radius", "$distance" ] }
}},
// Filter any false results
{ "$match": { "within": true } },
// Sort by shortest distance from origin
{ "$sort": { "distance": -1 } }
])
Basically this says,
*"out to 100 kilometers from a given location, find the places with their distance from that point. If the distance is within their "delivery radius" then return them, sorted by the closest"
There are other options you can pass to $geoNear in order to refine the result, as well as return more than the default 100 results if required and basically pass other options to query such as a "type" or "name" or whatever other information you have on the document.
I currently have a query on MongoDB which queries for devices located within a certain distance of a location. It's been working perfectly fine for months now. When I upgraded from MongoDB 2.4.10 to 2.6.3, the query no longer works and it returns no results.
{
"lastLocation": {
"$near": {
"$geometry": {
"type": "Point",
"coordinates": [ -122.195 , 37.423]
}
},
"$maxDistance": 10000
}
}
After playing around with it, it seems that without $maxDistance, the query works. However, I can't get $maxDistance to work at all.
I have two databases side by side with the exact same data, one with each version of MongoDB, 2.4 returns data, 2.6 does not (so it's not because there aren't devices within that distance).
Is there a known issue with maxDistance, or was there a change in units or how it works?
Index is as follows:
{
v: 1,
name: "lastLocation_2dsphere",
key: {
lastLocation: "2dsphere"
},
ns: "s-dev.devices",
background: true,
safe: true
}
Sample entry:
"lastLocation": [-122.19888, 37.42316]
Yes there have been a few changes in 2.6, kind of hidden in the documentation. In your case, in version 2.6 the $maxDistance needs to be inside of the $near document, not outside:
$maxDistance Changes¶
Description
For $near queries on GeoJSON data, if the queries specify a $maxDistance, $maxDistance must be inside of the $near document. In
previous version, $maxDistance could be either inside or outside the
$near document.
$maxDistance must be a positive value.
Solution
Update any existing $near queries on GeoJSON data that currently have the $maxDistance outside the $near document
Update any existing queries where $maxDistance is a negative value.
http://docs.mongodb.org/manual/release-notes/2.6-compatibility/
So in your case the query needs to look like this:
{
"lastLocation": {
"$near": {
"$geometry": {
"type": "Point",
"coordinates": [ -122.195 , 37.423]
},
"$maxDistance": 10000
}
}
}