I have a lot of different documents each containing a "savetime" field along with a point. Is it possible to query not only for all documents containing a location $near a point, but also within that same time (or within 1 second of each other)? What is the best way to perform the query to ensure maximal performance?
From the documentation:
A compound 2dsphere index can reference multiple location and non-location fields within a collection’s documents. You can arrange the fields in any order.
So in order for you to perform your queries that are constrained by a time, then you would create a compound index that has the "savetime" as it's first element. i.e:
db.collection.ensureIndex({ savetime: 1, loc: "2dsphere" })
That allows you to constrain dates normally while adding your additional constraints for finding near that point:
db.collection.find({
savetime: { $gte: new Date(<start>), $lte: new Date(<end>) },
loc: { $near :
{ $geometry :
{ type : "Point" ,
coordinates : [ <longitude> , <latitude> ] } ,
$maxDistance : <distance in meters>
}
}
}
})
For more information see the tutorials on creating and querying.
Related
I have a route that does a $near search on my mongo database. It returns documents that have a geo tag within 100 miles. This request searches each document for "bandLocation". I have a second field, that is indexed, called bandTour - It holds several other geo locations in the same format. I want the request to also include these locations but have been unsuccessful - How do I add the second query?
Here is my route - If "bandLocation" is the only request, it works... How would I add "bandTour"?
router.get('/allbands/:lng/:lat', (req, res) => {
quoteGenerator.find(
{
"bandLocation.geometry":
{ $near :
{
$geometry: {
type: "Point",
coordinates: [parseFloat(req.params.lng), parseFloat(req.params.lat)]
},
$maxDistance: 160934,
}
},
"bandTour.geometry":
{ $near :
{
$geometry: {
type: "Point",
coordinates: [parseFloat(req.params.lng), parseFloat(req.params.lat)]
},
$maxDistance: 160934,
}
}
})
.then(
function(bands){
res.send(bands)
}
)
});
MongoDB only supports a single geo-expressions such as $near in a query. While aggregation does support geo queries, it is only supported as the first stage of the pipeline, so aggregation is not a solution in this case.
You would need to implement this as 2 queries, then combine the results. Since your sample query is requiring both locations to match, you could use a find with $near on bandLocation, projecting just the _id field. Then use the returned _id values to build a second query that tests for _id:{$in:[_array of ids_]} along with the $near on bandTour
Given the collection locations consisting of ~20,000,000 documents with 3 properties:
{
_id,
name, // string
geo // coordinate pair, e.g. [-90.123456, 30.123456]
}
and an index of name: 1 and a geo index setup like so:
{
"geo" : "2dsphere"
},
{
"v" : 1,
"name" : "geo_2dsphere",
"ns" : "db.locations",
"min" : "-180.0",
"max" : "180.0",
"w" : 1.0,
"2dsphereIndexVersion" : 2
}
How can I performantly query against this collection both on the geo_2dsphere index and on the name index?
When I run a $box query on the geo index only, it takes over 20 seconds to return 50 results. When I include a search against the name property it goes up even further.
If I run a $near query, then things can perform very quickly, but sometimes queries seem to (very randomly) go from ~200ms to many seconds. See this example where the only difference is one additional character on the name index which actually increases the time:
200ms:
{name: /^mac/, geo: {$near: {$geometry: {type: "Point", coordinates: [ -90.123456, 30.123456 ]}, $maxDistance: 20000}}}
18,000ms:
{name: /^macy/, geo: {$near: {$geometry: {type: "Point", coordinates: [ -90.123456, 30.123456 ]}, $maxDistance: 20000}}}
I can't understand why being more specific with an index is slowing things down so much. When I get more specific with a phrase, I have to drastically reduce the $maxDistance to something like 7,000 meters before the query returns in any reasonable amount of time.
Is there a better setup I should be doing here?
As has been pointed out to me by Blakes Seven, you cannot search across multiple indexes in MongoDB:
There is a "highlander rule" (there can be only one) in the query
evaluation that denies the usage of more than "one" "special" index in
a query evaluation. So you cannot have multiple "text" or muliple
"geospatial" or any combination of "text" and "geospatial" or usage of
any of those within an $or condition, that results in multiple index
selection.
So, I've opted to move over to Elasticsearch for this specific query, indexing only what I need to complete these multi-index queries, and then use those results to load the necessary Mongo documents. Works quickly, works well.
If I have documents like this:
{firstname:"Jordan", lastname:"Snyder", age:6, homelocation:[<longitude, latitude>]}
In the mongo shell, how do I all the "distinct" firstname's across matching documents of people who live near a specific point (say 1 mile)? I see mongo has a distinct db.collection.distinct(field, query), but all the samples I see for finding anything "near" or "geowithin" (using homelocation field in my case) is using db.collection.find. I don't want all documents, I just want the distinct list of firstnames.
The query parameter of distinct uses the same format as the query selector parameter of find. So assuming a 2dsphere index on homelocation you can do something like:
db.test.distinct('firstname', {
homelocation: {
$near: {
$geometry: { type: "Point", coordinates: [ -73.9667, 40.78 ] },
$maxDistance: 1600 // In meters
}
}
})
Let's say user finds documents with in Wellington New Zealand and provide coordinates of Wellington city: lat: -41.2864603, lng: 174.77623600000004.
But if there are no documents within this city I need to search documents within nearest cities within New Zealand country.
How could I find documents within country?
Most of the "heavy lifting" if this is really done by the general geospatial query itself, so long as you are using operations like $near or ideally $nearSphere with a 2dsphere index to allow for the curvature of the earth. Such queries basically find the "nearest" points to the queried "point" or GeoJSON object supplied and sort the results in the response that way.
Original implementations with the geoNear command had a default "limit" of 100 documents in the response and the ability to set the limit of documents in response. This is still generally valid as you would usually not want too many responses from such a query as there is a point where things are not "near" at all.
But the general case is that if you want to have additional query parameters in your logic then you can just add them to the existing query:
db.collection.find({
"$nearSphere": {
"$geometry": {
"type": "Point",
"coordinates": [ 174.77623600000004, -41.2864603 ]
}
},
"$or": [
{ "city": "Wellington" },
{ "country": "New Zealand" }
]
})
The logic there is that the queried objects must be "near" the queried location data and also either have data matching the "city" of "Wellington" $or matching "country" of "New Zealand".
You could alternately represent a complete "or" condition where the object did not necessarily match the geolocation condition, as long as the other query parameters matched something:
db.collection.find({
"$or": [
{ "$nearSphere": {
"$geometry": {
"type": "Point",
"coordinates": [ 174.77623600000004, -41.2864603 ]
}
}},
{ "city": "Wellington" },
{ "country": "New Zealand" }
]
})
That is of course if you "really want to" as it should not be needed generally speaking as the selected objects will already be returned ordered by the "nearest" results and there are other ways to "bound" those results to within a certain "distance". But if you really want to specify additional "bounds" then you can.
Of course when issuing either a $near or $nearSphere query or other variation, a "2d" or "2dSphere" index must be in place. The other constraint is that the data must either be represented in your document in either the form of legacy coordinate pairs or in GeoJSON format. In all cases the representation is <longitude>, <latitude>.
So just having fields present in the document to represent "longitude" and "latitude" is not enough in itself and you need a supported storage format along with the "index" to use the operators as shown.
The other data in the conditions can be contained in other indexes, following the guidelines for query selection with and $or condition, or within a compound index as along as that data meets the rules for complexity of what can be combined. It isn't necessary for additional data fields to be indexed, but it is usually the most optimal way to query with addition query parameters.
I'm just trying to get a simple near query working. Here's a sample of my document.
{"point":
{"type": "Point",
"coordinates": [30.443902444762696, -84.27326978424058]},
"created_on": {"$date": 1398016710168},
"radius": 180,
"user": {"$oid": "53543188eebc5c0cc416b77c"},
"_id": {"$oid": "53544306eebc5c0ecac6cfba"},
"expires_on": {"$date": 1399831110168}
}
and with mongod I tried the command:
db.bar.find({point: {$near: [-84.26060492426588, 30.45023887165371]}});
but I get this error:
error: {
"$err" : "Unable to execute query: error processing query: ns=foo.bar skip=0\nTree: GEONEAR field=point maxdist=1.79769e+308 isNearSphere=0 || First: notFirst: full path: point\nSort: {}\nProj: {}\n planner returned error: unable to find index for $geoNear query",
"code" : 17007
}
Maybe my google fu is not so sharp today but I couldn't find anything. Also, I ran the ensure index command. My intention is that these are map locations.
db.bar.ensureIndex({a:1});
db.bar.ensureIndex({geo:"2d"});
Few problems, you created your indexes on the foo collection of the foo database, but are querying the bar collection. You need to be on the correct collection.
Reading the document you have inserted you need to add a "2dsphere" index to support the geoJson objects. This index needs to be on the "point" element of your documents, so try
db.bar.createIndex({point:"2dsphere"});
You can then query as follows by providing a geoJson obj for the query:
db.bar.find(
{ point :
{ $near :
{
$geometry : {
type : "Point" ,
coordinates : [-84.27326978424058, 30.443902444762696] },
$maxDistance : 1
}
}
}
)
db.prod.createIndex({ "location": "2d" })
This solved for the same issue for me.
Where prod is my collection name and location is name of column which stores geo location (GeoPoint)
Some discussion about the same can be found here
So there seems to be a couple of things wrong here:
From the data you are showing and also your query information the relevant information is contained under the field point and in GeoJSON format. Your index creation:
db.foo.createIndex({geo: "2d"})
Does not "fail" because there presently isn't a field called "geo" and the field with the data should have been in that place. If you had used "point" instead, which is the correct field, then you would have received an error telling you that this type of index is invalid for the GeoJSON data. You need a "2dsphere" index:
db.points.createIndex({ "point": "2dsphere" })
Extending the same problem, again the data is in GeoJSON format and the form of the query is that for a legacy coordinate pair. You need to change the query arguments so that no longer fails:
db.points.find({point: {
$near: {
$geometry:{
type: "Point",
coordinates: [-84.26060492426588, 30.45023887165371]
}
}
}})
See the documentation for $near
In addition to the answers above, if you've already tried to create an Index and got some syntax or field wrong, you can run
db.<yourcollection>.dropIndexes();
To clean up all indexes and re-create them properly.
Also, the index should be created on the parent of "coordinates", not on coordinates itself:
{
"_id": 59ac03d168eaaa14c2a57a00",
"location":{
"type":"Point",
"coordinates":[
131.6667,
57.8368
]
},
"age":53,
"username":"Brandi_Greenfelder"
}
db.<yourcollection>.createIndex({ location: '2dsphere' });
Attention, there is "2d" and "2dsphere", use the second as it's the new thing.
If you're using mongoose to connect, this would be right answer:
db.collections.<yourcollection>.createIndex({ location : "2dsphere" })
Notice there is a "collections" property before collection itself. If it's not working, check db object in console.log:
console.log(db)