MongoDB geospatial query with find() - mongodb

I have some documents with a "loc" field that looks like this:
mongos> db.foo.findOne()
{
// ... some other stuff
"loc" : {
"type" : "Point",
"coordinates" : [
-83.362342,
26.687779
]
}
}
It's indexed like so:
mongos> db.foo.getIndices()
[
// ... some other stuff
{
"v" : 1,
"key" : {
"loc" : "2dsphere"
},
"name" : "loc_2dsphere",
"ns" : "amitest.foo",
"2dsphereIndexVersion" : 2
},
]
According to the docs I should be able to query it like the following, but I get the error shown:
mongos> db.foo.find(
{loc :
{$near:
{$geometry:
{type: "Point",
coordinates: [-83.362342, 26.687779]
},
$maxDistance: 100
}
}
}
).limit(5)
error: { "$err" : "use geoNear command rather than $near query", "code" : 13501 }
I can successfully query the field using runCommand(), but that's not ideal because I can't combine it with other criteria:
mongos> db.runCommand(
{geoNear : "foo",
near : {type : "Point",
coordinates : [-83.362342, 26.687779]
},
spherical : true,
maxDistance : 10,
limit : 5
}
)
I've got MongoDB 2.6.0, on a hash-sharded collection with 8 shards.

In the geospatial index documentation, it states that: "For sharded collections, queries using $near are not supported. You can instead use either the geoNear command or the $geoNear aggregation stage."
There is a bug related to this, https://jira.mongodb.org/browse/SERVER-926, which is marked as closed and targets a future release of MongoDB, 1.7.2, which will enable the $near operator to be used with sharded collections. However, note, there is a big but which is that near queries will be routed to all shards via mongos, which is likely to be very inefficient. This is because sharding is not supported on a geo column in general, see the related and still open bug https://jira.mongodb.org/browse/SERVER-1982
This, in turn, is related to to the fact that MongoDB uses geohashing to convert two dimensional geographical objects to something that can be both indexed using a B-tree and something that could, in theory, be used as a shard key. It is a difficult problem to fix, as with geohash indexes, objects very close together, can end up with hash values very far apart, and so it is difficult to design something that is both appropriate to use as a shard key, but also supports efficient geospatial querying such as $near. See the limitations section in the Wikipedia geohash article for more on potential issues with geohashing.

Related

Geospatial index querying doesn't work on Azure CosmosDB (DocumentDB) using the Mongo API

I have an Azure CosmosDB with a collection restaurants where a field geoJSON specifies the location of a restaurant in geoJSON format. I am using the MongoDB API to access this.
When I log into the DB using the mongo shell, I am able to see all the documents using db.restaurants.find(). I created a 2dsphere geospatial index using db.restaurants.createIndex( {geoJSON: "2dsphere"}).
output:
{
"_t" : "CreateIndexesResponse",
"ok" : 1,
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 3,
"numIndexesAfter" : 4
}
However, upon running db.restaurants.getIndexes(), I still see only three indexes. This question says that indexing is not allowed. Is this still the case?
Furthermore, there seems to be an index called
{
"v" : 1,
"key" : {
"DocumentDBDefaultIndex" : "2dsphere"
},
"name" : "DocumentDBDefaultIndex_2dsphere",
"ns" : "<redacted>"
}
,but executing a geospatial query returns nothing.
db.restautants.find({DocumentDBDefaultIndex: {$near: {$geometry: {type: "Point", coordinates: [24.9430111, 60.166608]}, $maxDistance: 1000}}})
How can I execute geospatial queries against this index using the mongo shell? Is this a bug in CosmosDB?
According to your description, I checked this issue on my side. For a quick way, I use MongoChef with the Azure Cosmos DB. Based on my test, db.brucetest1.createIndex({loc:"2dsphere"}) could not be applied to the collection. Moreover, I found DocumentDB automatically creates a 2dsphere index on the location field DocumentDBDefaultIndex, then I drop my documents, and insert the documents as follows:
db.brucetest.insertMany([
{DocumentDBDefaultIndex : { type: "Point", coordinates: [ -73.97, 40.77 ] },name: "Central Park",category : "Parks"},
{DocumentDBDefaultIndex : { type: "Point", coordinates: [ -73.88, 40.78 ] },name: "La Guardia Airport",category : "Airport"}
])
With the following query, I could encounter your issue:
db.brucetest.find({DocumentDBDefaultIndex:{$near:{$geometry:{type:"Point",coordinates:[-73.88, 40.78]}}}})
Based on the similar issue your mentioned, I assumed that the creating/updating index and Geospatial indexes querying features have not been implemented in the MongoDB Compatibility layer of Azure CosmosDB. Moreover, you could add your feedback here or wait for these features released.
Geospatial IS currently supported with the MongoDB API. Please see the Geospatial operators section here.

What is a proper Pub/Sub for the nearest documents sorted by custom fields?

In my Meteor project, I have documents like this:
{
"_id" : "cgR25FAxb3tbYShjN",
"owner" : "6mQKnNnwkQYSaaAMr",
"username" : "admin",
"filename" : "5sQqwbDFBcZZ.png",
"title" : "Title",
"points" : 2,
"loc" : {
"type" : "Point",
"coordinates" : [
-119.981134343,
20.635934343
]
},
"createdAt" : ISODate("2016-09-15T10:27:20.956Z")
}
I am able to find the nearest documents by this query:
Photos.find({loc: {$near: { $geometry: {type: "Point", coordinates: [lng, lat]}}}});
What I want to be able to do is sort all nearby documents by something like points or createdAt. I am able to do this if I publish all documents at once.
The Problem:
Publishing 200+ documents at once isn't ideal in the least bit. I want to be able to limit the amount being returned to the client and request more as I need it. The problem is: if I do that with a simple {limit: 20} in the query, there will probably be a document that is a little farther away that has more points. This will basically screw up the all sorting.
I considered finding all documents on the server at once, then sorting and returning 20 at a time and so on, but won't this put a huge load on my server? Storing 200+ documents in memory per user doesn't seem like a good idea either.
Update:
The more I think about this the more I realize that it's a requirement to do a large query on the server first, then pass the data to the client manually. Even if I did a $geoWithin look up, if I'm limiting the pub to 20 documents, they will still be out of order when I sort them because they are fetched at different times.
You need to specify $minDistance or $maxDistance too along with the query.
Photos.find({loc: {$near: { $geometry: {type: "Point", coordinates: [lng, lat]}},$maxDistance: 0.01}});
The 0.01 in the query is in radians.
Refer docs.

mongodb geospacial query : $centerSphere not giving good result

I have a db with geodata (lng,lat) from Europe.
I'm trying to do a geospatial query to get all element within a circle.
So, i create a GeoJSON field, and a 2dsphere index on it.
This look like that :
{
"_id" : ObjectId("56c3484612aeb853a83ec336"),
"defaultLabel" : "Zoo de Mulhouse",
"weight" : NumberInt(1),
"country" : "FR",
"loc" : {
"type" : "Point",
"coordinates" : [
47.731673,
7.347819
]
}
}
and db.myCollection.createIndex({"loc":"2dsphere"});
So far, so good.
But then i tried some queries to check.
I took a point at ~1,12 km (1120m) from the data, and did a request with a radius of 1200m.
db.poi.find({loc:
{$geoWithin:
{$centerSphere:[
[47.728593,7.333486],
1200 / 6378100] // radius of earth ~6378.1Km
}
}
})
I got no result.
With the same request, i have to put about 1650m to get the result.
Obviously, this is not acceptable.
So, what did I do wrong?
Found the problem : coordinates were stored as [lat,lgn] in my legacy, and i didn't check before create my GeoJSON, so it was in the wrong order.

$and with $nearSphere in mongodb

I have a collection having from and to point locations. Now I wish to find documents which have both, to and from locations nearby the given source and destinations.
Here's the setup:
collection: db.t2.find():
{
"_id" : ObjectId("5..4"),
"uid" : "sdrr",
"valid_upto": 122334,
"loc" : {
"from" : {
"type" : "Point",
"coordinates" : [ 77.206672, 28.543347 ]
},
"to" : {
"type" : "Point",
"coordinates" : [ 77.1997687, 28.5567278 ]
}
}
}
Indices: db.t2.getIndices():
{
"v" : 1,
"name" : "_id_",
"key" : {
"_id" : 1
},
"ns" : "mydb.t2"
},
{
"v" : 1,
"name" : "uid_1_loc.from_2dsphere_loc.to_2dsphere_valid_upto_1",
"key" : {
"uid" : 1,
"loc.from" : "2dsphere",
"loc.to" : "2dsphere",
"valid_upto" : 1
},
"ns" : "mydb.t2"
}
Single queries for either to or from work good with the current settings give nice results. However, when I use to and from together in a single query with $and clause:
db.t2.find({
"$and" : [
{
"loc.from" : {
"$nearSphere" : [ 77.5454589,28.4621213 ],
"$maxDistance" : 0.18
}
},
{
"loc.to" : {
"$nearSphere" : [ 77.206672, 28.543347 ],
"$maxDistance" : 0.18
}
}
]
})
it throws the following error:
error: {
"$err" : "can't find any special indices: 2d (needs index), 2dsphere (needs index), for: { $and: [ { loc.from: { $nearSphere: [ 77.5454589, 28.4621213 ], $maxDistance: 0.18 } }, { loc.to: { $nearSphere: [ 77.206672, 28.543347 ], $maxDistance: 0.18 } } ] }",
"code" : 13038
}
I suppose the data has been indexed as evident from getIndices(), but still its unable to find indices! Where is the problem then and how can I fix it to have effect of a $and-ed operation?
The error appears to be present from a MongoDB 2.4 version where there indeed was a bug that would not allow a $near type of query within and $and operation that accessed another field.
But your particular problem here is that you just cannot do this.
The code and comments to test this can be vied on GitHub but essentially:
// There can only be one NEAR. If there is a NEAR, it must be either the root or the root
// must be an AND and its child must be a NEAR.
size_t numGeoNear = countNodes(root, MatchExpression::GEO_NEAR);
if (numGeoNear > 1) {
return Status(ErrorCodes::BadValue, "Too many geoNear expressions");
}
So that is an error that would be emitted from MongoDB 2.6 you tried to do this.
A brief look at all the surrounding code within the method will show you that "geo" queries are not alone in this and the other "special" index type of "text" is included in the same rules.
Part of the reason for this is the $meta "scoring" that is required, as in this case is $maxDistance. There really is no valid way to combine or discern which value would actually apply in combined results such as this.
On a bit more of a technical note, the other issue is with being able to "intersect" indexes in a query such as this. The required fuzzy matching makes this a very different prospect to something like the basic "Btree" index intersection.
For now at least, your best approach is to perform each query by itself and manually "union/intersect" your results in code, with of course your own tagging as to which results are for your origin and which are for your destination.
This was a known issue in version 2.4 and prior of MongoDB, fixed in version 2.5.5:
https://jira.mongodb.org/browse/SERVER-4572
Core ServerSERVER-4572 Geospatial index cannot be used in $and
criteria of a query?
Should be fixed as of 2.6 - if you're running 2.4 or previous I'd upgrade, if you're running 2.6.X I'd report it as a bug.

spring mongo support for geoJSOn for 2dsphere index

Mongo though provides 2dsphere index on legacy co-ordinates, the query requires to present to Point/Shapes in geoJSON format. For e.g., I have inserted the following records to address collection.
{ "city" : "First", "geo" : [ 13.45, 23.46 ] }
{ "city" : "Second", "geo" : [ 13.45, 20.46 ] }
Then I added 2dsphere index using following command as mongodb still allows 2dsphere index on legacy co-ordinates.
db.address.ensureIndex({"geo":"2dsphere"})
Then if I do $near query using legacy format, but got an exception.
> db.address.find({"geo":{$near:{"x":13.45,"y":23.45}}})
error: {
"$err" : "can't parse query (2dsphere): { $near: { x: 13.45, y: 23.45 } }",
"code" : 16535
}
But If do same query with geoJSON format, then I get result.
> db.address.find({"geo":{$near:{"type":"Point",coordinates:[13.45,23.45]}}})
{ "_id" : ObjectId("537306b4b8ac1f134d9efe89"), "city" : "First", "geo" : [ 13.45, 23.46 ] }
{ "_id" : ObjectId("537306c3b8ac1f134d9efe8a"), "city" : "Second", "geo" : [ 13.45, 20.46 ] }
My question is, GeoConverters has all conversion made to legacy format. So, obviously they wont' work if I use 2dsphere index. Are there any converts available for geoJSON format. Is there any workaround?
Currently spring-data-mongo doesn't support the new mongo (> 2.4) 2dsphere indexes. There is a open issue on Jira about it:
https://jira.spring.io/browse/DATAMONGO-1113?jql=project%20%3D%20DATAMONGO%20AND%20text%20~%20%22%24geometry%22
In the link you can find a gist link to example of how create such converters. You can use it or you can overcome this limitation creating a #Query with the query that you want that spring-data-mongo execute.
Regards.
avaz