After upgrading MongoDb to 2.6.1 in my System n i get sometimes the following error:
Legacy point is out of bounds for spherical query
ErrorCode 17444
Here: https://github.com/mongodb/mongo/blob/master/src/mongo/db/geo/geoquery.cpp#L73
I can see that this is raised by mongo db due to some invalid data.
// The user-provided point can be flat. We need to make sure that it's in bounds.
if (isNearSphere) {
uassert(17444,
"Legacy point is out of bounds for spherical query",
centroid.flatUpgradedToSphere || (SPHERE == centroid.crs));
}
But currently i can't figure why and how to prevent it.
My Code looks like this:
IEnumerable<BsonValue> cids = companyIds.ToBsonValueArray();
return Collection.Find(
Query.And(
Query.In("CompanyId", cids),
Query.Near("Location", location.Geography.Longitude, location.Geography.Latitude, location.Radius / 6371000, true))).ToList();
Stacktrace:
QueryFailure flag was Legacy point is out of bounds for spherical
query (response was { "$err" : "Legacy point is out of bounds for
spherical query", "code" : 17444 }).
at MongoDB.Driver.Internal.MongoReplyMessage1.ReadFrom(BsonBuffer buffer, IBsonSerializationOptions serializationOptions)
at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument](BsonBinaryReaderSettings
readerSettings, IBsonSerializer serializer, IBsonSerializationOptions
serializationOptions)
at MongoDB.Driver.Operations.QueryOperation1.GetFirstBatch(IConnectionProvider
connectionProvider)
You're using MongoDB 2.6.1 or higher because the code you're looking at was added as a fix for a JIRA-13666 issue.
The problem was that some $near queries would crash MongoDB server when called with legacy coordinates that are out of range.
You're probably sending coordinates that are out of range. The part of the code that checks longitude and latitude when doing $near queries with max distance (GeoParser::parsePointWithMaxDistance method in geoparser.cpp):
bool isValidLngLat(double lng, double lat) {
return lat >= -90 && lat <= 90 && lng >= -180 && lng <= 180;
}
If the coordinates are out of range centroid.flatUpgradedToSphere will be false and that will cause the error you're receiving.
You should either change your coordinates to be in range or set spherical parameter to false to avoid getting this error.
Query.Near("Location", location.Geography.Longitude,
location.Geography.Latitude, location.Radius / 6371000, false)
Related
I have an application that stores items with a latitude and longitude, I wanted to create a query that filters the items and paginates them by their distance to a given lat/lon pair.
I have read online and a lot of the solutions don't seem feasible within the constraints of fluent.
You'll want to use PostGIS for any kind of Geometric/Geographic query. Luckily there's a package for that! https://github.com/brokenhandsio/fluent-postgis
You can do filterGeograghyWithin() to find items within a certain distance. Note the geographic queries are better suited for these kind of transformations (and more accurate) as they take into account the curvature of the Earth.
The downside is you'll need to convert your lat/lon columns to a GeographicPoint2D which is the native PostGIS type, but given PostGIS is the de facto standard for this kind of work it's worth doing anyway
The best you could do is to calculate the angular distance (ang) from your given latitude (lat) and longitude (lng), and then select the square region that will include the circle. Then use Pythagoras to filter those lying within the required distance:
let ang2 = pow(ang, 2)
Point.query(on: req.db)
.filter(\.$lng >= lng - ang).filter(\.$lng <= lng + ang)
.filter(\.$lng >= lat - ang).filter(\.$lat <= lat + ang).all().flatMap { points in
let closePoints = points.filter { pow($0.lat - lat, 2) * pow($0.lng - lng, 2) <= ang2) }
// continue processing
}
}
EDIT: Following the OPs disclosure that he wants to paginate the results!
Capture the id values of the final set of Points and then filter on this. Something like:
let ang2 = pow(ang, 2)
Point.query(on: req.db)
.filter(\.$lng >= lng - ang).filter(\.$lng <= lng + ang)
.filter(\.$lng >= lat - ang).filter(\.$lat <= lat + ang).all().flatMap { points in
return points.filter { pow($0.lat - lat, 2) * pow($0.lng - lng, 2) <= ang2) }.map { $0.id }
}.flatMap { ids in
Point.query(on: req.db).filter(\.$id ~~ ids).paginate().flatMap { points in
// continue processing
}
}
}
So currently I am using geohashes to do location based queries as such (following this stackoverflow post: Finding geohashes of certain length within radius from a point)
public extension CLLocationCoordinate2D {
func boundingBox(radius: CLLocationDistance) -> (max: CLLocationCoordinate2D, min: CLLocationCoordinate2D) {
// 0.0000089982311916 ~= 1m
let offset = 0.0000089982311916 * radius
let latMax = self.latitude + offset
let latMin = self.latitude - offset
// 1 degree of longitude = 111km only at equator
// (gradually shrinks to zero at the poles)
// So need to take into account latitude too
let lngOffset = offset * cos(self.latitude * .pi / 180.0)
let lngMax = self.longitude + lngOffset
let lngMin = self.longitude - lngOffset
let max = CLLocationCoordinate2D(latitude: latMax, longitude: lngMax)
let min = CLLocationCoordinate2D(latitude: latMin, longitude: lngMin)
return (max, min)
}
func isWithin(min: CLLocationCoordinate2D, max: CLLocationCoordinate2D) -> Bool {
return
self.latitude > min.latitude &&
self.latitude < max.latitude &&
self.longitude > min.longitude &&
self.longitude < max.longitude
}
}
func getGeohashPrefix(){
let loc = CLLocationCoordinate2D(latitude: lat!, longitude: long!)
MBR = loc.boundingBox(radius: 16093.4) //16093.4 meters = 10 miles
//corners = [NorthWest, SouthWest, SouthEast, NorthEast] in lat n long
let corners = [CLLocationCoordinate2D(latitude: MBR.0.latitude,longitude: MBR.1.longitude),
MBR.1, CLLocationCoordinate2D(latitude: MBR.1.latitude, longitude: MBR.0.longitude),
MBR.0]
var geohashes_of_corners: [String] = []
for corner in corners {
geohashes_of_corners.append(corner.geohash(length: 12))
}
geohashes_prefix = geohashes_of_corners.longestCommonPrefix()
}
var query: Query = db.collection("Users").whereField("geohash",isGreaterThanOrEqualTo: geohashes_prefix).whereField("geohash",isLessThanOrEqualTo: geohashes_prefix + "~").order(by: "geohash", descending: false)
query.getDocuments { (querySnapshot, err) in
if err != nil{
print("error getting da documents")
}else{
if querySnapshot!.isEmpty{
return completion(arr_of_people)
}
for document in querySnapshot!.documents {
let d = document.data()
let isPersonWithin = CLLocationCoordinate2D(latitude: (d["loc"] as! GeoPoint).latitude, longitude: (d["loc"] as! GeoPoint).longitude).isWithin(min: self.MBR.1, max: self.MBR.0)
if !isPersonWithin{
continue
}
arr_of_people.append([d["firstName"] as! String, d["lastName"] as! String])
}
return completion(arr_of_people)
}
}
As you can see, I am querying for documents with a specific prefix and then filtering those documents AGAIN on the client. Is that safe? If not, what is the workaround? Use cloud functions, different algorithm (suggest one if you have it), or something else?
A query on geohashes returns points within a certain range of geohashes, which are (somewhat) rectangular regions.
A geoquery on a central point and a distance returns points that are in a circle.
Since the two shapes are different, your code uses a client-side check to cut off the points that are outside the circle, but inside the rectangles. This is a normal step when performing geoqueries for a max distance around a point when using geohashes.
Here's an example of this on a map:
The green pins are in a 250km circle around San Francisco, which is what I queried for. The red pins are outside of that circle, but within the set of geohash ranges ([["9q0","9qh"],["9nh","9n~"],["9r0","9rh"],["9ph","9p~"]] here) that needed to be queried to be sure we had all points in range.
As said: this so-called oversampling is inherent to using geohashes to perform point-and-distance queries. In my experience you'll end up reading between 2x and 8x too many documents.
It might be possible to reduce the number of false positives by querying for more-but-smaller ranges, but I don't know of any geo-libraries for Firestore that do that.
I mapped the extra cost as a mental model: finding documents within a certain distance from a point costs me 2x to 8x more than a regular document read.
Moving the operation to Cloud Functions on the server makes no difference on the number of documents that needs to be read, it just changes where they are read. So you can perform the operation on a server to reduce the bandwidth of transferring the documents from the database to the client. But it won't make a difference in the number of documents that need to be read.
As discussed in comments: performing the query on the server does allow you to remove client-side access from the data, so that you can ensure the application code will never see documents that are not within the requested range. So if you're worried about document access, performing the query in a trusted environment (such as a server you control or Cloud Functions) is a good option.
To not pay for the extra document reads, consider finding a service that natively supports geoqueries with a pricing model based on the number of results it returns - instead of the number of results it has to consider. Such a service will (very likely) still consider too many points, but if the pricing model matches what you want that might be worth it.
While trying to experiment with mongo performance I found strange behaviour of my mongo db.
First of all i filled it with the following query:
for (i=0; i < 10000000; i++){db.location.insert( {_id: Math.floor((Math.random()*10000000000)+1), position: [Math.round(Math.random()*10000)/10000, Math.round(Math.random()*10000)/10000]} )}
next:
db.location.ensureIndex( {position: "2d"} )
Then i execute query:
db.location.find( {position: { $near: [1,1], $maxDistance: 1/111.12 } } )
Whatever i try to do i get always size or count result 100.
I noticed in documentation that defualt limit is 100. I tried also to override it with bigger than 100 values. Unfortunately I failed.
Have you ever encountered such a case?
To get all document query like
cordinate = [1,1];
maxDistance = 1/111.12 ;
db.location.find({"position" : {"$within" :
{"$center" : [cordinate , maxDistance ]}
}
});
From oficial documentation:
The $near operator requires a geospatial index: a 2dsphere index for GeoJSON points; a 2d index for legacy coordinate pairs. By default, queries that use a 2d index return a limit of 100 documents; however you may use limit() to change the number of results.
And also look at 'Note' in the and of this tutorial page.
Update:
As Sumeet wrote in comment to his answer - it is open issue.
For be sure, that your query return correct count, that you specifying in limit method, you could try to use .limit(<some_number>).explain().n with your cursor, if you working in the shell.
Just tried the same scenario on my MongoDB and it works fine. I gave limit of 145 and it gave me 145 records. However as mentioned in the documentation it give me 100 records if I do not specify any limit.
db.location.find( {position: { $near: [1,1], $maxDistance: 1 } } ).limit(145)
I tried using above statement.
Please note that I changed the value of $maxDistance so that I get lot of records.
I am trying to use spatial search in my android app. I am new to mongodb and spatial search.
I am trying to implement this query in command prompt but it displays nothing
db.places.find( { loc: { $geoWithin : { $center : [ [x, y], 10 ] } } } )
Inplace of x and y i wrote longitude and latitude.
I want to do spatial search around a particular longitude and latitude. What should i write in loc, x, y?
Please help!!!!
Thanks in advance.
Look at this:
http://docs.mongodb.org/manual/reference/operator/center/
Check the http://docs.mongodb.org/manual manual, it helps a lot.
I have objects like the following:
{
"_id" : ObjectId("4f7f0d64e4b0db1e18790f10"),
"location" : [
0.674081,
23.473
],
"name" : "Item Name"
}
If I try and create a 2D index on the location field, I get an error:
> db.Place.ensureIndex({location:"2d"});
point not in interval of [ -180, 180 )
I've tried finding documents that are out of bounds but this returns no results:
> db.Place.find({"location.0":{"$lte":-180, "$gte":180}});
> db.Place.find({"location.1":{"$lte":-180, "$gte":180}});
I've also tried the following queries to make sure:
> db.Place.find({"location":{"$size":0}}).count();
0
> db.Place.find({"location":{"$size":1}}).count();
0
> db.Place.find({"location":{"$size":2}}).count();
363485
> db.Place.count();
363485
What other approaches can I use to find the bad document(s)?
For information I'm using MongoDB version 2.0.2 on 32 bit Linux (yes, I know that's not good enough for prod). The documents were imported by a Java document where the location objects are only ever represented as double primitives.
Your queries aren't quite right. You need:
db.Place.find({"location": {"$lte":-180 } })
And
db.Place.find({"location": {"$gte":180 } })
I've run into a similar issue, and I've been able to pinpoint that ensureIndex breaks when one of lat/lon are exactly equal to -180 or 180. Look for those cases if you want to get around this issue, otherwise it looks like there's a fix: https://github.com/mongodb/mongo/commit/86b5b89c2785f3f45ad2674f48fe3189c137904c