I have a collection of (X,Y,Z) points in Mongo database - is there any way to precisely query for points within given distance of some coordinate? For example, within 50 units of 0,0,0.
The most prevalent answer seems to be to strip the least significant coordinate and use 2D $near query, but this approach would lead to unacceptable loss of precision with my data set.
Are there any plugins available for 3D queries? Or some clever query tricks? Failing that, could you suggest some external service like elasticsearch that works well with 3D geometry?
If it changes anything, I'm dealing with ~10 pre-set distances, couple thousand points that change relatively rarely, always querying for points within a distance from another point, and reaching maximum ~20% of other points from any given one at the biggest distance. So pre-computing is an option, but I'd like to avoid it if possible.
Related
I have two large databases (500k to 3M rows) inside PostgreSQL, each containing a set of GPS lat longs. I need to compare all coordinates in one database with that of the other database, and find points that are within 300m of each other.
I started using PostgreSQL as I had heard about its spatial indexing which speeds up geometry-related tasks a lot. The idea is to use spatial indexing e.g. R Trees, to only check nodes that were already determined to be close to each other, instead of checking the entire database every time O(n^2)
However, I couldn't find anything related to this.
*Edit: I am not looking for a distance calculation algorithm, I am looking for optimizations to speed up the comparison of locations in my 2 tables. So it is not a duplicate question.
Just giving a tip here.
Distance is a result of a static function (calculation) that has input the 2 sets of long and lat.
Work with the formula you can find on the internet to find the deviation (sum of lat and long) bigger than which the distance is certainly greater than 300 km .
This deviation will be bigger the closer the point to the poles is and will be lesser the closer the point to the equator is. So divide the coords in 3-4 different areas based on how close to the equator they are, then calculate how different the other coords must be in order to be certain that distance is higher than 300km and then work only with the remaining coords.
that would drastically improve the performance of any type of search you plan to perform.
good luck!
Given an array of points in 3D space, I need to find the closest point to a given point p.
I am looking to optimize performance for this search. Currently, I am using square of magnitude as a metric to compare distances, because of the obvious optimization in comparison to Euclidean distance (which requires the calculation of the square root).
I am aware of the existence of the Manhattan distance. However, I am not sure whether this metric could be used in this case. I need to do this evaluation correctly.
Is there a better way of achieving this calculation, performance wise? Perhaps storing the points in a specialized data structure, or using a different metric?
I read the related question (Fastest way to find the closest point to a given point in 3D, in Python), but am still wondering about the optimal metric for comparison.
In Matlab I would like to calculate the (shortest) distances between a set of independent points (m-by-2 matrix of lat/lng) and a set of polylines (n-by-2 matrix of lat/lng). The resulting table should be an n-m matrix with distances in KM.
I have rewritten this JavaScript implementation (http://www.bdcc.co.uk/Gmaps/BdccGeo.js) to Matlab, but it does not seem to perform well.
Currently I am working on a project with a relatively large set of data and running into performance issues. I have roughly 40.000 points and 150 polylines. The polylines are subsets of the original set of 40.000 points. With about 15 seconds per polyline, calculating all these distances can take up to an hour. Also, the intermediate matrixes of 40000x150x3 cause out of memory errors on my lesser machines.
Instead of optimizing or revising this implementation I am wondering if Matlab doesn't already have some (smarter) functions built in for this. But as far as I can see, the documentation mainly has information on how to display geodata as opposed to doing calculations on it.
Does anyone know or have experience with these kind of calculations in Matlab? Has anything like this already been written which I can reuse so I don't have to reinvent the wheel. And finally, is this expected performance, given these numbers, or should my function be able to perform much better?
I have an unorganized point cloud and I want to compare ideal CAD geometry and profile measurement of an object. For example, I have a CAD data of ideal object and I have a point cloud like this;
How can I compare these two data? I know from CAD file, is point on the CAD data belongs to line or radius(Arc), but how can I derive radius error of an Arc or length error of a Line?
I tried to organize data with knnsearch but results are not satisfying.So, I tried to draw a line starting from a one point ( lets Say point 1) and I want to go next closest point ( Lets say Point 2). If closest neighbour of Point 2 is Point 1, then go to second closest point of Point 2. That algorithm seems to good for me but results are not satisfying also. Connection lines went one edge to other.
I also thought that, may be I should convert CAD data to point cloud and I have to compare each point of measurement with closest point on CAD point cloud. I know which points belong to line and which point belong the Arc and I can calculate mean error from a line or Arc. But end points of lines or arcs will be trouble I think. Comparison results at these points will have large error I think.
On the other hand, CAD geometry and measurements will not be convex and perfectly covered always. Some non-convex geometries can be measured. For example, you can see measurements of inverted V shape with lack of some points. It is the worst case;
If there are some errors on geometry estimation when measurements are not enough, it is acceptable for me.
CPU load is also important criteria for me. There are 10.000 points and I want to complete filterings and geometry matchings in 20 ms with i7 processors.
Are there any robust solution for this aim?
Ok, I'm answering my question. Matlab has built-in functions for computational geometry.
Boundary function of this module has solved my problem partially. I can use it for non-convex geometries. For convex geometries, I'm calculating center point of geometry by simple avaraging of points. Then, I'm sorting all points by atan2 function.
But I can't figure out how can I find geometries. One way is using CAD data and iterate described geometries in CAD data to minimize least square error between point cloud. Other way is, creating arc and lines from points directly. I dont which way will be faster and more robust.
I am working on a project where the following functions has to be implemented.
Predict the location of the ships (in maritime environment) into a future time (Can be done with Kalman filter, IMM filter and some other algorithms).
Ships can be any part of the world.
Avoiding landmass during prediction
Shortest path along the shorelines
I am totally done with the first part which is predicting without considering the shoreline information. I have
problem with the functions 2 and 3.
Problem in function 2
At times, your predicted location can fall into the landmass area which is totally unacceptable.
I am using following coastal area shp file http://openstreetmapdata.com/data/coastlines
This file has converted XY values of the world shoreline data.
I have loaded this shp file into postgreSQL and used postgis to read it from the database.
So my idea is to go through all the polygons (shoreline defined based on polygons) and checking whether the line connecting the present location and the predicted location
crosses the polygon. If it crosses, that means we have to find the where the ship intercept the shoreline first.
So if I follow this approach going through all the polygons, it is going to take time forever. (It has around 62000 polygons with each of them has 1000's of
points). So any advice on this? I thought about initially dividing the worldmap into hierachical areas (Level 1 : 10 polygons, Level 2: Each polygon has 10 polygons inside).
But I am not sure how to divide the world map with the above shp file into the level of polygons I require.
Or any functionality of postgis helpful for this? or any other libraries for this purpose. I believe this kind of functionality should be available already. But I could not
able to figure it out sofar.
Function 3
Since now we know where does the ship intercept the shoreline first, we can predict it along the shoreline using the shortest path algorithm given we know
the destination information. But to do this, you need to divide the above shoreline map into grids so the shortest path can be used.
So how can you make grids based on this along the shorelines? I am not doing image processing here. What I have is this shp file now. Any advice is appreciated.
or should I go with some image processing approach and make the grid shorelines. if so please provide some links.
First, PostGIS is pretty fast, and with the proper indexes, as long as you keep your polygons reasonably small, you should be able to make up for the number of them with good indexing and overlapping operator support (overlapping polygons can use GIST and GIN indexes, with the latter performing better than the former for reads and worse for writes).
62000 polygons globally is nothing. Write back when you are having to check more than a few thousand whose bounding boxes overlap with your line....
For the third problem, you are going one direction, right? I am wondering how hard it would be to write a tangent(point, vector, polygon) function which would return the closest tangent to a polygon along a certain vector (a vector could be represented by a (point, point) tuple). If you were to combine this with KNN searches, you ought to be able to plot a course using a WITH RECURSIVE query.