I have an app that will register the GPS coordinates on each check in made by the user. If the user comes back to make a new check in, as a speed saving option, we will show the closest place he had checked in.
Each check in will save the current position of the user, regardless if it is the first or 100th check in.
After some time we have a huge list of coordinates that we need to check each time.
How can I collapse this information? I want to turn a huge list of latitudes and longitudes into one or more polygons so I can speed up distance calculations.
# So this
1:0 100:200 2:0 2:2 2:1 1:1 2:1 0:1
# Could become this
100:200 1:1
# Or this
100:200,100:200 0:0,2:2
This is called the nearest neighbor search problem. You can have a look at the Wiki article which mentions faster algorithms. It seems that space partitioning with an R-Tree is the best approach.
Related
I'm a complete illiterate when it comes to working with geographical data, so bear with me.
For our application we will be tracking a fairly large amount of rapidly changing points on a map. It would be nice to be able to cache the location of these points in some kind of map-tile structure so it would be easy to find all points currently in the same tile or neighbouring tiles, making it easier to quickly determine the nearest neigbours and have special logic for specific tiles, etc.
Although we're working for one specific (but already large) location, it would be nice if a solution would scale to other locations as well. Since we would only cache tiles that concern the system, would just tiling the enitre planet be the best option? The dimensions of a tile would then be measured in arc seconds/minutes, or is that a bad idea?
We already work with Postgres and this seems like something that could be done with PostGIS (is this what rasters are?), but jumping in to the documentation/tutorials without knowing what exactly I'm looking for is proving difficult. Any ideas?
PostGIS is all that you need. It can store your points in any coordinate reference system, but you'll probably be using longitude/latitude. Are your points coming from a GPS device?
PostGIS uses GIST indexing, making the search for points close to a given point quite efficient. One option you might want to look at, seeing that you are interested in tiling, is to "geohash" your points. Basically, this turns an (X,Y) coordinate pair into a single "string" with a length depending on the level of partitioning. Nearby points will have the same geohash value (= 1 tile) and are then easily identified with standard database search tools. See this answer and related question for some more considerations and an example in PostgreSQL.
You do not want to look at rasters. These are gridded data, think aerial photography or satellite images, weather maps, etc.
But if you want a more specific answer you should give some more details:
How many points? How are they collected?
Do you have large clusters?
Local? Regional? Global?
What other data does this relate to?
Pseudo table structure? Data layout?
etc
More info = better answer
Cheers, hope you get your face back
I have a table that contains a bunch of Earth coordinates (latitude/longitude) and associated radii. I also have a table containing a bunch of points that I want to match with those circles, and vice versa. Both are dynamic; that is, a new circle or a new point can be added or deleted at any time. When either is added, I want to be able to match the new circle or point with all applicable points or circles, respectively.
I currently have a PostgreSQL module containing a C function to find the distance between two points on earth given their coordinates, and it seems to work. The problem is scalability. In order for it to do its thing, the function currently has to scan the whole table and do some trigonometric calculations against each row. Both tables are indexed by latitude and longitude, but the function can't use them. It has to do its thing before we know whether the two things match. New information may be posted as often as several times a second, and checking every point every time is starting to become quite unwieldy.
I've looked at PostgreSQL's geometric types, but they seem more suited to rectangular coordinates than to points on a sphere.
How can I arrange/optimize/filter/precalculate this data to make the matching faster and lighten the load?
You haven't mentioned PostGIS - why have you ruled that out as a possibility?
http://postgis.refractions.net/documentation/manual-2.0/PostGIS_Special_Functions_Index.html#PostGIS_GeographyFunctions
Thinking out loud a bit here... you have a point (lat/long) and a radius, and you want to find all extisting point-radii combinations that may overlap? (or some thing like that...)
Seems you might be able to store a few more bits of information Along with those numbers that could help you rule out others that are nowhere close during your query... This might avoid a lot of trig operations.
Example, with point x,y and radius r, you could easily calculate a range a feasible lat/long (squarish area) that could be used to help rule it out if needless calculations against another point.
You could then store the max and min lat and long along with that point in the database. Then, before running your trig on every row, you could Filter your results to eliminate points obviously out of bounds.
If I undestand you correctly then my first idea would be to cache some data and eliminate most of the checking.
Like imagine your circle is actually a box and it has 4 sides
you could store the base coordinates of those lines much like you have lines (a mesh) on a real map. So you store east, west, north, south edge of each circle
If you get your coordinate and its outside of that box you can be sure it won't be inside the circle either since the box is bigger than the circle.
If it isn't then you have to check like you do now. But I guess you can eliminate most of the steps already.
I have a database with the current coordinates of every online user. With a push of a button the user can update his/her coordinates to update his current location (which are then sent off to server). The app will allow you to set the radius of a circle (where the user is in the center) in which you can see the other users on a map. The users outside the circle are discarded.
What is the optimal way to find the users around you?
1) The easiest solution is to find the distance between you and every user and then see if it's less than the radius. This would place the sever under unnecessarily great load as comparison has to be made with every user in the world. In addition, how would one deal with changes in the locations?
2) An improved way would be to only calculate and compare the distance with other users who have similar latitude and longitude. Again in order to be efficient, if the radius is decreased the app should only target users with even closer coordinates. This is not as easy as it sounds. If one were to walk around the North Pole with, say, 10m radius then every step around the circumference would equal to a change of 9 degrees longitude. Every step along the equator would be marginal. Still, even being very rough and assuming there aren't many users visiting the Poles I could narrow it down to some extent.
Any ideas regarding finding users close-by and how to keep them up to date would be much appreciated! :)
Andres
Very good practice is to use GeoHash concept (http://geohash.org/) or GeoModel http://code.google.com/p/geomodel/ (better for BigTable like databases). Those are efficient ways of geospatial searches. I encourage you to read some of those at links I have provided, but in few words:
GeoHash translates lon and lat to unique hash string, than you can query database through those hashes. If points are closer to each other similar prefix will bi longer
GeoModel is similar to GegoHash with that difference that hashed are squares with set accuracy. If square is smaller the hash is longer.
Hope I have helped you. But decision, which you will pick, is yours :).
Lukasz
1) you would probably need a two step process here.
a) Assuming that all locations go into a database, you can do a compare at the sql level (very rough one) based on the lat & long, i.e. if you're looking for 100m distances you can safely disregard locations that differ by more than 0.01 degree in both directions. I don't think your North Pole users will mind ;)
Also, don't consider this unnecessary - better do it on the server than the iPhone.
b) you can then use, for the remaining entries, a comparison formula as outlined below.
2) you can find a way to calculate distances between two coordinates here http://snipplr.com/view/2531/calculate-the-distance-between-two-coordinates-latitude-longitude/
The best solution currently, in my opinion, is to wrap the whole earth in a matrix. Every cell will cover a small area and have a unique identifier. This information would be stored for every coordinate in the database and it allows me to quickly filter out irrelevant users (who are very far away). Then use Pythagoras to calculate the distance between all the other users and the client.
I'd like to know (from a high level view) what would be required to take a pdf floor plan of a building and determine where exactly you are on that floor plan using GPS coordinates? In addition to location, the user would be presented with a "turn by turn" directions to another point on the map, navigating down hallways, between cubicles, etc.
Use case: an iPhone app that determined a user's location and guided them to a conference room or person's office in the building.
I realize that this is by no means trivial, but any help is appreciated. Thanks!
It's an interesting problem. When you're using Core Location, you're not necessarily using GPS. Using WiFi and cell tower triangulation, you can get pretty good location results. So from Core Location you get a latitude and longitude fix. (You might also get altitude info, since GPS data is 3-dimensional. You also will get an accuracy value.)
So you have lat and lon. You need to map these coordinates to the PDF plan's coordinates. Assuming that the plan is aligned with the latitude and longitude lines, and that you have a lat-long fix for one of the points on the plan, you need to calculate the x-axis scale and y-axis scale. Then it's some calculations to map the lat-long to x-y coordinates on the PDF plan.
GPS may not be accurate enough for this purpose, especially indoors. Assuming errors on
the order of 10 meters, you'll have difficulty determining which floor the user is on.
Here's a neat (?) idea that might work: can you post some "You are here" placards
at various locations around the building? You could label each one with a unique,
machine-readable location code (maybe a QR code or something similar), then take an
image using the camera, have your app read that image and interpret the location code,
and use that instead of GPS to determine the start location.
GPS inside? That's your first -- and biggest -- hurdle.
Next hurdle is knowing the GPS coordinates of at least three points on that PDF to define the plane of of your map in the real world. (The PDF will need to be to scale, of course.)
So that gives you where you are on the PDF. Now you'll need to figure out some way to determine where you can walk (or where you can't) to get directions.
I have an application in which users interact with each-other. I want to visualize these interactions so that I can determine whether clusters of users exist (within which interactions are more frequent).
I've assigned a 2D point to each user (where each coordinate is between 0 and 1). My idea is that two users' points move closer together when they interact, an "attractive force", and I just repeatedly go through my interaction logs over and over again.
Of course, I need a "repulsive force" that will push users apart too, otherwise they will all just collapse into a single point.
First I tried monitoring the lowest and highest of each of the XY coordinates, and normalizing their positions, but this didn't work, a few users with a small number of interactions stayed at the edges, and the rest all collapsed into the middle.
Does anyone know what equations I should use to move the points, both for the "attractive" force between users when they interact, and a "repulsive" force to stop them all collapsing into a single point?
Edit: In response to a question, I should point out that I'm dealing with about 1 million users, and about 10 million interactions between users. If anyone can recommend a tool that could do this for me, I'm all ears :-)
In the past, when I've tried this kind of thing, I've used a spring model to pull linked nodes together, something like: dx = -k*(x-l). dx is the change in the position, x is the current position, l is the desired separation, and k is the spring coefficient that you tweak until you get a nice balance between spring strength and stability, it'll be less than 0.1. Having l > 0 ensures that everything doesn't end up in the middle.
In addition to that, a general "repulsive" force between all nodes will spread them out, something like: dx = k / x^2. This will be larger the closer two nodes are, tweak k to get a reasonable effect.
I can recommend some possibilities: first, try log-scaling the interactions or running them through a sigmoidal function to squash the range. This will give you a smoother visual distribution of spacing.
Independent of this scaling issue: look at some of the rendering strategies in graphviz, particularly the programs "neato" and "fdp". From the man page:
neato draws undirected graphs using ``spring'' models (see Kamada and
Kawai, Information Processing Letters 31:1, April 1989). Input files
must be formatted in the dot attributed graph language. By default,
the output of neato is the input graph with layout coordinates
appended.
fdp draws undirected graphs using a ``spring'' model. It relies on a
force-directed approach in the spirit of Fruchterman and Reingold (cf.
Software-Practice & Experience 21(11), 1991, pp. 1129-1164).
Finally, consider one of the scaling strategies, an attractive force, and some sort of drag coefficient instead of a repulsive force. Actually moving things closer and then possibly farther later on may just get you cyclic behavior.
Consider a model in which everything will collapse eventually, but slowly. Then just run until some condition is met (a node crosses the center of the layout region or some such).
Drag or momentum can just be encoded as a basic resistance to motion and amount to throttling the movements; it can be applied differentially (things can move slower based on how far they've gone, where they are in space, how many other nodes are close, etc.).
Hope this helps.
The spring model is the traditional way to do this: make an attractive force between each node based on the interaction, and a repulsive force between all nodes based on the inverse square of their distance. Then solve, minimizing the energy. You may need some fairly high powered programming to get an efficient solution to this if you have more than a few nodes. Make sure the start positions are random, and run the program several times: a case like this almost always has several local energy minima in it, and you want to make sure you've got a good one.
Also, unless you have only a few nodes, I would do this in 3D. An extra dimension of freedom allows for better solutions, and you should be able to visualize clusters in 3D as well if not better than 2D.