I'm a complete illiterate when it comes to working with geographical data, so bear with me.
For our application we will be tracking a fairly large amount of rapidly changing points on a map. It would be nice to be able to cache the location of these points in some kind of map-tile structure so it would be easy to find all points currently in the same tile or neighbouring tiles, making it easier to quickly determine the nearest neigbours and have special logic for specific tiles, etc.
Although we're working for one specific (but already large) location, it would be nice if a solution would scale to other locations as well. Since we would only cache tiles that concern the system, would just tiling the enitre planet be the best option? The dimensions of a tile would then be measured in arc seconds/minutes, or is that a bad idea?
We already work with Postgres and this seems like something that could be done with PostGIS (is this what rasters are?), but jumping in to the documentation/tutorials without knowing what exactly I'm looking for is proving difficult. Any ideas?
PostGIS is all that you need. It can store your points in any coordinate reference system, but you'll probably be using longitude/latitude. Are your points coming from a GPS device?
PostGIS uses GIST indexing, making the search for points close to a given point quite efficient. One option you might want to look at, seeing that you are interested in tiling, is to "geohash" your points. Basically, this turns an (X,Y) coordinate pair into a single "string" with a length depending on the level of partitioning. Nearby points will have the same geohash value (= 1 tile) and are then easily identified with standard database search tools. See this answer and related question for some more considerations and an example in PostgreSQL.
You do not want to look at rasters. These are gridded data, think aerial photography or satellite images, weather maps, etc.
But if you want a more specific answer you should give some more details:
How many points? How are they collected?
Do you have large clusters?
Local? Regional? Global?
What other data does this relate to?
Pseudo table structure? Data layout?
etc
More info = better answer
Cheers, hope you get your face back
Related
For a project on geospatial data analytics, we are currently extracting road type and speed limit data of certain roads along a track by using Overpass' polygon query (where we define the roads by a buffer zone around them). The problem is that in the case of separate tracks, we can end up with disconnected polygons which often lead to a significant increase in computation time. In this situation, we were wondering how Overpass' polygon query actually works. Does the algorithm actually query only the data inside this polygon/these polygons, or does it query inside a bounding box, after which it filters out the data inside the polygons?
The algorithm checks if nodes are inside the defined polygon, or if a way crosses the polygon. It's not based on bounding boxes as you mentioned.
From your description it's not quite clear why disconnected polygons pose an issue. You should get decent performance with a lz4-based backend and a reasonable number of lat/lon pairs in your (poly: ) filter (the more pairs you provide, the more expensive the computation gets).
BTW: The best approach to tackle this issue would be something I described in this blog post: https://www.openstreetmap.org/user/mmd/diary/42055 - unfortunately, this feature is not yet available in the official branch. If you see some use for it, please upvote here: https://github.com/drolbr/Overpass-API/issues/418
I need to store the geographic location of a few physical locations in my Grails/GORM/PostgreSQL based software.
Not a whole lot (about a 100), and there won't be any geo-spatial searches or calculations based on them (although I will have to render them onto a map - probably using a HTML5 canvas).
What are my options for representing and storing them in the GORM object model and in PostgreSQL? Which one is the best?
The incoming format for the coordinates is UTM so something like "29U E 323 314.000 N 3232 111.000". Storing this as text seems bad.
I've looked at PostGIS but I'm concerned it is a bit of overkill (as I need no real geospatial queries) and I'm not sure it integrates with GORM well.
Is there a Java (or Grails) standard simple library for working with coordinates? I've looked at GeoTools and OpenMap so far, and some others which appeared abandoned.
Right now I'm thinking using one of these libraries to transform my UTM coordinates to latitude/longitude, and store those as a pair of doubles - then figure out a way how to project those onto a map again when it comes time to render.
This tutorial and sample class solved my needs for coordinate conversion:
http://www.ibm.com/developerworks/java/library/j-coordconvert
Now I'm storing a location object with latitude/longitude doubles (as embedded) in database, and plan to stick with that.
Rendering to a HTML5 canvas (thus another set of coordinate translation) is still outstanding though.
P.S. GeoTools has a very complex API, OpenMap appear to hate Maven ("We prefer SVN").
I have a program that uses the API to get altitude data of the screen by running through hundreds of thousands of latitude and longitude points. However, Google Earth's data refines itself as one zooms in. So I need to know the minimum altitude I can use to get the best data Google has to offer. I am at a loss to figure out how to do this.
Furthermore, I assume that this depends on the part of the world. I'm going to assume the best part of the world, always.
Thanks for any feedback.
There is no API for determining the quality of the underlying imagery. While this question is not an exact duplicate of Get ground altitude in a reliable way, the same general algorithm can be applied:
Zoom to the required lat/lon at altitude Z
Wait for the imagery to come in (as described in the linked question)
Zoom in "closer" (Z - [delta])
If Google has better imagery, the loaded percentage should drop, wait for it load
Repeat steps 3-4 until the progress is always ~100% (e.g., until no loading is required)
Even with this hack, I'm not sure it will yield reliable results. Your question is flawed for several reasons:
What does "the best data Google has to offer" mean?
You have a lat/lon coordinate. That's a single, infinitely small, point on the Earth
Raw satellite imagery typically covers large sections (many lat/lon points), so the "best" height would be to set the GE camera so that your view convers exactly one satellite image tile
Why do you need the "best" data?
Satellite imagery is expressed in terms of "area covered by a single pixel" (e.g., 1m = 1 pixel in the image covers 1sq m)
Knowing this, the camera height will vary image by image, even for tiles within the same satellite imagery data set (it shouldn't vary much -- but it could slightly.)
Are you trying to cache the imagery off the globe? If so, you should carefully review the TOS to make sure you're not in violation.
Note that if you are just looking to obtain satellite imagery -- there are many good (free) sources. USGS National Map Viewer should be able to get you pretty good imagery for the US (including territories).
I have a table that contains a bunch of Earth coordinates (latitude/longitude) and associated radii. I also have a table containing a bunch of points that I want to match with those circles, and vice versa. Both are dynamic; that is, a new circle or a new point can be added or deleted at any time. When either is added, I want to be able to match the new circle or point with all applicable points or circles, respectively.
I currently have a PostgreSQL module containing a C function to find the distance between two points on earth given their coordinates, and it seems to work. The problem is scalability. In order for it to do its thing, the function currently has to scan the whole table and do some trigonometric calculations against each row. Both tables are indexed by latitude and longitude, but the function can't use them. It has to do its thing before we know whether the two things match. New information may be posted as often as several times a second, and checking every point every time is starting to become quite unwieldy.
I've looked at PostgreSQL's geometric types, but they seem more suited to rectangular coordinates than to points on a sphere.
How can I arrange/optimize/filter/precalculate this data to make the matching faster and lighten the load?
You haven't mentioned PostGIS - why have you ruled that out as a possibility?
http://postgis.refractions.net/documentation/manual-2.0/PostGIS_Special_Functions_Index.html#PostGIS_GeographyFunctions
Thinking out loud a bit here... you have a point (lat/long) and a radius, and you want to find all extisting point-radii combinations that may overlap? (or some thing like that...)
Seems you might be able to store a few more bits of information Along with those numbers that could help you rule out others that are nowhere close during your query... This might avoid a lot of trig operations.
Example, with point x,y and radius r, you could easily calculate a range a feasible lat/long (squarish area) that could be used to help rule it out if needless calculations against another point.
You could then store the max and min lat and long along with that point in the database. Then, before running your trig on every row, you could Filter your results to eliminate points obviously out of bounds.
If I undestand you correctly then my first idea would be to cache some data and eliminate most of the checking.
Like imagine your circle is actually a box and it has 4 sides
you could store the base coordinates of those lines much like you have lines (a mesh) on a real map. So you store east, west, north, south edge of each circle
If you get your coordinate and its outside of that box you can be sure it won't be inside the circle either since the box is bigger than the circle.
If it isn't then you have to check like you do now. But I guess you can eliminate most of the steps already.
I have a database with the current coordinates of every online user. With a push of a button the user can update his/her coordinates to update his current location (which are then sent off to server). The app will allow you to set the radius of a circle (where the user is in the center) in which you can see the other users on a map. The users outside the circle are discarded.
What is the optimal way to find the users around you?
1) The easiest solution is to find the distance between you and every user and then see if it's less than the radius. This would place the sever under unnecessarily great load as comparison has to be made with every user in the world. In addition, how would one deal with changes in the locations?
2) An improved way would be to only calculate and compare the distance with other users who have similar latitude and longitude. Again in order to be efficient, if the radius is decreased the app should only target users with even closer coordinates. This is not as easy as it sounds. If one were to walk around the North Pole with, say, 10m radius then every step around the circumference would equal to a change of 9 degrees longitude. Every step along the equator would be marginal. Still, even being very rough and assuming there aren't many users visiting the Poles I could narrow it down to some extent.
Any ideas regarding finding users close-by and how to keep them up to date would be much appreciated! :)
Andres
Very good practice is to use GeoHash concept (http://geohash.org/) or GeoModel http://code.google.com/p/geomodel/ (better for BigTable like databases). Those are efficient ways of geospatial searches. I encourage you to read some of those at links I have provided, but in few words:
GeoHash translates lon and lat to unique hash string, than you can query database through those hashes. If points are closer to each other similar prefix will bi longer
GeoModel is similar to GegoHash with that difference that hashed are squares with set accuracy. If square is smaller the hash is longer.
Hope I have helped you. But decision, which you will pick, is yours :).
Lukasz
1) you would probably need a two step process here.
a) Assuming that all locations go into a database, you can do a compare at the sql level (very rough one) based on the lat & long, i.e. if you're looking for 100m distances you can safely disregard locations that differ by more than 0.01 degree in both directions. I don't think your North Pole users will mind ;)
Also, don't consider this unnecessary - better do it on the server than the iPhone.
b) you can then use, for the remaining entries, a comparison formula as outlined below.
2) you can find a way to calculate distances between two coordinates here http://snipplr.com/view/2531/calculate-the-distance-between-two-coordinates-latitude-longitude/
The best solution currently, in my opinion, is to wrap the whole earth in a matrix. Every cell will cover a small area and have a unique identifier. This information would be stored for every coordinate in the database and it allows me to quickly filter out irrelevant users (who are very far away). Then use Pythagoras to calculate the distance between all the other users and the client.