I need to store the geographic location of a few physical locations in my Grails/GORM/PostgreSQL based software.
Not a whole lot (about a 100), and there won't be any geo-spatial searches or calculations based on them (although I will have to render them onto a map - probably using a HTML5 canvas).
What are my options for representing and storing them in the GORM object model and in PostgreSQL? Which one is the best?
The incoming format for the coordinates is UTM so something like "29U E 323 314.000 N 3232 111.000". Storing this as text seems bad.
I've looked at PostGIS but I'm concerned it is a bit of overkill (as I need no real geospatial queries) and I'm not sure it integrates with GORM well.
Is there a Java (or Grails) standard simple library for working with coordinates? I've looked at GeoTools and OpenMap so far, and some others which appeared abandoned.
Right now I'm thinking using one of these libraries to transform my UTM coordinates to latitude/longitude, and store those as a pair of doubles - then figure out a way how to project those onto a map again when it comes time to render.
This tutorial and sample class solved my needs for coordinate conversion:
http://www.ibm.com/developerworks/java/library/j-coordconvert
Now I'm storing a location object with latitude/longitude doubles (as embedded) in database, and plan to stick with that.
Rendering to a HTML5 canvas (thus another set of coordinate translation) is still outstanding though.
P.S. GeoTools has a very complex API, OpenMap appear to hate Maven ("We prefer SVN").
Related
I've created a base layer and 6 different overlay (Points of Interest) layers for a leaflet map.
The base layer of markers can appear on the map almost anywhere in the world, but I want the POI layers to appear only if they are in the same area (mapBounds) of the base layer. Probably the screen size.
All the data is pulled from a MySQL database and using Ajax I create the various sets of markers from two different tables, base and poi. This much is all done and working, you can see it at https://net-control.us/map2.php. The green and blue markers are from the base table, other markers are currently selected for view by clicking on the appropriate icon in the lower right. The only one active at the moment is 'Fire Station'. But if you zoom out far enough you will see additional fire stations in the Kansas City area, and in Florida. Those sets are not needed.
After the query runs I create a fitBounds variable of the base layer and another poiBounds for the poi layer. But I'm not sure I need the poiBounds. The number of base markers is generally less than 50 for the base query, but if all the poi markers are pulled world wide that number could be very large.
So I'm hoping someone can help me determine a best practice for this kind of scenario and maybe offer up an example of how it should be done. Should I...
1) Download all POIs and not worry about them appearing outside the base bounds layer? Should I inhibit them from showing in the javascript or in the SQL? How?
2) If I inhibit the unwanted points from SQL do I test one POI at a time to see if its included in the base bounds? How? Are there MySQL functions perhaps to work with this kind of data?
I'm fairly new at leaflet maps and would appreciate examples if appropriate.
2) If I inhibit the unwanted points from SQL do I test one POI at a time to see if its included in the base bounds? How? Are there MySQL functions perhaps to work with this kind of data?
You probably want a column of type POINT, a spatial index on such column (which internally is likely to be implemented as an R-Tree), and spatial relation functions on your SQL query to make use of that index.
Start by reading https://dev.mysql.com/doc/refman/8.0/en/spatial-types.html. Take your time, as spatial databases, spatial data types and spatial indices work a bit differently than their non-spatial equivalents.
For a small project I am allowing users to add areas to the database. Their query is sent to http://nominatim.openstreetmap.org and I store the latitude and longitude. If available, I also store the geoJSON polygon outline data.
Example output: http://nominatim.openstreetmap.org/search?q=wyoming&format=xml&polygon_geojson=1&addressdetails=1
This outlined area is then displayed on a map using leaflet.js. For a lot of polygons this works out just fine, but it seems that there is a limit to the amount of data the library can process. Some rather complex areas (that require a longtext to store in mysql) simply do not get displayed at all, without an error being thrown.
I guess my question has two parts:
1 - Am I right to assume that the large datasets are the root of the problem or should leaflet.js be able to handle those?
2 - What would be the best way of simplifying such datasets? Leaflet has such an algorithm for displaying areas, but that seems to be the failing point already.
And while we are on the topic: Right now I'm converting Nominatim's lnglat polygons to leaflet's latlng by splitting up the data and patching it back together in javascript. Is there an easier/safer way to do that? Should I rather move that task to the server and use some php library/function?
I appreciate your help!
Edit: Forgot to mention: on the occasion that the polygon fails to render, my console gives me this error: TypeError: t is null
are the large datasets the root of the problem or should leaflet.js be able to handle those?
Leaflet.js will handle whatever you throw at it. There is, however, a limit on what your web browser can handle without slowing down.
Remember that every modern web browser has performance analysis tools that you can use to see what parts of the Leaflet code (or of the browser's internals) is taking up most of your time.
What would be the best way of simplifying such datasets?
You probably want to look at the Douglas-Peucker algorithm for a starting point in these algorithms.
Keep in mind that Leaflet uses this algorithm internally to simplify polygons on each zoom level, up to a pixel.
For some big complex polygons, slicing them up with something like Leaflet.VectorGrid might improve the performance.
There is no silver bullet for simplifying datasets, however. The best way will depend on the specific data you are using.
on the occasion that the polygon fails to render, my console gives me this error: TypeError: t is null
This is a different matter, and might be a symptom of malformed data.
In order to display prettier error messages, use the leaflet-src.js file instead of the leaflet.js file. Starting with Leaflet 1.0.0-beta2, leaflet-src.js has a sourcemap, which can point to the individual original files, allowing for better debugging.
I'm a complete illiterate when it comes to working with geographical data, so bear with me.
For our application we will be tracking a fairly large amount of rapidly changing points on a map. It would be nice to be able to cache the location of these points in some kind of map-tile structure so it would be easy to find all points currently in the same tile or neighbouring tiles, making it easier to quickly determine the nearest neigbours and have special logic for specific tiles, etc.
Although we're working for one specific (but already large) location, it would be nice if a solution would scale to other locations as well. Since we would only cache tiles that concern the system, would just tiling the enitre planet be the best option? The dimensions of a tile would then be measured in arc seconds/minutes, or is that a bad idea?
We already work with Postgres and this seems like something that could be done with PostGIS (is this what rasters are?), but jumping in to the documentation/tutorials without knowing what exactly I'm looking for is proving difficult. Any ideas?
PostGIS is all that you need. It can store your points in any coordinate reference system, but you'll probably be using longitude/latitude. Are your points coming from a GPS device?
PostGIS uses GIST indexing, making the search for points close to a given point quite efficient. One option you might want to look at, seeing that you are interested in tiling, is to "geohash" your points. Basically, this turns an (X,Y) coordinate pair into a single "string" with a length depending on the level of partitioning. Nearby points will have the same geohash value (= 1 tile) and are then easily identified with standard database search tools. See this answer and related question for some more considerations and an example in PostgreSQL.
You do not want to look at rasters. These are gridded data, think aerial photography or satellite images, weather maps, etc.
But if you want a more specific answer you should give some more details:
How many points? How are they collected?
Do you have large clusters?
Local? Regional? Global?
What other data does this relate to?
Pseudo table structure? Data layout?
etc
More info = better answer
Cheers, hope you get your face back
I'd like to stack the images from this map: http://earthquake.usgs.gov/regional/nca/soiltype/map/ from linear projection into a leaflet map. The source tiles are in known but nonstandard zoom levels, and leaflet maps want mercator mercator XYZ tiles. In principle, I know how to do this - I have functions for changing XY coordinates into lat-lng coordinates in the two maps, and I just need to map pixels for the target map in terms of pixels in the source map.
This is unfortunately nontrivial, as the source pixels are spread across hundreds of different image files, and I am trying to put them into hundreds more images. Is there a software package that makes this a little bit more straightforward? If there is no library for dealing with this kind of data, it seems like there really should be...
Postgis has the RT_ST_Transform method, which under the hood uses GdalWarp. So, you have at least these two options. If you use Postgis, you will need to actually register/import the images into Postgis, using raster2pgsql and then call RT_ST_Transform on each one and then dump them out again -- which could be scripted to some extent using plpgsql (Postgres's scripting language). There is something of a learning curve involved with using Postgis raster, which may be worthwhile if you plan to do any other image processing analysis. You could also write a shell script (or similar) to automate gdalwarp if you don't wish to go the Postgis route.
For a less formal method than gdalwarp (an excellent program), you can check out the Leaflet plugin Leaflet.imageTransform, that can transform and image on the fly in the browser.
MY PROBLEM
We´re doing a Project right now where we have to display a huge image (containing chemical compounds and elements, so not geo referenced) as a map within a Web-Application (with Leaflet). The image itself is an Adobe Illustrator-File, so its actually a bunch of vector graphics. To makes things easy, we just converted it into a large .png (27.000x19.000 px) and then used MapTiler to create the needed MapRessources for Leaflet, easily included within a TileLayer.
The Problem is:
The user needs to be able to dynamically add and remove different Layers (== Filter) of the map to show more or less informationes from the picture. So we first created those layers within the Illustrator-File, then exported every layer as its own transparent .png-File, mapTiled it and included it as an own Leaflet-Layer.
Right now, we have 6 Filter-Layers and two more base Layers for the background and an overlay. This means that when all Filters are activated (which is the default), we have 8 Leaflet-Layers stacked on top of each other at once. As you can imagine, this causes some performance issues in the Browser, since Leaflet has to load and render 8 Layers with all its Tiles (depending on Screen size up to 25 at once) for every zoom or drag-action. Its still in a point that is not impossible to bear, but we are expecting several more filters to come and therefore wanna stay scalable in the future.
This means we will somehow have to change our approach of generating the Layers.
MY APPROACH SO FAR
Since we actually have a vector-graphics based map, i thought there have to be better alternatives. But it seems that we have a rare case of requirements, since my researches mostly ended in dead ends, especially since most of the cases only cover REAL geographical maps, but what we have is a raster map. I also thought about somehow putting the map into a GeoJSON or redraw it somehow directly with SVG, but since we have LOTS of single elements on the map (> 20k), I dont think this would perform much better.
So I kind of need to stay with the Bitmaps, and therfore my main goal is simple: I wanna reduce the number of layers by merging the tiles of the currently activated Filter into one single .png which then gets delivered to leaflet within ONE Layer. I spent some hours now researching, but I always run into dead ends since it seems we have a rare case of requirements here (especially since most people deal with georeferenced data, not with custom raster maps).
So right now, I can think of 2 different options:
Create ONE Layer for every Filter-Combination. This means we would have to create 2^n layers, so this would only work up to a certain number of filters (which probably will increase) - therefore, i would prefer another solution (this is only last case)
Use MapServer and somehow import my Layers. Then we could merge the Layers on runtime with a query (I read about Union Layer here) and therefore only deliver ONE Layer to leaflet.
MY QUESTION
I have absolutely no experience with MapServer and im therefore not even sure if that is a use-case or if its capabale of doing this, and more important: If it would really give us a performacne boost, since it probably requires a lot of logic ServerSide.
Before i start spending another hours to try this out:
Can someone who already worked with MapServer give me some feedback if that is even a good idea or if I am misunderstanding something with MapServer completely?
Also, if someone has another alternative or idea for me, you´re more than welcome to share it, im grateful for every input. :)
Thanks in advance!
You might want to look at OpenLayers where you can display a mix of raster and vector layers. another option might be mapcache a tile caching engine part of the mapserver project. This has the ability to do vertical assembly of tiles. So if you case where you have 8 layers you can ask mapcache to stack all the eight tiles into a single tile. You can give it a list of layers to stack and it takes care of it for you. You can also do this with mapserver. The difference being that mapcache is a lightweight apache module that just works with tiles and is probably a little faster. Mapserver is a cgi-bin process and work efficiently at rendering and combining raster layers but is probably not as fast as mapcache for simple assembly of tiles.