This is not really a programming question but more of looking for advice. I am trying to figure out the best way to load about 15 million polygons onto a web map. The file is large, about 20GB uncompressed.
The best solution that I have found is to use .mbtiles format to turn the geojson data into vector tileset. I did this with a portion of the data using MapBox with Tippecanoe but they do have upload limits of 25GB, I would assume all 15 million polygons with attributes are going to be larger than that.
Also, it seems a little slow when I load a sample of the .mbtiles into mapbox. Is there a better and faster solution than using MapBox to host the tiles? I am looking for faster tile loading times.
I am also wondering once the .mbtiles tiles are generated is it possible to edit the attributes of each individual polygon and if so, how? It would seem like a huge pain to have to go back and edit my geojson data, convert to .mbtiles and upload every time. I appreciate any help with this matter. Thank you.
You're asking a lot of separate questions there:
I would assume all 15 million polygons with attributes are going to be larger than [25GB, in mbtiles format]
I wouldn't assume that. You should try it.
Is there a better and faster solution than using MapBox to host the tiles? I am looking for faster tile loading times.
I'd very surprised if Mapbox's hosting is your bottleneck. Or rendering with Mapbox-GL-JS for that matter. But other solutions include cloud.tilehosting.com, or self-hosting with something like Tessera.
I am also wondering once the .mbtiles tiles are generated is it possible to edit the attributes of each individual polygon and if so, how?
You can use tile-join, in the same package as Tippecanoe. https://github.com/mapbox/tippecanoe#tile-join
Related
I have a mapbox tileset with lots of small data points and I regularly need to update one data point at a time. Is it possible to programmatically update a mapbox tileset in a partial way? These instructions from the mapbox docs and this stack overflow question explain how to overwrite a tileset file. This is useful but seems like overkill if I just want to update 1 point out of thousands.
I think what you're after is Mapbox's tippecanoe.
No. But probably the better way to update a source is using the tiling service API: https://docs.mapbox.com/api/maps/mapbox-tiling-service/#replace-a-tileset-source
For my non-commercial, low-traffic web site, I successfully use Leaflet with standard raster tile layers from well-known sources.
I'd like to add additional layers containing very localized high-resolution maps. I've succeeded in making a usable raster tile-set from such a map, hosting the tiles on my own server, and adding that as an additional layer. But this creates a huge file count. My cheap shared-hosting account promises unlimited storage but limits file (actually, inode) counts. If I add one more such tile-set, I risk getting thrown off my server.
Clearly I can look for a hosting account with higher limits, and I'm exploring Cloud alternatives, too. (Comments welcome!)
Any other ideas? Are there free or very low-cost alternatives for non-commercial ventures to use for low-traffic tile storage?
Or: As I look at the localized, high-resolution maps – I see I could fairly easily trace them to create vector artwork without much loss of data -- and some gains in clarity. I use Adobe Illustrator. Is there a reasonably painless way to get from an .ai file (or some similar vector format) to a Leaflet layer? With a substantially lower file count compared to the raster alternative?
Apologies if I've misused terminology --please correct me-- or if I've cluelessly missed some incredibly obvious way of solving this problem.
TIA
This sounds like a good use case for Leaflet.TileLayer.MBTiles plugin: (demo)
A LeafletJS plugin to load tilesets in .mbtiles format.
The idea is to write your Tiles into a single .mbtiles file (actually an SQLite database), so that you just need to host that single file on your server, instead of your thousands of individual tiles.
The drawback is that visitors now need to download the entire file before the map can actually display your tiles. But then navigation is extremely smooth, since tiles no longer need to be fetched from the network, but are all locally available to the browser.
As for generating the .mbtiles file, there are many implementations that can do the job.
I want to use custom styled Map using TileMill. Found a very good style (http://a.tiles.mapbox.com/v3/aj.Sketchy2/page.html#6/34.044/10.942) in which I want to some changes, host it and use it. (Already found its style).
I've before used mapbox and Tilemill for creating mbtiles for building level, which usually creates mbtiles of less then 10 MB, later I used to upload it to Mapbox free account and use. But this Time I need to work on world level map with Few Layers like country boundaries, city boundaries with roads for few cities (OSM Data).
I've already done changes to the style using TileMill software. Now I am not sure How and where I can host these map as if I export this world map it will be a huge.
What is the most feasible way to export customized world map from TileMill and where I can host it?
Yes, exporting such a world map at a high range of zoom levels is incredibly large. Mapbox isn't really optimized for that sort of map size, which is also why our OSM-based maps aren't available for MBTiles download -- it's not the format they are designed in.
For background, the way this works is OSM is baked down to vector tiles which are consumed by front-end servers which apply styling and render them much like TileMill does out into tiles directly that are served.
One way we're tackling this problem is with TileMill 2, which is still in development. It consumes these vector tiles from the backend server directly, allowing you to style the whole world of OSM in TileMill directly without setting up up any OSM data configurations. Then, you upload the style info to Mapbox to reproduce this on our servers.
Please contact support#mapbox.com about getting setup to upload TM2 styles, since again this isn't in full release yet. You can start playing with TM2 styling right away, though.
You can use TileStream or TileStache + Gunicorn + nginx. Here is good manual.
If you don't expect a much load you can try Amazon EC2 server with free tier for one year.
For my Windows Phone Mango app, I want to make overlay a heatmap on Bing Maps, and a tile overlay seems the best way to do it. I've been having trouble finding any good documentation or code samples to work from. It seems like most people are pointing the tile source to a web service. I'd rather render the heatmap on the phone itself - is that possible?
One of the main reasons to use tilelayers to represent data on a map is that the computation and rendering involved in creating the layer is performed in advance, generally as a one-off or infrequent task. Then, at runtime, the only work the client needs to do is to retrieve the pre-rendered tile images from the server and display them straight on the map, which is a simple, low-resource activity.
Rendering tiles can be a resource-intensive task, both in terms of processing and memory usage - for example, I can only render about 3 tiles per second on a quadcore desktop machine with 8Gb RAM. Even if it's technically possible to create the tiles dynamically on a handheld device, the performance is almost certainly going to be unacceptable for any user. You've also got the question of how you're going to store the data from which the layer is created. Since you're talking about plotting a heatmap, I'm guessing you have a reasonably large dataset of points - did you envisage these stored locally on the device, or retrieved over the network? (either will create different problems).
Basically, while it may be theoretically possible to create tile layers dynamically on the client, doing so would negate almost any benefit of using tilelayers in the first place, which is why you probably won't find any code samples explaining how to do so. Perhaps you could explain your comment why you'd rather create the heatmap on the phone?
It's pretty easy to create a server-side tile renderer using .NET or PHP that renders and server tile images to a Bing Maps client, or you can use an existing map rendering library such as mapnik.org or geoserver.org.
I have designed and developed couple of navigation apps using google API and osmdroid API for android powered devices. Now I am looking to create an Indoor navigation system using osmdroid API. But, in order to do so I need to create tiles similar to regular map tiles from an simple PNG file with naming convention similar to OpenStreetMap.
Please suggest me how to do this?
Cheers,
Susheel
You could design your indoor map using JOSM. Save it to a .osm file. Don't upload the data to OpenStreetMap unless it is a appropriate to do so (OpenStreetMap has some basic some indoor features, e.g. a highway=footway running through a shopping mall, but generally a lot of very detailed indoor stuff will be inappropriate for OSM) But...
With a .osm file you could then use one of the OpenStreetMap rendering tools to create a raster map, and chop it into tiles. For quick satisfaction I'd recommend Maperative, although I'm not sure how easy the last tile chopping step is. I've never done this with Maperative. Mapnik has a nice generate_tiles.py output, which will give you the tileset you want, but it's a bit tricky to set up in the first place.
Actually the last step is the main thing you're asking about. You can chop up any image into tiles. It may or may not be important to you that the tiles are geo-positioned in some meaningful way. For an old project I did a quick fudge solution using google tile cutter script, which is actually a wrapper around GDAL tools.
Have a look at the gdal library, and in particular gdal2tiles. This is a library designed to create maps from raster images, and serves exactly your purpose.
You can decide on a projection and what the bounds of your source image(s) are. The library allows you to reproject your image to the correct coordinate space.
It can also generate tiles at various zoom levels using gdal2tiles, either with or without reprojection.
Now you can check indoor rendering by drag and dropping OSM geojson data into https://app.openindoor.io web page.