I saw several places where OSM data is converted to MVT (mapbox vector tiles) for simple rendering.
Those include:
OSRM which provides MVT based image of the route for rendering
OpenMapTiles which enables downloading of OSM maps in MVT format
GeoServer which can import OSM and export MVT
Is there a simple tool to convert OSM data to MVT locally?
I have my own data in OSM format that I want to convert to MVT and store locally, without rendering to a map image.
The mapbox/awesome-vector-tiles repository at GitHub has a long list of Mapbox Vector Tile implementations. This is the list as of today:
Parsers & Generators
vector-tile-js - Parses vector tiles with JavaScript.
mapnik-vector-tile - C++ vector tile read/write implementation on top of Mapnik.
mbtiles-cpp - C++ library for decoding of mbtiles and vector data into function callbacks.
vector-tile-py - Python tool to convert a Mapnik vector tile to GeoJSON
node-mapnik - Node.js API for vector tiles which depends on mapnik-vector-tile
vector-tile-cs - Parses vector tiles with C# (native C# implementation, no dependencies).
mapbox-vector-tile-cs - Parses vector tiles with C# (uses protobuf-net).
tilelive-bridge - Implements Tilelive API for creating vector tiles from traditional Mapnik datasources in Node.js.
tilelive-vector - Implements Tilelive API for reading vector tiles and rendering to image tiles in Node.js.
mapbox-vector-tile is a Python package for vector tile encoding maintained by Mapzen. (It is used in Mapzen's vector tile service).
geojson-vt - Slice GeoJSON into vector tiles on the fly in the browser.
java-vector-tile - A java encoder and decoder for vector tiles.
mapbox-vector-tile-java - Encode and decode v2.1 Mapbox Vector Tiles. Convert JTS Geometry to and from MVT features, including simple user data support. Utility functions for converting world coordinates to MVT coordinates and clipping to a tile envelope.
cached-vector-tile - An alternative implementation of the vector-tile-js interface, backed by plain JS objects/arrays rather than parsed-on-demand protobuf data. Trades away memory efficiency for faster feature.loadGeometry() calls.
tilegrinder - A helper library for applying a data altering function on each vector tile in an MBTiles, using the native protobuf wrapper for de- and encoding, recompressing the results and storing them either in an MBTiles or as single files.
SwiftVectorTiles - A Swift encoder for vector tiles according to the Mapbox vector tile spec.
Clients
Mapbox GL Native - C++/OpenGL vector maps library with native SDKs for Android, iOS, Node.js, macOS, and Qt
Mapbox GL JS - JavaScript/WebGL vector maps library.
OpenLayers 3 - JavaScript vector & raster library.
WhirlyGlobe/Maply - Objective C code that is able to read and render vector tiles(and style with mapnik xml) on iOS devices.
Leaflet.MapboxVectorTile is able to read PBF MapboxVectorTiles from a REST endpoint and render them as a TileLayer on a Leaflet Map. Use this option if you want to utilize vector tiles on a standard Leaflet web map without needing WebGL.
CARTO Mobile SDK - C++ maps library focused on offline features, for iOS, Android, Windows Phone and Xamarin with bindings for Java, Objective-C and C#. Based on Nutiteq Maps SDK, but open source and uses CartoCSS.
Mapzen Tangram - JavaScript library for rendering 2D & 3D maps live in a web browser with WebGL, supports MVT, GeoJSON, TopoJSON
Mapzen Tangram-es - C++ library for rendering 2D and 3D maps using OpenGL ES 2 with custom styling and interactions
mapbox-gl-leaflet - Create Mapbox GL layers in Leaflet
react-native-mapbox-gl - Render Mapbox GL maps from React applications
hoverboard - Render vector tiles on canvas with Leaflet 0.7.x (supports GeoJSON, TopoJSON, and protobuf)
Leaflet.VectorGrid - Display gridded vector data (sliced GeoJSON, TopoJSON or Mapbox Vector Tiles) in Leaflet 1.0.0
ArcGIS API for JavaScript - Draw vector tile layers as part of your web map. Rendering done via mapbox-gl-js integration.
mapscii - A Vector Tile to Braille and ASCII renderer for xterm-compatible terminals
Applications / Command line tools
Mapbox Studio - Desktop design studio for both creating vector tiles from raw geodata and for rendering them on-the-fly into image tiles. Internally uses tilelive.js modules to handle vector tiles (see tilelive-bridge and tilelive-vector)
kosmtik - Design maps with CartoCSS and Mapnik.
ArcGIS Pro - Generate vector tiles from maps authored in ArcGIS Pro or imported from ArcMap.
MVT Styler - map style editor for vector tiles.
Maputnik - A visual style editor for the Mapbox GL style specification.
CLI Utilities
Datamaps C application that can be used to create vector tiles and store them in an mbtiles. See the render-vector command.
tilemaker - Command line tool to produce vector tiles directly from an .osm.pbf extract without an intermediate database.
vector-tiles-producer Command line tool in C++ to creates vector tiles for a given area at chosen zoom levels using a Mapnik XML.
tippecanoe - Build vector tilesets from large collections of GeoJSON features.
vt-geojson - decodes vector tiles to GeoJSON FeatureCollections
tl - An alternate command line interface to tilelive
tileshrink - Reduce the layer extent and simplify the resulting geometries of all vector tiles in an MBTiles
tiler - Command line tool for converting GeoJSON, Shapefiles or PostGIS layer to raw Vector Tiles (or MBTiles)
geojson2mvt - npm package for building a static vector tile tree for given xyz bounds from a geojson file (uses vt-geojson)
Mapbox GL JS Plugins
gl-draw - Adds support for drawing and editing features on Mapbox GL JS maps
Servers
tessera - Supports serving and rendering vector tiles. Uses the same core libraries as Mapbox Studio.
tilestrata - with tilestrata-vt, it can generate Mapnik Vector Tiles; with tilestrata-postgismvt, it can serve Mapbox Vector Tiles from a PostGIS db
SpatialServer (PGRestAPI) - A multi-purpose GeoSpatial NodeJS web server created at SpatialDev that not only serves MBTiles stuffed with vector tiles, it can also cut vector tiles on the fly from a PostGIS database.
Utilery Server to generate vector tiles from PostGIS queries. Python based
tileserver Mapzen Vector Tile Service.
TileStache added support for Mapbox Vector tiles via .pbf extension requests.
Kartotherian Wikipedia tile server with Tilerator backend tile pre-generator
ArcGIS Online - Supports serving vector tiles and rendering in the mapping application powered by the ArcGIS API for JavaScript
Portal for ArcGIS - Supports serving vector tiles and rendering in the mapping application powered by the ArcGIS API for JavaScript
tilesplash - A light and quick nodejs webserver for serving topojson or mapbox vector tiles from a postgis backend
go-vtile-example - An example server written in Go
Tegola - A MVT server written in pure Go that supports serving tiles from a PostGIS data provider.
t-rex - MVT server in a single executable written in Rust. Serves tiles from PostGIS supporting custom tile grids.
Low-level utilities
mapbox-gl-function - Mapbox GL style function evaluator
mapbox-gl-filter-simplify - Simplifies and complexifies filters in Mapbox GL Styles
vt-pbf serialize JavaScript objects representing vector tiles into binary Protocol Buffer encodings of vector tiles
Articles
Vector tiles remixed - guide to using tilemaker to generating vector tiles
Specific thing with MVT is that this is tiled and multi-resolution (zoom-based) display-optimized, more like "rendered" map for visual fast rendering and styling, it is not your typical geodata format really. For example it does not have geographical coordinates in it, it has "pixel space" coordinates in tiles for your vector objects. This makes 'simple conversion' from/to geodata quite tricky, and very typical conversion solution is a strange one - use use map server or service with your data and then scrape tiles from it. For tile scraping the best tool seems to be tilelive-copy.
However, closest what you may want from the long list above would be tippecanoe which converts GeoJSON to MVT (in a mbtiles file). If your data is in .osm format, then you need to convert it to geojson; but there are other tools for it, for example ogr2ogr. Be aware that OSM files include many data layers and the structure is quite specific, so you need to find well-working configurations for all conversion steps.
After some long time search, here is my conclusion at the very moment:
gdal should be useful with ogr2ogr cli tool converter. Theorically, it is able to convert from osm to mvt (geojson step should not be mandatory).
tippecanoe does a quiet equivalent thing, from geojson to mvt (ogr2ogr or osmium can help to convert from .osm to .geojson)
But the devil is in details: you need to be explicit on what data go on what mvt layer (water, roads, buildings, etc...).
I've not found a convenient "all-in-one" tool able to convert osm building or building parts polygons and relations into mvt buildings layer, and so on... It looks like each team (mapbox and co) keeps it secretely behind API services with API key system.
Related
Is there a way to use a custom DEM for 3D viewing in the new mapbox release? I can use custom 2D rasters tiled using MapTiler, so I was wondering if the same option is available for "raster-dem" then setTerrain. The goal is to create something like this, but with my own elevation data:
https://docs.mapbox.com/mapbox-gl-js/example/add-terrain/
Thanks
Good question. In theory, you ought to be able to produce your own raster tileset following the same Terrain RGB format like this:
map.addSource('mapbox-dem', {
'type': 'raster-dem',
'url': 'mapbox://yourusername.yourtilesetid',
'tileSize': 512,
'maxzoom': 14
});
map.setTerrain({ 'source': 'mapbox-dem', 'exaggeration': 1.5 });
However, the documentation does say:
Only supports Mapbox Terrain RGB (mapbox://mapbox.terrain-rgb):
It's unclear if they mean it only supports that format (seems more likely), or is somehow hardcoded to only support data from that exact data source (which seems a very strange choice).
Based on my very basic testing, Mapbox GL JS does attempt to fetch tiles from whatever source you provide, and doesn't give any weird error messages, but I haven't gone as far as actually producing such a tileset.
I'm trying out Mapbox for the first time, and playing around with drawing some polygons in the dataset editor for export to a tileset. However, the polygons in the resulting tileset are not the same as what I create in the editor. The polygons are only very rough, simplified approximations of the originals.
In dataset editor:
In map layer as tileset export:
I understand that Mapbox does vector simplification at certain zoom levels, but these changes are not zoom-dependent. I zoom in all the way and the shapes are still like this.
Moreover, such extreme degredation of the geometries makes tilesets essentially useless for features that require any sort of accuracy, like property lot lines.
Am I missing something, or is this really the expected behavior? Is there really no way to get accurate geometries into a tileset?
UPDATE: It appears this is only happening with shapes I create by drawing in the Mapbox data editor. So far the geometries that I've uploaded as geojson files have gotten converted to tilesets accurately...
I suspect this is because the maxzoom is too low.
When you create a Mapbox Tileset, either by uploading GeoJSON directly as a new Tileset, or by exporting your your Dataset to a Tileset, Mapbox will try to guess an appropriate minzoom and maxzoom of the Tileset.
Sometimes the min/max zoom's used aren't suitable for the map you're trying to create. Since there is no way to specify a maxzoom in either of the two approaches the only alternative is to create your Tileset locally with https://github.com/mapbox/tippecanoe specifying an appropriate maxzoom for your data and then uploading the resulting .mbtiles as a Mapbox Tileset.
Mapbox provides a kindle of map tiles--mapbox.mapbox-terrain-v2 which is stored in pbf format and saved in mvt suffix. The height data is represented by contour (line).
I want to generate terrain with satellite texture and this height data in Unity3D. How could I convert this pbf data to a height map(a pixel for a height value)?
There is an example
https://api.mapbox.com/v4/mapbox.mapbox-terrain-v2/12/1171/1566.jpg?access_token=pk.eyJ1Ijoib2xlb3RpZ2VyIiwiYSI6ImZ2cllZQ3cifQ.2yDE9wUcfO_BLiinccfOKg
And the mvt file
https://api.mapbox.com/v4/mapbox.mapbox-terrain-v2/12/1171/1566.mvt?access_token=pk.eyJ1Ijoib2xlb3RpZ2VyIiwiYSI6ImZ2cllZQ3cifQ.2yDE9wUcfO_BLiinccfOKg
And the document of Mapbox:
https://www.mapbox.com/vector-tiles/mapbox-terrain/
https://www.mapbox.com/vector-tiles/specification/
MapBox have buid a Unity3d package: MapBox-Unity-SDK
SDK here : https://www.mapbox.com/unity/
Just click download.
This is an asset you can open in Unity directly.
Launch Unity3d, goto Menu>Assets>ImportPackage>CustomPackage
and select your downloaded file.
It will unpack some files and the folders, you will find into some exemples scene files to help you.
The current vector terrain layer isn't designed to be turned into a heightmap: we've processed terrain into elevation contours and lines, so turning those back into raw data would be difficult (much like doing the opposite: we do a lot of processing because it would also be difficult to transfer raw data and derive visual data).
A new and improved vector terrain model that supports your usecase is on the way, but we've also introduced RGB terrain, which was actually designed specifically to address cases like Unity - decoding the RGB-encoded elevation tiles tends to be much simpler in software.
My map's source file is GeoJSON, but it is very large and very slow. Is there a way to convert this to vector tiles on the fly using MapBox GL JS? (Load the GeoJSON, pre-process the file into vector tiles, and show use the vector tiles as the base map.) It seems that vector tiles are much faster.
I've tried all the GeoJSON-VT tutorials and examples that I could, like the one on MapBox's site, but it just says that GeoJSON-VT works under the hood, so it isn't much help. The others mostly apply to Leaflet, not MapBox GL JS.
Meanwhile, every example I find that uses a large dataset always does so via vector tiles:
map.addSource('x', {
"type": "vector",
"url": "url"
});
For reference, I am loading my file using this method:
map.addSource('x', {
type: 'geojson',
data: 'file.geojson'
});
In case your GeoJSON file is static, you could use mapbox/tippecanoe to convert the GeoJSON to an .mbtiles file. You could then either upload the file to Mapbox as a tileset (about tilesets) or you could serve your own vector tile source from a web server using the .mbtiles file (reference implementation).
If your GeoJSON file is more dynamic, things get a little bit more complicated. I have never used it, but Mapbox Dataset API might be a good solution for that.
To the best of my knowledge, Mapbox-GL-JS uses GeoJSON-VT to automatically convert client-side-loaded GeoJSON files into vector tiles within the browser - so it's already doing what you're asking for.
If this is still "slow", probably the problem is the actual loading and processing - so pre-generating and serving vector tiles is the right answer.
I'd like to stack the images from this map: http://earthquake.usgs.gov/regional/nca/soiltype/map/ from linear projection into a leaflet map. The source tiles are in known but nonstandard zoom levels, and leaflet maps want mercator mercator XYZ tiles. In principle, I know how to do this - I have functions for changing XY coordinates into lat-lng coordinates in the two maps, and I just need to map pixels for the target map in terms of pixels in the source map.
This is unfortunately nontrivial, as the source pixels are spread across hundreds of different image files, and I am trying to put them into hundreds more images. Is there a software package that makes this a little bit more straightforward? If there is no library for dealing with this kind of data, it seems like there really should be...
Postgis has the RT_ST_Transform method, which under the hood uses GdalWarp. So, you have at least these two options. If you use Postgis, you will need to actually register/import the images into Postgis, using raster2pgsql and then call RT_ST_Transform on each one and then dump them out again -- which could be scripted to some extent using plpgsql (Postgres's scripting language). There is something of a learning curve involved with using Postgis raster, which may be worthwhile if you plan to do any other image processing analysis. You could also write a shell script (or similar) to automate gdalwarp if you don't wish to go the Postgis route.
For a less formal method than gdalwarp (an excellent program), you can check out the Leaflet plugin Leaflet.imageTransform, that can transform and image on the fly in the browser.