I'm looking to replace Google Maps by something self-hosted and free. After searching for some time I found OpenMapTiles. I'm very new to this stuff, so there are many things I don't understand, namely:
I'm not sure I qualify for pre-rendered tiles, can I generate my tiles myself from osm data?
I would like to have something like google terrain map, with contour lines, is that possible with OpenMapTiles?
I also have some GPS tracks that I would like to show like overlay over base maps (OSM or Google), and I want them clickable (clicking open a link or popup or something like that), is that possible?
Is OpenMapTiles server heavy to run?
Thanks
That's totally possible, I've done it myself. However, it takes a while to do everything.
1. Generate MBTiles
First of all, I'd suggest you start taking a look in the Github repo of OpenMapTiles: https://github.com/openmaptiles/openmaptiles
To generate an MBTiles file, for a country is quite manageable to do in your own computer. And basically you can do so by executing the main script with the name of the country as a parameter. Like this:
./quickstart.sh portugal
After the script creates a Docker machine, downloads the OpenStreetMap data for the country, processes it, and generates the MBTiles file, then you'll be done.
For me, with an average computer, for countries like Spain or Portugal, it takes about 4-12 hours to do so. But it really depends on the country. Norway takes forever (about 4 days), I believe due to its very long coastline.
If you want an MBTiles file that will work for the whole planet, then you need a more powerful computer, that specially has a big SSD and lots of RAM. Between 32 and 64 GB of RAM and 250-300 GB of SSD.
2. Set up a tile server
After that, you can use this project https://github.com/urbica/tessella to run your own tile server to be consumed form a website or mobile app.
The resources of the server will depend on the size of the MBTiles file you want to use in the server.
3. Enable contour lines
This is possible too, but a bit more trickier. I'd suggest you use Maptiler as a service for this.
Whenever I achieve this matter, I'll go back to you with the information.
If the tiles you generate don't contain the contour / elevation data, then styles won't help. The tiles from OpenMapTiles do not have the contours inside.
4. Web and mobile libraries (to display GPS tracks, waypoints, etc)
This is possible as well. What technology you want to use here?
For web apps:
There's a well established library, Leaflet: https://leafletjs.com
More recently, looks like MapLibre is taking the lead (fork from Mapbox GL JS): https://github.com/maplibre/maplibre-gl-js
For mobile apps:
VTM is an open source iOS and Android library: https://github.com/mapsforge/vtm
There's Carto, which is more recent but it has a great support: https://github.com/CartoDB/mobile-sdk
And MapLibre has a native Android and iOS library too: https://github.com/maplibre/maplibre-gl-native
Related
we would like to make sure that the MLFLOW experiment management platform fits our needs and workflow.
We work with image processing CNNs like Yolo, UNET, and RetinaNet based on an NVIDIA TAO framework.
What we actually need is a tool that concentrates on one place (in a nice and representative way comfortable for comparison) at least the three following things for each experiment:
a- chosen by user typical meta parameters that were used to train a network (such as batches, subdivisions, max batches, etc)
b- a link to the dataset the network was trained on, located on our cloud storage (such as one-drive, google drive or google cloud) or a list of filenames or a link to a file storage cloud or online drive suggested by MLFLOW service if there is such a thing.
c- a result of running the trained network - the number of detected objects
Thus the question is:
Does the MLFLOW fit our needs?
If not ill be glad if anyone could suggest a relevant alternative.
Thank you
I use Comet.ml and it addresses all 3 of your points.
With Comet, parameter tracking is as easy as calling the experiment.log_parameter function. You can also use the diff tool to compare two experiments by their hyper parameters and even group experiments by hyper-paremeters!
Comet has the concept of artifacts. You can upload your dataset as an artifact and version it. You can also have remote artifacts!
Comet has a feature call the Image Panel. This allows users to visualize their model performance on the data across different experiment runs. For the object detection use case, use experiment.log_image to log your images in which you have you drawn your model predicted bounding box on! You will then see in the Image Panel different experiments and how they each one of them are drawing their predictions side by side
Currently I am trying to develop a multi-layered Leaflet map using GeoJSON (for country, states/provinces) and CSV (for city data). I want it to go down all the way to city level and that requires layers of Country, State/Province, and City data. I have all the Country data I need in the format I require (GeoJSON) and I have a decent source for City data in a CSV format.
However, I only have USA, Canada, Brazil, and Australian states/provinces and I have been looking around but haven't been able to find a reliable source such as NaturalEarth (which is where I initially got my states/province data from).
Does anyone have a resource they could point me towards? Even if it is multiple, hopefully I can merge them together in mapshaper/other open source applications. I've been looking for the past month but I am new to geographic visualization so I don't know the good spots to look yet.
Thank you so much for any help
You can try BBBike exports of OpenStreetMap data. Probably you will have to export bit by bit as there is a size limit to what you can export.
I recommend you also OpenStreet map. Osm also uses NaturalEarth for higher layers.
But I would download their database (for test, just download a small country, then you can download continents at a time).
Then I would select the features I'm interested. In your case the boundaries (region, districts, cities/municipalities), and city/hamlets names. Then you should look at the tools they have (and what you need). My simple and stupid way would be just to import such features in a GIS database, and then use it to get data. But you may find shortcuts and use directly the data without importing in a database.
Check OSM wiki, Downloading data and Planet.osm page, and other linked pages about tools. You may need to look the Feature page to known what features you want (just to discard most of the data).
If you want to use the tiles (pre-rendered images), you may need to read the term of services. It may be easier to download all data and render in your server, or to buy a service which offer you tiles. (all in the wiki).
I'm using leaflet map in my ionic application.
On offline, by default, open street map layer loads but when changing the zoom level, the map disappears.
How can I get the map to show when zoomed offline?
You need to download (pre-cache) the tile images to work with offline mode.
Download the tiles at the zoom levels you want. Map data is quite large and so you may want to limit to a small subset of zoom levels, or a geographic boundary (a city, country, etc.)
OSM's specific tile usage policy is here: http://wiki.openstreetmap.org/wiki/Tile_usage_policy But the part that applies to your app is this:
In particular, downloading significant areas of tiles at zoom levels
17 and higher for offline or later usage is forbidden without prior
consultation with a System Administrator.
That page also lists numerous "free" tile providers. Because I expect it to change "significantly" over time, I'm not gonna copy and paste it for you.
Unless you enter into some kind of contract/agreement with the tile provider, you may need to render your own or find a different tile service that will allow you to pre-cache tiles.
The details for downloading map data that can be rendered as tiles can be found on the OSM website: http://wiki.openstreetmap.org/wiki/Downloading_data
There are multiple ways to get the data from OSM, but these services and options will undoubtedly change over time so I think it safest to just refer to their website (google "download open street map data" if above link is broken).
I have used a .NET library called BruTile to manage the tile cache. But basically it is just a bunch of image files organized in a zoom/grid structure.
It would also be good for you to google search topics such as "ionic leaflet tile cache" and "pre cache map tiles ionic" and such. There isn't a lot out there yet, but this is a growing area of development.
Thoughts regarding mobile apps
If you are deploying to mobile devices it probably doesn't make sense to deploy the pre-cached tiles because (a) they will become out of date and require constant upkeep, and (b) large file size resulting in slow downloads. It would be better to download tiles while online after the app is installed.
Windows and Android phones both allow download of offline map data in their maps apps. It may be possible to leverage that data. Otherwise you would make your app work similarly: prompt user to download maps for their region, and then find a way to reasonably specify the region (geographic area) for which they need maps. It is also a good idea to let the user know how much data will be downloaded for metered data plans and device storage.
It also would be better for you to use a server other than the OSM servers, such as a paid Microsoft or MapQuest or Bing account. The OSM servers aren't capable of handling production load across every SPA that wants maps to work offline. Better to rely on the device's capabilities and built-in maps app if possible. Amazon and Azure (for example) services may be feasible for this. If you wrote your own program to run as a service on your own sever it could pre-cache tiles from the tile service (thus reducing your usage fees and server load) and then the apps would just get map data from your server. This also gives you the opportunity to get creative with your own custom tiles.
Is there an offline Map layer available for Leaflet? I don't need in detail, but basic geometry would be sufficient.
For sure you can set up your own offline map (raster tiles and/or vector shapes). The difficulty or out-of-the-box availability depends on what kind of information and level of details you want.
GeoJSON:
The easiest is if you need just world countries borders with little detail, just to get the outline. In that case, you can find GeoJSON files on Internet that contain that data for a few hundreds kB (the weight of a single normal big image), e.g. https://github.com/johan/world.geo.json
Then simply use L.geoJson(geoJsonData).addTo(map) to have it shown.
Demo: http://plnkr.co/edit/aB6p7IC2cF7xW41Ju8m7?p=preview
Downloaded tiles:
A more complex (technically and contractually) but still manageable situation is if you want raster tiles (like the OSM website for example). You can download tiles (which are just normal images) from an online server, then use them offline. This is called "tiles scraping" or "tiles bulk downloading".
As for the technical side (you may have to download thousands of individual images, depending on to which zoom level / details you want to use offline), several software are available (have a look at the above OSM Wiki link).
As for the contractual side, many tile servers (including OSM for instance) specifically require in their Terms of Use not to perform bulk downloading (as it generates high demand and uses high bandwidth on their servers). So you should look for a service that accepts this usage.
Render tiles locally:
A perfectly authorized solution (but the most technically complex) is to download the raw OSM data, and to use it through software to generate your map (whether raster tiles or vector shapes).
You can probably find services on Internet that offer to download simplified OSM data (the full database for the entire planet is ~30 GB compressed…) or for a small geographic area (see the above OSM Wiki link).
Regarding the software, the link provided by chrki in the question comment (http://wiki.openstreetmap.org/wiki/Rendering) should get you started.
In particular, you can very well generate raster tiles once, save them and get rid of the rendering software, so you can use those tiles as if you had scraped them.
You can build your own local osm server. It is quite easy using Ubuntu, because there is special packages for it. Installation process is described on this great resource: switch2osm.org
Here is an example using NeDB and modifications to this project/examples https://github.com/tbicr/OfflineMap: https://github.com/KD0NKS/APRSViewJS/blob/master/js/techpire/LayerManager.js. This only caches what the user has already viewed and avoids bulk downloading from osm servers.
There is also a Leaflet Plugin: https://github.com/MazeMap/Leaflet.TileLayer.PouchDBCached
Afternoon all,
I'm looking for a way of presenting near realtime data using line charts within my web application. Specifically, my requirements are as follows:
It should support zooming (with the mouse wheel too!), scrolling, selecting different time ranges, etc.
It should support presenting data at multiple resolutions within a single chart. The default view would be the past 24 hours (using data at one minute resolution), but if the user zoomed out or browsed back further in time then lower resolution data would be used. On the daily view, we'll have 1440 points per line (there'll be at least two). If we lower this to hourly averages when zoomed out further, we're looking at 720 points per month per line. We can lower the resolution further after that too.
Ideally, if the user zoomed into a historical date the graph would poll a server to see if high resolution data was available for that period, and if so, fetch it and update the graph area.
Streaming data support. Namely, the graph would fetch deltas every X minutes and append them to the graph.
We're open to using Flash, Java, Silverlight or pure Javascript for the presentation layer, although Flash would probably be the strongest preference.
I've spent a fair bit of time looking around for something that would fit the bill here and have (surprisingly) not found much. Here's some brief notes from my research:
a) Google Finance charts are pretty much exactly what I want to achieve, but it seems that their public version of these charts (the Annotated Timeline Chart) is comparatively very limited. It does not allow for streaming data, and if you want to redraw graph data (e.g. when changing the selected date range) you get a nasty flicker as the chart area reloads with the new data.
b) Timepedia Chronoscope charts sound promising at first glance, with support for incremental data loading. However, there seems to be no documentation or examples of this (all examples use a static dataset from .js files)
c) We've seen charts such as Open Flash Chart and others like this, but they don't have the "wow factor" that Google Finance charts do.
Any suggestions welcome!
I would suggest Highcharts - very neat SVG and javascript-based charts. There's also an example called Live Random Data which updates a spline ever second - this would be useful for your streaming requirements. The zooming and scrolling is available: Master Detail Chart, but everything depends on how much data you'll be pointing out.
In my experiments earlier this year, I managed to have around 20 graphs with over 40 plots each running smoothly in Firefox, but more than that could give out tiny delays, so in that sense yeah, you'll be better off with Flash, although I doubt that anybody would be able to read 20 splines at a time ;)
Hope that helped. Cheers!