Currently I am trying to develop a multi-layered Leaflet map using GeoJSON (for country, states/provinces) and CSV (for city data). I want it to go down all the way to city level and that requires layers of Country, State/Province, and City data. I have all the Country data I need in the format I require (GeoJSON) and I have a decent source for City data in a CSV format.
However, I only have USA, Canada, Brazil, and Australian states/provinces and I have been looking around but haven't been able to find a reliable source such as NaturalEarth (which is where I initially got my states/province data from).
Does anyone have a resource they could point me towards? Even if it is multiple, hopefully I can merge them together in mapshaper/other open source applications. I've been looking for the past month but I am new to geographic visualization so I don't know the good spots to look yet.
Thank you so much for any help
You can try BBBike exports of OpenStreetMap data. Probably you will have to export bit by bit as there is a size limit to what you can export.
I recommend you also OpenStreet map. Osm also uses NaturalEarth for higher layers.
But I would download their database (for test, just download a small country, then you can download continents at a time).
Then I would select the features I'm interested. In your case the boundaries (region, districts, cities/municipalities), and city/hamlets names. Then you should look at the tools they have (and what you need). My simple and stupid way would be just to import such features in a GIS database, and then use it to get data. But you may find shortcuts and use directly the data without importing in a database.
Check OSM wiki, Downloading data and Planet.osm page, and other linked pages about tools. You may need to look the Feature page to known what features you want (just to discard most of the data).
If you want to use the tiles (pre-rendered images), you may need to read the term of services. It may be easier to download all data and render in your server, or to buy a service which offer you tiles. (all in the wiki).
Related
I understand that most of the data in OSM for USA is coming from TIGER. It seems that this doesn't have complete road key attributes such as speed limit, maximum weight, allowed / restricted turns, lane change etc.
I found that some state level DOT do have public datasets available for the roads within their jurisdiction, however some states don't have that. I wonder if the OSM community knows if the data already available at these state DOT's was loaded in the OSM?
Please also let me know if there was a netter place to ask this question?
Many thanks,
It is stated in the OSM webpage as well as in other GIS blogs that OSM doesn't have much data on key attributes of roads ie truck restrictions, however some DOT do have data on that.
In OSM pages it wasn't clear if that data was loaded.
There's been an awful lot of discussion about data imports, especially within the USA, over the years. I suspect that if data is available and hasn't been imported into OSM it's for one of two reasons:
the quality isn't very good
the licence isn't compatible
The relevant OSM wiki page that explains what would someone suggesting a new import need to do is here. With regard to the "community buy-in" step, I'd suggest contacting the OSM USA community in the first instance via the forum, mailing list or OSM US' slack workspace.
We store an manipulate geodata in postgresql(postgis) and use QGIS to view and edit.
My team and I need to build reports around that data. The reports are quite text heavy and need some good visual formatting. My colleagues are used to MSword or adobe indesign, so we agreed to use one of these two programs to build our final reports.
Our documents will include around 10 indesign/MSword tables. The data for these tables lives in dedicated views in the pg database. The Tables inside the reports should grow with the underlying data when I do my editing. So that everyone can focus on "their" work and to reduce data redundancy.
What would be the best way to do this?
I looked at some (ODBC) catalog plugins for indesign but i'm not
really sure if this is the way to go.
I'm no developer. I need to learn/look into the respective technologies involved anyway
so I wanted to define my tech-stack first (maybe hire someone to do it properly).
Maybe someone can give me a general idea / point me in the right direction with this.
Thanks!
I'm looking to replace Google Maps by something self-hosted and free. After searching for some time I found OpenMapTiles. I'm very new to this stuff, so there are many things I don't understand, namely:
I'm not sure I qualify for pre-rendered tiles, can I generate my tiles myself from osm data?
I would like to have something like google terrain map, with contour lines, is that possible with OpenMapTiles?
I also have some GPS tracks that I would like to show like overlay over base maps (OSM or Google), and I want them clickable (clicking open a link or popup or something like that), is that possible?
Is OpenMapTiles server heavy to run?
Thanks
That's totally possible, I've done it myself. However, it takes a while to do everything.
1. Generate MBTiles
First of all, I'd suggest you start taking a look in the Github repo of OpenMapTiles: https://github.com/openmaptiles/openmaptiles
To generate an MBTiles file, for a country is quite manageable to do in your own computer. And basically you can do so by executing the main script with the name of the country as a parameter. Like this:
./quickstart.sh portugal
After the script creates a Docker machine, downloads the OpenStreetMap data for the country, processes it, and generates the MBTiles file, then you'll be done.
For me, with an average computer, for countries like Spain or Portugal, it takes about 4-12 hours to do so. But it really depends on the country. Norway takes forever (about 4 days), I believe due to its very long coastline.
If you want an MBTiles file that will work for the whole planet, then you need a more powerful computer, that specially has a big SSD and lots of RAM. Between 32 and 64 GB of RAM and 250-300 GB of SSD.
2. Set up a tile server
After that, you can use this project https://github.com/urbica/tessella to run your own tile server to be consumed form a website or mobile app.
The resources of the server will depend on the size of the MBTiles file you want to use in the server.
3. Enable contour lines
This is possible too, but a bit more trickier. I'd suggest you use Maptiler as a service for this.
Whenever I achieve this matter, I'll go back to you with the information.
If the tiles you generate don't contain the contour / elevation data, then styles won't help. The tiles from OpenMapTiles do not have the contours inside.
4. Web and mobile libraries (to display GPS tracks, waypoints, etc)
This is possible as well. What technology you want to use here?
For web apps:
There's a well established library, Leaflet: https://leafletjs.com
More recently, looks like MapLibre is taking the lead (fork from Mapbox GL JS): https://github.com/maplibre/maplibre-gl-js
For mobile apps:
VTM is an open source iOS and Android library: https://github.com/mapsforge/vtm
There's Carto, which is more recent but it has a great support: https://github.com/CartoDB/mobile-sdk
And MapLibre has a native Android and iOS library too: https://github.com/maplibre/maplibre-gl-native
Is there an offline Map layer available for Leaflet? I don't need in detail, but basic geometry would be sufficient.
For sure you can set up your own offline map (raster tiles and/or vector shapes). The difficulty or out-of-the-box availability depends on what kind of information and level of details you want.
GeoJSON:
The easiest is if you need just world countries borders with little detail, just to get the outline. In that case, you can find GeoJSON files on Internet that contain that data for a few hundreds kB (the weight of a single normal big image), e.g. https://github.com/johan/world.geo.json
Then simply use L.geoJson(geoJsonData).addTo(map) to have it shown.
Demo: http://plnkr.co/edit/aB6p7IC2cF7xW41Ju8m7?p=preview
Downloaded tiles:
A more complex (technically and contractually) but still manageable situation is if you want raster tiles (like the OSM website for example). You can download tiles (which are just normal images) from an online server, then use them offline. This is called "tiles scraping" or "tiles bulk downloading".
As for the technical side (you may have to download thousands of individual images, depending on to which zoom level / details you want to use offline), several software are available (have a look at the above OSM Wiki link).
As for the contractual side, many tile servers (including OSM for instance) specifically require in their Terms of Use not to perform bulk downloading (as it generates high demand and uses high bandwidth on their servers). So you should look for a service that accepts this usage.
Render tiles locally:
A perfectly authorized solution (but the most technically complex) is to download the raw OSM data, and to use it through software to generate your map (whether raster tiles or vector shapes).
You can probably find services on Internet that offer to download simplified OSM data (the full database for the entire planet is ~30 GB compressed…) or for a small geographic area (see the above OSM Wiki link).
Regarding the software, the link provided by chrki in the question comment (http://wiki.openstreetmap.org/wiki/Rendering) should get you started.
In particular, you can very well generate raster tiles once, save them and get rid of the rendering software, so you can use those tiles as if you had scraped them.
You can build your own local osm server. It is quite easy using Ubuntu, because there is special packages for it. Installation process is described on this great resource: switch2osm.org
Here is an example using NeDB and modifications to this project/examples https://github.com/tbicr/OfflineMap: https://github.com/KD0NKS/APRSViewJS/blob/master/js/techpire/LayerManager.js. This only caches what the user has already viewed and avoids bulk downloading from osm servers.
There is also a Leaflet Plugin: https://github.com/MazeMap/Leaflet.TileLayer.PouchDBCached
I am using Scantron Cognition Enterprise at work to capture data from scanned forms. Building these forms is tedious at best, especially when it would be nice to have a library of pre-built objects to use. Unfortunately, documentation and on-line resources are scarce.
Does anyone have any pointers to find some resources for this tool?
Hey Jason, believe it or not, Scantron is STILL the standard, but this is not the Scantron you probably remember. Although OMR (bubble) forms are still used extensively in education, there are a lot more advanced technologies available to be added to them today.
Concerning Cognition, I looked through the available tags and these would fit:
"document-imaging" - Cognition is a document imaging product and can feed images and index values into most commercially available document storage applications
"OCR" - Optical Character Recognition, or reading machine print.
"ICR" - Intelligent Character Recognition - reading hand writing, usually in a constrained print format (one letter per box like a credt card application.
"datacollection" - the key purpose of Cognition is data collection.
However, there is not a tag for "OMR" - Optical Mark Recognition, or reading bubble choices, similar to the basic Scantron forms of the past. Also, I could not find one for "Key From Image", another purpose that Cognition is used for.
I am a Cognition user as well as someone who markets it and I know that there are a large number of users in North America. Many corporations that use Cognition use it for sensitive HR functions and so might not have their usage of it posted in a searchable format. Many other organizations use it for safety inspections, insurance data entry, and also for testing and surveys - basically anywhere you have a large number of paper forms and need all of the data quickly entered into a database. Many users are using Cognition for sensitive applications are so are not likely to share, but I can share a few I have, you could also contact your Scantron rep and they might have something they could share as well. I have some decent ICR fields built for name, e-mail, address, etc. The ICR fields are best when you build in your own dictionary or database look-ups. The OMR fields are the hard ones to build, but I have a few of these as well. The easiest way to share these is to send you the form that already has the field built into it. You can build your own lookups from txt, xls or db files.