I want to serve a large spatial dataset, preferably without webgl support, over minimal bandwith. I've seen many examples doing so. For example:
https://interaktiv.morgenpost.de/europakarte/
What would be the best way to achieve something like this? Is it possible without hosting my data on mapbox, maybe host the data on AWS S3? How should I approach this?
I found a solution. There's a technology called an UTFGrid, which are tiles in JSON format. These tiles can be generated in many ways, for example by using TileMill and MB-util. These tiles are very small, but enable the user to see featureinfo on hover. Once these tiles are generated they can be hosted anywhere you want, AWS S3 or your own web-server.
Related
I'm currently building my first real project that includes Express and MongoDB. Since it's one of the first backend-heavy projects I've worked on outside of my Udemy course, I've run into a lot of questions.
My project is supposed to be a mock-online store that would display items I have created inside of my MongoDB server. The problem I'm having is that I don't know the proper way of serving those image files that should be associated with each item (such as the image of a hat, for a hat item). I could add them directly into the project's public folder, but I don't know if that would be feasible in terms of the scalability that I want this project to demonstrate. But it doesn't seem like MongoDB will let me store images within each item. How would I go about doing that?
Sorry in advance if any of this is unclear, it's my first time posting as well. I'll try and provide more information if I need too. Thanks!
If you want a scalable solution for images, you typically would use a separate service like AWS S3 or Imgix.
There are several benefits to using a 3rd party service. You don't bog down your web server with image requests, or image resizing. You get virtually unlimited space. Etc.
In your MongoDB document, you would then store a key like /item/1.jpg or whatever, rather than the image itself. Your front-end then uses the key to request the image when someone visits your website.
If you want a turn-key solution, I recommend starting with Imgix (or Cloudinary, or some similar service). It is more expensive than S3, but it is pretty cheap for a small project, and it will get you up and running a lot faster.
I've done quite a bit of searching, but haven't been able to find anything within this community that fits my problem.
I have a MongoDB collection that I would like to normalize and upload to Google Big Query. Unfortunately, I don't even know where to start with this project.
What would be the best approach to normalize the data? From there, what is recommended when it comes to loading that data to BQ?
I realize I'm not giving much detail here... but any help would be appreciated. Please let me know if I can provide any additional information.
If you're using python, easy way is to read collection chunky and use pandas' to_gbq method. Easy and quite fast to implement. But better to get more details.
Additionally to the answer provided by SirJ, you have multiple options to load data to BigQuery, including loading the data to Cloud Storage, local machine, Dataflow any more as mentioned here. Cloud Storage supports data in multiple formats such as CSV, JSON, Avro, Parquet and more. You also have various options to load data using Web UI, Command Line, API or using the Client Libraries which support C#, GO, Java, Node.JS, PHP, Python and Ruby.
What are the benefits and / or needs to use Geoserver in the development of a web mapping application?
In other words, is it required to use a server such as "Geoserver" for the optimal development of a web mapping application?
I have created a web mapping application with Leaflet to publish geological and geophysical data. All data are already conditioned to be displayed in an Internet browser (data formats and styles are ready). My data and assets are stored in folders in the Apache directory of my PC. The application works and runs "perfectly".
Why should I implement Geoserver (or MapServer)?
I will really appreciate suggestions/opinions.
Of course, there is no reason that you have to implement GeoServer. However, there are variety of reasons why others do. Here are just a few.
GeoServer would allow you to manage datasets that are far larger than those that can be managed within a browser.
GeoServer can serve data through a variety of services, including WMS, WFS, WCS, WPS, etc.
GeoServer / GeoWebCache continue to perform well in environments with lots of geospatial data and lots of users.
If you thought that you might want to consume a variety geospatial data sources, then GeoServer is useful. It can consume all sorts of other geospatial data sources, including ESRI, PostGIS, OGC, etc. In fact, via GeoWebCache, it can even cache that data within your local network and reduce traffic to the external servers. GeoServer can even unify data from these disparate sources onto single layers (group).
GeoServer has lots of great styling options. You can use SLD, CSS, and Mapbox styles. Styles can be property and scale-sensitive.
GeoServer can transform data from a source on the fly.
Hope this helps.
Does OpenStreetMap just provide GIS data? To render maps and have a browser based UI to show maps with data, would a developer need to use something like Mapnik?
OpenStreetMap provides data yes, and you'll need specific tools to make map with them. You first need to put the data in a true GIS format, which OSM is not. Think about PostGIS, an osm2pgsql tool is designed for that. Then, you'll need a rendering tool, i.e. a cartographic engine. You can use Mapnik or MapServer. Tilemill or Kosmtik can help you designing your map styles.
Take a look here.
yo can export data form OpenStreetMap and render your own map.
Take a look at
http://www.slideshare.net/jones139/rendering-openstreetmap-data-using-mapnik
http://www.slideshare.net/artemp/mapnik-sotm-2007
There's a fuller list of rendering libraries for OpenStreetMap data here, both free and commercial.
http://wiki.openstreetmap.org/wiki/Renderer
Disclosure: I am the developer of one of them.
OpenStreetMap provides map data out to a wide sprawling ecosystem of different downloads and tools offering developers different map-making and map using possibilities. Here is a diagram I drew which captures some of this:
So the data comes out of the core of OpenStreetMap, in the form of 'planet downloads', one big unwieldy blob of XML (and diffs), but from there it fans out into a range of other offerings to developers, many of which are provided by 3rd parties. Taken together "OpenStreetMap & Friends" provide lots of possibilities.
If you are inclined to run your own rendering system, then yes, Mapnik is a good choice. Powerful and scaleable (most successful OSM tile servers run Mapnik), but takes a bit of setting up. You normally load OSM XML data into PostGIS, and run mapnik from that (tutorials on switch2osm.org)
If you are not inclined to run your own rendering system (for more lightweight web development tasks this seams reasonable!) you don't have to! You can configure LeafletJS or OpenLayers or others to fetch map tiles from a tile server run by somebody else. There are many providers. See the providers list on switch2osm.org and the Tile Servers list.
I was looking at the Bing Heat Map Creator here.
It looks pretty cool, but I wanted to build something a little more dynamic with it. Unfortunately it appears as if the only way to use it is by uploading a CSV file manually. Does anyone know of another way to get the data loaded into this?
Thanks.
We developed the Bing Heat Map Creator, which is a Map App running on the Bing Maps platform. It is certainly possible to build similar heat mapping features in a more automated mapping application but likely would not be done as a map app due to limitations with map apps APIs. Ping me if you want to discuss further.
Regards
Steve