I have downloaded shapefile boundary data from the UK ordnance survey and have converted it to geojson format, using QGIS. There were no errors reported during the conversion, and given the fact that the data came from a governmental organisation, I think it is a reasonable assumption that the data made available is correct, coherent and intact (which has an effect on how I narrow down the error source).
I use mapbox to display this data, and retrieve it through a webservice. When I load the data with mapbox, it causes mapbox to "bluescreen", i.e. completely crashes the browser tab and disconnects the development console (Chrome, FF and IE). There are far too many polygons/features for me to investigate "manually", and I can't contain the error - even try/catch doesn't help.
I have started to systematically halve the data set, and it turns out that the error is contained somewhere in boundary areas beginning with letters a-m. I know that I could continue with this approach and at some point find the error(s), but I would like to know if you have some tips for me to
Check WKT/geojson/polygonal integrity. Technology is unimportant to me (C#, Java, JS, whatever...) as long as it's a concise process. (Apparently, Postgres/Postgis has this but all my data resides in an object database, and if there is ANY other means to check, I'd prefer that :-))
Check geodata integrity specifically with respect to mapbox consumption.
My assumption is that "correct" (i.e. error-free and correctly-formatted) geodata will render correctly in mapbox, but as my problem has shown: somewhere in between the official data set from a governmental website, error-free conversion using a well-established tool and rendering in mapbox using the latest version, it still goes wrong.
Thanks for your hints in solving this !
EDIT: amazingly, Firefox seems to handle this better than Chrome. Doesn't crash using the same data.
First of all I fear that there is no right or wrong answer to your question and I don't know how you want to accept an answer. This is normally a a bad sign for a question.
Second I would not take a data set for "must be valid" even if it comes from a government. Exporting data into a format always can have some edge cases or they might not follow the geojson specification as tight as you would have liked them to.
However this is what you can try to do:
Often the data is too precise to work with (over fitting) and you should try to simplify your geojson data with a tool of your choice to reduce its size. You could use one of the following to do that task (there are more ofc.)
QGIS
http://mapshaper.org/
https://www.npmjs.com/package/simplify-geojson
See if you can load the simplified version into the browser
You could try to validate geojson with an external tool like http://geojsonlint.com/ or http://geojson.io/ if your file is not too large.
Try another browser to see if it can handle data better. Its strange that your browser crashes. If chrome runs out of memory it will normally notify you about that issue.
Other than that I think it would stay a manual task to hunt down the problem.
Related
Good morning all,
I'm looking to know how variables are stored in a flutter application. I don't really know the world of mobile, so I'm a little lost. I'm on a project in which I have to store information when launching the application, which I retrieve via an API (they are then deleted when the user closes the application). Currently, the amount of data is reasonable, but I would like to know how much data I can temporarily store in my application (10kb? 10mo?), what are the risks with this way of doing things, does it have slow down the app...
I couldn't find anything on the flutter doc (https://docs.flutter.dev/perf/memory - At the time of writing this message, the line returns to an empty page with the letters TBD). I couldn't find any articles explaining the specifics of memory, performance related variable storage, related to flutter.
Thanks for your help !
The Dart compiler used by Flutter is actually very good at optimizing code, so variables have almost zero overhead.
Having said that I must add that using global variables is not a good way of storing data in any kind of software. I suppose you select a suitable state management technique to handle your data.
Regarding the amount of data, I don't think 10mo will be too much on a modern mobile device, but as always, ymmv.
I've only started using Graphstream a couple of weeks ago, and so far it is working pretty well, but I have one issue that I have not been able to solve:
I am displaying data which for smaller datasets is nice to view as a MultiGraph, and for larger datasets becomes to hectic and would be nicer to view as a SingleGraph. What I'm trying to accomplish is at runtime to either dump the data from a SingleGraph into a MultiGraph (and vice versa) on some user action. Either that or use MultiGraph all the time if there is some way to suggest to it to display only a single edge between nodes?
So far my only semi-reasonable-looking-to-implement-solution that I have thought up was maintaining both a SingleGraph and MultiGraph and swapping them out with the viewer when the user performs the swap action, but that seemed kludgy (and would have to do every operation on both graphs). I searched around the docs and web for quite awhile but have not come up with a reasonable solution so far...so if anyone knows of an easier way, please let me know :)
EDIT: Solution found (good enough for us anyway)
So what I ended up doing is adding another ui.class to the edges that are known to be duplicates at graph construction time, which for now I made a much less obvious color, so they fade into the background when you're trying to look at the singlegraph links. I could also use this to iterate over the graph and hide them (or just redefine the css to be the same as the background color...or something)...It's not quite what I wanted, but looks like it will serve us well enough for the near and medium term on this project.
Thanks,
-Dave
We need to build an image processing application for smartphones (mainly for iPhone). The operations consist of:
image filtering
composite geometrical transformation
region growing
Also, the user is required to specify (touch) some parts of the image. Those parts would serve as inputs for the app. Example: eyes and lip in a face.
We have built a desktop version of this app. The processing part is quite heavy and we extensively used BufferedImage class.
Should we use CodeNameOne for building this app? If not then what alternatives do you suggest?
Please consider the following factors:
Performance
Ease of writing the code (for image processing)
I gave an answer for this in our discussion forum but I think its a worthwhile question for a duplicate post:
Generally for most platforms you should be fine in terms of performance except for iOS & arguably Windows Phone.
Codename One is optimized for common use cases, since iOS doesn't allow for JIT's it can never be as fast as Java on the desktop since its really hard to do some optimizations e.g. array bound check elimination. So every access to an array will contain a branch check which can be pretty expensive for image processing.
Add to that the fact that we don't have any image processing API's other than basic ARGB and you can get the "picture", it just won't be efficient or easy.
The problem is that this is a very specific field, I highly doubt you will find any solution that will help you with this sort of code. So your only approach AFAIK is to build native code to do the actual image processing heavy lifting.
You can do this with Codename One by using the NativeInterface API which allows you to invoke critical code in native code and use cn1lib's to wrap them as libraries. You would then be able to get native performance for that portion of the code but this would only make sense for critical sections in the code. If you write a lot of native code the benefits of Codename One start to dissipate and you might as well go to native.
I am looking for a black-box type program/script/Selenium add-on/etc. that will generate different kinds of data, input it into a web form, hit SUBMIT, and test whether the correct response occurred.
I have found programs that would test for one kind of data or another, but I'm looking for a more comprehensive program that would test a wide variety of possible user inputs, so you can verify that the form is accepting or rejecting the inputs in accordance with program specifications.
For example, the program could verify that an error occurs when:
a number is input into a field that is supposed to be text
a required field is left blank
a SQL injection is present
(checks for other things I have not thought of)
Note 1: I started using Selenium IDE to insert random data, but realized I would have to write a great deal of loops of test cases to test a variety of combinations of different kinds of data. It seems like this would be a very common issue, so I suspected there must be some solution already out there. After much Googling, I could not find one.
Note 2: I realize this can be done via unit testing or other white box methods, but I am looking for a black box solution here.
Thanks!
Selenium IDE is great for prototyping your tests, but for anything past trivial solutions you really need to look to tools/frameworks like Watin or Webdriver (Selenium 2) to create useful tests which are maintainable over time. You should also have a look at Telerik's Test Studio (Disclosure: I'm their evangelist for that tool) which is another great functional test solution.
Regardless of which toolset you decide on, it sounds like you're looking for an approach to data drive your various tests: validating combinations of given inputs create expected output conditions. There are a number of posts and articles around on that. I have a blog post with a brief example of data driving an AJAX menu using C# and Webdriver which might be of help.
For my Windows Phone Mango app, I want to make overlay a heatmap on Bing Maps, and a tile overlay seems the best way to do it. I've been having trouble finding any good documentation or code samples to work from. It seems like most people are pointing the tile source to a web service. I'd rather render the heatmap on the phone itself - is that possible?
One of the main reasons to use tilelayers to represent data on a map is that the computation and rendering involved in creating the layer is performed in advance, generally as a one-off or infrequent task. Then, at runtime, the only work the client needs to do is to retrieve the pre-rendered tile images from the server and display them straight on the map, which is a simple, low-resource activity.
Rendering tiles can be a resource-intensive task, both in terms of processing and memory usage - for example, I can only render about 3 tiles per second on a quadcore desktop machine with 8Gb RAM. Even if it's technically possible to create the tiles dynamically on a handheld device, the performance is almost certainly going to be unacceptable for any user. You've also got the question of how you're going to store the data from which the layer is created. Since you're talking about plotting a heatmap, I'm guessing you have a reasonably large dataset of points - did you envisage these stored locally on the device, or retrieved over the network? (either will create different problems).
Basically, while it may be theoretically possible to create tile layers dynamically on the client, doing so would negate almost any benefit of using tilelayers in the first place, which is why you probably won't find any code samples explaining how to do so. Perhaps you could explain your comment why you'd rather create the heatmap on the phone?
It's pretty easy to create a server-side tile renderer using .NET or PHP that renders and server tile images to a Bing Maps client, or you can use an existing map rendering library such as mapnik.org or geoserver.org.