It took nearly 7 hours for me to download the whole tiles. Production-package shows that Business PLUS users can use the automated update service to update the tiles incrementally. But I can not find the tool or system. So how can I do that?
With the higher Production package, you can download the smaller incremental set of map tiles - containing only the places which have changed since the last update (typically a week).
This allows proceeding with the faster updates, where your locally stored 60 Gbyte planet file is actualized by a script using a small downloaded incremental tileset.
Please contact us (OpenMapTiles.com) to get the access to this system.
Related
I have been exploring using PouchDB as an offline cache for raster map tiles. Specifically, for Leaflet. I have just finished some preliminary tests which I thought I would share.
I have created a "JsFiddle" (actually I prefer CodePen these days), as a playground for showing how to use PouchDB to cache off-line raster map tiles.
http://codepen.io/DrYSG/pen/hpqoD
The Algorithm it uses is as follows:
Test for presence of XHR2, IndexedDB, and Chrome (which does not have
binary blobs, but Base64). and show this status info
Fetch a JSON manifest of PNG tiles from GoogleDrive (I have 171 PNG
tiles, each 256x256 in size). The manifest lists their names and
sizes.
Store the JSON manifest in the DB
MVVM and UI controls are from KendoUI (This time I did not use their
superb grid control, since I wanted to explore CSS3 Grid Styling).
XHR2 is from: https://github.com/p-m-p/xhr2-lib/blob/master/test/index.html
I am using the nightly build of PouchDB
All files PNG file are stored on Google Drive (NASA Blue Marble.
I created the tile pyramid with Safe FME 2013 Desktop.
http://www.safe.com/fme/fme-technology/fme-desktop/overview
Before Pressing the button "Download Tiles" Check to see that the manifest has been stored in the DB, and that 171 tiles are present. If you already ran the test then your PouchDB is going to already have tiles in the DB, and you will get errors. In that case, Press Delete DB, and then reload the page.
When you press "Download Tiles" The following steps occur:
The Manifest is fetched from the DB
A XHR2 Fetch loop grabs the PNG blobs from GoogleDrive.
As loop runs, it starts saving the Blobs into PouchDB.
Note: Fetching and Saving are on overlapped threads (think
co-routines), since those (fetch & store) operations are running
async on separate threads.
When the Fetch loop is done it reports the elapsed time.
Note: This time is not pure Fetch work, since PouchDB
putAttachments() are running in parallel.
When all the Tiles are saved, it will report full statistics, and
display a tile fetched from PouchDB.
The Blob-Rate the total fetch and store time per each png tile
Right now Chrome is running fine. Firefox is very slow. I found this out a few months when I did a native IndexedDB API. So I don't think this is a PouchDB issue. Probably more due to FireFox using SQLlite which is a relational approach to a no-SQL DB.
IE10 is not working. This is sad, since my prior tests with IE10 shows it has a fantastically fast IndexedDB solution: Storing Image Data for offline web application (client-side storage database)
A Must read article on "How the browsers store IndexedDB data"
http://www.aaron-powell.com/web/indexeddb-storage
Note: FireFox uses SQLlite for the NOSQL IndexedDB. That might be the
reason for the slow performance. (blobs stored separately)
Note: Microsoft IE uses the extensible storage engine:
http://en.wikipedia.org/wiki/Extensible_Storage_Engine
Note: Chrome uses LevelDB http://code.google.com/p/leveldb/
I will build a dashboard system for my apps, where a page will have several widgets that draw charts, tables and glyphs representing potentially unrelated data.
The client will be HTML5 and I can push for only modern web browser.
My big problem is what backend use for this. I want to store "tables" for use in the charts and in real-time update the widgets.
For example, a invoicing widget will show how much $$ have been collected today. In the "table" will have a row for each total of the invoice:
inv = 1; total = 50
Total: 50
and the widget will draw that. When new data is pushed:
inv = 2; total = 100
Total: 150
The widget will show in realtime the total to the end-user.
The data is private for the user company. Eventually I will need to purge too old data (ie: I only need to keep as much data is necessary to proper evaluation of the info need for the end-user. For example, only keep 1 month of invoicing totals).
I'm thinking in use something like http://www.firebase.com/ or http://pusher.com/ but I suspect only solve the "notify in realtime" part of the equation. As far as I understand, they not let me get past data (ie: If the data is update in the weekend and the user open his dashboard to see what happened)
Then I see http://derbyjs.com/ and the possibility to use mongodb.
I wonder which backend/platform will bring me closer to the build of this system. I have experience with python/django/.net/postgress but could accept the use of something else if solve best this kind of app behavior.
Firebase offers both the "notify in relatime" part that you mention, as well as persistent data storage. Take a look at the tutorial, which walks you through building a real-time persisted chat app (the past chat messages are stored in Firebase and are sent back to the client every time you reload). And you can do much more complicated stuff like the real-time charts / widgets that you mention as well.
The big limitation with Firebase right now is that we're in closed beta and the data is currently unprotected (anybody can read and write your data). The security features are coming soon though.
Some other backend platforms you may want to evaluate are: Meteor and Simperium. Firebase and Simperium are cloud services where your data is stored in the cloud and you don't have to manage any servers of your own, while Meteor and DerbyJS are platforms that you have to install and run on your own server.
I would recommend signalR. It's amazing and you can literally do anything with it. Check it out: www.signalr.net and if you have any problems simply go to www.jabbr.net You will find a very helpful community there. I implemented a notification mechanism similar to facebook together with real time monitoring and a small chat in the same web site.
Some discussions in Stackoverflow say that storing the picture in DB is a bad idea (a because overtime, the number of images get large & may lead to app crash). So, either
a. the image can be stored in the iPhone itself & only its location can be stored in the DB
Potential Issue: The image might get removed (outside of the app) & the app might not be able to load them the next time
b. the image can be shrunk to a small size (say, 100*100 pixels) and stored in the DB
Potential Issue: will size be an issue if the image is shrunk to just 100x100pixels?
c. Doing both (a) & (b). So, small versions of the images will be stored in the DB and then, retrieved & displayed in the App, whereas, if the user chooses to see the original version of the image (whose probability is low), then that'll be fetched from the local directory & shown.
Your suggestions please? In my opinion, (c) seems a good option to go in for, in case (a) has the potential issue mentioned.
It really depends on where the images are coming from. Are they being downloaded from the Internet (or imported from the user's library / camera) after the user installs the app, or are they bundled with the app from the App Store?
If the images are being downloaded / imported, the best solution is to store images in the filesystem following the recommendations in http://developer.apple.com/library/ios/#qa/qa1719/_index.html
Basically, if the images cannot be replaced or recreated, store them in the <Application_Home>/Documents directory and do not set the Do Not Backup attribute. These items will be backed up to iCloud, and this data does persist for at least some amount of time even if the app is deleted from the device. (Remember that your users do not have unlimited iCloud space. Be responsible.)
-However-
If the images are bundled with the app, the best solution is to import them directly into your Xcode project and reference them from there. This way you know they are always available even if the user deletes and reinstalls the app.
I would definitely stay away from storing image data in the databased whenever possible. There are simply better, more efficient ways in most any scenario.
I'm traveling all the time (every 2-3 months, I'm in a new city or country), with no real permanent address. I've managed to work out all the kinks over the last couple of years...except having a good backup/sync solution.
I have a macbook pro & a thinkpad w701 (which runs two different VMs). It's a pain in the ass because making changes on one machine (such as adding some new music or updating some presentations) requires me to keep track of what changed where. And then every couple of weeks, after syncing the three different images, I try to manually sync it out to a backup drive that I carry around.
It's pretty much the most annoying thing ever...especially when I sometimes make changes on the backup drive and I have to remember not to override them.
What I'd really like is something simple that has more of a version control like workflow:
I can push out changes to some
central server (like a commit.
Example: I add some changes to my
music directory and then I can just
commit those changes to backup)
Before the backup happens, I'd like to see a "diff": what files will be
overridden, which one's newer, etc
I can access my files off the server (if I'm making an audio mix and need
to pull out some songs, I'd like to get them from the server. All the
backups can't just be one big binary
compressed zip blob)
Dropbox comes pretty close but it lacks the "commit" & "diff" functionality. I thought about using Amazon AWS but that falls short because I can't see diffs and can't access my files directly off aws.
Any ideas? Or any other solutions? I guess what I'd really like is TimeMachine in the cloud or maybe even a NAS that's securely accessible through the internet
You might want to use rsync. It's a unix synchronization tool you can use it on windows and unix variants (including Mac OS X). It uses delta copying to minimize transfer and hardlinks to minimize backup size.
You can access all files in every backup as though they were normal files. Diffing can be done using traditional tools. It is all command-line based so if you don't want that you will need to find GUI tools, but I don't know which you could use.
You would need a server with a rsync deamon/service. I don't know if there are providers for it but you can set up your own VPS starting at a few dollars a month.
Have you looked at Amazon S3? S3 is data storage mechanism and there are bunch of tools to "sync" your local directory with S3. Some of the tools are:
http://www.vinodlive.com/2007/08/20/amazon-s3-storage-tools/
Out of these , S3Sync should do what you are looking for, I.e. submit only changed files and a mode that would tell you changes before submitting them.
Is there a limit to how much persistent storage a single iPhone app may consume?
What does save set the error argument to if the iPhone hits a per-app limit? What if it hits the hardware limit?
Is it possible to limit the number of objects stored for certain entities? If so, what's a good approach to doing this?
acani, an iPhone app I'm working on, downloads the nearest 20 users from the server and saves them to Core Data. After using the app for a while, the users SQLite table could become rather large. How could I limit it? What should I limit it to? Once this table has reached capacity, how could I make it so that newly downloaded users replace the oldest downloaded users?
Thanks!
Matt
I don't know the answer to the limits questions, but I would think you would want the maximum amount of data to be limited well ahead of that. There are some iPhone apps (games0 which take up a large amount of storage (I think Myst is something line 1.5G). But if you allowed your database to grow to those sorts of sizes you might start to impact on the storage the user has for their other applications.
I'd be inclined to suggest that your application needs to have some sort of database house keeping implemented. You will have to write this. Either automatically triggered or manually triggered by the user. For example you might want to setup a settings option where the user can specify how many "old" users it wants to preserve. If users are being added automatically based on location, what sort of algorythm would a user most likely want to cull the list with?
There is a 2GB limit for apps from the App Store but as far as user data goes, you should be able to basically fill the disk. When that happens, your saves will start to fail, I believe with 'NSFileWriteOutOfSpaceError' bubbled up from the PSC.
As far as limiting entity space, there's no Core Data support for this - you'd have to handle it programatically. You could extend the validation system to check for certain conditions (free space, number of entities) and fail an insert or update if these didn't match your criteria.
If you want to delete old users, just sort the results and delete the first/last one.