I am interacting with the cloudkit dashboard and looking at data collected by my app.
How can I export all the data from the dashboard (data-> csv or json) so that I can do some analytics on it?
Thanks!
I don't think Apple will ever provide an export feature.
The system is capable to collecting more than 40 events per second. This could very quickly be massive amount of data.
Instead there is a possibility to query the system via an API, so you can build an external website to query your results and possibly export your data from there.
Here is how I export my iCloud data on my Mac using this wonderful DB browser called DB Browser for SQLite.
I run my app on Simulator. The app syncs with iCloud and downloads both public and private databases. Once the sync is complete I have all the data on my simulator. The data is essentially in SQlite format stored in a file which I locate using command in my app:
print("\(NSHomeDirectory())/Library/Application Support/")
and the result is something like:
/Users/zeeshan/Library/Developer/CoreSimulator/Devices/34A4ADC6-1B67-4339-B67F-C4B6DDA46B07/data/Containers/Data/Application/E8F9486B-B40B-47D0-84C1-1043241E68EA/Library/Application Support/
And here .sqlite file is saved. In my case I have two files, private.sqlie and public.sqlite which is how I have named them in my code.
Now I open these files in DB Browser for SQLite. And then rest is simple, just export the opened file as you would do for any SQlite file.
Attached are the screenshots.
Related
I am new to sqlite. I successfully implemented CRUD methods to CloudKit in my app. I am not using Core Data. I read in the Apple developer documentation on Remote Records that:
CloudKit stores your records in iCloud and uses subscriptions to notify your app in real time about record changes. You then use change tokens to handle these changes efficiently. Additionally, you can improve your app’s performance and support offline use by storing records in a local cache.
I am interested in this local cache. After some digging around, it looks like this cache is automatically created and stored in the container referenced in AppDelegate.swift. I put a breakpoint in my code, and in the Debug Console, I typed: po (UIApplication.shared.delegate as! AppDelegate).persistentContainer
to discover where the .sqlite file is located. It shows:
<NSPersistentCloudKitContainer: 0x600002934c40>
CoreData: debug: CoreData+CloudKit: -NSCloudKitMirroringDelegate observeChangesForStore:inPersistentStoreCoordinator:: <NSCloudKitMirroringDelegate: 0x6000007e9d40>: Observing store: <NSSQLCore: 0x7f8276e1cc70> (URL: file:///Users/xxxxxxxx/Library/Developer/CoreSimulator/Devices/30CE52B9-E246-4B40-82F4-13DBE48EB7D6/data/Containers/Data/Application/5E26076E-12F0-4E86-9B1D-BED642C01BB6/Library/Application%20Support/MyAppName.sqlite)
In a Terminal window, I typed:
open file:///Users/xxxxxxxx/Library/Developer/CoreSimulator/Devices/30CE52B9-E246-4B40-82F4-13DBE48EB7D6/data/Containers/Data/Application/5E26076E-12F0-4E86-9B1D-BED642C01BB6/Library/Application%20Support/MyAppName.sqlite
It opened the .sqlite file in Liya. I see a bunch of tables like these:
.
I inspected each table in the list. None of the table names match the name of the Record Type and none of the field names match the names of the Record Fields that I see in the CloudKit Dashboard. So, is this .sqlite file really the cache that CloudKit created locally to "mirror" the database in iCloud? If so, how do I make use of this local cache when I can't even recognize the table name or field names used in my app?
The database you are viewing is, in fact, Core Data. When you use NSPersistentCloudKitContainer with CloudKit, your app will use Core Data as the local cache.
The SQLite database that Core Data uses is not meant to be manipulated directly since the Core Data framework manages how and when data is transacted with SQLite. You should only interface with the Core Data framework in your app and not worry about what is happening under the hood in SQLite.
If you want raw access to SQLite and full control over how your data is named and structured, then you will need to do things manually. You will need to setup and manage your own SQLite transactions and then manually manage how you sync with CloudKit.
Here are a couple good options for SQLite frameworks:
SQLite.swift: https://github.com/stephencelis/SQLite.swift
GRDB: https://github.com/groue/GRDB.swift
I currently have an iOS app that provides a sqlite DB for the data backend to the app. This data is basically a list of information. Within this app I allow the users to mark records as bookmarked (sets a value in the DB). The problem here is when I post updates to the app via the internet (update through software) for data changes, the new downloaded DB wipes out the old one thereby removing the user customizations.
Any ideas on an easy method to search the current sqlite DB for those changes, store them temporarily, then import the new DB and transfer the changes to the new DB? Could I for instance use a Core Data element at the same time as using the sqlite DB backend? Maybe a key/value pair system?
Any suggestions would be greatly appreciated.
Thanks
-LK
The way I've done it is this:
For your new version, rename your .sqlite file e.g. foo-v2.sqlite. Then, during initialization, check to see if the sqlite file for the old version is there - if so, copy over the necessary info and then delete the old sqlite file.
I'm using Core Data to store some sensitive information. So far, I've applied hardware file encryption to the SQLite file used by the Persistent Store Coordinator. I've done this by setting its file Attributes (NSFileProtectionKey to NSFileProtectionComplete).
I'm storing some image data as Binary Data in the Core Database and I've checked off the "Allows External Storage" and "Store in External Record File" to prevent bloating of my SQLite datastore and to improve performance.
I'm presuming that the data files automatically stored outside of the SQLite database by Core Data will NOT be encrypted and that I need to encrypt these files myself. Does anyone know if this is correct?
Thanks
luckman777,
Every version preinstalled of iOS will hardware encrypt every file when the user uses a screen lock. With respect to your question about external Core Data storage, why don't you just look at the files? It is quite straightforward to move the data from the phone to your dev system. Then try to open one of the external files. I expect that it is encrypted. (If not, that is a rather big and obvious hole in Core Data's encryption policy. I doubt that it exists.)
Andrew
Hidden, but not encrypted!
The folder it currently (iOS 11.2) holds the data is under Documents/.SingleViewCoreData_SUPPORT/_EXTERNAL_DATA
There, you can see all the files, without their extension in a token-name. However, the data is all there unchanged. You can view any file by simply adding the file extension or using the right App.
Yes, the device data is encrypted when the screen is locked, but connected to Xcode, you can very easily download the container and access all the data. If your app holds sensitive data, the 'device is encrypted' will simply not hold.
Only the SingleViewCoreData.sqlite file seems to be encrypted.
Iam using SQLite for storing around 3MB of data, the catch is that i cannot use iCloud oe Web Service for storing this data, because the application will be used by users, who will not be having access to net mostly.
Is there any other way to fix this problem? Need suggestions.
if you download data from web server and store in SQLite or not need that data to user than you should save that data to temp dir not in documents dir because when we backup app by iCould than its get data of document dir not temp dir
and see also this : not backup
I want to have on-the-device database for my iphone app. It concerns with the data, which comes from dictionary consisting of 200.000 things and their definitions. It is only related with text-type data as appeared. My questions:
1- Does SQLite hold all of these data?
2- When the client downloaded my app, he/she will also have the db in his/her device. Does app store allow me to update my app's db and upload my new release? (i don't know these issues well by the way)
3- And can any client, who downloaded my app, hack and obtain my database? Is there any prevention methods? Is SQLite resilient enough against these?
1- Does SQLite hold all of these data?
Yes, SQLite can cope with this amount of data.
2- When the client downloaded my app,
he/she will also have the db in
his/her device. Does app store allow
me to update my app's db and upload my
new release? (i don't know these
issues well by the way)
The general approach is to store the SQLite database in your application bundle and then copy the database into the application's document directory on the device when the application is first run. On subsequent updates to your applciation, you should check if the database within the document directory is the same version and update it if necessary. See the existing Run NSBundle from the documents folder question/answer for more information on this.
3- And can any client, who downloaded
my app, hack and obtain my database?
Is there any prevention methods? Is
SQLite resilient enough against these?
It's fairly trivial to open up an app (the deployment package is just a zip file after all), so yes, it will be possible to obtain access to your database data. Unfortunately there's no easy way around this that I'm aware of. (You could I suppose download the data from a server when you first run you app, but it'll still be accessible on a jailbroken device.)
Sometimes, you just have to bite the bullet and accept the fact that your data is going to be ripped off.
1) sqllite can definitely hold that amount of data.
2) You can put up an option of refreshing the database in your app. That can be used to sync the local db with the server copy. Updated db can also be added with the new version of the app.
3) You can encrypt your local db using SQLCipher for protecting your application db against hacks.