CloudKit: Query returns partial results, no errors - swift

I have some kind of CloudKit indexing issue. When I save records to iCloud using CKModifyRecordsOperation, modifyRecordsCompletionBlock returns no errors. When I query those records using NSPredicate(value: true) or Dashboard, most of the time it misses one or two records.
So say I upload 5 records (no errors), wait some time (~15 secs) to make sure that indexes are updated, and then query them (through dashboard or app's CKQueryOperation). Most of the time it will show 4 records out 5. Again, no errors. Records are in privateDB in customZone.
Here is what's strange: I'm always able to get records that query didn't return by manually typing recordNames in Dashboard (development) under 'Fetch' menu. So it stores them, just doesn't query. When I delete indexes in a dashboard and reassign them, dashboard query will start to return all the results (with previously omitted records too), but after a few more uploads, some will start to be missing from query again.
Here is my CKModifyRecordsOperation:
let operation = CKModifyRecordsOperation(recordsToSave: records, recordIDsToDelete: [])
operation.modifyRecordsCompletionBlock =
{ [weak self] savedRecords, deletedRecordIDs, error in
guard error == nil else { // no errors here ... }
...
//for each item uploaded to iCloud, mark as synced
if let savedRecords = savedRecords{ // all attempted to save records are here
PendingCloudOperations.shared.markAsUploaded(
savedRecords.map{ $0.recordID.recordName })
}
completion(...)
}
operation.savePolicy = .changedKeys // tried .allKeys too
operation.qualityOfService = .userInitiated
self.privateDB.add(operation)
I experimented with record fields (originally date, asset, and reference) trying to see if any of the fields make a problem. But even if I remove all field's (creating a record with no extra fields, just system meta), problem persists. I didn't include CKQueryOperation code, because Dashboard acts same way as the app.
Any ideas?
EDIT:
Here are bare-bones of my fetching function:
var receipts:[FS_Receipt] = []
let query = CKQuery(recordType: myRecordType, predicate: NSPredicate(value: true))
let operation = CKQueryOperation(query: query)
//completion block
operation.queryCompletionBlock = { [weak self] cursor, error in
guard error == nil else {
// doesn't have any errors here
}
completion(...)
}
operation.recordFetchedBlock = { record in
// doesn't return all records here most of the time.
}
operation.qualityOfService = .userInitiated // without this, 'no internet' will NOT return error
operation.resultsLimit = 5000
operation.zoneID = customZoneID
self.privateDB.add(operation)
}

Related

Swift Firebase get batches of documents in order

For context, I have a bunch of documents that hold fields similar to a social media post. (photo url link, like count, date uploaded, person who uploaded it, etc.) And I am showing this data in a gallery (lazyvgrid). I do not want to get all of the documents at once so when the user scrolls down the gallery I am getting 20 documents at a time based on how far the user scrolls down the gallery view. I am sorting my get request with:
self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20).getDocuments
I have no problem getting the first 20 using this code. How can I get the next 20 and the 20 after that, and so on?
With query cursors in Cloud Firestore, you can split data returned by a query into batches according to the parameters you define in your query.
Query cursors define the start and end points for a query, allowing you to:
Return a subset of the data.
Paginate query results.
Use the startAt() or startAfter() methods to define the start point for a query. Use the endAt() or endBefore() methods to define an endpoint for your query results.
As Dharmaraj mentioned for your case, it will be best if we use Pagination with Firestore.
Paginate queries by combining query cursors with the limit() method to limit the number of documents you would want to show in the gallery. And as you want no definite numbers, but the user should be able to scroll through as long as he wants, and as long as there are documents, I would suggest to put a cursor until the last document, like in the below code sample.
To get the last document,
let first = db.collection("collectionname")
.order(by: "fieldname")
first.addSnapshotListener { (snapshot, error) in
guard let snapshot = snapshot else {
print("Error retrieving cities: \(error.debugDescription)")
return
}
guard let lastSnapshot = snapshot.documents.last else {
// The collection is empty.
return
}
I ended up referencing Dharmaraj's link in his comment.
#Published var isFetchingMoreDocs: Bool = false
private var lastDocQuery: DocumentSnapshot!
public func getUpdatedEventMedias(currentEventID: String, eventMedias: [EventMedia], completion: #escaping (_ eventMedias: [EventMedia]) -> Void) {
self.isFetchingMoreDocs = true
var docQuery: Query!
if eventMedias.isEmpty {
docQuery = self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20)
} else if let lastDocQuery = self.lastDocQuery {
docQuery = self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20).start(afterDocument: lastDocQuery)
}
if let docQuery = docQuery {
print("GET DOCS")
docQuery.getDocuments { (document, error) in
if let documents = document?.documents {
var newEventMedias: [EventMedia] = []
for doc in documents {
if let media = try? doc.data(as: EventMedia.self) {
newEventMedias.append(media)
}
}
self.lastDocQuery = document?.documents.last
self.isFetchingMoreDocs = false
completion(newEventMedias)
} else if let error = error {
print("Error getting updated event media: \(error)")
self.isFetchingMoreDocs = false
completion([])
}
}
} else {
self.isFetchingMoreDocs = false
completion([])
}
}
As seen in my code, by utilizing:
.order(by: "savesCount", descending: true).limit(to: 20).start(afterDocument: lastDocQuery)
I am able to start exactly where I left off. I should also note that I am only calling this function if !isFetchingMoreDocs - otherwise the func will be called dozens of times in a matter of seconds while scrolling. The most important thing about this code is that I am checking lastDocQuery if it is nil. After the user scrolls all the way to the bottom, the lastDocQuery will no longer be valid and cause a fatal error. Also I am using a custom scroll view that tracks the scroll offset in order to fetch more media and make more calls to firebase.

Trouble finding out if this counts as a read/many reads/will I get charged loads on database costs?

I am currently developing an iOS app with a google cloud firestore as a backend and I am using a few listeners to find out if data is updated and then pushing it to my device accordingly. I wrote this function that listens for a value if true or not and according to so will update an animation in my app. The trouble is I don't know if I wrote it properly and don't want to incur unnecessary reads from my database if I don't have to.
func dingAnimation() {
let identifier = tempDic![kBOUNDIDENTIFIER] as! String
if identifier != "" {
dingListener = reference(.attention).document(identifier).addSnapshotListener({ (snapshot, error) in
if error != nil {
SVProgressHUD.showError(withStatus: error!.localizedDescription)
return
}
guard let snapshot = snapshot else { return }
let data = snapshot.data() as! NSDictionary
for dat in data {
let currentId = FUser.currentId() as! String
let string = dat.key as! String
if string == currentId {
} else {
let value = dat.value as! Bool
self.shouldAnimate = value
self.animateImage()
}
}
})
}
}
This might help you.
From Firestore DOCS - Understand Cloud Firestore billing
https://firebase.google.com/docs/firestore/pricing
Listening to query results
Cloud Firestore allows you to listen to the results of a query and get realtime updates when the query results change.
When you listen to the results of a query, you are charged for a read each time a document in the result set is added or updated. You are also charged for a read when a document is removed from the result set because the document has changed. (In contrast, when a document is deleted, you are not charged for a read.)
Also, if the listener is disconnected for more than 30 minutes (for example, if the user goes offline), you will be charged for reads as if you had issued a brand-new query.

How to Fetch Records from CloudKit Not in My Local Cache

I have an app that uses CloudKit for sync and I maintain a local cache of records. I have run into a sync scenario that I can't figure out.
I'm using the Public database and when my app is opened, I want to be able to go get all the updated records that my app missed while it was closed, or on a device where the app was just installed.
I can get the updated records by creating a NSPredicate to compare the modificationDate like this:
let predicate = NSPredicate(format: "modificationDate > %#", syncTimestamp as CVarArg)
let query = CKQuery(recordType: recordType, predicate: predicate)
But the part I can't figure out is how to get only the records that have been added to, or removed from, the CloudKit server.
Does anyone know how to do this? I know Apple provides a way for this in the Private database but I'm using the Public one here.
The only thing I can think of so far is to query all the reocrds of a certain recordType, collect all their recordNames and compare to my local cache. But it'd be nice to have smarter way than just pull large amounts of data and comparing huge arrays of recordNames.
CKQuerySubscription(recordType: myRecordType, predicate: predicate, options: [.firesOnRecordCreation, .firesOnRecordDeletion]) works perfectly on public DB.
Here's a code snippet (saving subscription is done with RxCloudKit, but this is beyond the point) -
let predicate = NSPredicate(format: "TRUEPREDICATE")
// the options are different from what you need, just showcasing the possibilities:
let subscription = CKQuerySubscription(recordType: recordTypeTest, predicate: predicate, options: [.firesOnRecordCreation, .firesOnRecordUpdate, .firesOnRecordDeletion, .firesOnce])
let info = CKNotificationInfo()
info.alertLocalizationKey = "NEW_PARTY_ALERT_KEY"
info.soundName = "NewAlert.aiff"
info.shouldBadge = true
subscription.notificationInfo = info
self.publicDB.rx.save(subscription: subscription).subscribe { event in
switch event {
case .success(let subscription):
print("subscription: ", subscription)
case .error(let error):
print("Error: ", error)
default:
break
}
}.disposed(by: self.disposeBag)

CloudKit - CKQueryOperation results different each time the same query is ran

This is the case - I'm using a simple UITableView that renders records from the CloudKit publicDB. When I run the app, the query operation returns for example returns 2 results (that's all it has currently).
My table view has a refresh control and when I pull to refresh I got zero results, if I continue to do reloads, eventually a result might come out but now always.
The same thing happens with more results as well, I used to have CKLocation type which I queried and the response was always different without any common sense
Some example code (the predicate in this case is TRUEPREDICATE - nothing fancy):
let sort = NSSortDescriptor(key: "creationDate", ascending: false)
let query = CKQuery(recordType: "Tests", predicate: DiscoveryMode.getPredicate())
query.sortDescriptors = [sort]
var operation = CKQueryOperation(query: query)
if lastCursor != nil {
operation = CKQueryOperation(cursor: lastCursor)
}
operation.resultsLimit = 15
operation.recordFetchedBlock = recordFetchBlock
operation.queryCompletionBlock = { [weak self] (cursor:CKQueryCursor!, error:NSError!) in
if cursor != nil {
self!.lastCursor = cursor
}
dispatch_async(dispatch_get_main_queue(), { () -> Void in
Misc.hideLoadingInView(view: self!.view)
self!.tableView.reloadData()
self!.refreshControl.endRefreshing()
if error != nil {
Misc.showErrorInView(view: self!.view, message: error.localizedDescription)
}
})
}
CloudKit.sharedInstance.publicDB.addOperation(operation)
All the recordFetchBlock does is to add objects to a mutable array that the table view uses as dataSource.
I'm new to CloudKit and I'm puzzled is this by design (not returning all the results but some random) or I'm doing something wrong?
I see that you are using a cursor. because of that the 2nd call will start at the point where the first call ended. You have a resultsLimit of 15. When using a cursor, you will only receive records the 2nd time you execute the query if there were more than 15 records. To test if this is the issue just comment out the line where you set the cursor: operation = CKQueryOperation(cursor: lastCursor)
I found the issue, I was trying to do (in the NSPredicate) a radius with a value I read somewhere is in kilometers. Therefore I was trying to query records within 500 meters instead of 500 kilometers and the GPX file I'm using in the simulator has multiple records with a larger distance. Since it simulates movement, that was the reason not to get consistent results.
Now, when I'm using a proper value for the radius all seems to be just fine!

swift load all record from cloud kit avoiding limit result

guys need help i can't find something similar online even setting the limit result to a fixed value still doesn't work no matter what i always get 100 record max i need all the records in my database right now is 377 record and will be way more probably 10k but this process will only run once could someone help me catch all the record from the database avoiding the 100 limit please
i added the cursor as you suggested and now i have an idea but now it loads 200 results not the total any tip please
if foodList.count == 0{
loadingView.hidden = false
loadMainDatabaseActivity.startAnimating()
let predicate = NSPredicate(value: true)
let query = CKQuery(recordType: "FoodListDataBase", predicate: predicate)
let operation = CKQueryOperation(query: query)
//operation.resultsLimit = CKQueryOperationMaximumResults
operation.recordFetchedBlock = { (record: CKRecord!) in
if record != nil{
count++
foodList.append(record.objectForKey("foodname") as! String)
categories.append(record.objectForKey("category") as! String)
}
}
operation.queryCompletionBlock = {(cursor: CKQueryCursor!, error: NSError!) in
if cursor != nil {
let newOperation = CKQueryOperation(cursor: cursor)
newOperation.recordFetchedBlock = operation.recordFetchedBlock
newOperation.queryCompletionBlock = operation.queryCompletionBlock
newOperation.resultsLimit = 300
publicDB!.addOperation(newOperation)
println(count)
}
self.loadingView.hidden = true
self.loadMainDatabaseActivity.stopAnimating()
}
publicDB.addOperation(operation)
}
I believe this was answered in the following post:
CKQuery from private zone returns only first 100 CKRecords from in CloudKit
Changing the CKQueryOperationMaximumResults to a larger number.