I was hoping I could get some help optimising my code. I´m new to development so please be kind.
Currently it works, but it uses quite some time (10-15 sec) to load the first table view I need in my app.
First I thought that I had not activated "persistence" properly, but I am starting to suspect that it is the way I am loading data that is suboptimal.
The "large" (12k + items) data set I use dont change that frequently, so the ideal solution would be to load that once, then listen for changes. I thought that was what I am doing, but if so I dont understand why it is so slow? So I now suspect that it is the way that I append the data every time, instead of just "reading/loading" from "somewhere local" and then listen for changes from the sever?
Any help is appreciated
//read From Firebase adjusted to whiskies
func startObservingDB () {
dbRef.queryOrdered(byChild: "brand_name").observe(.value, with: { (snapshot:FIRDataSnapshot) in
var newWhisky = [WhiskyItem]()
//forloop to iterate through the snapshot
for whiskyItem in snapshot.children {
let whiskyObject = WhiskyItem(snapshot: whiskyItem as! FIRDataSnapshot)
newWhisky.append(whiskyObject)
}
//update
self.whiskies = newWhisky
print("WhiskyItem")
self.tableView.reloadData()
}) { (error: Error) in
print(error.localizedDescription)
}
}
Firebase structure: /Results/Index/name: xxx, "other thing1": xxxx,..., "other thing32": xxxx
I'm not sure, that it is a good idea to store all 12 000 items on your phone.
May be it will be good solution for you:
You can use this lib for:
(example)
1) load data for 100 rows
2) scroll to the end
3) do another load of 100 rows.
Hope it helps
Related
For testing purposes, I created a specific node on my Firebase database. I copy a user over to that node and then can futz with it without worrying about corrupting data or ruining a user's info. It works really well for my purposes.
I've run into a problem, however. If a user has an extremely large set of data, the copy function won't work. It just stalls. I don't get any errors, though. I read that Firebase has copy limits of 1MB, and I'm guessing that's the problem. I'm running up against that wall, I think.
Here is my code:
func copyToTestingNode() {
let start = Date()
// 1 . create copy of user and then modify the copy
guard var copiedUser = user else { print("copied user error"); return }
copiedUser.userID = MP.adminID
copiedUser.householdInfo.subscriptionExpiryDate = 2500000000
// 2. get a snapshot of the copied user's info
ref.child(user.userID).observeSingleEvent(of: .value) { (userSnapshot) in
print("Step 2 TRT:", Date().timeIntervalSince(start))
// 3. remove any existing data at admin node, and then...
self.ref.child(MP.adminID).removeValue { (error, dbRef) in
print("Step 3 TRT:", Date().timeIntervalSince(start))
// 4. ...copy the new user info to the admin node
self.ref.child(MP.adminID).setValue(userSnapshot.value, withCompletionBlock: { (error, adminRef) in
print("Step 4 TRT:", Date().timeIntervalSince(start))
// 5. then send user alert and stop activity indicator
self.activityIndicator.stopAnimating()
self.showSimpleAlert(alertTitle: "Copy Complete", alertMessage: "Your copy of \(copiedUser.householdInfo.userName) is complete and can be found under the new node:\n\n\(copiedUser.householdInfo.userName) Family")
})
}
}
}
Options:
Is there a simple way to check the size of the DataSnapshot to alert me that the dataset is too large to copy over?
Is there a simple way to split up the snapshot into smaller pieces and overcome the 1MB limit that way?
Should I use Cloud Functions instead of trying to trigger this on a device?
Is there a way to somehow "compress" the snapshot to be smaller so that I can copy it easier?
I'm open to suggestions.
UPDATE #1
I read about the size limitation HERE. Judging from Frank's reaction, I'm guessing my understanding of that limitation is wrong.
I downloaded the node from the Firebase console and checked its size. It's 799 KB on my hard drive. It's a large JSON tree, and so I thought that its size must be the reason why it won't copy over. The smaller nodes copy over no problem. Just the large ones have trouble.
UPDATE #2
I'm not sure how to show the actual data, other than a screenshot, seeing how large the JSON tree is. So here is a screenshot:
As you can see, the data has multiple nodes, some of which are larger than others. I suppose I can cut down the 'Job Jar' node, but the rest really need to be that size for everything to work properly.
Granted, this is one of the largest datasets I have among all my users, but the structure doesn't change.
As for the speed of execution for each line of code, here are the simulator times for each numbered step:
Step 2 TRT: 0.5278879404067993
Step 3 TRT: 0.6249579191207886
Step 4 TRT: 1.8466829061508179
ALL DONE COPYING!!
This only works for the smaller datasets. For the larger ones, I never get to step 4. It just hangs. I let it run for several minutes, but no change.
Final version that seems to work:
func copyToTestingNode() {
// 1 . create copy of user and then modify the copy
guard var copiedUser = user else { print("copied user error"); return }
let adminRef = ref.child(MP.adminID)
copiedUser.userID = MP.adminID
copiedUser.householdInfo.subscriptionExpiryDate = 2500000000
// 2. get a snapshot of the copied user's info
ref.child(user.userID).observeSingleEvent(of: .value) { (userSnapshot) in
// 3. remove any existing data at admin node, and then...
adminRef.removeValue { (error, dbRef) in
if (error != nil) { print("Yikes!") }
// 4. ...copy the new user info to the admin node one node at a time (if user has a lot of data)
var totalNodesCopied = 0
for item in userSnapshot.children {
guard let snap = item as? DataSnapshot else { print("snap error"); return }
self.ref.child(MP.adminID).child(snap.key).setValue(snap.value) { (error, adminRef) in
totalNodesCopied += 1
if totalNodesCopied == userSnapshot.childrenCount {
print("ALL DONE COPYING!!")
}
}
}
}
}
}
In my new app (Project Control, iOS App Store ;)) I want users to take part of development decisions. For this I have added a path in my Firebase database called "claps". I would like to enter the number of the following in my TableView for the different concepts. I have tried the following
self.posts.append(Post(title: post_title, des: post_description, info: "\(post_date) - \(post_user) - \(post_claps) 👏", claps: Int(post_claps)))
for var item in self.posts {
g.ref.child("concepts").child(item.title).queryOrdered(byChild: "claps").observe(.childAdded) { (snapshotClaps: DataSnapshot!) in
item.claps = Int(snapshotClaps.childrenCount)
}
}
DispatchQueue.main.async() {
self.tableView.reloadData()
}
However, it does not yet represent the right one, but is one before it. I don't know how to make the reference more specific to really get only what's under claps.
This ist my Database:
Currently my output is 5 but it should be 4. You see its observing one "layer" to early. Help will be appreciated. Improvements too :)
UPDATE:
Through testing I could reveal that the problem is in the reference. The Int five is coming from the 5 Childs of "top-layer" "Journal". My problem is that I cant get any deeper in the structure because I don't have a specific String for .child()
Since you're observing the .childAdded event, your closure gets called for each matching child node. If you want to count the number of matching child nodes, you'll want to observe the .value event, which ensures your closure gets called for all matching nodes at once.
Something like:
g.ref.child("concepts").child(item.title).observe(.value) { (snapshotClaps: DataSnapshot!) in
item.claps = Int(snapshotClaps.childrenCount)
}
Note that I also removed the orderBy clause, since that has no useful meaning if all you use is the count.
create an Array and allow the firebase to populate it. or do something like
g.ref.child("concepts").child(item.title).observe(.value) { (snapshotClaps: DataSnapshot!) in
item.claps = Int(snapshotClaps.childrenCount)
}
observing value makes sure your closure gets its matching nodes.
There's a couple of great solutions but the issue in reading a node by .value is it reads in everything in that node.
While that would be fine for nodes that have a limited amount of data, it would overwhelm the device when the node contains a lot of data.
So another option is to leverage that Firebase executes all .childAdded events before .value events. That way, we can use a .value as a trigger that all nodes have been read.
Here's a function that uses .childAdded to iterate and count all of the users in the users node. Also, there's a .value observer that reads in just the last node, removes the .childAdded observer and passes the count back to the calling function via a completion handler. Remember that even though we are attaching both observers, the .childAdded events will all fire before the .value event.
func countUsers( completion: #escaping(Int) -> Void) {
var count = 0
let usersRef = self.ref.child("users")
usersRef.observe(.childAdded, with: { snapshot in
count+=1
})
let query = usersRef.queryOrderedByKey().queryLimited(toLast: 1)
query.observeSingleEvent(of: .value, with: { snapshot in
usersRef.removeAllObservers()
completion(count)
})
}
to call the function, here's the code
func getUserCount() {
self.countUsers(completion: { userCount in
print("number of users: \(userCount)")
})
}
I am using CloudKit to download CoreML (machine learning) models. They are about 90MB each. I have the public database and default zone with one custom 'ML' record type.
I query this 'ML' by id, and it takes more than a minute to get a response on the completion block (it should be a matter of seconds). I've tried production environment, setting quality of service, and different ways of querying with the same result (very slow).
I wonder if I'm missing something or if there is any other way of downloading the ML models that is faster?
Here's my current code:
let arrayPredicate = NSPredicate(format: "id == %#", id)
let query = CKQuery(recordType: "ML", predicate: arrayPredicate)
let queryOperation = CKQueryOperation(query: query)
queryOperation.qualityOfService = .userInteractive
queryOperation.resultsLimit = 1
queryOperation.recordFetchedBlock = { record in
// This gets called +60 sec after
}
queryOperation.queryCompletionBlock = { record, error in
// Same here
}
publicDB.add(queryOperation)
I switched to Firebase Storage to test and the result was slightly faster, but not much faster. rmdaddy and TommyBs were right on their line of thought: CloudKit might be a bit slower because you need to query a record, but the download is on a similar speed.
My final solution was so use Firebase Storage as it's easy to handle download progress and then show it on UI for the user to wait.
I am facing a strange issue with CoreData. I am starting a operation to fill initial data in a table. I am starting the operation in applicationDidBecomeActive.
// Creating child context
let context = NSManagedObjectContext(concurrencyType: .PrivateQueueConcurrencyType)
let delegate = UIApplication.sharedApplication().delegate as! AppDelegate
context.parentContext = delegate.managedObjectContext
// Reading data from database and printing here, shows zero number of entities always
context.performBlockAndWait({
// Performing batch delete, to remove duplicacy
})
context.performBlockAndWait({
// Creating entities from the JSON read from App bundle
...
...
do {
// Saving local context
try context.save()
context.parentContext?.performBlockAndWait({
do {
try context.parentContext?.save()
// Reading data from database and printing here, shows correct number of entities
} catch {
printDebug("Unable to save main context: \(error)")
}
})
} catch {
printDebug("Unable to save main context: \(error)")
}
})
// Reading data from database and printing here also, shows correct number of entities
I am starting this operation only from once place i.e applicationDidBecomeActive, and also accessing the entity from this operation only.
Any idea, what is the problem ?
So the problem was batch delete using NSBatchDeleteRequest. I was performing same code for multiple type of NSManagedObjectContext, and all are sub-classes of a single NSManagedObjectContext. So that might be the issue.
When I perform fetch-all-and-delete-in-loop, everything works fine, i.e entities are getting stored. But when I use NSBatchDeleteRequest to delete all at once, the entities that are inserted before the batch-delete operation of the next type of NSManagedObjectContext are getting deleted.
So the culprit was NSBatchDeleteRequest. And I don't know why ? I searched but didn't find any solution. So I will post another question regarding this issue.
I have developed a simple Swift iOS game with high scores stored in Parse. Saving and retrieving data works fine. What I'd like to do now is to implement a "user ranking" -feature, which would show how the user's high score ranks against other players. In practice this would mean that I'd need:
The total count of high scores in Parse
The ranking of the user's high score on that list
If Parse did not have any query limits, this would be relatively easy to implement for even a newbie coder like myself. However, it does, and I just can't figure out how to implement this in a way that this would still work (1) efficiently, and (2) even if there were e.g., 100,000 high scores.
What would be a workable way of identifying the current user's ranking amongst all other high scores stored in Parse? I don't want to use countObjects as I believe it fails when the number of objects gets high.
Were you using the countObjectsInBackground without a completion block? Because then I'd believe that you could get messed up with asynchronous issues, but it's very easy, and very safe with the countObjectsInBackgroundWithBlock method.
func getUserPosition() {
let totalQuery = PFQuery(className: "highScores")
totalQuery.countObjectsInBackgroundWithBlock { (number, error) -> Void in
if error == nil {
//your total number is the number var passed here
let positionQuery = PFQuery(className: "highScores")
positionQuery.whereKey("score", greaterThanOrEqualTo: userHighScore)
positionQuery.countObjectsInBackgroundWithBlock({ (position, error) -> Void in
//your user's position is the position var passed here
})
}
}
}