Firebase realtime database: load & monitor children (Swift) - best practice? - swift

This seems like it ought to have a very simple answer.
I am building an app using SwiftUI and firebase realtime database.
The database will have a node called items with a large number of children - on the order of 1,000.
I'd like the app to load the contents of that node children when it launches, then listen to firebase for future additions of children. Imagine the following:
struct Item { … } // some struct
var items: [Item] = []
let itemsRef = Database.database().reference().child("items")
According to firebase, this call ought to load in all the children one at a time, and then add new ones as they are added to the items node in firebase:
itemsRef.observe(.childAdded) { snapshot in
// add a single child based on snapshot
items.append(Item(fromDict: snapshot.value as! [String:String])
}
That gets the job done, but seems hugely inefficient compared to using the getData() method they supply, which hands us a dictionary containing all the children:
itemsRef.getData() { error, snapshot in
// set all children at once base on snapshot
items = Item.itemArray(fromMetaDict: snapshot.value as! [String:[String:String]])
}
It would seem best to use getData() initially, then observe(.childAdded) to monitor additions. But then how do we prevent the observe completion block from running 1,000 times when it fires up? The firebase docs say that that's what will happen:
This event is triggered once for each existing child and then again every time a new child is added to the specified path.
Thanks in advance!
PS I didn't think it necessary to include definitions for the functions Item.init(fromDict:) or Item.itemArray(fromMetaDict:) — hopefully it's clear what they are meant to do.

There is (or: should be) no difference between listening for .childAdded on a path versus listening for .value or calling `getData() on that same path. The distinction is purely client-side and the wire traffic is (or: should be) the same.
It is in fact quite common to listen for .child* events to manipulate some data structure/UI, and then also listen for the .value event to commit those changes, as .value is guaranteed to fire after all corresponding .child*.
A common trick to do the first big batch of data in .value or getData(), and then use .child* for granular updates is to have a boolean flag to indicate whether you got the initial data already, and set if to false initially.
In the .child* handlers, only process the data if the flag is true.
itemsRef.observe(.childAdded) { snapshot in
if isInitialDataProcessed {
items.append(Item(fromDict: snapshot.value as! [String:String])
}
}
And then in your .value/getData handler, process the data and then set the flag to true.

Related

How to know when all initial data is fetched from a root node using listeners in Realtime Firebase database

Goal:
My goal here is to fetch initial data, update UI after I am sure all data is fetched, and continue observing changes immediately:
The Problem
I see two ways to do this:
To use getData method and to fetch all data at once (bulky). This is ok cause I know I have fetched all data at once, and I can accordingly update UI and continue listening for changes (CRUD).
The problem with this approach is that I can't just attach listener after, to listen to a for new additions (inserts), and wait for new items. It works differently (which makes sense cause I fetched data without being in sync with a database through listeners), and immediately after I attach the listener, I get its callback triggered as many times as how many items are currently in a root node. So I am getting most likely the same data.
So this seems like overkill.
Second way to do this, is just to attach the listener, and get all data. But the problem with this is that I don't know when all data is fetched, cause it comes sequentially, one item by another. Thus I can't update UI accordingly.
Here are some code examples:
I am currently fetching all previous data with a getData method, like this:
func getInitialData(completion: #escaping DataReadCompletionHandler){
rootNodeReference.getData { optionalError, snapshot in
if let error = optionalError {
completion([])
return
}
if let value = snapshot.value,
let models = self.parseData(type: [MyModel].self, data: value) as? [MyModel]{
completion([MyModel](models.values))
}
}
}
As I said, with this, I am sure I have all previous data and I can set up my UI accordingly.
After this, I am interested in only new updates (updates, deletions, inserts).
And later I connect through listeners. Here is an example for a listener that listens when something new is added to a root node:
rootNodeReference.observe(DataEventType.childAdded, with: {[weak self] snapshot in
guard let `self` = self else {return}
if let value = snapshot.value,
let model = self.parseData(type: MyModel.self, data: value) as? MyModel{
self.firebaseReadDelegate?.event(type: .childAdded, model: model)
}
})
This would be great if with this listener I would somehow be able to continue only updates when something new is added.
Though, I guess option 2. would be a better way to go, but how to know when I have got all data through listeners?
There are two ways to do this, but they both depend on the same guarantee that Firebase makes about the order in which events are fired.
When you observe both child events and value events on the same path/query, the value event fires after all corresponding child events.
Because if this guarantee, you can add an additional listener to .value
rootNodeReference.observeSingleEvent(of: DataEventType.value, with: { snapshot in
... the initial data is all loaded
})
Adding the second listener doesn't increase the amount of data that is read from the database, because Firebase deduplicates them behind the scenese.
You can also forego the childAdded listener and just use a single observe(.value as shown in the documentation on reading a list by observing value events:
rootNodeReference.observe(.value) { snapshot in
for child in snapshot.children {
...
}
}

Deleting, adding and updating Firestore documents when offline

I have been using the following code in my iOS Swift app:
class ProfileController
{
func remove(pid: String, completion: #escaping ErrorCompletionHandler)
{
guard let uid = self.uid else
{
completion(Errors.userIdentifierEmpty)
return
}
let db = Firestore.firestore()
let userDocument = db.collection("profiles").document(uid)
let collection = userDocument.collection("profiles")
let document = collection.document(pid)
document.delete()
{
error in
completion(error)
}
}
}
When the device is online, everything works fine. The completion handler of the deletecall is properly executed. However, when I am offline, I have noticed that the completion handler will not be executed as long as I am offline. As soon as I get back online, the completion handler will be called almost immediately.
I don't want to wait until the user is back online (which could take forever), so I changed the code a little bit and added a ListenerRegistration:
class ProfileController
{
func remove(pid: String, completion: #escaping ErrorCompletionHandler)
{
guard let uid = self.uid else
{
completion(Errors.userIdentifierEmpty)
return
}
let db = Firestore.firestore()
let userDocument = db.collection("profiles").document(uid)
let collection = userDocument.collection("profiles")
let document = collection.document(pid)
var listener: ListenerRegistration?
listener = document.addSnapshotListener(includeMetadataChanges: false)
{
snapshot, error in
listener?.remove() // Immediately remove the snapshot listener so we only receive one notification.
completion(error)
listener = nil
}
document.delete()
}
}
Although this works fine, I am not sure if this is the right way to go. I have read online that a snapshot listener can be used in real-time scenarios, which is not really what I am looking for (or what I need).
Is this the right approach or is there another (better) way? I only want to get notified once (thus I added the includeMetadataChanged property and set it to false). I also remove the ListenerRegistration once the completion handler was called once.
If the first approach does not work properly when being offline - what are the use cases of this approach? Before I change my entire codebase to use listeners, is there any way of executing the completion handler of the first approach when the device is offline?
TL;DR: The second implementation works fine, I am simply unsure if this is the proper way of receiving notifications when the device is offline.
If the first approach does not work properly when being offline - what are the use cases of this approach?
It depends on what you mean by "work properly". The behavior you're observing is exactly as intended. Write operations are not "complete" until they're registered at the server.
However, all writes (that are not transactions) are actually committed locally before they hit the server. These local changes will eventually be synchronized with the server at some point in the future, even if the app is killed and restarted. The change is not lost. You can count on the synchronization of the change to eventually hit the server as long as the user continues to launch the app - this is all you can expect. You can read more about offline persistence in the documentation.
If you need to know if a prior change was synchronized, there is no easy way to determine that if the app was killed and restarted. You could try to query for that data, if you know the IDs of the documents written, and you could check the metadata of the documents to find out the source of the data (cache or server). But in the end, you are really supposed to trust that changes will be synchronized with the server at the earliest convenience.
If your use case requires more granularity of information, please file a feature request with Firebase support.

Firebase child nodes are randomly deleted

Firebase child nodes are randomly deleted in random times
I have a firebase project with a lot of children per node (in the range of 100k+) and it uses real time database. The problem is, sometimes, completely randomly, few child items simply get removed.
For example, given the following database structure:
**Project** (root)
users
user1
user2
user3
.
.
.
user100000
one of the nodes among user1, user2... user100000 would just randomly get removed. This happens so randomly out of nowhere, I cannot reproduce it, and only affects less than 1% of the time, so out of more than 100k children, only 200ish would disappear in the span of 3 month. I went through every line of the code, but there is no function that removes a node. I only use updateChildValues, observe(.childAdded..., observeSingleEvent, and queryOrdered. There is no single method that starts with remove. Any ideas on why this is happening?
My first intuition is I am calling some write API like setValue or updateValue with Nil, but I am on Swift and that is not possible. I also looked at Firebase Web update() deletes all other child nodes, but this actually removes every child other than the one being added, which is definitely not happening to me. Has anyone also experienced this issue? Any help would be really appreciated. Thank you.
-----EDIT-----
Here are some example APIs that I use with write function. item variable is a dictionary defined by let.
let databaseRef = Database.database().reference()
newReferece = databaseRef.child("users").child(UID)
newReferece.setValue(item)
Another one:
theAPI.REF?.updateChildValues(dict, withCompletionBlock: { (error, ref) in
if error != nil {
onError(error!.localizedDescription)
} else {
onSuccess()
}
})

Possible to get latest data with observeSingleEvent + persistence enabled?

I am trying to get some data from firebase. Any idea how can I get the latest data (not from cache) when I have persistence enabled? I tried keepSynced; I still get stale data. Is this the correct usage?
userRef = FIRDatabase.database().reference().child("<path>")
userRef.keepSynced(true)
userRef.observeSingleEvent(of: .value, with: { snapshot in
...stale data here...
})
Or the only option is to use observe instead of observeSingleEvent? I don't like the fact that with observe I get the cache data first, and then the event triggers a second time with data from the server. So with observe, when I navigate to this screen, first I see a blank table, then I see the table with stale data, and then I see the table with latest data.
Thanks.
EDIT:
https://stackoverflow.com/a/34487195/1373592 -
This post says keeySynced should work. But it's not working for me. I would like to know if I am doing something wrong.
I retrieve some explanation, I think it might help you in your case :
ObserveSingleEventType with keepSycned will not work if the Firebase
connection cannot be established on time. This is especially true
during appLaunch or in the appDelegate where there is a delay in the
Firebase connection and the cached result is given instead. It will
also not work at times if persistence is enabled and
observeSingleEvent might give the cached data first. In situations
like these, a continuous ObserveEventType is preferred and should be
used if you absolutely need fresh data.
I think you don't have the choice to use a continuous listener. But to avoid performance issues why you don't remove yourself your listeners when you don't it anymore.
Here is an example on how to ALWAYS get latest data from firebase when persistence is turned on. Use observe event, keepSynced on your ref and terminate listener if you don't want to keep it always. After several trials, I came up with this and it is working.
func readFromFB() {
let refHandle: DatabaseHandle?
let ref: DatabaseReference? = firebase.child(nodeName)
ref?.keepSynced(true)
refHandle = ref!.observe(.value, with:
{ snapshot in
if snapshot.exists() {
for item in ((snapshot.value as! NSDictionary).allValues as Array) {
//do whatever tasks
}
}
})
if let rf = ref {
rf.removeObserver(withHandle: refHandle!)
}
}

When are Realm notifications delivered to main thread after writes on a background thread?

I'm seeing crashes that either shouldn't be possible, or are very much possible and the documentation just isn't clear enough as to why.
UPDATE:
Although I disagree with the comment below asking me to separate this into multiple SO questions, if someone could focus on this one I think it would help greatly:
When are notifications delivered to the main thread? Is it possible that the results on the main thread are different than they were in a previous runloop without being notified yet of the difference?
If the answer to this question is yes the results could be different than a previous runloop without notifying then I would argue it is CRUCIAL to get this into the documentation somewhere.
Background Writes
First I think it's important to go over what I am already doing for writes. All of my writes are performed through a method that essentially looks like this (error handling aside):
func write(block: #escaping (Realm) -> ()) {
somePrivateBackgroundSerialQueue.async {
autoreleasepool {
let realm = try! Realm()
realm.refresh()
try? realm.write { block(realm) }
}
}
}
Nothing crazy here, pretty well documented on your end.
Notifications and Table Views
The main question I have here is when are notifications delivered to the main thread after being written from a background thread? I have a complex table view (multiple sections, ads every 5th row) backed by realm results. My main data source looks like:
enum StoryRow {
case story(Story) // Story is a RealmSwift.Object subclass
case ad(Int)
}
class StorySection {
let stories: Results<Story>
var numberOfRows: Int {
let count = stories.count
return count + numberOfAds(before: count)
}
func row(at index: Int) -> StoryRow {
if isAdRow(at: index) {
return .ad(index)
} else {
let storyIndex = index - numberOfAds(before: index)
return .story(stories[storyIndex])
}
}
}
var sections: [StorySection]
... sections[indexPath.section].row(at: indexPath.row) ...
Before building my sections array I fetch the realm results and filter them based on the type of stories for the particular screen, sort them so they are in the proper order for their sections, then I build up the sections by passing in results.filter(...date query...) to the section constructor. Finally, I results.observe(...) the main results object (not any of the results passed into the section) and reload the table view when the notification handler is called. I don't bother observing the results in the sections because if any of those results changed then the parent had to change as well and it should trigger a change notification.
The ad slots have callbacks when an ad is filled or not filled and when that happens instead of calling tableView.reloadData() I am doing something like:
guard tableView.indexPathsForVisibleRows?.contains(indexPath) == true else { return }
tableView.beginUpdates()
tableView.reloadRows(at: [indexPath], with: .automatic)
tableView.endUpdates()
The problem is, I very rarely see a crash either around an index being out of bound when accessing the realm results or an invalid table view update.
QUESTIONS
Is it possible the realm changed on the main thread before any notifications were delivered?
Should table view updates other than reloadData() simply not be used anywhere outside of a realm notification block?
Anything else crucial I am missing?
There's nothing in your code snippets or description of what you're doing that jumps out at me as obviously wrong. Having a separate callback mechanism that updates specific slots independent of Realm change notifications has a lot of potential for timing related bugs, but since you're explicitly checking if the indexPath is visible before reloading the row, I would expect that to at worst manifest as reloading the wrong row and not a crash.
The intended behavior is that refreshing the Realm and delivering notifications is an atomicish operation: anything that causes the read version to advance will deliver all notifications before returning. In simple cases, this means that you'll never see the new data without the associated notification firing first. However, there's some caveats to this:
Nested notification delivery doesn't work correctly, so beginning a write transaction from within a notification block can result in a notification being skipped (merely calling refresh() can't cause this, as it's just a no-op within a notification). If you're performing all writes on background threads you shouldn't be hitting this.
If you have multiple notification blocks, then obviously anything which gets invoked from the first one will run before the second notification block gets a chance to do things, and a call to tableView.reloadData() may result in quite a lot of things happening within the notification block. If this is the source of problems, you would hopefully see exceptions being thrown with a stack trace coming from within a notification block.