How can I make retrieving data from firestore cloud faster when refreshing the page [duplicate] - google-cloud-firestore

I am new in programming and in iOS development. I am trying to make an app using Firestore database from Firebase. I don't know if it is normal or not, but when I am trying to get a data from firestore database, it seems too long for me. I don't know if I make a mistake or not
here is my code to get all city data from firestore
reference :
import Foundation
import FirebaseFirestore
import Firebase
enum FirestoreCollectionReference {
case users
case events
case cities
private var path : String {
switch self {
case .users : return "users"
case .events : return "events"
case .cities : return "cities"
}
}
func reference () -> CollectionReference {
return Firestore.firestore().collection(path)
}
}
I use getAllCitiesDataFromFirestore method in CityKM class to get the city data that stored in firestore
class CityKM {
var name : String
var coordinate : GeoPoint
init (name: String , coordinate: GeoPoint ) {
self.name = name
self.coordinate = coordinate
}
init (dictionary: [String:Any]) {
// this init will be used if we get data from firebase observation to construct an event object
name = dictionary["name"] as! String
coordinate = dictionary["coordinate"] as! GeoPoint
}
static func getAllCitiesDataFromFirestore (completion: #escaping ( [CityKM]? )->Void) {
// to retrieve all cities data from Firebase database by one read only, not using realtime fetching listener
let startTime = CFAbsoluteTimeGetCurrent() // to track time consumption of this method
FirestoreCollectionReference.cities.reference().getDocuments { (snapshot, error) in
if let error = error {
print("Failed to retrieve all cities data: \(error.localizedDescription)")
} else {
print("Sucessfully get all cities data from firestore")
guard let documentsSnapshot = snapshot, !documentsSnapshot.isEmpty else {
completion(nil)
return
}
let citiesDocuments = documentsSnapshot.documents
var cityArray = [CityKM]()
for document in citiesDocuments {
guard let cityName = document.data()["name"] as? String,
let cityCoordinate = document.data()["coordinate"] as? GeoPoint else {return}
let theCity = CityKM(name: cityName, coordinate: cityCoordinate)
cityArray.append(theCity)
}
completion(cityArray)
let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime // to track time consumption of this method
print("Time needed to get all cities data from Firestore : \(timeElapsed) s.") // to track time consumption of this method
}
}
}
}
extension CityKM {
// MARK: - User Helper Methods
func toDictionary() -> [String:Any]{
return [
"name" : name,
"coordinate" : coordinate
]
}
}
from my debugging area, it is printed
"Time needed to get all cities data from Firestore : 1.8787678903 s."
is it possible to make it faster? Is 1.8s normal? am i make a mistake in my code that make the request data takes too long time ? I hope that I can make request time is below one second
I don't think the internet speed is the problem, since I can open video on youtube without buffering

That performance sounds a bit worse than what I see, but nothing excessive. Loading data from the cloud simply takes time. A quick approach to hide that latency is by making use of Firebase's built-in caching.
When you call getDocuments, the Firebase client needs to check on the server what the document's value is before it can call your code, which then shows the value to the user. As said: there is no way to speed up this reading in your code, so it'll always take at least 1.8s before the user sees a document.
If instead you listen for realtime updates from the database with addSnapshotListener, the Firebase client may be able to immediately call your code with values from its local cache, and then later re-invoke your code in case there has been an update to the data on the server.

Related

Swift Firebase get batches of documents in order

For context, I have a bunch of documents that hold fields similar to a social media post. (photo url link, like count, date uploaded, person who uploaded it, etc.) And I am showing this data in a gallery (lazyvgrid). I do not want to get all of the documents at once so when the user scrolls down the gallery I am getting 20 documents at a time based on how far the user scrolls down the gallery view. I am sorting my get request with:
self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20).getDocuments
I have no problem getting the first 20 using this code. How can I get the next 20 and the 20 after that, and so on?
With query cursors in Cloud Firestore, you can split data returned by a query into batches according to the parameters you define in your query.
Query cursors define the start and end points for a query, allowing you to:
Return a subset of the data.
Paginate query results.
Use the startAt() or startAfter() methods to define the start point for a query. Use the endAt() or endBefore() methods to define an endpoint for your query results.
As Dharmaraj mentioned for your case, it will be best if we use Pagination with Firestore.
Paginate queries by combining query cursors with the limit() method to limit the number of documents you would want to show in the gallery. And as you want no definite numbers, but the user should be able to scroll through as long as he wants, and as long as there are documents, I would suggest to put a cursor until the last document, like in the below code sample.
To get the last document,
let first = db.collection("collectionname")
.order(by: "fieldname")
first.addSnapshotListener { (snapshot, error) in
guard let snapshot = snapshot else {
print("Error retrieving cities: \(error.debugDescription)")
return
}
guard let lastSnapshot = snapshot.documents.last else {
// The collection is empty.
return
}
I ended up referencing Dharmaraj's link in his comment.
#Published var isFetchingMoreDocs: Bool = false
private var lastDocQuery: DocumentSnapshot!
public func getUpdatedEventMedias(currentEventID: String, eventMedias: [EventMedia], completion: #escaping (_ eventMedias: [EventMedia]) -> Void) {
self.isFetchingMoreDocs = true
var docQuery: Query!
if eventMedias.isEmpty {
docQuery = self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20)
} else if let lastDocQuery = self.lastDocQuery {
docQuery = self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20).start(afterDocument: lastDocQuery)
}
if let docQuery = docQuery {
print("GET DOCS")
docQuery.getDocuments { (document, error) in
if let documents = document?.documents {
var newEventMedias: [EventMedia] = []
for doc in documents {
if let media = try? doc.data(as: EventMedia.self) {
newEventMedias.append(media)
}
}
self.lastDocQuery = document?.documents.last
self.isFetchingMoreDocs = false
completion(newEventMedias)
} else if let error = error {
print("Error getting updated event media: \(error)")
self.isFetchingMoreDocs = false
completion([])
}
}
} else {
self.isFetchingMoreDocs = false
completion([])
}
}
As seen in my code, by utilizing:
.order(by: "savesCount", descending: true).limit(to: 20).start(afterDocument: lastDocQuery)
I am able to start exactly where I left off. I should also note that I am only calling this function if !isFetchingMoreDocs - otherwise the func will be called dozens of times in a matter of seconds while scrolling. The most important thing about this code is that I am checking lastDocQuery if it is nil. After the user scrolls all the way to the bottom, the lastDocQuery will no longer be valid and cause a fatal error. Also I am using a custom scroll view that tracks the scroll offset in order to fetch more media and make more calls to firebase.

Firebase's ref().child(stringPath: String) returning the entire top level collection

I'm trying to retrieve a specific child of my Firebase database using swiftUI. To do that I use the simple expression
func addListeners() {
let database = Database.database(url: "https://someUrl")
let ref = database.reference(withPath: "users")
let currentUserId = "u3Ebr6M3BAbP7PBSYYJ7q9kEe1l2"
let drivingTowardsRef = database.reference(withPath: "users/\(currentUserId)/drivingTowardsUsers")
print("Loading data from \(drivingTowardsRef)")
//THIS RIGHT HERE IS CAUSING THE PROBLEM
ref.observe(.childAdded) { snapshot in
print("Got TOP LEVEL data for user \(snapshot.key): \(String(describing: snapshot.value))")
}
//---------------------------------------
drivingTowardsRef.observe(.childAdded) { snapshot in
ref.child(snapshot.key).getData { (error, userSnapshot) in
if let error = error {
print(error)
} else {
print("Got arriving user data \(snapshot.key): \(String(describing: userSnapshot.value))")
}
}
}
}
The function will just return the entire database data
EDIT: The function returns the data from the first observer ref top level in this case users/ which in my case has two elements: niixi6iORjNn8gWq6tKvSi3Bxfc2, u3Ebr6M3BAbP7PBSYYJ7q9kEe1l2
Got arriving user data niixi6iORjNn8gWq6tKvSi3Bxfc2: Optional({
niixi6iORjNn8gWq6tKvSi3Bxfc2 = {
aproxTime = 0;
distance = 0;
latitude = "37.33070704";
longitude = "-122.03039943";
parkingMode = searching;
userId = niixi6iORjNn8gWq6tKvSi3Bxfc2;
username = testeroNumero;
};
u3Ebr6M3BAbP7PBSYYJ7q9kEe1l2 = {
aproxTime = 0;
distance = 0;
drivingTowardsUsers = {
niixi6iORjNn8gWq6tKvSi3Bxfc2 = {
approxTime = 0;
distance = "560.1447571016249";
};
};
latitude = "37.32984184";
longitude = "-122.02018095";
parkingMode = offering;
userId = u3Ebr6M3BAbP7PBSYYJ7q9kEe1l2;
username = cleoBadu;
};
The key for the child path I pass him seems to be correct but it's still returning the entire top level collection instead of the single item...
EDIT: The problem seems to be on the first observer which messes up the .getData() of the ref.child(snapshot.key). Is that even possible?
Just commenting out that ref.observe(.childAdded) will automatically make the second ref.child(snapshot.key) behave totally normally
What am I missing?
I could get the entire database as a single mega dictionary and then get the child I want from there but it doesn't seem really conventional, especially when google's library offers the possibility to not do that.
EDIT: I added a printing statement that prints the url of the database ref. If I then type in the url on my browser, it redirects me on the FRT database and landing me on the correct object. So the url it's generating is correct and works perfectly fine.
Still the object returned by the getData() is the entire db
SN: I removed all codable structs as that is not the problem, so the question is more focused on the actual problem
EDIT: Created a simple view as that. On a clean project it works on my project it doesn't. I guess it's some sort of configuration but's it's hard to look into it.
PROBLEM: Whatever child(string) I pass him it returns the entire top level data either way (replacing so snapshot.key). For example: I pass the key "something" -> all users are returned, I pass the key "" all users are returned
I just tried to reproduce the problem with (mostly) your code and data, but am not getting the same behavior.
I put the equivalent data into a database of mine at: https://stackoverflow.firebaseio.com/68956236.json?print=pretty
And used this code in Xcode 1.2 with Firebase SDK version 8.6.1:
let ref: DatabaseReference = Database.database().reference().child("68956236")
let currentUserId: String = "u3Ebr6M3BAbP7PBSYYJ7q9kEe1l2"
let drivingTowardsRef: DatabaseReference! = ref.child("\(currentUserId)/drivingTowardsUsers");
print("Loading data from \(drivingTowardsRef)")
drivingTowardsRef.observe(.childAdded) { snapshot in
ref.child(snapshot.key).getData { (error, userSnapshot) in
if let error = error {
print(error)
} else {
do {
//let parkingUser = try userSnapshot.data(as: ParkingUser.self)
print("Got data for user \(snapshot.key): \(String(describing: userSnapshot.value))")
} catch {
print("There has been an error while decoding the user location data with uid \(snapshot.key), the object to be decoded was \(userSnapshot). The decode failed with error: \(error)")
}
}
}
}
The output I get is:
Loading data from Optional(https://stackoverflow.firebaseio.com/68956236/u3Ebr6M3BAbP7PBSYYJ7q9kEe1l2/drivingTowardsUsers)
2021-08-27 10:39:09.578043-0700 Firebase10[36407:3458780] [] nw_protocol_get_quic_image_block_invoke dlopen libquic failed
Got data for user niixi6iORjNn8gWq6tKvSi3Bxfc2: Optional({
aproxTime = 0;
distance = 0;
latitude = "37.32798355";
longitude = "-122.01982712";
parkingMode = searching;
userId = niixi6iORjNn8gWq6tKvSi3Bxfc2;
username = testeroNumero;
})
As far as I can see this behavior is correct, but different from what you get. I hope knowing that I don't see the same behavior, and what versions I use, may be helpful to you.
This is not an issue with Firebase but rather client-side handling of the data returned, You’re expecting a Double within your Codable struct but supplying a String in the other end— Can you try:
public struct ParkingUser: Codable {
var latitude: String
var longitude: String
}

Removing an array item from Firestore not working when array contains date

I've spent days researching this including various answers like: Firebase Firestore: Append/Remove items from document array and my previous question at: Removing an array item from Firestore
but can't work out how to actually get this working. Turns out the issue is when there is a date property in the object as shown below:
I have two structs:
struct TestList : Codable {
var title : String
var color: String
var number: Int
var date: Date
var asDict: [String: Any] {
return ["title" : self.title,
"color" : self.color,
"number" : self.number,
"date" : self.date]
}
}
struct TestGroup: Codable {
var items: [TestList]
}
I am able to add data using FieldValue.arrayUnion:
#objc func addAdditionalArray() {
let testList = TestList(title: "Testing", color: "blue", number: Int.random(in: 1..<999), date: Date())
let docRef = FirestoreReferenceManager.simTest.document("def")
docRef.updateData([
"items" : FieldValue.arrayUnion([["title":testList.title,
"color":testList.color,
"number":testList.number,
"date": testList.date]])
])
}
The above works as reflected in the Firestore dashboard:
But if I try and remove one of the items in the array, it just doesn't work.
#objc func deleteArray() {
let docRef = FirestoreReferenceManager.simTest.document("def")
docRef.getDocument { (document, error) in
do {
let retrievedTestGroup = try document?.data(as: TestGroup.self)
let retrievedTestItem = retrievedTestGroup?.items[1]
guard let itemToRemove = retrievedTestItem else { return }
docRef.updateData([
"items" : FieldValue.arrayRemove([itemToRemove.asDict])
]) { error in
if let error = error {
print("error: \(error)")
} else {
print("successfully deleted")
}
}
} catch {
}
}
}
I have printed the itemToRemove to the log to check that it is correct and it is. But it just doesn't remove it from Firestore. There is no error returned, yet the "successfully deleted" is logged.
I've tried different variations and this code works as long as I don't have a date property in the struct/object. The moment I add a date field, it breaks and stops working. Any ideas on what I'm doing wrong here?
Please note: I've tried passing in the field values as above in FieldValue.arrayUnion as well as the object as per FieldValue.arrayRemove and the same issue persists regardless of which method I use.
The problem is, as you noted, the Date field. And it's a problem because Firestore does not preserve the native Date object when it's stored in the database--they are converted into date objects native to Firestore. And the go-between these two data types is a token system. For example, when you write a date to Firestore from a Swift client, you actually send the database a token which is then redeemed by the server when it arrives which then creates the Firestore date object in the database. Conversely, when you read a date from Firestore on a Swift client, you actually receive a token which is then redeemed by the client which you then can convert into a Swift Date object. Therefore, the definition of "now" is not the same on the client as it is on the server, there is a discrepancy.
That said, in order to remove a specific item from a Firestore array, you must recreate that exact item to give to FieldValue.arrayRemove(), which as you can now imagine is tricky with dates. Unlike Swift, you cannot remove items from Firestore arrays by index. Therefore, if you want to keep your data architecture as is (because there is a workaround I will explain below), the safest way is to get the item itself from the server and pass that into FieldValue.arrayRemove(). You can do this with a regular read and then execute the remove in the completion handler or you can perform it atomically (safer) in a transaction.
let db = Firestore.firestore()
db.runTransaction { (trans, errorPointer) -> Any? in
let doc: DocumentSnapshot
let docRef = db.document("test/def")
// get the document
do {
try doc = trans.getDocument(docRef)
} catch let error as NSError {
errorPointer?.pointee = error
return nil
}
// get the items from the document
if let items = doc.get("items") as? [[String: Any]] {
// find the element to delete
if let toDelete = items.first(where: { (element) -> Bool in
// the predicate for finding the element
if let number = element["number"] as? Int,
number == 385 {
return true
} else {
return false
}
}) {
// element found, remove it
docRef.updateData([
"items": FieldValue.arrayRemove([toDelete])
])
}
} else {
// array itself not found
print("items not found")
}
return nil // you can return things out of transactions but not needed here so return nil
} completion: { (_, error) in
if let error = error {
print(error)
} else {
print("transaction done")
}
}
The workaround I mentioned earlier is to bypass the token system altogether. And the simplest way to do that is to express time as an integer, using the Unix timestamp. This way, the date is stored as an integer in the database which is almost how you'd expect it to be stored anyway. This makes locating array elements that contain dates simpler because time on the client is now equal to time on the server. This is not the case with tokens because the actual date that is stored in the database, for example, is when the token is redeemed and not when it was created.
You can extend Date to conveniently convert dates to timestamps and extend Int to conveniently convert timestamps to dates:
typealias UnixTimestamp = Int
extension Date {
var unixTimestamp: UnixTimestamp {
return UnixTimestamp(self.timeIntervalSince1970 * 1_000) // millisecond precision
}
}
extension UnixTimestamp {
var dateObject: Date {
return Date(timeIntervalSince1970: TimeInterval(self / 1_000)) // must take a millisecond-precision unix timestamp
}
}
One last thing is that in my example, I located the element to delete by its number field (I used your data), which I assumed to be a unique identifier. I don't know the nature of these elements and how they are uniquely identified so consider the filter predicate in my code to be purely an assumption.

Trouble finding out if this counts as a read/many reads/will I get charged loads on database costs?

I am currently developing an iOS app with a google cloud firestore as a backend and I am using a few listeners to find out if data is updated and then pushing it to my device accordingly. I wrote this function that listens for a value if true or not and according to so will update an animation in my app. The trouble is I don't know if I wrote it properly and don't want to incur unnecessary reads from my database if I don't have to.
func dingAnimation() {
let identifier = tempDic![kBOUNDIDENTIFIER] as! String
if identifier != "" {
dingListener = reference(.attention).document(identifier).addSnapshotListener({ (snapshot, error) in
if error != nil {
SVProgressHUD.showError(withStatus: error!.localizedDescription)
return
}
guard let snapshot = snapshot else { return }
let data = snapshot.data() as! NSDictionary
for dat in data {
let currentId = FUser.currentId() as! String
let string = dat.key as! String
if string == currentId {
} else {
let value = dat.value as! Bool
self.shouldAnimate = value
self.animateImage()
}
}
})
}
}
This might help you.
From Firestore DOCS - Understand Cloud Firestore billing
https://firebase.google.com/docs/firestore/pricing
Listening to query results
Cloud Firestore allows you to listen to the results of a query and get realtime updates when the query results change.
When you listen to the results of a query, you are charged for a read each time a document in the result set is added or updated. You are also charged for a read when a document is removed from the result set because the document has changed. (In contrast, when a document is deleted, you are not charged for a read.)
Also, if the listener is disconnected for more than 30 minutes (for example, if the user goes offline), you will be charged for reads as if you had issued a brand-new query.

Upvote/Downvote system within Swift via Firebase

I've looked over hours of code and notes and I'm struggling to find any documentation that would help me with upvoting and downvoting an object in a swift app with firebase.
I have a gallery of photos and I'm looking to add an instagram style upvote to images. The user has already logged with firebase auth so I have their user ID.
I'm just struggling to figure the method and what rules need to be set in firebase.
Any help would be awesome.
I will describe how I implemented such a feature in social networking app Impether using Swift and Firebase.
Since upvoting and downvoting is analogous, I will describe upvoting only.
The general idea is to store a upvotes counter directly in the node corresponding to an image data the counter is related to and update the counter value using transactional writes in order to avoid inconsistencies in the data.
For example, let's assume that you store a single image data at path /images/$imageId/, where $imageId is an unique id used to identify a particular image - it can be generated for example by a function childByAutoId included in Firebase for iOS. Then an object corresponding to a single photo at that node looks like:
$imageId: {
'url': 'http://static.example.com/images/$imageId.jpg',
'caption': 'Some caption',
'author_username': 'foobarbaz'
}
What we want to do is to add an upvote counter to this node, so it becomes:
$imageId: {
'url': 'http://static.example.com/images/$imageId.jpg',
'caption': 'Some caption',
'author_username': 'foobarbaz',
'upvotes': 12,
}
When you are creating a new image (probably when an user uploads it), then you may want to initialize the upvote counter value with 0 or some other constant depending on what are you want to achieve.
When it comes to updating a particular upvotes counter, you want to use transactions in order to avoid inconsistencies in its value (this can occur when multiple clients want to update a counter at the same time).
Fortunately, handling transactional writes in Firebase and Swift is super easy:
func upvote(imageId: String,
success successBlock: (Int) -> Void,
error errorBlock: () -> Void) {
let ref = Firebase(url: "https://YOUR-FIREBASE-URL.firebaseio.com/images")
.childByAppendingPath(imageId)
.childByAppendingPath("upvotes")
ref.runTransactionBlock({
(currentData: FMutableData!) in
//value of the counter before an update
var value = currentData.value as? Int
//checking for nil data is very important when using
//transactional writes
if value == nil {
value = 0
}
//actual update
currentData.value = value! + 1
return FTransactionResult.successWithValue(currentData)
}, andCompletionBlock: {
error, commited, snap in
//if the transaction was commited, i.e. the data
//under snap variable has the value of the counter after
//updates are done
if commited {
let upvotes = snap.value as! Int
//call success callback function if you want
successBlock(upvotes)
} else {
//call error callback function if you want
errorBlock()
}
})
}
The above snipped is actually almost exactly the code we use in production. I hope it helps you :)
I was very surprised, but this code from original docs works like a charm. There is one disadvantage with it: the json grows pretty big if there are a lot of likes.
FirebaseService.shared.databaseReference
.child("items")
.child(itemID!)
.runTransactionBlock({ (currentData: MutableData) -> TransactionResult in
if var item = currentData.value as? [String : AnyObject] {
let uid = SharedUser.current!.id
var usersLikedIdsArray = item["liked_who"] as? [String : Bool] ?? [:]
var likesCount = item["likes"] as? Int ?? 0
if usersLikedIdsArray[uid] == nil {
likesCount += 1
usersLikedIdsArray[uid] = true
self.setImage(self.activeImage!, for: .normal)
self.updateClosure?(true)
} else {
likesCount -= 1
usersLikedIdsArray.removeValue(forKey: uid)
self.setImage(self.unactiveImage!, for: .normal)
self.updateClosure?(false)
}
item["liked_who"] = usersLikedIdsArray as AnyObject?
item["likes"] = likesCount as AnyObject?
currentData.value = item
return TransactionResult.success(withValue: currentData)
}
return TransactionResult.success(withValue: currentData)
}) { (error, committed, snapshot) in
if let error = error {
self.owner?.show(error: error)
}
}
Not a Swift fella myself (pun!) but I think this stackoverflow question has most of your answers.
Then you would simply use a couple of if statements to return the correct value from the transaction based on whether you want to up vote or down vote.