Right now I query Firestore arrays based on another array of strings. This works fine but I would like to know if this is correct and if there is a way to optimise it to make it faster.
Here is my code.
var exampleArray = [test]()
func loadData(textArray : [String]){
let db = Firestore.firestore()
for i in 0..<textArray.count{
db.collection("testCollection").whereField("testField", arrayContains: textArray[i]).getDocuments{ (querySnapshot, err) in
if let err = err {
print("\(err.localizedDescription)")
print("Test Error")
} else {
if (querySnapshot!.isEmpty == false){
let res = querySnapshot!.documents.compactMap({test(dictionary: $0.data())})
self.exampleArray.append(contentsOf: res)
self.summaryTableView.reloadData()
SVProgressHUD.dismiss()
print(self.textArray[i])
}
}
}
}
}
Thanks in advance.
UPDATE:
Some info:
Right now the VisionCloudRecognition process and query take about 10-11sec. 7 sec the recognition and 3-4 the query.
The Database has 7700 documents, 7 fields each.
Basically my app does the following:
User takes a photo
With OCR the image becomes text and i append the text in an array of strings.We don't know how many strings are going to be. Maybe 0 or maybe 50.
Then i query Firestore arrays based on the array of strings.
As pointed out in the comments, you're running a bunch of async requests in a loop (that's always bad) and you're reloading the data as each one resolves causing the tableView to keep repainting with new data.
What you want to do is add all of those db requests to a DispatchGroup. As they resolve, add the data to your array. Once they have all finished, do a single tableView.reloadData() and dismiss your SVProgressHud.
This should not only speed things up because multiple calls can happen in parallel, but your user will not have table rows bouncing around as new data is added and the tableView reloads X number of times in rapid fire.
EDIT: You asked for an example - this is very quick and written here on SO, so look for XCode to notify you of any typos or errors.
class MyVC: UIViewController {
var exampleArray = [test]()
let db = Firestore.firestore()
let dispatchGroup = DispatchGroup()
override func viewDidLoad(animated: bool) {
super.viewDidLoad(animated)
myTableView.delegate = self
myTableView.dataSource = self
for i in 0..<textArray.count {
getTestFieldRecords(i)
}
dispatchGroup.notify(queue: .main) {
self.myTableView.reloadData()
}
}
private func getTestFieldRecords(_ record: Int) {
dispatchGroup.enter()
db.collection("testCollection").whereField("testField", arrayContains: textArray[i]).getDocuments{ (querySnapshot, err) in
if let err = err {
print("\(err.localizedDescription)")
print("Test Error")
} else {
// rest of your code because I'm too lazy to format it
}
self.dispatchGroup.leave()
}
}
}
Make sure you call leave() every time - error or not. The DispatchGroup will not finish until all enter() commands are finished with a leave() command. Hope this helps point you in the right direction!
Related
For context, I have a bunch of documents that hold fields similar to a social media post. (photo url link, like count, date uploaded, person who uploaded it, etc.) And I am showing this data in a gallery (lazyvgrid). I do not want to get all of the documents at once so when the user scrolls down the gallery I am getting 20 documents at a time based on how far the user scrolls down the gallery view. I am sorting my get request with:
self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20).getDocuments
I have no problem getting the first 20 using this code. How can I get the next 20 and the 20 after that, and so on?
With query cursors in Cloud Firestore, you can split data returned by a query into batches according to the parameters you define in your query.
Query cursors define the start and end points for a query, allowing you to:
Return a subset of the data.
Paginate query results.
Use the startAt() or startAfter() methods to define the start point for a query. Use the endAt() or endBefore() methods to define an endpoint for your query results.
As Dharmaraj mentioned for your case, it will be best if we use Pagination with Firestore.
Paginate queries by combining query cursors with the limit() method to limit the number of documents you would want to show in the gallery. And as you want no definite numbers, but the user should be able to scroll through as long as he wants, and as long as there are documents, I would suggest to put a cursor until the last document, like in the below code sample.
To get the last document,
let first = db.collection("collectionname")
.order(by: "fieldname")
first.addSnapshotListener { (snapshot, error) in
guard let snapshot = snapshot else {
print("Error retrieving cities: \(error.debugDescription)")
return
}
guard let lastSnapshot = snapshot.documents.last else {
// The collection is empty.
return
}
I ended up referencing Dharmaraj's link in his comment.
#Published var isFetchingMoreDocs: Bool = false
private var lastDocQuery: DocumentSnapshot!
public func getUpdatedEventMedias(currentEventID: String, eventMedias: [EventMedia], completion: #escaping (_ eventMedias: [EventMedia]) -> Void) {
self.isFetchingMoreDocs = true
var docQuery: Query!
if eventMedias.isEmpty {
docQuery = self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20)
} else if let lastDocQuery = self.lastDocQuery {
docQuery = self.eventsDataCollection.document(currentEventID).collection("eventMedias").order(by: "savesCount", descending: true).limit(to: 20).start(afterDocument: lastDocQuery)
}
if let docQuery = docQuery {
print("GET DOCS")
docQuery.getDocuments { (document, error) in
if let documents = document?.documents {
var newEventMedias: [EventMedia] = []
for doc in documents {
if let media = try? doc.data(as: EventMedia.self) {
newEventMedias.append(media)
}
}
self.lastDocQuery = document?.documents.last
self.isFetchingMoreDocs = false
completion(newEventMedias)
} else if let error = error {
print("Error getting updated event media: \(error)")
self.isFetchingMoreDocs = false
completion([])
}
}
} else {
self.isFetchingMoreDocs = false
completion([])
}
}
As seen in my code, by utilizing:
.order(by: "savesCount", descending: true).limit(to: 20).start(afterDocument: lastDocQuery)
I am able to start exactly where I left off. I should also note that I am only calling this function if !isFetchingMoreDocs - otherwise the func will be called dozens of times in a matter of seconds while scrolling. The most important thing about this code is that I am checking lastDocQuery if it is nil. After the user scrolls all the way to the bottom, the lastDocQuery will no longer be valid and cause a fatal error. Also I am using a custom scroll view that tracks the scroll offset in order to fetch more media and make more calls to firebase.
I am trying to fetch bunch of data with for in loop function, but it doesn't return data in correct orders. It looks like some data take longer to fetch and so they are mixed up in an array where I need to have all the data in correct order. So, I used DispatchGroup. However, it's not working. Can you please let me know what I am doing wrong here? Spent past 10 hours searching for a solution... below is my code.
#IBAction func parseXMLTapped(_ sender: Any) {
let codeArray = codes[0]
for code in codeArray {
self.fetchData(code)
}
dispatchGroup.notify(queue: .main) {
print(self.dataToAddArray)
print("Complete.")
}
}
private func fetchData(_ code: String) {
dispatchGroup.enter()
print("count: \(count)")
let dataParser = DataParser()
dataParser.parseData(url: url) { (dataItems) in
self.dataItems = dataItems
print("Index #\(self.count): \(self.dataItems)")
self.dataToAddArray.append(self.dataItems)
}
self.dispatchGroup.leave()
dispatchGroup.enter()
self.count += 1
dispatchGroup.leave()
}
The problem with asynchronous functions is that you can never know in which order the blocks return.
If you need to preserve the order, use indices like so:
let dispatchGroup = DispatchGroup()
var dataToAddArray = [String](repeating: "", count: codeArray.count)
for (index, code) in codeArray.enumerated() {
dispatchGroup.enter()
DataParser().parseData(url: url) { dataItems in
dataToAddArray[index] = dataItems
dispatchGroup.leave()
}
}
dispatchGroup.notify(queue: .main) {
print("Complete"
}
Also in your example you are calling dispatchGroup.leave() before the asynchronous block has even finished. That would also yield wrong results.
Using semaphores to eliminate all concurrency solves the order issue, but with a large performance penalty. Dennis has the right idea, namely, rather than sacrificing concurrency, instead, just sort the results.
That having been said, I would probably use a dictionary:
let group = DispatchGroup()
var results: [String: [DataItem]] // you didn't say what `dataItems` was, so I'll assume it's an array of `DataItem` objects; but this detail isn't material to the broader question
for code in codes {
group.enter()
DataParser().parseData(url: url) { dataItems in
results[code] = dataItems // if parseData doesn't already uses the main queue for its completion handler, then dispatch these two lines to the main queue
group.leave()
}
}
group.notify(queue: .main) {
let sortedResults = codes.compactMap { results[$0] } // this very efficiently gets the results in the right order
// do something with sortedResults
}
Now, I might advise constraining the degree of concurrency (e.g. maybe you want to constrain this to the number of CPUs or some reasonable fixed number (e.g. 4 or 6). That is a separate question. But I would advise against sacrificing concurrency just to get the results in the right order.
In this case, using DispatchSemaphore:
let semaphore = DispatchSemaphore(value: 0)
DispatchQueue.global().async {
for code in codeArray {
self.fetchData(code)
semaphore.wait()
}
}
private func fetchData(_ code: String) {
print("count: \(count)")
let dataParser = DataParser()
dataParser.parseData(url: url) { (dataItems) in
self.dataItems = dataItems
print("Index #\(self.count): \(self.dataItems)")
self.dataToAddArray.append(self.dataItems)
semaphore.signal()
}
}
I have a function adding documents to a collection in firebase. It is done using a for loop. I have a DispatchGroup and I am calling enter() at the start of each iteration of the loop. After each document has been added I want to call the completion handler of the addDocument method. In the completion handler I want to call leave() on my DispatchGroup, so that I eventually can perform a segue when all documents have been added. My problem is that the completion handler never seems to get called as the messages never get printed. I can see that the documents get added to my collection in firebase every time I run the code. Have I misunderstood something or is there something wrong with my approach? Any help would be very appreciated. A simplified example of my code looks something like this:
func uploadDocumentToFirebase(names: String[])
{
for name in names
{
dispatchGroup.enter()
collection.addDocument(data: ["name": name], completion: {error in
print("Document: \(name) was uploaded to firebase")
self.dispatchGroup.leave()
})
}
}
The actual documents I'm adding have 6 fields instead of the 1 shown in my example, if that makes any difference.
There are many ways to do this - here's two. First is using a dispatch group and the second is using and index technique.
I have an array of words and want to write them to Firestore, notifying as each one is written and then when they are all written.
let arrayOfWords = ["boundless", "delicious", "use", "two", "describe", "hilarious"]
Here's the dispatch group code. We enter the group, write the data and in the completion handler, when done, leave. When all have been left group.notify is called.
func writeWordUsingDispatchGroup() {
let group = DispatchGroup()
let wordCollection = self.db.collection("word_collection")
for word in self.arrayOfWords {
group.enter()
let dataToWrite = ["word": word]
wordCollection.addDocument(data: dataToWrite, completion: { error in
print("\(word) written")
group.leave()
})
}
group.notify(queue: .main) {
print("all words written")
}
}
And then the index code. All this does is calculates the index of the last object in the array and then iterates over the array enumerated (so we get the index). Then when the index of the current loop matches the last index, we know we're done.
func writeWordsUsingIndex() {
let wordCollection = self.db.collection("word_collection")
let lastIndex = self.arrayOfWords.count - 1
for (index, word) in self.arrayOfWords.enumerated() {
let dataToWrite = ["word": word]
wordCollection.addDocument(data: dataToWrite, completion: { error in
print("\(word) written")
if index == lastIndex {
print("all words written")
}
})
}
}
Edit:
Maybe you can run a completion handler so that your exits are in the same place as your group? I generally write completion handlers in situations like this this and call them where you have self.dispatchGroup.leave(). You can put self.dispatchGroup.leave() in the completion block which might help? It seems like your group has an uneven number of entry points and exit points. Organizing with a completion block might help find it?
completion: (#escaping (Bool) -> ()) = { (arg) in })
Original:
Would you mind using this setData code instead of addDcoument to see if it helps? You can add your dispatch to this code and see if it all works. If not I will keep thinking it through...
Also maybe check to make sure the input array isn't empty (just print it to console in the method).
let db = Firestore.firestore()
db.collection("your path").document("\(your document name)").setData([
"aName": "\(name)",
"anEmail": "\(email)",
]) { err in
if let _ = err {
print("Error writing document:")
} else {
print("Document successfully written!")
}
}
My problem is that the completion handler never seems to get called as
the messages never get printed.
It seems you're not calling completion inside your addDocument method, at least for the case, when a document is successfully added.
I am new to Realm DataBase and I need a way to read data from realmCloud, but from two different app projects. The way I have tried to implement this is by using query-synced realm. At the moment I'm using a singe realm user to write the data in one app, and the same realm user to read data from another app. The problem is that making a query from the second app(the one used for reading) doesn't return any realm objects ( I have also noticed that user identifier is different from the first one, and also the user permissions are nil.
I have tried setting permissions directly from RealmStudio since documentation is not precise on how to set them from code
func openRealm() {
do {
realm = try Realm(configuration: SyncUser.current!.configuration())
let queryResults = realm.objects(*className*.self)
let syncSubscription = queryResults.subscribe()
let notificationToken = queryResults.observe() { [weak self] (changes) in
switch (changes) {
case .initial: print(queryResults)
case .error(let error): print(error)
default: print("default")
}
}
for token in queryResults {
print(token.tokenString)
}
syncSubscription.unsubscribe()
notificationToken.invalidate()
} catch {
print(error)
}
}
This function prints the data in one app project, but used in another app project with the same user logged in, and the same classFile referenced in the project, it does not. (note that SyncUser.current.identifier is different also
There are a couple of issues.
Some of these calls are asynchronous and the code in your question is going out of scope before the data is sync'd (retreived). The bottom line is code is faster than the internet and you need to design the flow of the app around async calls; don't try to work with the data until it's available.
For example
let notificationToken = queryResults.observe() { [weak self] (changes) in
//here is where results are fully populated
}
// this code may run before results are populated //
for token in queryResults {
print(token.tokenString)
}
Also, let notificationToken is a local var and goes out of scope before the results are populated as well.
These issues are super easy to fix. First is to keep the notification token alive while waiting for results to be populated and the second is to work with the results inside the closure, as that's when they are valid.
var notificationToken: NotificationToken? = nil //a class var
func openRealm() {
do {
let config = SyncUser.current?.configuration()
let realm = try Realm(configuration: config!)
let queryResults = realm.objects(Project.self)
let syncSubscription = queryResults.subscribe(named: "my-projects")
self.notificationToken = queryResults.observe() { changes in
switch changes {
case .initial:
print("notification: initial results are populated")
queryResults.forEach { print($0) }
case .update(_, let deletions, let insertions, let modifications):
print("notification: results, inserted, deleteed or modified")
insertions.forEach { print($0) } //or mods or dels
case .error(let error):
fatalError("\(error)")
}
}
} catch {
print(error)
}
}
deinit {
self.notificationToken?.invalidate()
}
The other advantage of keeping that token (and its corresponding code) alive is when there are further changes, your app will be notified. So if another project is added for example, the code in the 'changes' section will run and display that change.
I have a CloudKit database with some data. By pressing a button my app should check for existence of some data in the Database. The problem is that all processes end before my app get the results of its search. I found this useful Answer, where it is said to use Closures.
I tried to follow the same structure but Swift asks me for parameters and I get lost very quick here.
Does someone can please help me? Thanks for any help
func reloadTable() {
self.timePickerView.reloadAllComponents()
}
func getDataFromCloud(completionHandler: #escaping (_ records: [CKRecord]) -> Void) {
print("I begin asking process")
var listOfDates: [CKRecord] = []
let predicate = NSPredicate(value: true)
let query = CKQuery(recordType: "Riservazioni", predicate: predicate)
let queryOperation = CKQueryOperation(query: query)
queryOperation.resultsLimit = 20
queryOperation.recordFetchedBlock = { record in
listOfDates.append(record)
}
queryOperation.queryCompletionBlock = { cursor, error in
if error != nil {
print("error")
print(error!.localizedDescription)
} else {
print("NO error")
self.Array = listOfDates
completionHandler(listOfDates)
}
}
}
var Array = [CKRecord]()
func generateHourArray() {
print("generate array")
for hour in disponibleHours {
let instance = CKRecord(recordType: orderNumber+hour)
if Array.contains(instance) {
disponibleHours.remove(at: disponibleHours.index(of: hour)!)
}
}
}
func loadData() {
timePickerView.reloadAllComponents()
timePickerView.isHidden = false
}
#IBAction func checkDisponibility(_ sender: Any) {
if self.timePickerView.isHidden == true {
getDataFromCloud{ (records) in
print("gotData")
self.generateHourArray()
self.loadData()
}
print(Array)
}
}
Im struggling to understand your code and where the CloudKit elements fit in to it, so Im going to try and give a generic answer which will hopefully still help you.
Lets start with the function we are going to call to get our CloudKit data, lets say we are fetching a list of people.
func getPeople() {
}
This is simple enough so far, so now lets add the CloudKit code.
func getPeople() {
var listOfPeople: [CKRecord] = [] // A place to store the items as we get them
let query = CKQuery(recordType: "Person", predicate: NSPredicate(value: true))
let queryOperation = CKQueryOperation(query: query)
queryOperation.resultsLimit = 20
// As we get each record, lets store them in the array
queryOperation.recordFetchedBlock = { record in
listOfPeople.append(record)
}
// Have another closure for when the download is complete
queryOperation.queryCompletionBlock = { cursor, error in
if error != nil {
print(error!.localizedDescription)
} else {
// We are done, we will come back to this
}
}
}
Now we have our list of people, but we want to return this once CloudKit is done. As you rightly said, we want to use a closure for this. Lets add one to the function definition.
func getPeople(completionHandler: #escaping (_ records: [CKRecord]) -> Void) {
...
}
This above adds a completion hander closure. The parameters that we are going to pass to the caller are the records, so we add that into the definition. We dont expect anyone to respond to our completion handler, so we expect a return value of Void. You may want a boolean value here as a success message, but this is entirely project dependent.
Now lets tie the whole thing together. On the line I said we would come back to, you can now replace the comment with:
completionHandler(listOfPeople)
This will then send the list of people to the caller as soon as CloudKit is finished. Ive shown an example below of someone calling this function.
getPeople { (records) in
// This code wont run until cloudkit is finished fetching the data!
}
Something to bare in mind, is which thread the CloudKit API runs on. If it runs on a background thread, then the callback will also be on the background thread - so make sure you don't do any UI changes in the completion handler (or move it to the main thread).
There are lots of improvements you could make to this code, and adapt it to your own project, but it should give you a start. Right off the bat, Id image you will want to change the completion handler parameters to a Bool to show whether the data is present or not.
Let me know if you notice any mistakes, or need a little more help.