swift parse query limit - swift

Got a question about parse query.limit with swift
using below settings and limit to pull objects from parse
func getUsernames()
query.limit = self.limit
query.findObjectsInBackgroundWithBlock //display on TableView
self.usernamesArray.append(someuser)
self.tableView.reloadData()
}
then in tableView:willDisplayCell
if indexPath.row == self.usernamesArray.count - 1 {
self.limit += 5
self.getUsernames()
}
and in viewDidAppear
if self.usernames.count == 0 {
self.limit = self.limit + 10
self.getUsernames()
}
this works fine. whenever my table scrolls through the second last Cell another 5 is ready which is what i expected yay!.
problem is if usernamesArray.count has total value of 50 and when the last cell(50th cell/count) has reached/scrolled the tableView:willDisplayCell is keep getting called and the self.limit is keep increasing 55, 60, 65 etc .... it doesn't stop when it reaches the LAST Cell or Last data in array. it keeps using the LTE data and query.limit number increases 5 by 5 (when there isn't anymore array value available)
am i doing this right? or should i try different approach?
any master of swift will be appreciated! Thanks

No, you aren't doing it right. First, by just increasing the limit and querying you're always getting another copy of the data you already have and maybe a few additional items. You should be changing the offset (skip) for each new query, not the limit.
You also need to check the response to see how many items were returned. If the number of items returned is less than the request limit then you know you're at the end of the data and you shouldn't make any more requests. Set a flag so that when scrolling you know there isn't anything else you can load.

I found it easier to create a resultSet class that handles all this information.
class ParseResultSet: NSObject {
var limit = 20
var skip = 0
var total = 0
var limitReached = false
var orderByAscendingKey: String?
var orderByDescendingKey: String?
var searchActive: Bool?
func increaseSkipByLimit() {
skip += limit
}
func increaseTotal(byNumber: Int) {
total += byNumber
if byNumber == 0 {
self.limitHasBeenReached()
skip = total
}
else {
self.increaseSkipByLimit()
}
}
func limitHasBeenReached() {
limitReached = true
}
}
I then use a method where I get the objects from Parse in a completion block. I check if the limit has been reached and increment the total if it hasn't
func getObjects(classname: String, include: [String], completion: #escaping (_ result: [PFObject]) -> Void) {
if self.resultSet.limitReached == false || self.resultSet.searchActive == true {
fetchObjectsFromClass(parseClass: classname, include: include, completion: { [weak self] (objects) in
self?.resultSet.increaseTotal(byNumber: objects.count)
completion(objects)
})
}
else {
print("LIMIT REACHED")
}
}
In my case, the fetchObjectsFromClass is a global function where the query is generated and returns an array of PFObjects.
Hopefully this gives you an idea of what you need to do

Related

Why doesn't adding an index to a Realm Object speed up my count query?

I have the following code I'm using to try to test indexing with Realm. I have an object class with a single Int value that is indexed, also an identical class, but unindexed.
After inserting 1 million rows of each class, with evenly distributed values, I run some queries to see the time it takes with and without indexes.
The time is basically the same (sometimes unindexed will be slightly faster, sometimes indexed will be slightly faster -- but within about 5% or something).
the results I get are these:
realm init6 took: 79.721125 ms
1000000
baseCountIndexed took: 0.233 ms
1000000
baseCountUnindexed took: 0.075833 ms
500000
getCountIndexedObjsViaFilterString took: 18.982542 ms
500000
getCountIndexedObjsViaWhereClosure took: 16.156041 ms
500000
getCountUnindexedObjsViaFilterString took: 17.985084 ms
500000
getCountUnindexedObjsViaWhereClosure took: 16.031917 ms
I would expect the indexed version to be faster -- seems like it should be close to the base count time, since an index should result in O(Log N) time -- the indexed version should at least be faster than unindexed.
What am I doing wrong?
The code I'm using follows: (long so I could make it complete / runnable)
The code basically inserts 1 million indexed objects, and 1 million unindexed objects, then gets the count of both types -- which is very fast -- and then does 2 different kinds of where clauses, one using a .filter(String), and one using a .where closure, to select 1/2 the objects, then do a count.
It is my understanding that the results coming from realm are lazy, and so the count should be done without loading the objects into memory.
The times for all 4 queries (indexed with .filter and indexed with .where, and unindexed for .filter & .where) all take about the same amount of time.
Edit:
After the answer from jay below, I ran the same code, with results not massively dissimilar -- I got about 33% speed increase by indexing, rather than the 66% he got. Still I would expect the results to be near instant for an indexed field, so something isn't right. I'll update if / when I figure it out. For now I'm moving on, since even the unindexed speed is good enough for my current usage. This is just really weird to see it so slow with indexes.
import RealmSwift
class TimeIt {
let val = DispatchTime.now()
func elapsed() -> DispatchTimeInterval {
return val.distance(to: DispatchTime.now())
}
static func time(_ desc: String, aclosure: () -> Void) {
let t = TimeIt()
aclosure()
print("\(desc) took: \(Double(t.elapsed().nanoseconds) / 1000000.0) ms")
}
}
class RealmIndexed: Object {
#Persisted var id = UUID().uuidString
#Persisted(indexed: true) var val: Int = 0
}
class RealmUnindexed: Object {
#Persisted var id = UUID().uuidString
#Persisted var val: Int = 0
}
func generateObjs(_ hundredsOfThousands: Int = 1) {
let realm = try! Realm()
var objs: [RealmIndexed] = []
for _ in 0..<hundredsOfThousands {
for i in 0..<100_000 {
let obj = RealmIndexed()
obj.val = i
objs.append(obj)
}
}
try! realm.write {
realm.add(objs)
}
var objs2: [RealmUnindexed] = []
for _ in 0..<hundredsOfThousands {
for i in 0..<100_000 {
let obj = RealmUnindexed()
obj.val = i
objs2.append(obj)
}
}
try! realm.write {
realm.add(objs2)
}
}
func baseCountIndexed() -> Int {
return (try! Realm().objects(RealmIndexed.self)).count
}
func baseCountUnindexed() -> Int {
return (try! Realm().objects(RealmUnindexed.self)).count
}
func getCountIndexedObjsViaFilterString(_ minVal: Int = 50000) -> Int {
let count = (try! Realm().objects(RealmIndexed.self).filter("val >= %#", minVal)).count
return count
}
func getCountUnindexedObjsViaFilterString(_ minVal: Int = 50000) -> Int {
let count = (try! Realm().objects(RealmUnindexed.self).filter("val >= %#", minVal)).count
return count
}
func getCountIndexedObjsViaWhereClosure(_ minVal: Int = 50000) -> Int {
let count = (try! Realm().objects(RealmIndexed.self).where { $0.val >= minVal }).count
return count
}
func getCountUnindexedObjsViaWhereClosure(_ minVal: Int = 50000) -> Int {
let count = (try! Realm().objects(RealmUnindexed.self).where { $0.val >= minVal }).count
return count
}
func testRealmSpeed() {
TimeIt.time("realm init6") { _ = try! Realm() }
TimeIt.time("baseCountIndexed") { print(baseCountIndexed()) }
TimeIt.time("baseCountUnindexed") { print(baseCountUnindexed()) }
TimeIt.time("getCountIndexedObjsViaFilterString") { print(getCountIndexedObjsViaFilterString()) }
TimeIt.time("getCountIndexedObjsViaWhereClosure") { print(getCountIndexedObjsViaWhereClosure()) }
TimeIt.time("getCountUnindexedObjsViaFilterString") { print(getCountUnindexedObjsViaFilterString()) }
TimeIt.time("getCountUnindexedObjsViaWhereClosure") { print(getCountUnindexedObjsViaWhereClosure()) }
}
generateObjs(10)
testRealmSpeed()
This is not exactly an answer but perhaps additional info.
I don't think you're doing anything wrong but perhaps a simpler test will be more revealing. I set up two similar objects, one using indexing on a val property and one not. Testing for equality; val == 5, here's the setup:
For brevity, I'm omitting the writing code but it creates 1 Million of each object containing the values 5 and 9 e.g. Realm will contain a million indexed 5,9, 5,9, 5,9 etc and a million not indexed 5,9, 5,9 etc. (5 & 9 are just arbitrary numbers I picked)
And then a function to test each object type. I queried for '5' so it would return a 1/2 million results
func testNotIndexed() {
Task {
let realm = Realm()
let startTime = Date()
let results = try await realm.objects(NotIndexedClass.self).where { $0.val == 5 }
let elapsed = Date().timeIntervalSince(startTime)
print("Not Indexed took: \(elapsed * 1000) ms")
}
}
func testIndexed() {
Task {
let realm = Realm()
let startTime = Date()
let results = try await realm.objects(IndexedClass.self).where { $0.val == 5 }
let elapsed = Date().timeIntervalSince(startTime)
print("Indexed took: \(elapsed * 1000) ms")
}
}
and the repeatable results
Not Indexed took: 1.2680292129516602 ms
Indexed took: 0.44596195220947266 ms
So the indexed query took roughly 1/3 the time.
If the test parameters are changed to be objects containing numbers from 0 to 999,999, and then query for all numbers > 50,000 (not 500k to increase the returned dataset size), the results are similar.

How to fetch objects from results one by one without do .count

I need to fetch from Realm Results 20 objects or less. A database can be heavy, so Results.count is a long time for calling.
So, what I need is to fetch objects from Results one by one until I get 20 or until last object.
But, when I'm trying to fetch index after the last object it's throwing Realm exception 'Index x is out of bounds (must be less than x)'.
So, this one isn't working:
let searchResult = Ticket().get(filter: "base == nil && deleted == 0 AND orderPaidAt > 0 AND (\(query))").sorted(byKeyPath: "orderPaidAt")
for i in 0..<20 {
if let ticket = searchResult[i] as? Ticket {
...
} else {
break
}
}
If I'm trying to use searchResult.count or searchResult.endIndex it increases a time a lot, especially on old devices. That's why I want to avoid it.
The results are lazily loaded, so you could loop through the results one by one, until the end, or until you hit a self-set count:
let searchResult = Ticket().get(filter: "base == nil && deleted == 0 AND orderPaidAt > 0 AND (\(query))").sorted(byKeyPath: "orderPaidAt")
var count = 0
for thisTicket in searchResult {
// do something
count += 1
if count > 20 { break }
}
This way you are only loading the values that you need, and never calling count or accessing the results out of bounds.
You can use prefix(maxLenght: Int) method to get a subCollection with specified maxLenght.
Example:
realm.objects(ObjectModel.self).prefix(20).count

How to iterate thru array with asynchronous method, and put return values into an array in the PROPER order

I have an array that holds x number of values(no more than 25). Each value corresponds to an item object that i wish to retrieve from a remote endpoint. I use the following method to retrieve the item object for each corresponding identifier...
func getValues(valueIDs: [Int]){
var values = [Item]()
let group = dispatch_group_create()
for i in 0...valueIDs.count-1 {
dispatch_group_enter(group)
Item.special(valueIDs[i], completion: ({ result in
if let value = result.response.result {
values.append(value)
dispatch_group_leave(group)
}
})
)
}
dispatch_group_notify(group, dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0)) {
print("the values are \(values)")
}
}
The method is asynchronous, so each call to it in the for-loop returns immediately and the loop finishes iterating thru all the values before any of the values return. I use a dispatch_group to be notified of when all the calls have returned.
The problem I have is that I need to have the retrieved values be put into my array, in the same order that they were called for. Currently, they are appended to the array when they are returned so they are randomly ordered. I do not want to put all the calls on a serial queue and make them wait for each other, that would take too long. Any advice would be great!!
Check if this works, declaring a fixed length array and accessing the array element by index:
func getValues(valueIDs: [Int]){
var items = [Item?](count: valueIDs.count, repeatedValue: nil)
let group = dispatch_group_create()
for i in 0...valueIDs.count-1 {
dispatch_group_enter(group)
Item.special(valueIDs[i], completion: ({ result in
if let value = result.response.result {
items[i] = value
dispatch_group_leave(group)
}
})
)
}
dispatch_group_notify(group, dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0)) {
print("the values are \(items)")
}
}
#muneeb's answer it the standard approach. For an alternate, maybe more readable, approach, use a dictionary to map index to item.
func getValues(valueIDs: [Int]) {
var keyedValues = [Int: Item]()
let group = dispatch_group_create()
for i in 0 ..< valueIDs.count {
dispatch_group_enter(group)
Item.special(valueIDs[i]) { result in
if let value = result.response.result {
keyedValues[i] = value
dispatch_group_leave(group)
}
}
}
dispatch_group_notify(group, dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0)) {
var values = [Item]()
for i in 0 ..< valueIDs.count {
values.append(keyedValues[i]!)
}
print("the values are \(values)")
}
}

Query Parse to preload all objectID data in Swift

I'm building a sort of hot or not style app in Swift where the user can vote: HOT, NOT and MAYBE on an image, respectively.
For every time the user gets to a image, they vote and then for each respective vote the IBAction triggers a query that shows the result of the total votes and total hots from Parse as shown in my code below.
I plan to have 1,000 images.
Can I preload all the objectIDs that correspond to each respective image and then when the user votes on the image, the data is already preloaded/queried from parse? How would i go about that?
For now, I'm writing a query for each ObjectID which would take 1000 queries from 1000 different images... Obviously unscalable.
The swipePosition variable is just a counter that counts which image the user is on. The images being stored are in an Array for now stored on Xcode. Maybe they can be preloaded as well if they are stored on Parse?
(I am only showing the "hotButtonQuery" function, but there is also a Not and Maybe buttonQuery function...)
Is there a way to simply this code so that it's scalable? Because as of now there's no way I can scale past 25 images...
Thanks a lot!
func hotButtonQuery() {
if swipePosition == 0 {
var query = PFQuery(className:"UserData")
query.getObjectInBackgroundWithId("RlvK3GhfqE") {
(userData: PFObject!, error: NSError!) -> Void in
if error != nil {
println(error)
}
else {
userData.incrementKey("totalVotes", byAmount: 1)
userData.incrementKey("hot", byAmount: 1)
var updateTotalVotesUILabel = userData.objectForKey("totalVotes") as NSInteger
var updateHotsUILabel = userData.objectForKey("hot") as NSInteger
userData.saveInBackground()
println("parse was updated!")
self.totalVotesLabel.text = String(updateTotalVotesUILabel)
self.totalHotsLabel.text = String(updateHotsUILabel)
}
}
} else if swipePosition == 1 {
var query = PFQuery(className:"UserData")
query.getObjectInBackgroundWithId("30WlVtgurP") {
(userData: PFObject!, error: NSError!) -> Void in
if error != nil {
println(error)
}
else {
userData.incrementKey("totalVotes", byAmount: 1)
userData.incrementKey("hot", byAmount: 1)
var updateTotalVotesUILabel = userData.objectForKey("totalVotes") as NSInteger
var updateHotsUILabel = userData.objectForKey("hot") as NSInteger
//println(userData.objectForKey("totalVotes"))
//println("total HOTs:")
//println(userData.objectForKey("hot"))
userData.saveInBackground()
println("parse was updated!")
self.totalVotesLabel.text = String(updateTotalVotesUILabel)
self.totalHotsLabel.text = String(updateHotsUILabel)
}
}
} else if swipePosition == 3 {
var query = PFQuery(className:"UserData")
query.getObjectInBackgroundWithId("5D6ARjk3xS") {
(userData: PFObject!, error: NSError!) -> Void in
if error != nil {
println(error)
}
else {
userData.incrementKey("totalVotes", byAmount: 1)
userData.incrementKey("hot", byAmount: 1)
var updateTotalVotesUILabel = userData.objectForKey("totalVotes") as NSInteger
var updateHotsUILabel = userData.objectForKey("hot") as NSInteger
//println(userData.objectForKey("totalVotes"))
//println("total HOTs:")
//println(userData.objectForKey("hot"))
userData.saveInBackground()
println("parse was updated!")
self.totalVotesLabel.text = String(updateTotalVotesUILabel)
self.totalHotsLabel.text = String(updateHotsUILabel)
}
}
}
Respectfully, I would suggest rethinking your whole approach. Don't store objectIds in your code. Don't repeat essentially the same block of code over and over. Keep it simple:
Retrieve and display image
Capture vote and save object
Query for next object and repeat process
If you want to reduce the number of queries, grab them in batches of 10 or so. No sense loading everything since the user is not likely to click through all 1000.
The only thing left to figure out is how you will be keeping track of which images the user has already voted on. You could do this client-side or within Parse. There are a few decent approaches.

Process Array in parallel using GCD

I have a large array that I would like to process by handing slices of it to a few asynchronous tasks. As a proof of concept, I have the written the following code:
class TestParallelArrayProcessing {
let array: [Int]
var summary: [Int]
init() {
array = Array<Int>(count: 500000, repeatedValue: 0)
for i in 0 ..< 500000 {
array[i] = Int(arc4random_uniform(10))
}
summary = Array<Int>(count: 10, repeatedValue: 0)
}
func calcSummary() {
let group = dispatch_group_create()
let queue = dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)
for i in 0 ..< 10 {
dispatch_group_async(group, queue, {
let base = i * 50000
for x in base ..< base + 50000 {
self.summary[i] += self.array[x]
}
})
}
dispatch_group_notify(group, queue, {
println(self.summary)
})
}
}
After init(), array will be initialized with random integers between 0 and 9.
The calcSummary function dispatches 10 tasks that take disjoint chunks of 50000 items from array and add them up, using their respective slot in summary as an accummulator.
This program crashes at the self.summary[i] += self.array[x] line. The error is:
EXC_BAD_INSTRUCTION (code = EXC_I386_INVOP).
I can see, in the debugger, that it has managed to iterate a few times before crashing, and that the variables, at the time of the crash, have values within correct bounds.
I have read that EXC_I386_INVOP can happen when trying to access an object that has already been released. I wonder if this has anything to do with Swift making a copy of the array if it is modified, and, if so, how to avoid it.
This is a slightly different take on the approach in #Eduardo's answer, using the Array type's withUnsafeMutableBufferPointer<R>(body: (inout UnsafeMutableBufferPointer<T>) -> R) -> R method. That method's documentation states:
Call body(p), where p is a pointer to the Array's mutable contiguous storage. If no such storage exists, it is first created.
Often, the optimizer can eliminate bounds- and uniqueness-checks within an array algorithm, but when that fails, invoking the same algorithm on body's argument lets you trade safety for speed.
That second paragraph seems to be exactly what's happening here, so using this method might be more "idiomatic" in Swift, whatever that means:
func calcSummary() {
let group = dispatch_group_create()
let queue = dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)
self.summary.withUnsafeMutableBufferPointer {
summaryMem -> Void in
for i in 0 ..< 10 {
dispatch_group_async(group, queue, {
let base = i * 50000
for x in base ..< base + 50000 {
summaryMem[i] += self.array[x]
}
})
}
}
dispatch_group_notify(group, queue, {
println(self.summary)
})
}
When you use the += operator, the LHS is an inout parameter -- I think you're getting race conditions when, as you mention in your update, Swift moves around the array for optimization. I was able to get it to work by summing the chunk in a local variable, then simply assigning to the right index in summary:
func calcSummary() {
let group = dispatch_group_create()
let queue = dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)
for i in 0 ..< 10 {
dispatch_group_async(group, queue, {
let base = i * 50000
var sum = 0
for x in base ..< base + 50000 {
sum += self.array[x]
}
self.summary[i] = sum
})
}
dispatch_group_notify(group, queue, {
println(self.summary)
})
}
You can also use concurrentPerform(iterations: Int, execute work: (Int) -> Swift.Void) (since Swift 3).
It has a much simpler syntax and will wait for all threads to finalise before returning.:
DispatchQueue.concurrentPerform(iterations: iterations) { i in
performOperation(i)
}
I think Nate is right: there are race conditions with the summary variable. To fix it, I used summary's memory directly:
func calcSummary() {
let group = dispatch_group_create()
let queue = dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)
let summaryMem = UnsafeMutableBufferPointer<Int>(start: &summary, count: 10)
for i in 0 ..< 10 {
dispatch_group_async(group, queue, {
let base = i * 50000
for x in base ..< base + 50000 {
summaryMem[i] += self.array[x]
}
})
}
dispatch_group_notify(group, queue, {
println(self.summary)
})
}
This works (so far).
EDIT
Mike S has a very good point, in his comment below. I have also found this blog post, which sheds some light on the problem.
Any solution that assigns the i'th element of the array concurrently risks race condition (Swift's array is not thread-safe). On the other hand, dispatching to the same queue (in this case main) before updating solves the problem but results in a slower performance overall. The only reason I see for taking either of these two approaches is if the array (summary) cannot wait for all concurrent operations to finish.
Otherwise, perform the concurrent operations on a local copy and assign it to summary upon completion. No race condition, no performance hit:
Swift 4
func calcSummary(of array: [Int]) -> [Int] {
var summary = Array<Int>.init(repeating: 0, count: array.count)
let iterations = 10 // number of parallel operations
DispatchQueue.concurrentPerform(iterations: iterations) { index in
let start = index * array.count / iterations
let end = (index + 1) * array.count / iterations
for i in start..<end {
// Do stuff to get the i'th element
summary[i] = Int.random(in: 0..<array.count)
}
}
return summary
}
I've answered a similar question here for simply initializing an array after computing on another array