Realm writes are blocking main thread - swift

I have a json file on a remote server that I'm grabbing data from (36k records) using dataTaskWithURL. I'm the resulting JSON (SwiftyJSON) is returned. Below is my completion handler.
The problem is that as soon as I start rolling through the realm create's, all my other callbacks stop executing until the realm commit fires. My desired outcome is that this task simply runs and inserts the data in the background allowing the user to continue on their merry way while this rolls.
The blocking seems to happen once the realm.beginWrite() fires.
RemoteAPI().getMetafile({JSONData, error -> Void in
if (JSONData != nil) {
do {
print("***** Loading realm")
let realm = try Realm()
realm.beginWrite()
for (_, subJSON) in JSONData {
realm.create(Meta.self, value: ["xxxxxx": subJSON["xxxxx"].int!, "xxxxx": subJSON["xxxxx"].stringValue, "xxxxx": subJSON["xxxxx"].stringValue, "xxxxxx": subJSON["xxxxxx"].int!, "xxxxxx": subJSON["xxxxx"].stringValue, "xxxxx": subJSON["xxxxx"].stringValue, "xxxxxx": subJSON["xxxxx"].stringValue], update: true)
}
try realm.commitWrite()
print("***** Finished loading realm")
} catch _ {}
} else {
print("api data fetch failed")
print(error)
}
})
While the above call is happening, I have another call:
RemoteAPI().getLatestActivityData({JSONData, error -> Void in
if (JSONData != nil) {
// do stuff
dispatch_async(dispatch_get_main_queue(), {
NSNotificationCenter.defaultCenter().postNotificationName("refreshTableView", object: nil)
})
}
})
Both of these calls fire from the app delegate. The problem however, is that the observer in this second call does not fire until after the first call above completes.

You should perform the transaction in the background if you don't want to block the main thread.
...
do {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)) {
let realm = try! Realm()
realm.beginWrite()
for (_, subJSON) in JSONData {
realm.create(Meta.self, value: [...], update: true)
}
try realm.commitWrite()
}
}
...
See Realm docs about threading.

The issue had nothing to do with thread blocking. Even though I was running the requests asynchronously something I figured out is if I have Request A and Request B running asynchronously and they both start trying to do Realm writes. They basically get treated as serial requests and the writes will be queued up until the initial writes are completed.
As noted in the documentation (not in the threads section) -- Realm writes will block each other. They note that you can use threading to resolve this, but even using the dispatch_async methods, the writes continued to block each other.
However what I realized was that the initial request is simply loading data to be used for an in-app search and not for anything else. The rest of the system uses Realm for storing user specific data.
My Solution
Request A represents my search meta data request populating, for that I created a new realm file:
let config = Realm.Configuration(
path: utility.getDocumentsDirectory().stringByAppendingPathComponent("Meta.realm"),
readOnly: false)
let realm = try! Realm(configuration: config)
Request B Which was writing all my user data, and then triggering a tableview refresh I had read/write from the default realm.
As soon as I implemented this change in usage, no more blocking and everything worked correctly and as expected.
Honestly I don't know if this is best practice for Swift/iOS or not (I'm not a Swift developer) but it works, and the performance is quite acceptable.

You can try this:
for (_, subJSON) in JSONData {
realm.beginWrite()
realm.create(Meta.self, value: ["xxxxxx": subJSON["xxxxx"].int!, "xxxxx": subJSON["xxxxx"].stringValue, "xxxxx": subJSON["xxxxx"].stringValue, "xxxxxx": subJSON["xxxxxx"].int!, "xxxxxx": subJSON["xxxxx"].stringValue, "xxxxx": subJSON["xxxxx"].stringValue, "xxxxxx": subJSON["xxxxx"].stringValue], update: true)
try realm.commitWrite()
}
I don't know why, but it works for me.

Related

Swift, URLSession, viewDidLoad, tableViewController

I've never really gotten the nuances of async operations so time and again, I get stymied. And I just can't figure it out.
I'm trying to do some very simple web scraping.
My local volleyball association has a page (verbose HTML, not responsive, not mobile-friendly, yaddah, yaddah, yaddah) which shows the refs assigned to each game of the season. I'm trying to write a silly little app which will scrape that page (no API, no direct access to db, etc.) and display the data in a grouped table. The first group will show today's matches (time, home team, away team). The second group will show tomorrow's matches. Third group shows the entire season's matches.
Using code I found elsewhere, my viewDidLoad loads the page, scrapes the data and parses it into an array. Once I've parsed the data, I have three arrays: today, tomorrow, and matches, all are [Match].
override func viewDidLoad() {
super.viewDidLoad()
let url = URL(string: urlString)!
let request = NSMutableURLRequest(url: url)
let task = URLSession.shared.dataTask(with: request as URLRequest) {
data, response, error in
if let error = error {
print (error)
} else {
if let unwrappedData = data {
// scrape, scrape, parse, parse
matchRow = ...
self.matches.append(matchRow)
if matchRow.date == todaysDate {
self.today.append(matchRow)
} else if matchRow.date == tomorrowsDate {
self.tomorrow.append(matchRow)
}
}
}
}
task.resume()
}
As I'm sure is no surprise to anyone who understands async operations, my table is empty. I've checked and I see the the data is there and properly parsed, etc. But I can't for the life of me figure out how get the data in my table. The way I have it now, the data is not ready when numberOfSections or numberOfRowsInSection is called.
I've found the Ray Wenderlich tutorial on URLSession and I also have a Udemy course (Rob Percival) that builds an app to get the weather using web scraping, but in both those instances, the app starts and waits for user input before going out to the web to get the data. I want my app to get the data immediately upon launch, without user interaction. But I just can't figure out what changes I need to make so that those examples work with my program.
Help, please.
You can simply reload the tableviews once the data arrays are getting populated from the URLSession completion block. Have you tried that. Sample snippet may be like the one follows.
let task = URLSession.shared.dataTask(with: request as URLRequest) {
data, response, error in
if let error = error {
print (error)
} else {
if let unwrappedData = data {
// scrape, scrape, parse, parse
matchRow = ...
self.matches.append(matchRow)
if matchRow.date == todaysDate {
self.today.append(matchRow)
} else if matchRow.date == tomorrowsDate {
self.tomorrow.append(matchRow)
}
}
DispatchQueue.main.async { [weak self] in
self?.todayTableView.reloadData()
self?.tomorrowTableView.reloadData()
}
}
}

Reading Data from Realm Database (Swift)

I am new to Realm DataBase and I need a way to read data from realmCloud, but from two different app projects. The way I have tried to implement this is by using query-synced realm. At the moment I'm using a singe realm user to write the data in one app, and the same realm user to read data from another app. The problem is that making a query from the second app(the one used for reading) doesn't return any realm objects ( I have also noticed that user identifier is different from the first one, and also the user permissions are nil.
I have tried setting permissions directly from RealmStudio since documentation is not precise on how to set them from code
func openRealm() {
do {
realm = try Realm(configuration: SyncUser.current!.configuration())
let queryResults = realm.objects(*className*.self)
let syncSubscription = queryResults.subscribe()
let notificationToken = queryResults.observe() { [weak self] (changes) in
switch (changes) {
case .initial: print(queryResults)
case .error(let error): print(error)
default: print("default")
}
}
for token in queryResults {
print(token.tokenString)
}
syncSubscription.unsubscribe()
notificationToken.invalidate()
} catch {
print(error)
}
}
This function prints the data in one app project, but used in another app project with the same user logged in, and the same classFile referenced in the project, it does not. (note that SyncUser.current.identifier is different also
There are a couple of issues.
Some of these calls are asynchronous and the code in your question is going out of scope before the data is sync'd (retreived). The bottom line is code is faster than the internet and you need to design the flow of the app around async calls; don't try to work with the data until it's available.
For example
let notificationToken = queryResults.observe() { [weak self] (changes) in
//here is where results are fully populated
}
// this code may run before results are populated //
for token in queryResults {
print(token.tokenString)
}
Also, let notificationToken is a local var and goes out of scope before the results are populated as well.
These issues are super easy to fix. First is to keep the notification token alive while waiting for results to be populated and the second is to work with the results inside the closure, as that's when they are valid.
var notificationToken: NotificationToken? = nil //a class var
func openRealm() {
do {
let config = SyncUser.current?.configuration()
let realm = try Realm(configuration: config!)
let queryResults = realm.objects(Project.self)
let syncSubscription = queryResults.subscribe(named: "my-projects")
self.notificationToken = queryResults.observe() { changes in
switch changes {
case .initial:
print("notification: initial results are populated")
queryResults.forEach { print($0) }
case .update(_, let deletions, let insertions, let modifications):
print("notification: results, inserted, deleteed or modified")
insertions.forEach { print($0) } //or mods or dels
case .error(let error):
fatalError("\(error)")
}
}
} catch {
print(error)
}
}
deinit {
self.notificationToken?.invalidate()
}
The other advantage of keeping that token (and its corresponding code) alive is when there are further changes, your app will be notified. So if another project is added for example, the code in the 'changes' section will run and display that change.

How do I asynchronously initialize firebase firestore listeners while also knowing when all of the tasks are done?

Basically, at the launch of my app, I want to load the latest data from firebase from about 4-5 different documents. Then I also want to set up a listener to monitor data changes. I do this by calling 4-5 similar functions that take a dispatchGroup as an argument. I may be approaching this completely wrong but I could not think of any other way to do it. I just want to load those documents, set up listeners, and take certain action whenever those docs are loaded at the launch of the app.
// app launch
let dispatch = DispatchGroup()
getFirebaseDocument1(dispatch: dispatch)
getFirebaseDocument2(dispatch: dispatch)
getFirebaseDocument3(dispatch: dispatch)
getFirebaseDocument4(dispatch: dispatch)
getFirebaseDocument5(dispatch: dispatch)
dispatch.notify(queue:main) {
// execute some code to execute after all the documents are fetched
}
// typical getFirebaseDocument code
dispatch.enter()
let ref = someFirestoreReference
ref.addSnapshotListener { (snapshot, error) in
if let error = error{
// handle error
} else {
// load the document
}
dispatch.leave()
}
The code works fine when it's launched but crashes whenever the listener receives an update. I know this is because the dispatch.leave() is called in the listener function. However, I cannot seem to figure out a clever solution to where I can asynchronously load the data from firebase at launch while also setting up listeners. I would also prefer not to nest closures within one another as it wouldn't be asynchronous and it would also be a pain.
I may be wrong, but you should leave the group inside your block, here is the example how I do it using group
class func getRates(completion: #escaping EmptyBlock) {
let eurRequest = APIConfigs.request(part: "rs/price/history")
let usdRequest = APIConfigs.request(part: "rs/price/history/usd")
let group = DispatchGroup()
group.enter()
sendRequest(request: eurRequest, method: .get, parameters: ServerParameters.rates()) { response in
Course.current.addRate(rates: ratesRequest(response: response), type: .eur)
group.leave()
}
group.enter()
sendRequest(request: usdRequest, method: .get, parameters: ServerParameters.rates()) { response in
Course.current.addRate(rates: ratesRequest(response: response), type: .usd)
group.leave()
}
group.notify(queue: .main) {
completion()
}
}

Multi-threaded core data sometimes returns nil properties

I am new to core data. I have an app that uses core data as local store. Writing to and reading from core data is done by background threads. While this works generally, in rare cases are the fetched data wrong, i.e. properties of a fetched entity are nil.
To check the situation, I wrote a unit test that starts 2 async threads: One fetches continuously from core data, and the other one overwrites continuously these data by first deleting all data, and then storing new data.
This test pretty quickly provokes the error, but I have no idea why. Of course I guess this is a multi-threading problem, but I don’t see why, because fetches and deletion+writes are done in separate managed contexts of a single persistentContainer.
I am sorry that the code below is pretty long, although shortened, but I think without it one cannot identify the problem.
Any help is highly welcome!
Here is my function to fetch data:
func fetchShoppingItems(completion: #escaping (Set<ShoppingItem>?, Error?) -> Void) {
persistentContainer.performBackgroundTask { (managedContext) in
let fetchRequest: NSFetchRequest<CDShoppingItem> = CDShoppingItem.fetchRequest()
do {
let cdShoppingItems: [CDShoppingItem] = try managedContext.fetch(fetchRequest)
for nextCdShoppingItem in cdShoppingItems {
nextCdShoppingItem.managedObjectContext!.performAndWait {
let nextname = nextCdShoppingItem.name! // Here, sometimes name is nil
} // performAndWait
} // for all cdShoppingItems
completion(nil, nil)
return
} catch let error as NSError {
// error handling
completion(nil, error)
return
} // fetch error
} // performBackgroundTask
} // fetchShoppingItems
I have commented the line that sometimes crashes the test, since name is nil.
Here are my functions to store data:
func overwriteCD(shoppingItems: Set<ShoppingItem>,completion: #escaping () -> Void) {
persistentContainer.performBackgroundTask { (managedContext) in
self.deleteAllCDRecords(managedContext: managedContext, in: "CDShoppingItem")
let cdShoppingItemEntity = NSEntityDescription.entity(forEntityName: "CDShoppingItem",in: managedContext)!
for nextShoppingItem in shoppingItems {
let nextCdShoppingItem = CDShoppingItem(entity: cdShoppingItemEntity,insertInto: managedContext)
nextCdShoppingItem.name = nextShoppingItem.name
} // for all shopping items
self.saveManagedContext(managedContext: managedContext)
completion()
} // performBackgroundTask
} // overwriteCD
func deleteAllCDRecords(managedContext: NSManagedObjectContext, in entity: String) {
let deleteFetch = NSFetchRequest<NSFetchRequestResult>(entityName: entity)
let deleteRequest = NSBatchDeleteRequest(fetchRequest: deleteFetch)
deleteRequest.resultType = .resultTypeObjectIDs
do {
let result = try managedContext.execute(deleteRequest) as? NSBatchDeleteResult
let objectIDArray = result?.result as? [NSManagedObjectID]
let changes = [NSDeletedObjectsKey: objectIDArray]
NSManagedObjectContext.mergeChanges(fromRemoteContextSave: changes as [AnyHashable: Any], into: [managedContext])
} catch let error as NSError {
// error handling
}
} // deleteAllCDRecords
func saveManagedContext(managedContext: NSManagedObjectContext) {
if !managedContext.hasChanges { return }
do {
try managedContext.save()
} catch let error as NSError {
// error handling
}
} // saveManagedContext
Are you sure that name isn't nil for all requested entities? Just use guard-let to avoid ! for optional variables. Also ! it isn't safe way to unwrap optional variable especially if you can't be sure for source of data.
The problem with my code was apparently a race condition:
While the „fetch“ thread fetched the core data records, and tried to assign the attributes to the properties, the „store“ thread deleted the records.
This apparently released the attribute objects, so that nil was stored as property.
I thought that the persistentContainer would automatically prevent this, but it does not.
The solution is to execute both background threads of the persistentContainer in a concurrent serial queue, the „fetch“ thread synchronously, and the „store“ thread asynchronously with a barrier.
So, concurrent fetches can be executed, while a store waits until all current fetches are finished.
The concurrent serial queue is defined as
let localStoreQueue = DispatchQueue(label: "com.xxx.yyy.LocalStore.localStoreQueue",
attributes: .concurrent)
EDIT:
In the following fetch and store functions, I moved the core data function persistentContainer.performBackgroundTask inside the localStoreQueue. If it were outside as in my original answer, the store code in localStoreQueue.async(flags: .barrier) would setup a new thread and thus use managedContext in another thread that it was created in, which is a core data multi-threading error.
The „fetch“ thread is modified as
localStoreQueue.sync {
self.persistentContainer.performBackgroundTask { (managedContext) in
let fetchRequest: NSFetchRequest<CDShoppingItem> = CDShoppingItem.fetchRequest()
//…
} // performBackgroundTask
} // localStoreQueue.sync
and the „store“ thread as
localStoreQueue.async(flags: .barrier) {
self.persistentContainer.performBackgroundTask { (managedContext) in
self.deleteAllCDRecords(managedContext: managedContext, in: "CDShoppingItem")
//…
} // performBackgroundTask
} // localStoreQueue.async

Realm write transaction failing, despite being in transaction

I have this code, which should append messages with a new message:
func addMessage(_ message: Message) {
do {
try Realm().write {
self.messages.append(message)
}
} catch let error {
print("could not add message due to error:\n\(error)")
}
}
However, I get an exception Cannot modify managed RLMArray outside of a write transaction It doesn't make any sense to me, because I'm already in a write transaction...
You need to create a Realm object before applying the write module.
According to the GitHub documentation, you can try code like this:
func addMessage(_ message: Message) {
do {
let realm = try! Realm()
try! realm.write {
self.messages.append(message)
}
} catch let error {
print("Could not add message due to error:\n\(error)")
}
}
Hope it helps!
You can use let realm = try! Realm() with a custom default configuration by setting a default configuration see here:
var config = Realm.Configuration()
// Set this as the configuration used for the default Realm
Realm.Configuration.defaultConfiguration = config
The trouble was I was using a plain Realm object with no special configuration. Since I am using Realm Mobile Platform, I needed to create a Realm object with the same config each time I want to write to that DB:
let configuration = Realm.Configuration(
syncConfiguration: SyncConfiguration(user: user, realmURL: URL(string: "realm://127.0.0.1:9080/~/speciail")!)
)
self.realm = try! Realm(configuration: configuration)
//now do the write transaction!
It took a bit of refactoring, but I have it now. My thanks to those of you who took the time to help me.