Generic Grand Central Dispatch - swift

I have code set up like below. It is my understanding that queue1 should finish all operations THEN move to queue2. However, as soon as my async operation starts, queue2 begins. This defeats the purpose of GCD.. what am I doing wrong? This outputs:
did this finish
queue2
then some time later, prints success from image download
..I want to make it clear that if I put in other code in queue1, such as print("test") or a loop 0..10 printing i, all those operations will complete before moving to queue2. It seems the async download is messing with it, how can I fix this? There is no documentation anywhere, I used This very hepful guide from AppCoda http://www.appcoda.com/grand-central-dispatch/
let queue1 = DispatchQueue(label: "com.matt.myqueue1")
let queue2 = DispatchQueue(label: "com.matt.myqueue2")
let group1 = DispatchGroup()
let group2 = DispatchGroup()
let item = DispatchWorkItem{
// async stuff happening like downloading an image
// print success if image downloads
}
queue1.sync(execute: item)
item.notify(queue1, execute: {
print("did this finish?")
})
queue2.sync {
print("queue2")
}

let item = DispatchWorkItem{
// async stuff happening like downloading an image
// print success if image downloads
}
OK, defines it, but nothing runs yet.
queue1.sync(execute: item)
Execute item and kick off its async events. Immediately return after that. Nothing here says "wait for those unrelated asynchronous events to complete." The system doesn't even have a way to know that there are additional async calls inside of functions you call. How would it know whether object.doit() includes async calls or not (and whether those are async calls you meant to wait for)? It just knows when item returns, continue.
This is what group1 is supposed to be used for (you don't seem to use it for anything). Somewhere down inside these "async stuff happening" you're supposed to tell the system that it finished by leaving the group. (I have no idea what group2 is for. It's never used either.)
item.notify(queue1, execute: {
print("did this finish?")
})
item already finished. We know it has to have finished already, because it was run with sync, and that doesn't return until its item has. So this block will be immediately scheduled on queue1.
queue2.sync {
print("queue2")
}
Completely unrelated and could run before or after the "did this finish" code, we schedule a block on queue2.
What you probably meant was:
let queue1 = DispatchQueue(label: "com.matt.myqueue1")
let group1 = DispatchGroup()
group1.enter()
// Kick off async stuff.
// These usually return quickly, so there's no need for your own queue.
// At some point, when you want to say this is "done", often in some
// completion handler, you call group1.leave(), for example:
... completionHandler: { group1.leave() }
// When all that finishes, print
group.notify(queue: queue1) { print("did this finish?") }

EVERYTHING is initially queued from the main queue, however at some point you switch from main queue to a background queue and you should NOT expect a synchronized queue would wait for what is has enqueued on another queue. They are irrelevant. If that was the case then always and always no matter what, everything is to wait for whatever it asked to run.*
so here's what I see is happening.
queue1 is happily finished, it has done everything it was suppose to by enqueuing item on another queue <-- that's all it was suppose to do. Since the 'async stuff' is async... it won't wait for it to finish. <-- actually if you use breakpoints inside the async you would see that the breakpoints would jump to } meaning they don't wait for a background queue they just jump to the end of the queue, since they are no longer on the main thread
then since it was a sync queue it wall wait till it's done. Once done it will go through its notify...now here's where it get's tricky: depending on how fast what you do in the async... 'print success' will either get called first or "queue2" though here obviously queue2 is returning/finishing sooner.
similar to what you see in this answer.
*: A mother (main queue) tells it's child1 to your room and bring book1 back, then tells child2 to your room and bring book2 back, then tells child3 to your room and bring book3 back. Each child is being ran from its own queue ( not the mother's queue).
The mother doesn't wait for child1 to come back...so it can tell child2 to go. The mother only tells child 1 go...then child2 go...then child 3 go.
However child2 is not told (to go) before child 1, nor child3 is told before child2 or child1 <-- this is because of the serial-ness of main queue. They get dispatched serially, but their completion order depends on how fast each child/queue finishes

Related

How to interrupt the awaiting in a `for-await` loop / Swift

I'm wondering how to stop a for-await loop from "awaiting".
Here's the loop. I use it to listen to new Transactions with storekit2:
transactionListener = Task(priority: .background) { [self] in
// wait for transactions and process them as they arrive
for await verificationResult in Transaction.updates {
if Task.isCancelled { print("canceled"); break }
// --- do some funny stuff with the transaction here ---
await transaction.finish()
}
print("done")
}
As you can see, Transaction.updates is awaited and returns a new transaction whenever one is created. When the App finishes, I cancel the loop with transactionListener.cancel() - but the cancel is ignored as the Transaction.updates is waiting for the next transaction to deliver and there's no direct way in the API to stop it (like, e.g. Task.sleep() does)
The issues starts, when I run unit-tests. The listener from a previous test is still listening while the next test is already running. This produces very unreliable test results and crashes our CI/CD pipeline. I nailed it down to the shown piece of code and the described issue.
So, question: Is it possible to interrupt a for-await loop from awaiting? I have something like the Unix/Linux command kill -1 in mind. Any ideas?

Background Process as NSOperation or Thread to monitor and update File

I want to check if a pdf file is changed or not, and if is changed i want to update the corresponding view. I don't know if it's more suitable to use a background process as a Thread or as an NSOperation to do this task. The Apple Documentation says: "Examples of tasks that lend themselves well to NSOperation include network requests, image resizing, text processing, or any other repeatable, structured, long-running task that produces associated state or data.But simply wrapping computation into an object doesn’t do much without a little oversight".
Also, if I understood correctly from the documentation, a Thread once started can't be stopped during his execution while an NSOperation could be paused or stopped and also they could rely on dependency to wait the completion of another task.
The workflow of this task should be more or less this diagram:
Task workflow
I managed to get the handler working after the notification of type .write has been sent. If i monitor for example a *.txt file everything works as expected and i receive only one notification. But i am monitoring a pdf file which is generated from terminal by pdflatex and thus i receive with '.write' nearly 15 notification. If i change to '.attrib' i get 3 notification. I need the handler to be called only once, not 15 or 3 times. Do you have any idea how can i do it or is not possible with a Dispatch Source? Maybe there is a way to execute a dispatchWorkItem only once?
I have tried to implement it like this(This is inside a FileMonitor class):
func startMonitoring()
{
....
let fileSystemRepresentation = fileManager.fileSystemRepresentation(withPath: fileStringURL)
let fileDescriptor = open(fileSystemRepresentation, O_EVTONLY)
let newfileMonitorSource = DispatchSource.makeFileSystemObjectSource(fileDescriptor: fileDescriptor,
eventMask: .attrib,
queue: queue)
newfileMonitorSource.setEventHandler(handler:
{
self.queue.async
{
print(" \n received first write event, removing handler..." )
self.newfileMonitorSource.setEventHandler(handler: nil)
self.test()
}
})
self.fileMonitorSource = newfileMonitorSource
fileMonitorSource!.resume()
}
func test()
{
fileMonitorSource?.cancel()
print(" restart monitoring ")
startMonitoring()
}
I have tried to reassign the handler in test(), but it's not working(if a regenerate the pdf file, what is inside the new handler it's not executed) and to me, doing in this way, it seems a bit boilerplate code. I have also tried the following things:
suspend the DispatchSource in the setEventHandler of startMonitoring() (passing nil), but then when i am resuming it, i get the remaining .write events.
cancel the DispatchSource object and recall the startMonitoring() as you can see in the code above, but in this way i create and destroy the DispatchSource object everytime i receive an event, which i don't like because the cancel() function shoul be called in my case only when the user decide to disable this feauture i am implementing.
I will try to write better how the workflow of the app should be so you can have an more clear idea of what i am doing:
When the app starts, a functions sets the default value of some checkboxes of the window preference. The user can modify this checkboxes. So when the user open a pdf file, the idea is to launch in a background thread the following task:
I create a new queue call it A and launch asynch an infinite while where i check the value of the UserDefault checkboxe (that i use to reload and update the pdf file) and two things could happen
if the user set the value to off and the pdf document has been loaded there could be two situations:
if there is no current monitoring of the file (when the app starts): continue to check the checkboxe value
if there is currently a monitoring of the file: stop it
if the user set value to on and the pdf document has been loaded in this background thread (the same queue A) i will create a class Monitor (that could be a subclass of NSThread or a class that uses DispatchSourceFileSystemObject like above), then i will call startMonitoring() that will check the date or .write events and when there is a change it will call the handler. Basically this handler should recall the main thread (the main queue) and check if the file can be loaded or is corrupted and if so update the view.
Note: The infinite while loop(that should be running in the background), that check the UserDefault related to the feature i am implementing it's launched when the user open the pdf file.
Because of the problem above (multiple handlers calls), i should use the cancel() function when the user set checkboxe to off, and not create/destroy the DispatchSource object everytime i receive a .write event.

When are Realm notifications delivered to main thread after writes on a background thread?

I'm seeing crashes that either shouldn't be possible, or are very much possible and the documentation just isn't clear enough as to why.
UPDATE:
Although I disagree with the comment below asking me to separate this into multiple SO questions, if someone could focus on this one I think it would help greatly:
When are notifications delivered to the main thread? Is it possible that the results on the main thread are different than they were in a previous runloop without being notified yet of the difference?
If the answer to this question is yes the results could be different than a previous runloop without notifying then I would argue it is CRUCIAL to get this into the documentation somewhere.
Background Writes
First I think it's important to go over what I am already doing for writes. All of my writes are performed through a method that essentially looks like this (error handling aside):
func write(block: #escaping (Realm) -> ()) {
somePrivateBackgroundSerialQueue.async {
autoreleasepool {
let realm = try! Realm()
realm.refresh()
try? realm.write { block(realm) }
}
}
}
Nothing crazy here, pretty well documented on your end.
Notifications and Table Views
The main question I have here is when are notifications delivered to the main thread after being written from a background thread? I have a complex table view (multiple sections, ads every 5th row) backed by realm results. My main data source looks like:
enum StoryRow {
case story(Story) // Story is a RealmSwift.Object subclass
case ad(Int)
}
class StorySection {
let stories: Results<Story>
var numberOfRows: Int {
let count = stories.count
return count + numberOfAds(before: count)
}
func row(at index: Int) -> StoryRow {
if isAdRow(at: index) {
return .ad(index)
} else {
let storyIndex = index - numberOfAds(before: index)
return .story(stories[storyIndex])
}
}
}
var sections: [StorySection]
... sections[indexPath.section].row(at: indexPath.row) ...
Before building my sections array I fetch the realm results and filter them based on the type of stories for the particular screen, sort them so they are in the proper order for their sections, then I build up the sections by passing in results.filter(...date query...) to the section constructor. Finally, I results.observe(...) the main results object (not any of the results passed into the section) and reload the table view when the notification handler is called. I don't bother observing the results in the sections because if any of those results changed then the parent had to change as well and it should trigger a change notification.
The ad slots have callbacks when an ad is filled or not filled and when that happens instead of calling tableView.reloadData() I am doing something like:
guard tableView.indexPathsForVisibleRows?.contains(indexPath) == true else { return }
tableView.beginUpdates()
tableView.reloadRows(at: [indexPath], with: .automatic)
tableView.endUpdates()
The problem is, I very rarely see a crash either around an index being out of bound when accessing the realm results or an invalid table view update.
QUESTIONS
Is it possible the realm changed on the main thread before any notifications were delivered?
Should table view updates other than reloadData() simply not be used anywhere outside of a realm notification block?
Anything else crucial I am missing?
There's nothing in your code snippets or description of what you're doing that jumps out at me as obviously wrong. Having a separate callback mechanism that updates specific slots independent of Realm change notifications has a lot of potential for timing related bugs, but since you're explicitly checking if the indexPath is visible before reloading the row, I would expect that to at worst manifest as reloading the wrong row and not a crash.
The intended behavior is that refreshing the Realm and delivering notifications is an atomicish operation: anything that causes the read version to advance will deliver all notifications before returning. In simple cases, this means that you'll never see the new data without the associated notification firing first. However, there's some caveats to this:
Nested notification delivery doesn't work correctly, so beginning a write transaction from within a notification block can result in a notification being skipped (merely calling refresh() can't cause this, as it's just a no-op within a notification). If you're performing all writes on background threads you shouldn't be hitting this.
If you have multiple notification blocks, then obviously anything which gets invoked from the first one will run before the second notification block gets a chance to do things, and a call to tableView.reloadData() may result in quite a lot of things happening within the notification block. If this is the source of problems, you would hopefully see exceptions being thrown with a stack trace coming from within a notification block.

Semaphore is not waiting swift

I'm trying to do 3 async requests and control the load with semaphores to know when all have loaded.
I Init the semaphore in this way:
let sem = dispatch_semaphore_create(2);
Then send to background the waiting for semaphore code:
let backgroundQueue = dispatch_get_global_queue(QOS_CLASS_BACKGROUND, 0)
dispatch_async(backgroundQueue) { [unowned self] () -> Void in
println("Waiting for filters load")
dispatch_semaphore_wait(sem, DISPATCH_TIME_FOREVER);
println("Loaded")
}
Then I signal it 3 times (on each request onSuccess and onFailure):
dispatch_semaphore_signal(sem)
But when the signal code arrives it already passed the semaphore wait code, it never waits to subtract the semaphore count.
why?
You've specified dispatch_semaphore_create with a parameter of 2 (which is like calling dispatch_semaphore_signal twice), and then signal it three more times (for a total of five), but you appear to have only one wait (which won't wait at all because you started your semaphore with a count of 2).
That's obviously not going to work. Even if you fixed that (e.g. use zero for the creation of the semaphore and then issue three waits) this whole approach is inadvisable because you're unnecessarily tying up a thread waiting for the the other requests to finish.
This is a textbook candidate for dispatch groups. So you would generally use the following:
Create a dispatch_group_t:
dispatch_group_t group = dispatch_group_create();
Then do three dispatch_group_enter, once before each request.
In each of the three onSuccess/onFailure blocks pairs, do a dispatch_group_leave in both block.
Create a dispatch_group_notify block that will be performed when all of the requests are done.

Code with a potential deadlock

// down = acquire the resource
// up = release the resource
typedef int semaphore;
semaphore resource_1;
semaphore resource_2;
void process_A(void) {
down(&resource_1);
down(&resource_2);
use_both_resources();
up(&resource_2);
up(&resource_1);
}
void process_B(void) {
down(&resource_2);
down(&resource_1);
use_both_resources();
up(&resource_1);
up(&resource_2);
}
Why does this code causes deadlock?
If we change the code of process_B where the both processes ask for the resources in the same order as:
void process_B(void) {
down(&resource_1);
down(&resource_2);
use_both_resources();
up(&resource_2);
up(&resource_1);
}
Then there is no deadlock.
Why so?
Imagine that process A is running and try to get the resource_1 and gets it.
Now, process B takes control and try to get resource_2. And gets it. Now, process B tries to get resource_1 and does not get it, because it belongs to resource A. Then, process B goes to sleep.
Process A gets control again and try to get resource_2, but it belongs to process B. Now he goes to sleep too.
At this point, process A is waiting for resource_2 and process B is waiting for resource_1.
If you change the order, process B will never lock resource_2 unless it gets resource_1 first, the same for process A.
They will never be dead locked.
A necessary condition for a deadlock is a cycle of resource acquisitions. The first example constructs this a cycle 1->2->1. The second example acquires the resources in a fixed order which makes a cycle and henceforth a deadlock impossible.