We have hanging problem (the app freezes due of main thread lock) with our iOS (swift) native app with OpenVidu implementation (which uses GoogleWebRTC under the hood). The specific conditions required: need to join existing room with at least 8 participants already streaming. With 6 participants it happens less often and almost never with less than 6. It doesn't hang if participants join one by one, only if you join the room with all other participants already streaming. This indicates concurrent nature of the issue.
The GoogleWebRTC hangs on setRemoteDescription call:
func setRemoteDescription(sdpAnswer: String) {
let sessionDescription: RTCSessionDescription = RTCSessionDescription(type: RTCSdpType.answer, sdp: sdpAnswer)
self.peerConnection!.setRemoteDescription(sessionDescription, completionHandler: {(error) in
print("Local Peer Remote Description set: " + error.debugDescription)
})
}
As you can see on the screenshot above, the main thread hangs on __psynch_cvwait. No any other threads seems being locked. The lock never releases leaving the app completely frozen.
In the attempt to solve it I was trying the following:
I moved OpenVidu signaling server processing (RPC protocol) from the main thread into separate threads. This only caused the lock now occurs in the one of separate threads I created. It now doesn't block the UI, but blocks OV signaling. The problem persists.
I added the lock to process each signaling event (participant join event, publish video, etc) synchronously (one by one). This doesn't help either (it actually made the situation worse).
Instead of using GoogleWebRTC v. 1.1.31999 from Cocoapods, I downloaded the latest GoogleWebRTC sources, built them in release configuration and included into my project. This didn't help to solve the issue.
Any suggestions/comments would be appreciated.
Thanks!
EDIT 1:
The signaling_thread and worker_thread are both is waiting for something in the same kind of lock. Nothing of them execute any of my code at the moment of the lock.
I also tried to run in DEBUG build of GoogleWebRTC, in this case no locks happen, but everything works much slower (which is OK for debug, but we can't use this in Production).
EDIT 2:
I tried to wrap in additional DispatchQueue for offer and setLocalDescription callbacks, but this changes nothing. The problem still well reproducible (almost 100% of time, if I have 8 participants with streams):
self.peerConnection!.offer(for: constrains) { (sdp, error) in
DispatchQueue.global(qos: .background).async {
guard let sdp = sdp else {
return
}
self.peerConnection!.setLocalDescription(sdp, completionHandler: { (error) in
DispatchQueue.global(qos: .background).async {
completion(sdp)
}
})
}
}
The WebRTC Obj-C API can be called from any thread, but most method calls are passed to WebRTC's internal thread called signalling thread.
Also, callbacks/observers like SetLocalDescriptionObserverInterface or RTCSetSessionDescriptionCompletionHandler are called from WebRTC on the signaling thread.
Looking at the screenshots, it seems that the signaling thread is currently blocked and can no longer call WebRTC API calls.
So, to avoid deadlocks, it's a good idea to create your own thread / dispatch_queue and handle callbacks.
See
https://webrtc.googlesource.com/src/+/0a52ede821ba12ee6fff6260d69cddcca5b86a4e/api/g3doc/index.md and
https://webrtc.googlesource.com/src/+/0a52ede821ba12ee6fff6260d69cddcca5b86a4e/api/g3doc/threading_design.md
for details.
After the comment from OpenVidu team, the problem was solved by adding 100ms delay between adding participants who are already in the room. I would consider this more like a hack than a real solution, but I can confirm that it works both in test and in Production environment:
DispatchQueue.global(qos: .background).async {
for info in dict.values {
let remoteParticipant = self.newRemoteParticipant(info: info)
if let streamId = info.streamId {
remoteParticipant.createOffer(completion: {(sdp) in
self.receiveVideoFrom(sdp: sdp, remoteParticipant: remoteParticipant, streamId: streamId)
})
} else {
print("No streamId")
}
Thread.sleep(forTimeInterval: 0.1)
}
}
Related
I have been using the following code in my iOS Swift app:
class ProfileController
{
func remove(pid: String, completion: #escaping ErrorCompletionHandler)
{
guard let uid = self.uid else
{
completion(Errors.userIdentifierEmpty)
return
}
let db = Firestore.firestore()
let userDocument = db.collection("profiles").document(uid)
let collection = userDocument.collection("profiles")
let document = collection.document(pid)
document.delete()
{
error in
completion(error)
}
}
}
When the device is online, everything works fine. The completion handler of the deletecall is properly executed. However, when I am offline, I have noticed that the completion handler will not be executed as long as I am offline. As soon as I get back online, the completion handler will be called almost immediately.
I don't want to wait until the user is back online (which could take forever), so I changed the code a little bit and added a ListenerRegistration:
class ProfileController
{
func remove(pid: String, completion: #escaping ErrorCompletionHandler)
{
guard let uid = self.uid else
{
completion(Errors.userIdentifierEmpty)
return
}
let db = Firestore.firestore()
let userDocument = db.collection("profiles").document(uid)
let collection = userDocument.collection("profiles")
let document = collection.document(pid)
var listener: ListenerRegistration?
listener = document.addSnapshotListener(includeMetadataChanges: false)
{
snapshot, error in
listener?.remove() // Immediately remove the snapshot listener so we only receive one notification.
completion(error)
listener = nil
}
document.delete()
}
}
Although this works fine, I am not sure if this is the right way to go. I have read online that a snapshot listener can be used in real-time scenarios, which is not really what I am looking for (or what I need).
Is this the right approach or is there another (better) way? I only want to get notified once (thus I added the includeMetadataChanged property and set it to false). I also remove the ListenerRegistration once the completion handler was called once.
If the first approach does not work properly when being offline - what are the use cases of this approach? Before I change my entire codebase to use listeners, is there any way of executing the completion handler of the first approach when the device is offline?
TL;DR: The second implementation works fine, I am simply unsure if this is the proper way of receiving notifications when the device is offline.
If the first approach does not work properly when being offline - what are the use cases of this approach?
It depends on what you mean by "work properly". The behavior you're observing is exactly as intended. Write operations are not "complete" until they're registered at the server.
However, all writes (that are not transactions) are actually committed locally before they hit the server. These local changes will eventually be synchronized with the server at some point in the future, even if the app is killed and restarted. The change is not lost. You can count on the synchronization of the change to eventually hit the server as long as the user continues to launch the app - this is all you can expect. You can read more about offline persistence in the documentation.
If you need to know if a prior change was synchronized, there is no easy way to determine that if the app was killed and restarted. You could try to query for that data, if you know the IDs of the documents written, and you could check the metadata of the documents to find out the source of the data (cache or server). But in the end, you are really supposed to trust that changes will be synchronized with the server at the earliest convenience.
If your use case requires more granularity of information, please file a feature request with Firebase support.
I have a piece of code that spams long running tasks 5-6 times per second. Each task takes some time to finish. I want to ignore all the other tasks while 1 is being executed. After it finishes a fresh one should take its place.
There are a bunch of tools being used for concurrency in Swift 4.2. What would work the best?
For solving of this problem you can use GCD or Operation. In case that you have describ I would use Operation. Using this approach you can have a bit more user friendly control over Operation that are executing (stoping, cancelling....).
Small example:
let queue = OperationQueue()
queue.maxConcurrentOperationCount = 1
queue.addOperation { print("🤠") }
queue.addOperation { print("🤓") }
queue.addOperation { print("👺") }
In this case operations are executed one by one.
I want to check if a pdf file is changed or not, and if is changed i want to update the corresponding view. I don't know if it's more suitable to use a background process as a Thread or as an NSOperation to do this task. The Apple Documentation says: "Examples of tasks that lend themselves well to NSOperation include network requests, image resizing, text processing, or any other repeatable, structured, long-running task that produces associated state or data.But simply wrapping computation into an object doesn’t do much without a little oversight".
Also, if I understood correctly from the documentation, a Thread once started can't be stopped during his execution while an NSOperation could be paused or stopped and also they could rely on dependency to wait the completion of another task.
The workflow of this task should be more or less this diagram:
Task workflow
I managed to get the handler working after the notification of type .write has been sent. If i monitor for example a *.txt file everything works as expected and i receive only one notification. But i am monitoring a pdf file which is generated from terminal by pdflatex and thus i receive with '.write' nearly 15 notification. If i change to '.attrib' i get 3 notification. I need the handler to be called only once, not 15 or 3 times. Do you have any idea how can i do it or is not possible with a Dispatch Source? Maybe there is a way to execute a dispatchWorkItem only once?
I have tried to implement it like this(This is inside a FileMonitor class):
func startMonitoring()
{
....
let fileSystemRepresentation = fileManager.fileSystemRepresentation(withPath: fileStringURL)
let fileDescriptor = open(fileSystemRepresentation, O_EVTONLY)
let newfileMonitorSource = DispatchSource.makeFileSystemObjectSource(fileDescriptor: fileDescriptor,
eventMask: .attrib,
queue: queue)
newfileMonitorSource.setEventHandler(handler:
{
self.queue.async
{
print(" \n received first write event, removing handler..." )
self.newfileMonitorSource.setEventHandler(handler: nil)
self.test()
}
})
self.fileMonitorSource = newfileMonitorSource
fileMonitorSource!.resume()
}
func test()
{
fileMonitorSource?.cancel()
print(" restart monitoring ")
startMonitoring()
}
I have tried to reassign the handler in test(), but it's not working(if a regenerate the pdf file, what is inside the new handler it's not executed) and to me, doing in this way, it seems a bit boilerplate code. I have also tried the following things:
suspend the DispatchSource in the setEventHandler of startMonitoring() (passing nil), but then when i am resuming it, i get the remaining .write events.
cancel the DispatchSource object and recall the startMonitoring() as you can see in the code above, but in this way i create and destroy the DispatchSource object everytime i receive an event, which i don't like because the cancel() function shoul be called in my case only when the user decide to disable this feauture i am implementing.
I will try to write better how the workflow of the app should be so you can have an more clear idea of what i am doing:
When the app starts, a functions sets the default value of some checkboxes of the window preference. The user can modify this checkboxes. So when the user open a pdf file, the idea is to launch in a background thread the following task:
I create a new queue call it A and launch asynch an infinite while where i check the value of the UserDefault checkboxe (that i use to reload and update the pdf file) and two things could happen
if the user set the value to off and the pdf document has been loaded there could be two situations:
if there is no current monitoring of the file (when the app starts): continue to check the checkboxe value
if there is currently a monitoring of the file: stop it
if the user set value to on and the pdf document has been loaded in this background thread (the same queue A) i will create a class Monitor (that could be a subclass of NSThread or a class that uses DispatchSourceFileSystemObject like above), then i will call startMonitoring() that will check the date or .write events and when there is a change it will call the handler. Basically this handler should recall the main thread (the main queue) and check if the file can be loaded or is corrupted and if so update the view.
Note: The infinite while loop(that should be running in the background), that check the UserDefault related to the feature i am implementing it's launched when the user open the pdf file.
Because of the problem above (multiple handlers calls), i should use the cancel() function when the user set checkboxe to off, and not create/destroy the DispatchSource object everytime i receive a .write event.
I'm seeing crashes that either shouldn't be possible, or are very much possible and the documentation just isn't clear enough as to why.
UPDATE:
Although I disagree with the comment below asking me to separate this into multiple SO questions, if someone could focus on this one I think it would help greatly:
When are notifications delivered to the main thread? Is it possible that the results on the main thread are different than they were in a previous runloop without being notified yet of the difference?
If the answer to this question is yes the results could be different than a previous runloop without notifying then I would argue it is CRUCIAL to get this into the documentation somewhere.
Background Writes
First I think it's important to go over what I am already doing for writes. All of my writes are performed through a method that essentially looks like this (error handling aside):
func write(block: #escaping (Realm) -> ()) {
somePrivateBackgroundSerialQueue.async {
autoreleasepool {
let realm = try! Realm()
realm.refresh()
try? realm.write { block(realm) }
}
}
}
Nothing crazy here, pretty well documented on your end.
Notifications and Table Views
The main question I have here is when are notifications delivered to the main thread after being written from a background thread? I have a complex table view (multiple sections, ads every 5th row) backed by realm results. My main data source looks like:
enum StoryRow {
case story(Story) // Story is a RealmSwift.Object subclass
case ad(Int)
}
class StorySection {
let stories: Results<Story>
var numberOfRows: Int {
let count = stories.count
return count + numberOfAds(before: count)
}
func row(at index: Int) -> StoryRow {
if isAdRow(at: index) {
return .ad(index)
} else {
let storyIndex = index - numberOfAds(before: index)
return .story(stories[storyIndex])
}
}
}
var sections: [StorySection]
... sections[indexPath.section].row(at: indexPath.row) ...
Before building my sections array I fetch the realm results and filter them based on the type of stories for the particular screen, sort them so they are in the proper order for their sections, then I build up the sections by passing in results.filter(...date query...) to the section constructor. Finally, I results.observe(...) the main results object (not any of the results passed into the section) and reload the table view when the notification handler is called. I don't bother observing the results in the sections because if any of those results changed then the parent had to change as well and it should trigger a change notification.
The ad slots have callbacks when an ad is filled or not filled and when that happens instead of calling tableView.reloadData() I am doing something like:
guard tableView.indexPathsForVisibleRows?.contains(indexPath) == true else { return }
tableView.beginUpdates()
tableView.reloadRows(at: [indexPath], with: .automatic)
tableView.endUpdates()
The problem is, I very rarely see a crash either around an index being out of bound when accessing the realm results or an invalid table view update.
QUESTIONS
Is it possible the realm changed on the main thread before any notifications were delivered?
Should table view updates other than reloadData() simply not be used anywhere outside of a realm notification block?
Anything else crucial I am missing?
There's nothing in your code snippets or description of what you're doing that jumps out at me as obviously wrong. Having a separate callback mechanism that updates specific slots independent of Realm change notifications has a lot of potential for timing related bugs, but since you're explicitly checking if the indexPath is visible before reloading the row, I would expect that to at worst manifest as reloading the wrong row and not a crash.
The intended behavior is that refreshing the Realm and delivering notifications is an atomicish operation: anything that causes the read version to advance will deliver all notifications before returning. In simple cases, this means that you'll never see the new data without the associated notification firing first. However, there's some caveats to this:
Nested notification delivery doesn't work correctly, so beginning a write transaction from within a notification block can result in a notification being skipped (merely calling refresh() can't cause this, as it's just a no-op within a notification). If you're performing all writes on background threads you shouldn't be hitting this.
If you have multiple notification blocks, then obviously anything which gets invoked from the first one will run before the second notification block gets a chance to do things, and a call to tableView.reloadData() may result in quite a lot of things happening within the notification block. If this is the source of problems, you would hopefully see exceptions being thrown with a stack trace coming from within a notification block.
On start my app I do some http requests, some heavy http requests (downloading some images) and some heavy tasks with UIGraphics (for example doing icon for GMSMarker from two UIImages and other operations with GraphicsContext). It costs some time, so I want to do all that tasks simultaneously. Can you show me best way make it?
On start I have to:
Download and write to local database all devices
Download and write to local database all geofences
Download and write to local database all users
Download and write to local database all positions
Download images for devices, users and geofences
Setup GMSMarkers for devices, users and geofences (after images for that objects will be available - for setting icon of marker)
Code of my login function (it works, but too slow):
func loginPressed(_ sender: UIButton) {
guard
let username = self.usernameTextField.text,
let password = self.passwordTextField.text,
!username.isEmpty,
!password.isEmpty
else {
return
}
self.loginButton.isEnabled = false
self.activityIndicator.startAnimating()
WebService.shared.connect(email: username, password: password) { error, loggedUser in
guard
error == nil,
let loggedUser = loggedUser
else {
self.showAlert(title: "Ошибка подключения", message: error?.localizedDescription ?? "", style: .alert)
self.activityIndicator.stopAnimating()
self.loginButton.isEnabled = true
return
}
DB.users.client.insert(loggedUser)
print("Start loading user photo...")
loggedUser.loadPhoto() { image in
if let image = image {
loggedUser.photo = UIImageJPEGRepresentation(image, 0.0)
}
print("User photo loaded...")
loggedUser.marker = UserMarker(loggedUser, at: CLLocation(latitude: 48.7193900, longitude: 44.50183))
DB.users.client.modify(loggedUser)
}
DB.geofences.server.getAll() { geofences in
DB.devices.server.getAll() { devices in
DB.positions.server.getAll() { positions in
for device in devices {
device.loadPhoto() { image in
if let image = image {
device.photo = UIImageJPEGRepresentation(image, 0.0)
}
if let position = positions.findById(device.positionId) {
device.marker = DeviceMarker(device, at: position)
}
device.attributes.battery = device.lastKnownBattery(in: positions)
}
}
geofences.forEach({$0.marker = GeofenceMarker($0)})
DB.geofences.client.updateAddress(geofences) { geofences in
if DEBUG_LOGS {
print("Geofences with updated addresses: ")
geofences.forEach({print("\($0.name), \($0.address ?? "")")})
}
DB.devices.client.insert(devices)
DB.geofences.client.insert(geofences)
DB.positions.client.insert(positions)
self.activityIndicator.stopAnimating()
WebService.shared.addObserver(DefaultObserver.shared)
self.performSegue(withIdentifier: "toMapController", sender: self)
}
}
}
}
}
}
Not sure it's good idea post here code snippets of all classes and objects, hope you'll get idea.
Any help would be appreciated.
P.S. In case you wonder what is DB, it's database, which consists of two parts - server side and client side for each group of objects, so first task is get all objects from server and write them in memory (in client database)
P.S. I've changed logic from "download everything" on login to "download all I need right now and download rest later". So now after I've got all devices, geofences and positions I'm performing segue to MapController, on which I show all those objects. Just after login I'm showing deviceMarkers (GMSMarker) with default iconView. SO question is - can I after show map with all objects start download photos of devices in background and refresh markers with that photos after that (in main thread of course)?
Network requests are asynchronous by default, so what you are asking for has already the intended behavior.
You can, however, make your life much more simple by using a Promises library such as then.
An example usage might look like:
login(email: "foo#bar.com", password: "pa$$w0rd")
.whenAll(syncDevices(), syncGeofences(), syncUsers(), syncPositions(), syncImages())
.onError { err in
// process error
}
.finally {
// update UI
}
Your login function is slow because you are downloading AND writing to disk (as you mentioned in your question) "all" the data in your closures (geofences, devices and/or positions). Furthermore, all your operations are being executed in the main thread. You should never do I/O (networking, writing to disk) in the main thread, as this thread is used mainly for UI updates. You should offload the expensive tasks to another thread using GCD.
Also, it is worth mentioning that writing to disk is a relatively slow operation, especially if you are doing it for EVERY item you are downloading.
I would recommend you to JUST download any data to be displayed and then use an async task in a DispatchQueue (GCD) to persist the data downloaded to disk AFTER you've displayed the data in your UI.
I am not sure what the DB.geofences.server.getAll() lines do for geofences, devices and positions (in regards to how you handle your networking or database fetching), so I cannot advice you on that. What I can advice you on, is to structure your code the following way:
When user logs in, do the validation against the DB (remote) and guard against a valid login. Transition to your next view controller next (don't execute all your logic), since I can see that you are delegating way too much responsibility to your login action (for your login button).
From that second view controller, get your data through networking calls asynchronously on another thread using a .UserInitiated priority (to get results fast).
After doing all your networking operations, call DispatchQueue.main.async { ... } to update your UI Asynchronously in the main thread with the data you just got.
AFTER you've displayed the downloaded data, you can persist it to your local DB, ideally using another DispatchQueue async task.
If anything I said above does not make any sense to you, please read AppCoda's article about GCD here and RayWenderlich's GCD article here. They will give you the basic knowledge about GCD in iOS. Once you've done that, come back and try to structure your code the way I recommended above.
I hope this helps!