SFSpeechURLRecognitionRequest Doesn't Output Results from Speech Recognition Until After Thread Calling It Completes - swift

I am trying to use SFSpeechURLRecognitionRequest to transcribe audio files in a terminal application. While I have it working in a preliminary form, I'm running into an odd issue. It appears to only output the voice recognition results (both partial and complete) after the main thread terminates in my test applications. Note I am a Swift noob, so I might be missing something obvious.
Below I have a complete Xcode Playground application which demonstrates the issue. The output from the playground writes Playground Execution Complete and then I begin receiving partial outputs followed by the final output. Note that if I add a sleep(5) prior to the print it will wait 5 seconds and then output the print, and only then after the main thread has concluded begin processing the text. I have seen similar behavior in a GUI test application, where it only begins processing the text after the method call kicking off the request completes.
I have tried repeatedly checking the state of the task that is returned, sleeping between each check with no luck.
I have also tried calling the recognition task inside a DispatchQueue, which appears to run successfully in the background based on CPU usage, but the Partial and Final prints never appear until the application completes, at which point the console fills up with Partials followed by the Final.
Does anyone know of a way to have the speech recognition begin processing without the application thread completing? Ideally I would like to be able to kick it off and sleep for brief periods repeatedly, checking if the recognition task has completed in between each.
Edited below to match version immediately prior to figuring out the solution.
import Speech
var complete = false
SFSpeechRecognizer.requestAuthorization {
authStatus in
DispatchQueue.main.async {
if authStatus == .authorized {
print("Good to go!")
} else {
print("Transcription permission was declined.")
exit(1)
}
}
}
guard let myRecognizer = SFSpeechRecognizer() else {
print("Recognizer not supported for current locale!")
exit(1)
}
if !myRecognizer.isAvailable {
// The recognizer is not available right now
print("Recognizer not available right now")
exit(1)
}
if !myRecognizer.supportsOnDeviceRecognition {
print("On device recognition not possible!")
exit(1)
}
let path_to_wav = NSURL.fileURL(withPath: "/tmp/harvard.wav", isDirectory: false)
let request = SFSpeechURLRecognitionRequest(url: path_to_wav)
request.requiresOnDeviceRecognition = true
print("About to create recognition task...")
myRecognizer.recognitionTask(with: request) {
(result, error) in
guard let result = result else {
// Recognition failed, so check error for details and handle it
print("Recognition failed!!!")
print(error!)
exit(1)
}
if result.isFinal {
print("Final: \(result.bestTranscription.formattedString)")
complete = true
} else {
print("Partial: \(result.bestTranscription.formattedString)")
}
}
print("Playground execution complete.")

I figured it out! sleep doesn't actually let background tasks execute. Instead by adding the following:
let runLoop = RunLoop.current
let distantFuture = NSDate.distantFuture as NSDate
while complete == false && runLoop.run(mode: RunLoop.Mode.default, before: distantFuture as Date) {}
to the end just before the last print works (results begin appearing immediately, and the final print prints right after the final results).

Related

Asynchronous DispatchSemaphore.wait(timeout:) Task handle alternative

I'm currently looking for a Task based alternative for my code using DispatchSemaphore:
func checkForEvents() -> Bool
let eventSemaphore = DispatchSemaphore(value: 0)
eventService.onEvent = { _ in
eventSemaphore.signal()
}
let result = eventSemaphore.wait(timeout: .now() + 10)
guard result == .timedOut else {
return true
}
return false
}
I'm trying to detect if any callback events happen in a 10-second time window. This event could happen multiple times, but I only want to know if it occurs once. While the code works fine, I had to convert the enclosing function to async which now shows this warning:
Instance method 'wait' is unavailable from asynchronous contexts; Await a Task handle instead; this is an error in Swift 6`
I'm pretty sure this warning is kind of wrong here since there's no suspension point in this context, but I would still like to know what's the suggested approach in Swift 6 then.
I couldn't find anything about this "await a Task handle" so the closest I've got is this:
func checkForEvents() async -> Bool {
var didEventOccur = false
eventService.onEvent = { _ in
didEventOccur = true
}
try? await Task.sleep(nanoseconds: 10 * 1_000_000_000)
return didEventOccur
}
But I can't figure out a way to stop sleeping early if an event occurs...

Is it possible to use the beginBackgroundTask() API within SwiftUI lifecycle?

I need to run some code when the app is closed to remove the client from a game. To do this I'm wanting to execute a Google Cloud Function for the server to do the cleanup - the function works, I guess similar to this question I just do not have enough time, and I'm running a completion handler so it's not like iOS thinks the function is finished straight away.
I have seen multiple questions on this, many of which are rather old and do not include answers for the SwiftUI Lifecycle. I have seen this exact issue and a potential answer here, however I'm not using the Realtime Database, I'm using Firestore so there is no equivalents for the onDisconnect methods.
I have seen that you can increase the time you need when the application finishes through beginBackgroundTask(expirationHandler:), I just can't find anywhere to state this can be done through SwiftUI Lifecycle, what I have so far:
.onReceive(NotificationCenter.default.publisher(for: UIApplication.willTerminateNotification), perform: { output in
Backend().removeFromGame(gameCode: otp, playerName: "name", completion: { res, error in
if error != nil{
print(error)
}
})
})
The function called is as follows:
func removeFromGame(gameCode: String, playerName: String, completion: #escaping (Bool?, Error?) -> Void){
Functions.functions().httpsCallable("removeFromGame").call(["gameCode": gameCode, "playerName": playerName]){ result, error in
if let error = error as NSError? {
if error.domain == FunctionsErrorDomain{
_ = FunctionsErrorCode(rawValue: error.code)
let errorDesc = error.localizedDescription
_ = error.userInfo[FunctionsErrorDetailsKey]
print(errorDesc)
}
}else{
print("Removed successfully")
}
}
}
I have seen in this Apple doc how to use the API:
func sendDataToServer( data : NSData ) {
// Perform the task on a background queue.
DispatchQueue.global().async {
// Request the task assertion and save the ID.
self.backgroundTaskID = UIApplication.shared.
beginBackgroundTask (withName: "Finish Network Tasks") {
// End the task if time expires.
UIApplication.shared.endBackgroundTask(self.backgroundTaskID!)
self.backgroundTaskID = UIBackgroundTaskInvalid
}
// Send the data synchronously.
self.sendAppDataToServer( data: data)
// End the task assertion.
UIApplication.shared.endBackgroundTask(self.backgroundTaskID!)
self.backgroundTaskID = UIBackgroundTaskInvalid
}
}
Just cannot seem to implement it correctly within the new way of getting these system notifications?

Why is a process suspended by another process behind it?

The code is in a simple way, only read and parse an xml file into an array. I did not notice the problem until one day I tried to open a big xml file.
I added a blur view with NSProgressIndicator when the data is parsing, but the blur view did not show up until the parsing was completed.
self.addBlurView()
let file = HandleFile.shared.openFile(filePath)
self.removeBlurView()
guard let name = file.name, let path = file.path, let data = file.data else {
return
}
So I tried to delay parsing data. The blur view can be showed up, and removed when completed.
self.addBlurView()
DispatchQueue.main.asyncAfter(deadline: .now() + 0.3, execute: {
let file = HandleFile.shared.openFile(filePath)
self.removeBlurView()
guard let name = file.name, let path = file.path, let data = file.data else {
return
}
})
I thought it might be a problem fo thread, so I tried this in func addBlurView(), failed. I also tried to add an counting in addBlurView(), it counted to a certain number and paused, and continue counting after parsing data.
DispatchQueue.main.async {
self.blurView.isHidden = false
self.spinner.startAnimation(self)
}
Have no idea why this happen. Can anyone help to solve this problem?
Thanks.
As I mentioned in the comments above, main queue is a serial queue and all the tasks assigned to it are executed serially by main thread. In general, You should not perform any heavy lifting task (like loading file to memory) on main thread as it would block the main thread and render UI unresponsive.
Typically all the heavy lifting tasks like loading a file to a memory (anything which does not deal with UI rendering directly) should be delegated to one of dispatch queues. Try wrapping your openFile(filePath) call inside DispatchQueue
self.addBlurView()
DispatchQueue.global(qos: .default).async {
let file = HandleFile.shared.openFile(filePath)
}
Personally I would expect openFile function to have a completion block which is triggered on main queue when it finished loading file so that you can remove your blurView, but in your case it seems like its a synchronous statement so you can try
self.addBlurView()
DispatchQueue.global(qos: .default).async {
let file = HandleFile.shared.openFile(filePath)
DispatchQueue.main.async {
self.removeBlurView()
}
}

Swift: iBeacon Run a Code Before iOS Ends the Process

I implemented iBeacon recognition in my project, everything works fine. When the application connects to the BLE device that has IBeacon, then I launch a whole process that sends data to a server.
I'm looking for a way to know when iOS kill the application after 10 seconds to perform one last action.
I'm looking at the functions in the CLLocationManagerDelegate protocol, but I can not find a function that might look like willDisconnectApplication
How can I know a second before iOS kill my application to run a code?
You can use a code snippet like below to track how much background time you have available.
Just by starting the background task shown, iOS will also give you extra background running time, so you get 180 seconds instead of 10 seconds. After doing this, you also get to track when that time is about to expire by looking at UIApplication.shared.backgroundTimeRemaining.
private var backgroundTask: UIBackgroundTaskIdentifier = UIBackgroundTaskInvalid
func extendBackgroundRunningTime() {
if (self.backgroundTask != UIBackgroundTaskInvalid) {
// if we are in here, that means the background task is already running.
// don't restart it.
return
}
self.backgroundTask = UIApplication.shared.beginBackgroundTask(withName: "DummyTask", expirationHandler: {
NSLog("Background running expired by iOS. Cannot detect beacons again until a new region event")
UIApplication.shared.endBackgroundTask(self.backgroundTask)
self.backgroundTask = UIBackgroundTaskInvalid
})
if threadStarted {
NSLog("Background task thread already started.")
}
else {
threadStarted = true
DispatchQueue.global().async {
let startedTime = Int(Date().timeIntervalSince1970) % 10000000
NSLog("Background task thread started")
while (true) {
let backgroundTimeRemaining = UIApplication.shared.backgroundTimeRemaining;
if (backgroundTimeRemaining < 200.0) {
if (backgroundTimeRemaining.truncatingRemainder(dividingBy: 30) < 1) {
NSLog("Thread \(startedTime) background time remaining: \(backgroundTimeRemaining)")
}
else {
NSLog("Thread \(startedTime) background time remaining: \(backgroundTimeRemaining)")
}
}
Thread.sleep(forTimeInterval: 1);
}
}
}
}

How to exit a runloop?

So, I have a Swift command-line program:
import Foundation
print("start")
startAsyncNetworkingStuff()
RunLoop.current.run()
print("end")
The code compiles without error. The async networking code runs just fine, fetches all its data, prints the result, and eventually calls its completion function.
How do I get that completion function to break out of above current runloop so that the last "end" gets printed?
Added:
Replacing RunLoop.current.run() with the following:
print("start")
var shouldKeepRunning = true
startAsyncNetworkingStuff()
let runLoop = RunLoop.current
while ( shouldKeepRunning
&& runLoop.run(mode: .defaultRunLoopMode,
before: .distantFuture ) ) {
}
print("end")
Setting
shouldKeepRunning = false
in the async network completion function still does not result in "end" getting printed. (This was checked by bracketing the shouldKeepRunning = false statement with print statements which actually do print to console). What is missing?
For a command line interface use this pattern and add a completion handler to your AsyncNetworkingStuff (thanks to Rob for code improvement):
print("start")
let runLoop = CFRunLoopGetCurrent()
startAsyncNetworkingStuff() { result in
CFRunLoopStop(runLoop)
}
CFRunLoopRun()
print("end")
exit(EXIT_SUCCESS)
Please don't use ugly while loops.
Update:
In Swift 5.5+ with async/await it has become much more comfortable. There's no need anymore to maintain the run loop.
Rename the file main.swift as something else and use the #main attribute like in a normal application.
#main
struct CLI {
static func main() async throws {
let result = await startAsyncNetworkingStuff()
// do something with result
}
}
The name of the struct is arbitrary, the static function main is mandatory and is the entry point.
Here's how to use URLSession in a macOS Command Line Tool using Swift 4.2
// Mac Command Line Tool with Async Wait and Exit
import Cocoa
// Store a reference to the current run loop
let runLoop = CFRunLoopGetCurrent()
// Create a background task to load a webpage
let url = URL(string: "http://SuperEasyApps.com")!
let task = URLSession.shared.dataTask(with: url) { (data, response, error) in
if let data = data {
print("Loaded: \(data) bytes")
}
// Stop the saved run loop
CFRunLoopStop(runLoop)
}
task.resume()
// Start run loop after work has been started
print("start")
CFRunLoopRun()
print("end") // End will print after the run loop is stopped
// If your command line tool finished return success,
// otherwise return EXIT_FAILURE
exit(EXIT_SUCCESS)
You'll have to call the stop function using a reference to the run loop before you started (as shown above), or using GCD in order to exit as you'd expect.
func stopRunLoop() {
DispatchQueue.main.async {
CFRunLoopStop(CFRunLoopGetCurrent())
}
}
References
https://developer.apple.com/documentation/corefoundation/1542011-cfrunlooprun
Run loops can be run recursively. You can call CFRunLoopRun() from within any run loop callout and create nested run loop activations on the current thread’s call stack.
https://developer.apple.com/documentation/corefoundation/1541796-cfrunloopstop
If the run loop is nested with a callout from one activation starting another activation running, only the innermost activation is exited.
(Answering my own question)
Adding the following snippet to my async network completion code allows "end" to be printed :
DispatchQueue.main.async {
shouldKeepRunning = false
}