I am trying to use SFSpeechURLRecognitionRequest to transcribe audio files in a terminal application. While I have it working in a preliminary form, I'm running into an odd issue. It appears to only output the voice recognition results (both partial and complete) after the main thread terminates in my test applications. Note I am a Swift noob, so I might be missing something obvious.
Below I have a complete Xcode Playground application which demonstrates the issue. The output from the playground writes Playground Execution Complete and then I begin receiving partial outputs followed by the final output. Note that if I add a sleep(5) prior to the print it will wait 5 seconds and then output the print, and only then after the main thread has concluded begin processing the text. I have seen similar behavior in a GUI test application, where it only begins processing the text after the method call kicking off the request completes.
I have tried repeatedly checking the state of the task that is returned, sleeping between each check with no luck.
I have also tried calling the recognition task inside a DispatchQueue, which appears to run successfully in the background based on CPU usage, but the Partial and Final prints never appear until the application completes, at which point the console fills up with Partials followed by the Final.
Does anyone know of a way to have the speech recognition begin processing without the application thread completing? Ideally I would like to be able to kick it off and sleep for brief periods repeatedly, checking if the recognition task has completed in between each.
Edited below to match version immediately prior to figuring out the solution.
import Speech
var complete = false
SFSpeechRecognizer.requestAuthorization {
authStatus in
DispatchQueue.main.async {
if authStatus == .authorized {
print("Good to go!")
} else {
print("Transcription permission was declined.")
exit(1)
}
}
}
guard let myRecognizer = SFSpeechRecognizer() else {
print("Recognizer not supported for current locale!")
exit(1)
}
if !myRecognizer.isAvailable {
// The recognizer is not available right now
print("Recognizer not available right now")
exit(1)
}
if !myRecognizer.supportsOnDeviceRecognition {
print("On device recognition not possible!")
exit(1)
}
let path_to_wav = NSURL.fileURL(withPath: "/tmp/harvard.wav", isDirectory: false)
let request = SFSpeechURLRecognitionRequest(url: path_to_wav)
request.requiresOnDeviceRecognition = true
print("About to create recognition task...")
myRecognizer.recognitionTask(with: request) {
(result, error) in
guard let result = result else {
// Recognition failed, so check error for details and handle it
print("Recognition failed!!!")
print(error!)
exit(1)
}
if result.isFinal {
print("Final: \(result.bestTranscription.formattedString)")
complete = true
} else {
print("Partial: \(result.bestTranscription.formattedString)")
}
}
print("Playground execution complete.")
I figured it out! sleep doesn't actually let background tasks execute. Instead by adding the following:
let runLoop = RunLoop.current
let distantFuture = NSDate.distantFuture as NSDate
while complete == false && runLoop.run(mode: RunLoop.Mode.default, before: distantFuture as Date) {}
to the end just before the last print works (results begin appearing immediately, and the final print prints right after the final results).
I'm using AudioKit to run an AKSequencer() that plays both mp3 and wav files using AKMIDISampler(). Everything works great, except in cases when the app has entered background state for more than 30+ min, and then brought back up again for use. It seems to then lose all of it's audio connections and plays the "missing file" sinewave tone mentioned in other threads. The app can happily can enter background momentarily, user can quit, etc without the tone. It seems to only happen when left in background for long periods of time and then brought up again.
I've tried changing the order of AudioKit.start() and file loading, but nothing seems to completely eliminate the issue.
My current workaround is simply to prevent the user's display from timing out, however that does not address many use-cases of the issue occurring.
Is there a way to handle whatever error I'm setting up that creates this tone? Here is a representative example of what I'm doing with ~40 audio files.
//viewController
override func viewDidLoad() {
sequencer.setupSequencer()
}
class SamplerWav {
let audioWav = AKMIDISampler()
func loadWavFile() {
try? audioWav.loadWav("some_wav_audio_file")
}
class SamplerMp3 {
let audioMp3 = AKMIDISampler()
let audioMp3_akAudioFile = try! AKAudioFile(readFileName: "some_other_audio_file.mp3")
func loadMp3File() {
try? audioMp3.loadAudioFile(audioMp3_akAudioFile)
}
class Sequencer {
let mixer = AKMixer()
let subMix = AKMixer()
let samplerWav = SamplerWav()
let samplerMp3 = SamplerMp3()
var callbackTrack: AKMusicTrack!
let callbackInstr = AKMIDICallbackInstrument()
func setupSequencer{
AudioKit.output = mixer.mixer
try! AudioKit.start()
callbackTrack = sequencer.newTrack()
callbackTrack?.setMIDIOutput(callbackInstr.midiIn)
samplerWav.loadWavFile()
samplerMp3.loadMp3File()
samplerWav.audioWav >>> subMix
samplerMp3.audioMp3 >>> submix
submix >>> mixer
}
//Typically run from a callback track
func playbackSomeSound(){
try? samplerWav.audioWav.play(noteNumber: 60, velocity: 100, channel: 1)
}
}
Thanks! I'm a big fan of AudioKit.
After some trial and error, here's a workflow that seems to address the issue for my circumstance:
-create my callback track(s) -once- from viewDidLoad
-stop AudioKit, and call .detach() on all my AKMIDISampler tracks and any routing in willResignActive
-start AudioKit (again), and reload and reroute all of the audio files/tracks from didBecomeActive
I'm trying to understand how to correctly use TestScheduler from RxTest 4.2 for RxSwift 4.2.
How do I correctly create TestScheduler and pass it events which would terminate the scheduler after all events are executed?
The code below goes into infinite loop upon starting the test scheduler. Because it keeps looping synchronously, I have no opportunity to stop the scheduler from the test
var testScheduler: TestScheduler = TestScheduler(initialClock: 0)
let stateObserver = testScheduler.createObserver(Bool.self)
subject.outputRelay.bind(to: stateObserver).disposed(by: bag)
let events = [next(1,true), next(10,false)]
let hotObservable = testScheduler.createHotObservable(events)
//direct events to the given relay
hotObservable.bind(to: subject.inputRelay).disposed(by: bag)
testScheduler.start() //infinite loop
//do not get called
XCTAssertEqual(stateObserver.events, [next(0, InterfaceState.started)])
testScheduler.stop()
I've narrowed the issued down to this method call within TestScheduler. It seems there's something wrong with the subscription disposal
func findNext() -> VirtualSchedulerItem<VirtualTime>? {
while let front = _schedulerQueue.peek() {
if front.isDisposed {
//this never gets called
_schedulerQueue.remove(front)
continue
}
return front
}
return nil
}
What I do in this cases is schedule a block in specific time that I know all test should have finish and there stop the subscription. I think in your case should something like this:
//...
let hotObservable = testScheduler.createHotObservable(events)
let disposable = hotObservable.bind(to: subject.inputRelay)
testScheduler.scheduleAt(1) {
disposable.dispose()
}
testScheduler.start()
XCTAssertEqual(stateObserver.events, [next(0, InterfaceState.started)])
//...
I want to stop a the user's Macbook from automatically sleeping while one the app is running. How would I do that? I have seen this
https://developer.apple.com/library/content/qa/qa1340/_index.html
and this:
Disable sleep mode in OS X with swift
and this:
Using Swift to disable sleep/screen saver for OSX
But there is no simple line of code that I can put or a delegate method that makes this simple like there is in iOS? I just want to have the entire app work properly and stop the computer from automatically going to sleep. If someone puts in it to sleep manually, then obviously that is fine. I don't want to prevent the user from putting the computer to sleep. But as long as my app is running I don't want the computer to sleep. Like when you are watching a movie, the app does not go to sleep. How do I do that?
I feel like this is the code that I need but do I just run this function in the viewdidload and it will work? Is it that simple?
var assertionID: IOPMAssertionID = 0
var success: IOReturn?
func disableScreenSleep(reason: String = "Unknown reason") -> Bool? {
guard success != nil else { return nil }
success = IOPMAssertionCreateWithName( kIOPMAssertionTypeNoDisplaySleep as CFString,
IOPMAssertionLevel(kIOPMAssertionLevelOn),
reason as CFString,
&assertionID )
return success == kIOReturnSuccess
}
As I already mentioned in comments you need to remove the guard otherwise it will simply return nil and will never disable the screen sleep. You can also simplify your method, return void and change your success property to a Bool property like disabled to monitor the screen sleep state:
var assertionID: IOPMAssertionID = 0
var sleepDisabled = false
func disableScreenSleep(reason: String = "Disabling Screen Sleep") {
if !sleepDisabled {
sleepDisabled = IOPMAssertionCreateWithName(kIOPMAssertionTypeNoDisplaySleep as CFString, IOPMAssertionLevel(kIOPMAssertionLevelOn), reason as CFString, &assertionID) == kIOReturnSuccess
}
}
func enableScreenSleep() {
if sleepDisabled {
IOPMAssertionRelease(assertionID)
sleepDisabled = false
}
}
To disable the screen simply just call the method inside viewDidLoad of your first view controller:
override func viewDidLoad() {
super.viewDidLoad()
disableScreenSleep()
print("sleep Disabled:", sleepDisabled)
}
How do I keep my application running in the background?
Would I have to jailbreak my iPhone to do this? I just need this app to check something from the internet every set interval and notify when needed, for my own use.
Yes, no need to jailbreak. Check out the "Implementing long-running background tasks" section of this doc from Apple.
From Apple's doc:
Declaring Your App’s Supported Background Tasks
Support for some types of background execution must be declared in advance by the app that uses them. An app declares support for a service using its Info.plist file. Add the UIBackgroundModes key to your Info.plist file and set its value to an array containing one or more of the following strings: (see Apple's doc from link mentioned above.)
I guess this is what you required
When an iOS application goes to the background, are lengthy tasks paused?
iOS Application Background Downloading
This might help you ...
Enjoy Coding :)
Use local notifications to do that. But this will not check every time. You will have to set a time where you will check your specific event, you may shorten this by decreasing your time slot. Read more about local notification to know how to achieve this at:
http://developer.apple.com/library/mac/#documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Introduction/Introduction.html
I found a way, to keep app running in background by playing silence
Make sure, that you selected audio playback in background modes
Also, don't use this method for long time, since it consumes CPU resources and battery juice, but I think it's a suitable way to keep app alive for a few minutes.
Just create an instance of SilencePlayer, call play() and then stop(), when you done
import CoreAudio
public class SilencePlayer {
private var audioQueue: AudioQueueRef? = nil
public private(set) var isStarted = false
public func play() {
if isStarted { return }
print("Playing silence")
let avs = AVAudioSession.sharedInstance()
try! avs.setCategory(AVAudioSessionCategoryPlayback, with: .mixWithOthers)
try! avs.setActive(true)
isStarted = true
var streamFormat = AudioStreamBasicDescription(
mSampleRate: 16000,
mFormatID: kAudioFormatLinearPCM,
mFormatFlags: kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked,
mBytesPerPacket: 2,
mFramesPerPacket: 1,
mBytesPerFrame: 2,
mChannelsPerFrame: 1,
mBitsPerChannel: 16,
mReserved: 0
)
let status = AudioQueueNewOutput(
&streamFormat,
SilenceQueueOutputCallback,
nil, nil, nil, 0,
&audioQueue
)
print("OSStatus for silence \(status)")
var buffers = Array<AudioQueueBufferRef?>.init(repeating: nil, count: 3)
for i in 0..<3 {
buffers[i]?.pointee.mAudioDataByteSize = 320
AudioQueueAllocateBuffer(audioQueue!, 320, &(buffers[i]))
SilenceQueueOutputCallback(nil, audioQueue!, buffers[i]!)
}
let startStatus = AudioQueueStart(audioQueue!, nil)
print("Start status for silence \(startStatus)")
}
public func stop() {
guard isStarted else { return }
print("Called stop silence")
if let aq = audioQueue {
AudioQueueStop(aq, true)
audioQueue = nil
}
try! AVAudioSession.sharedInstance().setActive(false)
isStarted = false
}
}
fileprivate func SilenceQueueOutputCallback(_ userData: UnsafeMutableRawPointer?, _ audioQueueRef: AudioQueueRef, _ bufferRef: AudioQueueBufferRef) -> Void {
let pointer = bufferRef.pointee.mAudioData
let length = bufferRef.pointee.mAudioDataByteSize
memset(pointer, 0, Int(length))
if AudioQueueEnqueueBuffer(audioQueueRef, bufferRef, 0, nil) != 0 {
AudioQueueFreeBuffer(audioQueueRef, bufferRef)
}
}
Tested on iOS 10 and Swift 4
I know this is not the answer to your question, but I think it is a solution.
This assumes that your trying to check something or get data from the internet on a regular basis?
Create a service that checks the internet every set interval for whatever it is you want to know, and create a push notification to alert you of it, if the server is down, or whatever it is your trying to monitor has changed state. Just an idea.
Yes you can do something like this. For that you need to set entry in info.plist to tell os that my app will run in background. I have done this while I wanted to pass user's location after particular time stamp to server. For that I have set "Required background modes" set to "App registers for location updates".
You can write a handler of type UIBackgroundTaskIdentifier.
You can already do this in the applicationDidEnterBackground Method