Is there a more detailed way to debug SFSpeechRecognizer? - swift

Updated info below, and new code
I am trying to incorporate SFSpeechRecognizer into my app, and the errors/results I am getting from three pre-recorded audiofiles aren't enough for me to figure out what's going on. From the results I am getting I can't figure out what's wrong, and info via Google is sparse.
The code where I loop through three files is at the bottom. Here are the responses I get for my three audio files. I've made sure in each file to speak loudly and clearly, yet I still get: No speech detected or no text returned.
SS-X-03.m4a : There was an error: Optional(Error
Domain=kAFAssistantErrorDomain Code=1110 "No speech detected"
UserInfo={NSLocalizedDescription=No speech detected})
SS-X-20221125000.m4a : There was an error: Optional(Error
Domain=kAFAssistantErrorDomain Code=1110 "No speech detected"
UserInfo={NSLocalizedDescription=No speech detected})
SS-X-20221125001.m4a : (there is some text here if I set
request.requiresOnDeviceRecognition to false)
My code:
func findAudioFiles(){
let fm = FileManager.default
var aFiles : URL
print ("\(urlPath)")
do {
let items = try fm.contentsOfDirectory(atPath: documentsPath)
let filteredInterestArray1 = items.filter({$0.hasSuffix(".m4a")})
let filteredInterestArray2 = filteredInterestArray1.filter({$0.contains("SS-X-")})
let sortedItems = filteredInterestArray2.sorted()
for item in sortedItems {
audioFiles.append(item)
}
NotificationCenter.default.post(name: Notification.Name("goAndRead"), object: nil, userInfo: myDic)
} catch {
print ("\(error)")
}
}
#objc func goAndRead(){
audioIndex += 1
if audioIndex != audioFiles.count {
let fileURL = NSURL.fileURL(withPath: documentsPath + "/" + audioFiles[audioIndex], isDirectory: false)
transcribeAudio(url: fileURL, item: audioFiles[audioIndex])
}
}
func requestTranscribePermissions() {
SFSpeechRecognizer.requestAuthorization { [unowned self] authStatus in
DispatchQueue.main.async {
if authStatus == .authorized {
print("Good to go!")
} else {
print("Transcription permission was declined.")
}
}
}
}
func transcribeAudio(url: URL, item: String) {
guard let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US")) else {return}
let request = SFSpeechURLRecognitionRequest(url: url)
if !recognizer.supportsOnDeviceRecognition { print ("offline not available") ; return }
if !recognizer.isAvailable { print ("not available") ; return }
request.requiresOnDeviceRecognition = true
request.shouldReportPartialResults = true
recognizer.recognitionTask(with: request) {(result, error) in
guard let result = result else {
print("\(item) : There was an error: \(error.debugDescription)")
return
}
if result.isFinal {
print("\(item) : \(result.bestTranscription.formattedString)")
NotificationCenter.default.post(name: Notification.Name("goAndRead"), object: nil, userInfo: self.myDic)
}
}
}
Updated info
It appears that I was calling SFSpeechURLRecognitionRequest too often, and before I completed the first request. Perhaps I need to create a new instance of SFSpeechRecognizer? Unsure.
Regardless I quickly/sloppily adjusted the code to only run it once the previous instance returned its results.
The results were much better, except one audio file still came up as no results. Not an error, just no text.
This file is the same as the previous file, in that I took an audio recording and split it in two. So the formats and volumes are the same.
So I still need a better way to debug this, to find out what it going wrong with that file.

Related

How to get the value of searchBar.text of one VC and use it in another file of the project in swift?

I've been looking for the answer everywhere and could't find any.. Is there any way I can access the value of searchBar.text in another file? I have the delegate set in my SearchVC but I also have a custom tableView cell in another file.
I need the value of the SearchBar of my SearchVC to use in FirstDefinitionVC for decoding the word from the searchBar and use it for finding the audio URL.
All works fine while I call the function inside the searchBarSearchButtonClicked method but I can find no way to pass that String into FirstDefintionVC.
The relevant searchVC code :
var word = ""
`
extension SearchVC: UISearchBarDelegate {
func searchBarSearchButtonClicked(_ searchBar: UISearchBar) {
// { (data: [WordData], [Definitions])
word = searchBar.text!
wordManager.performRequest(word: word) { data in
self.wordData = data
self.searchButtonPressed = true
// print(data)
DispatchQueue.main.async {
self.tableView.reloadData()
}
}
fetchAudio(word: word) { data in //this one works fine
DispatchQueue.main.async {
self.wordData = data
}
}
}
func fetchAudio(word: String, comp: #escaping ([WordData])-> Void) {
let wordURL = "https://api.dictionaryapi.dev/api/v2/entries/en/"
let urlString = "\(wordURL)\(word)"
if let url = URL(string: urlString) {
let dataTask = URLSession.shared.dataTask(with: url, completionHandler: {
(data,response,error) in
guard let data = data, error == nil else {
print("Error occured while accessing data with URL")
return
}
do {
let decoded = try JSONDecoder().decode([WordData].self, from: data)
comp(decoded)
if let sound = decoded[0].phonetics[0].audio,
let sound2 = decoded[0].phonetics[1].audio {
print("sound = \(sound)")
let nonEmpty = (sound != "") ? sound : sound2 //write switch cases or another ternary with more urls to choose from if both are empty
self.audioUrl = URL(string: nonEmpty)
// url = URL(string: sound2)
do {
try AVAudioSession.sharedInstance().setMode(.default)
try AVAudioSession.sharedInstance().setActive(true, options: .notifyOthersOnDeactivation)
self.player = AVPlayer(url: self.audioUrl!)
guard let player = self.player else { return }
player.play()
} catch let error {
print(error.localizedDescription)
}
}
//comp(decoded, entries.self)
} catch {
print("Error occured while decoding JSON into Swift structure \(error)")
}
})
dataTask.resume()
}
}
I need to call the searchBar.text value in another file inside this IBAction of class FirstDefinitionVC:
`
#IBAction func pronunciationButton(_ sender: UIButton) {
searchVC.fetchAudio(word: searchVC.word) { data in
self.wordData = data
}
}
This was one of my approaches to this, I tried to create a global model Word with an initializer also and it didn't work. Is there any way around it?

Listing all files in a directory on macOS Big Sur

In a different question I asked how to save files on a directory of the user's choosing. The reply was the following code, which works great.
func resolveURL(for key: String) throws -> URL {
if let data = UserDefaults.standard.data(forKey: key) {
var isStale = false
let url = try URL(resolvingBookmarkData: data, options:[.withSecurityScope], bookmarkDataIsStale: &isStale)
if isStale {
let newData = try url.bookmarkData(options: [.withSecurityScope])
UserDefaults.standard.set(newData, forKey: key)
}
return url
} else {
let panel = NSOpenPanel()
panel.allowsMultipleSelection = false
panel.canChooseDirectories = true
panel.canCreateDirectories = true
panel.canChooseFiles = false
if panel.runModal() == .OK,
let url = panel.url {
let newData = try url.bookmarkData(options: [.withSecurityScope])
UserDefaults.standard.set(newData, forKey: key)
return url
} else {
throw ResolveError.cancelled
}
}
}
func saveFile(filename: String, contents: String) {
do {
let directoryURL = try resolveURL(for: "savedDirectory")
let documentURL = directoryURL.appendingPathComponent (filename + ".txt")
print("saving " + documentURL.absoluteString)
try directoryURL.accessSecurityScopedResource(at: documentURL) { url in
try contents.write (to: url, atomically: false, encoding: .utf8)
}
} catch let error as ResolveError {
print("Resolve error:", error)
} catch {
print(error)
}
}
Now, the next step is to go to the directory the user chose when the app loads, and if any files are there, ready each one and add the contents of those files to the struct I use to hold the data.
Googling a little bit I found that you can read all files in a directory using FileManager.default.contentsOfDirectory so I wrote:
func loadFiles() {
do {
let directoryURL = try resolveURL(for: "savedDirectory")
let contents = try FileManager.default.contentsOfDirectory(at: directoryURL,
includingPropertiesForKeys: nil,
options: [.skipsHiddenFiles])
for file in contents {
print(file.absoluteString)
}
} catch let error as ResolveError {
print("Resolve error:", error)
return
} catch {
print(error)
return
}
}
But, I get the following error:
Error Domain=NSCocoaErrorDomain Code=257 "The file “myFiles” couldn’t be opened because you don’t have permission to view it." UserInfo={NSURL=file:///Users/aleph/myFiles, NSFilePath=/Users/aleph/myFiles, NSUnderlyingError=0x600000704ba0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}
which looking at my code I would guess it's happening because I'm not using directoryURL.accessSecurityScopedResource. I tried to add that, or find any other way, but I'm running into a block and I don't know how to get to the directory saved in savedDirectory, and go through every file, reading its contents.
Thank you for any help
If I use:
directoryURL.startAccessingSecurityScopedResource()
// load the files
directoryURL.stopAccessingSecurityScopedResource()
Then it works.

AVAudioEngine MIDI file play (Current progress + MIDI end callback) Swift

Am playing MID using AVAudioEngine, AVAudioSequencer, AVAudioUnitSampler.
AVAudioUnitSampler loads Soundfont and AVAudioSequencer load MIDI file.
My initial configurations are
engine = AVAudioEngine()
sampler = AVAudioUnitSampler()
speedControl = AVAudioUnitVarispeed()
pitchControl = AVAudioUnitTimePitch()
engine.attach(sampler)
engine.attach(pitchControl)
engine.attach(speedControl)
engine.connect(sampler, to: speedControl, format: nil)
engine.connect(speedControl, to: pitchControl, format: nil)
engine.connect(pitchControl, to: engine.mainMixerNode, format: nil)
Here is how my sequence loads MIDI file
func setupSequencer() {
self.sequencer = AVAudioSequencer(audioEngine: self.engine)
let options = AVMusicSequenceLoadOptions()
let documentsDirectoryURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let introurl = URL(string: songDetails!.intro!)
let midiFileURL = documentsDirectoryURL.appendingPathComponent(introurl!.lastPathComponent)
do {
try sequencer.load(from: midiFileURL, options: options)
print("loaded \(midiFileURL)")
} catch {
print("something screwed up \(error)")
return
}
sequencer.prepareToPlay()
if sequencer.isPlaying == false{
}
}
And here is how sampler load SoundFont
func loadSF2PresetIntoSampler(_ preset: UInt8,bankURL:URL ) {
do {
try self.sampler.loadSoundBankInstrument(at: bankURL,
program: preset,
bankMSB: UInt8(kAUSampler_DefaultMelodicBankMSB),
bankLSB: UInt8(kAUSampler_DefaultBankLSB))
} catch {
print("error loading sound bank instrument")
}
}
And it's playing fine no issue with this. I have 2 other requirements, and am having problem in those
I have to play another MIDI file after first MID ends playing, for that i need to get the complete/finish MIDI file callback from either Engine or Sequence OR How can i load multiple MIDI files in Sequence? I have tried many ways but didn't help.
I need to show the progress of MIDI file play, like current time and total time. For this i have tried a method found in stack answers somewhere which is:
var currentPositionInSeconds: TimeInterval {
get {
guard let offsetTime = offsetTime else { return 0 }
guard let lastRenderTime = engine.outputNode.lastRenderTime else { return 0 }
let frames = lastRenderTime.sampleTime - offsetTime.sampleTime
return Double(frames) / offsetTime.sampleRate
}
}
Here offsetTime is
offsetTime = engine.outputNode.lastRenderTime
And it always return nil.
Glad to see someone using my example code.
It's a missing feature. Please file a Radar. Nothing will happen without a Radar.
They do pay attention to them to schedule what to work on.
I fake it by getting the length of the sequence.
if let ft = sequencer.tracks.first {
self.lengthInSeconds = ft.lengthInSeconds
} else {
self.lengthInSeconds = 0
}
Then in my play function,
do {
try sequencer.start()
Timer.scheduledTimer(withTimeInterval: self.lengthInSeconds, repeats: false) {
[weak self] (t: Timer) in
guard let self = self else {return}
t.invalidate()
self.logger.debug("sequencer finished")
self.sequencer.stop()
}
...
You can use a DispatchSourceTimer if you want more accuracy.

Speech to Text SWIFT

I have the following code:
func createStringFromAudio () {
SFSpeechRecognizer.requestAuthorization {_ in
DispatchQueue.main.async {
switch SFSpeechRecognizer.authorizationStatus() {
case .authorized :
let audioURL = Bundle.main.url(forResource: "ConversionTest", withExtension: "mp3")!
let recognizer = SFSpeechRecognizer()
let request = SFSpeechURLRecognitionRequest(url: audioURL)
recognizer?.recognitionTask(with: request) { result, error in
guard error == nil else { print("Error"); return}
guard let result = result else {print("No result"); return}
print(result.bestTranscription.formattedString)
}
break
default :
break
}
}
}
}
Here is my question:
1: How do I know when the file is done? Currently it continually updates the result print but doesn't notify me when the transcription is 100% completed. How do I know when the transcription is done and I can save the string for parsing?
2: It seems to cut off before finishing. Is there a time limit? Character limit?
You can save the task to check its status, even cancell it.
Actually, the transcribing process is not controlled by us and when the system think its finished, it's finished.
If you do need precise controls, use a delegate:
func recognitionTask(with request: SFSpeechRecognitionRequest, delegate: SFSpeechRecognitionTaskDelegate) -> SFSpeechRecognitionTask
This may provide more informations during transcribing, at least it will let you know when it is finished.

Swift Google Drive downloading file

I can't figure out how to download a Google Drive file in Swift. I followed the modified quickstart from Google Objective-C API 'GTL' with Swift and that worked. I can't translate the objective C code from the google drive API on downloading files. I've searched around and can't find anything. How can I get this to work?
You can use this function for downloading files with the Google Drive API in Swift:
func downloadFile(file: GTLDriveFile){
let url = "https://www.googleapis.com/drive/v3/files/\(file.identifier!)?alt=media"
let fetcher = drive.fetcherService.fetcherWithURLString(url)
fetcher.beginFetchWithDelegate(
self,
didFinishSelector: #selector(ViewController.finishedFileDownload(_:finishedWithData:error:)))
}
(In this case drive is the GTLServiceDrive - the same as in the Documentation)
Then you need to implement the function finishedFileDownload that will be called once the download is completed:
func finishedFileDownload(fetcher: GTMSessionFetcher, finishedWithData data: NSData, error: NSError?){
if let error = error {
//show an alert with the error message or something similar
return
}
//do something with data (save it...)
}
Actual for Swift 5.
func download(file: GTLRDrive_File) {
let url = "https://www.googleapis.com/drive/v3/files/\(file.identifier!)?alt=media"
let fetcher = drive.fetcherService.fetcher(withURLString: url)
fetcher.beginFetch(completionHandler: { data, error in
if let error = error {
print(error.localizedDescription)
}
//Do something with data
})
}
Swift 5 with progress block. file.size returns nil for some reason so I used fetcher.response?.expectedContentLength instead.
func download(file: GTLRDrive_File, service: GTLRDriveService) {
let url = "https://www.googleapis.com/drive/v3/files/\(file.identifier)?alt=media"
let fetcher = service.fetcherService.fetcher(withURLString: url)
fetcher.beginFetch(completionHandler: { fileData, error in
if error == nil {
print("finished downloading Data...")
print(fileData as Any)
} else {
print("Error: \(String(describing: error?.localizedDescription))")
}
})
fetcher.receivedProgressBlock = { _, totalBytesReceived in
if let fileSize = fetcher.response?.expectedContentLength {
let progress: Double = Double(totalBytesReceived) / Double(fileSize)
// update progress bar here
print(progress)
}
}
}