I want to get the volume of AKAmplitudeTracker but getting -inf what is wrong with me please help out.
AKAudioFile.cleanTempDirectory()
AKSettings.audioInputEnabled = true
AKSettings.bufferLength = .medium
AKSettings.defaultToSpeaker = true
AKSettings.playbackWhileMuted = true
AKSettings.enableRouteChangeHandling = true
AKSettings.enableCategoryChangeHandling = true
AKSettings.enableLogging = true
do {
try AKSettings.setSession(category: .playAndRecord, with: .allowBluetoothA2DP)
} catch {
print("error \(error.localizedDescription)")
}
microphone = AKMicrophone()!
tracker = AKAmplitudeTracker(microphone)
booster = AKBooster(tracker, gain: 0)
AudioKit.output = booster
try AudioKit.start()
=================
extension AKAmplitudeTracker {
var volume: Decibel {
return 20.0 * log10(amplitude)
}
}
=================
OutPut print(tracker. amplitude)
0.0
Had a quick look, seems that you followed the basic setup, you do seem to fail to trace the data generated in time correctly! Amplitude data is provided during the time period for the computation that is taken from the microphone, so to look at what it looks like in timeline you can use a timer, as such:
func reset() throws {
do {
self.timer.invalidate()
self.timer = nil
} catch {
throw error
}
}
func microphoneTracker() {
guard self.timer == nil else { return }
self.watcher()
let timer = Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { _ in
log.info(self.akMicrophoneAmplitudeTracker.amplitude)
}
self.timer = timer
}
Change the withTimeInterval to how frequently you want to check the amplitude.
I think it's quite readable what I put there for you, but I'll break it down in a few words:
Keep a reference for the AKAmplitudeTracker in a property, here I've named it akMicrophoneAmplitudeTracker
Keep a reference for your timed event, that will check the amplitude value during a period
Compute the data in the closure body, the property holding value is .amplitude
The computation in the example is a logger that prints .amplitude
As required, use the .invalidate method to stop the timer
A few other things you may want to double-check on your code is to make sure that the tracker is part of the signal chain, as that's an AVAudioEngine engine requirement; I've also noticed in some other people's code a call for the method .start in the AKAmplitudeTracker, as follows:
akMicrophoneAmplitudeTracker.start()
To finish, have in mind that if you are testing it through Simulator, look at the microphone settings of your host-machine and expect amplitudes that might be different then the actual hardware.
Related
I'm having a pathfinder class in a SpriteKit game that a I want to use to process every path request in the game. So I have my class stored in my SKScene and I access it from different parts of the game always from the main thread. The pathfinder uses a GKGridGraph of a pretty good size (288 x 224). The class holds an array of requests processed one after another at each update() call from the main scene. Here is the code :
class PathFinder {
var isLookingForPath = false
var groundGraph : GKGridGraph<MyNode>?
var queued : [PathFinderRequest] = []
var thread: DispatchQoS.QoSClass = .userInitiated
func generate(minPoint: CGPoint) {
// generate the groundGraph grid
}
func update() {
// called every frame
if !self.isLookingForPath {
findPath()
}
}
func findPath(from start: TuplePosition, to end: TuplePosition, on layer: PathFinderLayer, callBack: PathFinderCallback) {
// Generating request
let id = String.randomString(length: 5)
let request = PathFinderRequest(id: id, start: start, end: end, layer: layer, callback: callBack)
// Append the request object at the end of the array
queued.append(request)
}
func findPath() {
self.isLookingForPath = true
guard let request = queued.first else {
isLookingForPath = false
return
}
let layer = request.layer
let callback = request.callback
let start = request.start
let end = request.end
let id = request.id
var graph = self.groundGraph
queued.removeFirst()
let findItem = DispatchWorkItem {
if let g = graph, let sn = g.node(atGridPosition: start.toVec()), let en = g.node(atGridPosition: end.toVec()) {
if let path = g.findPath(from: sn, to: en) as? [GKGridGraphNode], path.count > 0 {
// Here we have the path found
// it worked !
}
}
// Once the findPath() method execution is over,
// we reset the "flag" so we can call it once again from
// the update() method
self.isLookingForPath = false
}
// Execute the findPath() method in the chosen thread
// asynchronously
DispatchQueue.global(qos: thread).async(execute: findItem)
}
func drawPath(_ path: [GKGridGraphNode]) {
// draw the path on the scene
}
}
Well the code works quite good as it is. If I send random path request within (x+-10, y+-10) length it will return them to each object holding the callback in the request object pretty quickly, but suddenly one request is randomly taking a huge amount of time (approximatively 20s compared to 0.001s) and despite everything I tried I wasn't able to find out what happens. It's never on the same path, never the same caller, never after a certain amount of time... here is a video of the issue : https://www.youtube.com/watch?v=-IYlLOQgJrQ
It sure happens more quickly when there is too much entities requesting but I can't figure why I'm sure it has to deal with the DispacthQueue async calls that I use to prevent the game from freezing.
With delay on every call, the error appear later but is still here :
DispatchQueue.global(qos: thread).asyncAfter(deadline: .now() + 0.1, execute: findItem)
When I look for what is taking so much time to process it is a sub method of the GKGridGraph class :
So I really don't know how to figure this out, I tried everything I could think of but it always happens whatever the delay, the number of entities, the different threads, etc...
Thank you for your precious help !
I am trying to capture FFT data from a microphone. I've managed to get it to work before with a similar codebase but since macOS Mojave it's broken - the fft data constantly stays 0.
Relevant Code:
var fft: AKFFTTap?
var inputDevice: AKDevice? {
didSet {
inputNode = nil
updateAudioNode()
}
}
var inputNode: AKNode? {
didSet {
if fft != nil {
// According to AKFFTTap class reference, it will always be on tap 0
oldValue?.avAudioNode.removeTap(onBus: 0)
}
fft = inputNode.map { AKFFTTap($0) }
}
}
[...]
guard let device = inputDevice else {
inputNode = ViewController.shared.player.mixer
return
}
do {
try AudioKit.setInputDevice(device)
}
catch {
print("Error setting input device: \(error)")
return
}
let microphoneNode = AKMicrophone()
do {
try microphoneNode.setDevice(device)
}
catch {
print("Failed setting node input device: \(error)")
return
}
microphoneNode.start()
microphoneNode.volume = 3
print("Switched Node: \(microphoneNode), started: \(microphoneNode.isStarted)")
inputNode = microphoneNode
try! AudioKit.start()
All the code is called, no errors are output, but the fft simply stays blank. With some code reordering I get varying errors.
A full version of the class, for completeness, is here.
Finally, I also tried implementing one to one the examples from the playground. Since XCode playgrounds seem to crash with AudioKit, I tried it in my own codebase, but there's no difference there either. AKFrequencyTracker, for example, gets 0s for both amplitude and frequency.
I am not 100% positive of this, but I'd like you to try AudioKit v4.5.1 out. We definitely fixed a bug in AKMicrophone, and that could have downstream consequences. I'll withdraw this answer and keep looking if it is not fixed. Let me know.
I have a question how to correctly call CIDetector correctly I'm trying to run the face detection in real-time this works very well. However the memory consumption of the app increases linearly with time how you can see in the image below I'm thinking this is due to objects being created but they're not released can anyone advise how to do it correctly.
I have pinpointed the issue down to this function as every time it's invoked memory increases linearly when it terminated it quickly drops down to almost 80 MB instead of 11 GB rising also check for memory leaks however none were found.
My target development platform is Mac OS I'm trying to extractthe mouth position from the CA detector and then use it to compute a Delta in the mouse function for a Game.
I also Looked that this post however I have tried their approach but it did not work for me
CIDetector isn't releasing memory
fileprivate func faceDetection(){
// setting up dispatchQueue
dispatchQueue.async {
// checking if sample buffer is equal to nil if not assign its value to sample
if let sample = self.sampleBuffers {
// if allfeatures is not equal to nil. if yes assign allfeatures to features otherwise return
guard let features = self.allFeatures(sample: sample) else { return }
// loop to cycle through all features
for feature in features {
// checks if the feature is a CIFaceFeature if yes assign feature to face feature and go on.
if let faceFeature = feature as? CIFaceFeature {
if !self.hasEnded {
if self.calX.count > 30 {
self.sens.append((self.calX.max()! - self.calX.min()!))
self.sens.append((self.calY.max()! - self.calY.min()!))
print((self.calX.max()! - self.calX.min()!))
self.hasEnded = true
} else {
self.calX.append(faceFeature.mouthPosition.x)
self.calY.append(faceFeature.mouthPosition.y)
}
} else {
self.mouse(position: CGPoint(x: (faceFeature.mouthPosition.x - 300 ) * 2, y: (faceFeature.mouthPosition.y + 20 ) * 2), faceFeature: faceFeature)
}
}
}
}
if !self.faceTrackingEnds {
self.faceDetection()
}
}
}
This problem was caused by repeatedly calling the function without waiting for its completion the fix was implementing a dispatch group and then calling the function on its completion
like this Now the CIdetector runs comfortably at 200 MB memory
fileprivate func faceDetection(){
let group = DispatchGroup()
group.enter()
// setting up dispatchQueue
dispatchQueue.async {
// checking if sample buffer is equal to nil if not assign its value to sample
if let sample = self.sampleBuffers {
// if allfeatures is not equal to nil. if yes assign allfeatures to features otherwise return
guard let features = self.allFeatures(sample: sample) else { return }
// loop to cycle through all features
for feature in features {
// checks if the feature is a CIFaceFeature if yes assign feature to face feature and go on.
if let faceFeature = feature as? CIFaceFeature {
self.mouse(position: faceFeature.mouthPosition, faceFeature: faceFeature)
}
}
}
group.leave()
}
group.notify(queue: .main) {
if !self.faceTrackingEnds {
self.faceDetection()
}
}
}
I have recently done the migration from AudioKit 3.7 to 4.2 (using Cocoapods), needed for XCode 9.3. I followed the migration guide and changed AKAudioPlayer to AKPlayer.
The issue
When AKPlayer plays an audio file, AudioKit is crashing with this error:
2018-04-17 09:32:43.042658+0200 hearfit[3509:2521326] [avae] AVAEInternal.h:103:_AVAE_CheckNoErr: [AVAudioEngineGraph.mm:3632:UpdateGraphAfterReconfig: (AUGraphParser::InitializeActiveNodesInOutputChain(ThisGraph, kOutputChainFullTraversal, *conn.srcNode, isChainActive)): error -10875
2018-04-17 09:32:43.049372+0200 hearfit[3509:2521326] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'error -10875'
*** First throw call stack:
(0x1847d6d8c 0x1839905ec 0x1847d6bf8 0x18a0ff1a0 0x18a11bf58 0x18a12aab0 0x18a128cdc 0x18a1a1738 0x18a1a160c 0x10519192c 0x10519d2f4 0x10519d64c 0x10503afdc 0x10507c4a0 0x10507c01c 0x104f6d9cc 0x1852233d4 0x18477faa8 0x18477f76c 0x18477f010 0x18477cb60 0x18469cda8 0x18667f020 0x18e67d78c 0x10504dfd4 0x18412dfc0)
libc++abi.dylib: terminating with uncaught exception of type NSException
Sometimes it happens on the first play, and sometimes the first play is done correctly, but not the second one.
Everything was working great before the migration. I also tried to keep AKAudioPlayer: sounds are played correctly but AKFrequencyTracker does not work anymore.
Context
This is my setup:
Quick explanation:
AKPlayer 1 plays short audio files (between 1 and 5 seconds)
AKFrequencyTracker is used to display a plot
AKPlayer 2 plays background sound (volume must be configurable)
AKWhiteNoise allows to do some manual volume measurements (using AKMixer 2 volume property)
Use case example
The user starts an exercise. A sound is played continuously (with looping) using AKPlayer 2 and the user listens a word (played with AKPlayer 1), the plot is displayed. Next, several words are displayed on screen and the user must pick the right one. And a new word is listened... and so on.
So I have to change dynamically the played file of AKPlayer 1. All the code is written in a dedicated class, a singleton. All the nodes are setup in the init() function.
// singleton
static let main = AudioPlayer()
private init() {
let silenceUrl = Bundle.main.url(forResource: "silence", withExtension: "m4a", subdirectory: "audio")
self.silenceFile = silenceUrl!
self.mainPlayer = AKPlayer(url: self.silenceFile)!
self.mainPlayer.volume = 1.0
self.freqTracker = AKFrequencyTracker(self.mainPlayer, hopSize: 256, peakCount: 10)
let noiseUrl = Bundle.main.url(forResource: "cocktail-party", withExtension: "m4a", subdirectory: "audio")
self.noiseFile = noiseUrl!
self.noisePlayer = AKPlayer(url: self.noiseFile)!
self.noisePlayer.volume = 1.0
self.noisePlayer.isLooping = true
let mixer = AKMixer(self.freqTracker, self.noisePlayer)
self.whiteNoise = AKWhiteNoise(amplitude: 1.0)
self.whiteNoiseMixer = AKMixer(self.whiteNoise)
self.whiteNoiseMixer.volume = 0
self.mixer = AKMixer(mixer, self.whiteNoiseMixer)
AudioKit.output = self.mixer
do {
try AudioKit.start()
} catch (let error) {
print(error)
}
// stop directly the white noise mixer
self.whiteNoise.stop()
self.whiteNoiseMixer.volume = self.whiteNoiseVolume
self.mainPlayer.completionHandler = {
DispatchQueue.main.async {
if let timer = self.timer {
timer.invalidate()
self.timer = nil
}
if let completion = self.completionHandler {
Timer.scheduledTimer(withTimeInterval: 0.5, repeats: false, block: { (_) in
completion()
self.completionHandler = nil
})
}
}
}
}
To change the AKPlayer 1 audio file, I use this function, on the same class:
func play(fileUrl: URL, tracker: #escaping TrackerCallback, completion: (() -> Void)?) throws {
self.completionHandler = completion
let file = try AKAudioFile(forReading: fileUrl)
self.mainPlayer.load(audioFile: file)
self.mainPlayer.preroll()
self.timer = Timer.scheduledTimer(withTimeInterval: self.trackerRefreshRate, repeats: true) { (timer) in
tracker(self.freqTracker.frequency, self.freqTracker.amplitude)
}
self.mainPlayer.play()
}
Thank you.
I'm not sure what you are replacing into the player, but if the format of the file is different from what you had before, channels, samplerate, etc -- you should create a new AKPlayer instance rather than load into the same one. If your files are all the same format then it should work ok.
That said, I haven't seen the crash you show.
Another thing that is dangerous in your code is force unwrapping those optionals - you should guard against things being nil. AKPlayer actually uses AVAudioFile, no need for AKAudioFile.
guard let akfile = try? AVAudioFile(forReading: url) else { return }
if akfile.channelCount != player?.audioFile?.processingFormat.channelCount ||
akfile.sampleRate != player?.audioFile?.processingFormat.sampleRate {
AKLog("Need to create new player as formats have changed.")
}
Getting some weird results when trying to render a sequence of AKAudioPlayers with AudioKit 4.0, Swift 4 on iOS 11.1
I'm aware of AudioKit.renderToFile alternative on the development branch (https://github.com/AudioKit/AudioKit/commit/09aedf7c119a399ab00026ddfb91ae6778570176) but would like to cover iOS 9+ if possible
Expected result:
A long audio file with the each file (URL) rendered in sequence
Actual result:
Only the last scheduled file is rendered (at the correct offset in the resulting wav file)
Weirdly, if I schedule them all at the 0 offset, they all get rendered. Also, if I play things back without rendering, it sounds correct (though I have to adjust the AVAudioTime to use mach_absolute_time)
It almost seems like scheduling an AKAudioPlayer cancels the previous one.
Setup:
class func initialize (){
// ....
do {
try AKSettings.setSession(category: .playAndRecord, with: .allowBluetoothA2DP)
} catch {
AKLog("Could not set session category.")
}
//AKSettings.playbackWhileMuted = true
AKSettings.defaultToSpeaker = true
mainMixer = AKMixer()
offlineRender = AKOfflineRenderNode()
mainMixer! >>> offlineRender!
AudioKit.output = offlineRender!
AudioKit.start()
// ....
Rendering:
class func testRender(urls: [URL], dest: URL, offset: TimeInterval = 2){
// Stop / Start AudioKit when switching internalRenderEnabled, otherwise I get the following error:
// *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'player started when engine not running'
AudioKit.stop()
var players = [AKAudioPlayer]()
var scheduleTime : TimeInterval = 0
// create players
for url in urls {
do {
let file = try AKAudioFile(forReading: url)
let player = try AKAudioPlayer(file: file)
players.append(player)
player.connect(to: mainMixer!)
print("Connecting player")
} catch {
print("error reading")
}
}
offlineRender!.internalRenderEnabled = false
AudioKit.start()
for player in players{
do {
// 0 instead of mach_absolute_time(), otherwise the result is silent
let avTime = AKAudioPlayer.secondsToAVAudioTime(hostTime: 0, time: scheduleTime)
// schedule and play according to:
// https://stackoverflow.com/questions/45799686/how-to-apply-audio-effect-to-a-file-and-write-to-filesystem-ios/45807586#45807586
player.schedule(from: 0, to: player.duration, avTime: nil)
player.play(at: avTime);
scheduleTime += offset
} catch {
print("error reading")
}
}
// add some padding
scheduleTime += 3
let duration = scheduleTime
do {
try offlineRender!.renderToURL(dest, seconds: duration)
} catch {
print("error rendering")
}
// cleanup
players.forEach { $0.schedule(from: 0, to: $0.duration, avTime: nil)}
players.forEach({$0.stop()})
players.forEach({$0.disconnectOutput()})
offlineRender!.internalRenderEnabled = true
}
Appreciate any help!
AKOfflineRenderNode has been deprecated as of iOS 11.0. Version 4.0.4 has an AudioKit.renderToFile method to replace it. It was updated recently (in late 2017).
So it looks like the AKOFflineRednerNode is indeed deprecated in the coming versions of AudioKit and is not working on iOS11. Reading comments discussing the issue on GitHub it sounds like the plan is to encapsulate both the new (iOS11+) offline rendering and the old (iOS9-10) under a common interface (AudioKit.renderToFile). However for now it seems to be iOS11 only.
After some testing with the dev version (install instructions here: https://github.com/audiokit/AudioKit/blob/master/Frameworks/README.md) I got the following code to work as intended:
try AudioKit.renderToFile(outputFile, seconds: duration, prerender: {
var scheduleTime : TimeInterval = 0
for player in players{
let dspTime = AVAudioTime(sampleTime: AVAudioFramePosition(scheduleTime * AKSettings.sampleRate), atRate: AKSettings.sampleRate)
player.play(at: dspTime)
scheduleTime += offset
}
})
Unless someone can provide a workaround that gets the OfflineRenderNode working on iOS11 and until the official release of AudioKit with the renderToFile implemented this is the best answer I could find.