I have successfully used AVAudioPlayerNode to play stereo and mono files. I would like to use files with 3+ channels (surround files) and be able to route the audio in a non-linear way. For instance, I could assign file channel 0 to output channel 2, and file channel 4 to output channel 1.
The number of outputs of the audio interface will be unknown (2-40), and this is why I need to be able to allow the user to route the audio as they see fit. And the solution in WWDC 2015 507 of having the user change the routing in Audio Midi Setup is not a viable solution.
There's only 1 possibility that I can think of (and I am open to others): creating one player per channel, and loading each with only one channels' worth of buffers similar to this post. But even by the posters admission, there are issues.
So I'm looking for a way to copy each channel of a file into an AudioBuffer like:
let file = try AVAudioFile(forReading: audioURL)
let fullBuffer = AVAudioPCMBuffer(pcmFormat: file.processingFormat,
frameCapacity: AVAudioFrameCount(file.length))
try file.read(into: fullBuffer)
// channel 0
let buffer0 = AVAudioPCMBuffer(pcmFormat: file.processingFormat,
frameCapacity: AVAudioFrameCount(file.length))
// this doesn't work, unable to get fullBuffer channel and copy
// error on subscripting mBuffers
buffer0.audioBufferList.pointee.mBuffers.mData = fullBuffer.audioBufferList.pointee.mBuffers[0].mData
// repeat above buffer code for each channel from the fullBuffer
I was able to figure it out, so here's the code to make it work. Note: the code below separates a stereo (2 channel) file. This could easily be expanded to handle an unknown number of channels.
let file = try AVAudioFile(forReading: audioURL)
let formatL = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: file.processingFormat.sampleRate, channels: 1, interleaved: false)
let formatR = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: file.processingFormat.sampleRate, channels: 1, interleaved:
let fullBuffer = AVAudioPCMBuffer(pcmFormat: file.processingFormat, frameCapacity: AVAudioFrameCount(file.length))
let bufferLeft = AVAudioPCMBuffer(pcmFormat: formatL, frameCapacity: AVAudioFrameCount(file.length))
let bufferRight = AVAudioPCMBuffer(pcmFormat: formatR, frameCapacity: AVAudioFrameCount(file.length))
try file.read(into: fullBuffer)
bufferLeft.frameLength = fullBuffer.frameLength
bufferRight.frameLength = fullBuffer.frameLength
for i in 0..<Int(file.length) {
bufferLeft.floatChannelData![0][i] = fullBuffer.floatChannelData![0][i]
bufferRight.floatChannelData![0][i] = fullBuffer.floatChannelData![1][i]
}
Related
we are working on a project which records voice from an external microphone. For analysis purposes, we need to have a sample rate of about 5k Hz.
We are using AvAudioEngine to record a voice.
We know Apple devices want able to record at a specific rate, so we are using AVAudioConverter to downgrade the sample rate.
But as you know it is similar to the compression, so the lower we reduce sample rate, file size and file duration affect the same. Which is currently happening(Correct me if I am wrong in this).
Issue
**Issue is downgrading sample rate shorter the file length and its effects on calculation & analysis.
For example, a 1-hour recording was downgraded to 45 mins. So suppose if we are making analysis on 5 minute period interval, it goes wrong
What will be the best solution for this?**
Query
We have searched over the internet but we could not figure out how buffer size on installTap affects? In the current code, we have set it to 2688.
Can anyone clarify?
Code
let bus = 0
let inputNode = engine.inputNode
let equalizer = AVAudioUnitEQ(numberOfBands: 2)
equalizer.bands[0].filterType = .lowPass
equalizer.bands[0].frequency = 3000
equalizer.bands[0].bypass = false
equalizer.bands[1].filterType = .highPass
equalizer.bands[1].frequency = 1000
equalizer.bands[1].bypass = false
engine.attach(equalizer) //Attach equalizer
// Connect nodes
engine.connect(inputNode, to: equalizer, format: inputNode.inputFormat(forBus: 0))
engine.connect(equalizer, to: engine.mainMixerNode, format: inputNode.inputFormat(forBus: 0))
// call before creating converter because this changes the mainMixer's output format
engine.prepare()
let outputFormat = AVAudioFormat(commonFormat: .pcmFormatInt16,
sampleRate: 5000,
channels: 1,
interleaved: false)!
// Downsampling converter
guard let converter: AVAudioConverter = AVAudioConverter(from: engine.mainMixerNode.outputFormat(forBus: 0), to: outputFormat) else {
print("Can't convert in to this format")
return
}
engine.mainMixerNode.installTap(onBus: bus, bufferSize: 2688, format: nil) { (buffer, time) in
var newBufferAvailable = true
let inputCallback: AVAudioConverterInputBlock = { inNumPackets, outStatus in
if newBufferAvailable {
outStatus.pointee = .haveData
newBufferAvailable = false
return buffer
} else {
outStatus.pointee = .noDataNow
return nil
}
}
let convertedBuffer = AVAudioPCMBuffer(pcmFormat: outputFormat, frameCapacity: AVAudioFrameCount(outputFormat.sampleRate) * buffer.frameLength / AVAudioFrameCount(buffer.format.sampleRate))!
var error: NSError?
let status = converter.convert(to: convertedBuffer, error: &error, withInputFrom: inputCallback)
assert(status != .error)
if status == .haveData {
// Process with converted buffer
}
}
do {
try engine.start()
} catch {
print("Can't start the engine: \(error)")
}
Expecting Result
We are fine with compression of buffer but We would like to have the same recording duration in the output file. If we record for 10 minutes output file should have 10 minutes of data.
Digitized audio doesn't have an intrinsic duration since it can be played back at any sample rate.
In order for the resulting file's duration to be what you expect, the sample rates have to be what you expect at each stage: Recording, processing, and playback.
I suspect that one of two possible things is happening:
A) the sample rate of the buffer you receive inside installtap is not what you assumed it would be... and you are converting from the wrong format.
B) You are playing back your audio at sample rates than are different that what you are assuming they are. (How do you know that your player is playing at 5000hz)?
In order to check this, you would have to break the process down into smaller pieces and check the sample rate at each stage.
The question is how it is possible to save audio samples of an audio file into a text file. I have the special case that I have stored the samples in an array of UnsafeMutableRawBufferPointers and now I wonder how I can put them into a text file. In C++ I would have taken a stream operator here, however I am new to Swift and am wondering how to do this.
My code so far looks like this:
let Data = Array(UnsafeMutableRawBufferPointer(start:buffer.mData, count: Int(buffer.mDataByteSize))
.... your buffer .....
let yourBuffer = Array(UnsafeMutableRawBufferPointer(start:buffer.mData, count: Int(buffer.mDataByteSize))
let dd = Data.withUnsafeBytes(yourBuffer)
let fileName = FileManager.default.urls(for: .downloadsDirectory, in: .userDomainMask).first!.appendingPathComponent("yourFile")
let result = try? dd.write(to: fileName)
This should create the file "yourFile" in your downloadsDirectory
Please let me know.
You already have code that creates an UnsafeMutableRawBufferPointer. Use that to create Data, and write that to disk:
let buffer = UnsafeRawBufferPointer(start:buffer.mData, count: Int(buffer.mDataByteSize)
let data = Data.withUnsafeBytes(buffer)
let fileURL = //the path to which you want to save
data.write(to: fileURL, options: []
I solved the problem by refraining from the special case of using the array of pointers. If anyone else wants to write audio samples to a text file, this is how I approached the problem via AVAudioFile:
if FileManager.default.fileExists(atPath: getFileURL().path)
{
do{
//Load Audio into Buffer and then write it down to a .txt
let file = try AVAudioFile(forReading: getFileURL())
let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: file.fileFormat.sampleRate, channels: file.fileFormat.channelCount, interleaved: false)
guard let buf = AVAudioPCMBuffer(pcmFormat: format!, frameCapacity: AVAudioFrameCount(file.length)) else{
throw NSError()
}
try file.read(into: buf)
guard buf.floatChannelData != nil else{print("Channel Buffer was not able to be created")
throw NSError()}
let arraySize = Int(buf.frameLength)
print(arraySize, "Samples saved")
let samples = Array(UnsafeBufferPointer(start:buf.floatChannelData![0],count:arraySize))
//Array is going to be encoded and safe
let encoded = try! JSONEncoder().encode(samples)
try encoded.write(to: outputURL())
print("Textfile created.\n")
}catch {print(error)}
}else {print("AudioFile is missing")}
I am working on a push to talk functionality where sender can send an audio in form of bytes array to server and receiver can listen it at realtime through socket connection.
when i try to play video at receiver end using AVAudioEngine, it's not working.
let buffer = dataToPCMBuffer(format: format16KHzMono!, data: data)
let player = AVAudioPlayerNode()
self.audioEngine?.attach(audioPlayerNode)
let mixer = self.audioEngine?.mainMixerNode
self.audioEngine?.connect(player, to: mixer!, format: AVAudioFormat.init(commonFormat: AVAudioCommonFormat.pcmFormatInt16, sampleRate: 16000, channels: 1, interleaved: true) )
self.playerQueue.async {
self.audioPlayerNode.scheduleBuffer(buffer!) {
print("stopping")
if self.audioEngine!.isRunning {
self.audioPlayerNode.play()
}else {
try? self.audioEngine?.start()
}
}
And, i am facing crash at below given line.
self.audioEngine?.connect(player, to: mixer!, format: AVAudioFormat.init(commonFormat: AVAudioCommonFormat.pcmFormatInt16, sampleRate: 16000, channels: 1, interleaved: true) )
Any help will be appreciated.
I think it’s the format in your connection. Try using nil instead. There are some magic numbers needed for sample rates, maybe 16000 is not one of them.
Is it possible to increase/decrease volume of AVAsset track or AVMutableComposition of an audio file?
I have two audio files (background instrumental and recorded song), I want to decrease one file's volume and merge it with the other.
1. Change the Track's volume
To do this to the physical file, you will need to load the raw PCM data into Swift. Below is an example of getting the floating point data thanks to this SO post:
import AVFoundation
// ...
let url = NSBundle.mainBundle().URLForResource("your audio file", withExtension: "wav")
let file = try! AVAudioFile(forReading: url!)
let format = AVAudioFormat(commonFormat: .PCMFormatFloat32, sampleRate: file.fileFormat.sampleRate, channels: 1, interleaved: false)
let buf = AVAudioPCMBuffer(PCMFormat: format, frameCapacity: 1024)
try! file.readIntoBuffer(buf)
// this makes a copy, you might not want that
let floatArray = Array(UnsafeBufferPointer(start: buf.floatChannelData[0], count:Int(buf.frameLength)))
print("floatArray \(floatArray)\n")
Once you have the data in your floatArray, simply multiply every value in the array by a number between 0 and 1 to adjust the gain. If you are more familiar with decibels then put your decibel value into the following line, and multiply every array value by the linGain:
var linGain = pow(10.0f, decibelGain/20.0f).
Then it's a question of writing the audio file back again before you load it (credit):
let SAMPLE_RATE = Float64(16000.0)
let outputFormatSettings = [
AVFormatIDKey:kAudioFormatLinearPCM,
AVLinearPCMBitDepthKey:32,
AVLinearPCMIsFloatKey: true,
// AVLinearPCMIsBigEndianKey: false,
AVSampleRateKey: SAMPLE_RATE,
AVNumberOfChannelsKey: 1
] as [String : Any]
let audioFile = try? AVAudioFile(forWriting: url, settings: outputFormatSettings, commonFormat: AVAudioCommonFormat.pcmFormatFloat32, interleaved: true)
let bufferFormat = AVAudioFormat(settings: outputFormatSettings)
let outputBuffer = AVAudioPCMBuffer(pcmFormat: bufferFormat, frameCapacity: AVAudioFrameCount(buff.count))
// i had my samples in doubles, so convert then write
for i in 0..<buff.count {
outputBuffer.floatChannelData!.pointee[i] = Float( buff[i] )
}
outputBuffer.frameLength = AVAudioFrameCount( buff.count )
do{
try audioFile?.write(from: outputBuffer)
} catch let error as NSError {
print("error:", error.localizedDescription)
}
2. Mix the Tracks Together
Once you have your new .wav files of your audio, you can load both into your AVAssets like before, but this time with the desired gain you applied before.
Then it looks like you will want to be using the AVAssetReaderAudioMixOutput which has a method specifically for mixing together two audio tracks.
AVAssetReaderAudioMixOutput.init(audioTracks: [AVAssetTrack], audioSettings: [String : Any]?)
Note: I would not continuously use steps 1 & 2 for example if you wanted to mix the song with a slider and hear the result, I would recommend using AVPlayer's and adjust their volume and then when the user is ready, call this file IO and mixing.
I'm having trouble writing wav files in 24bits with AVAudioEngine in swift.
For my usage, my input is an array of Float.
I have the audio format of the input file (retrieved with AVAudioFile).
So, I need to convert my input Float array to a value that will be writable for the buffer. Also, I want to find the right channel to write my data.
My code is working with 16bit and 32 bit files, but I don't know how to handle 24 bit files...
Here it is :
//Static func to write audiofile
fileprivate func writeAudioFile(to outputURL : URL,
withFormat format : AVAudioFormat,
fromSamples music : [Float] )
{
var outputFormatSettings = format.settings
guard let bufferFormat = AVAudioFormat(settings: outputFormatSettings) else{
return
}
var audioFile : AVAudioFile?
do{
audioFile = try AVAudioFile(forWriting: outputURL,
settings: outputFormatSettings,
commonFormat: format.commonFormat,
interleaved: true)
} catch let error as NSError {
print("error:", error.localizedDescription)
}
let frameCount = music.count / Int(format.channelCount)
let outputBuffer = AVAudioPCMBuffer(pcmFormat: bufferFormat,
frameCapacity: AVAudioFrameCount(frameCount))
//We write the data in the right channel
guard let bitDepth = (outputFormatSettings["AVLinearPCMBitDepthKey"] as? Int) else {
return
}
switch bitDepth {
case 16:
for i in 0..<music.count {
var floatValue = music[i]
if(floatValue > 1){
floatValue = 1
}
if(floatValue < -1){
floatValue = -1
}
let value = floatValue * Float(Int16.max)
outputBuffer?.int16ChannelData!.pointee[i] = Int16(value)
}
case 24:
//Here I am not sure of what I do ... Could'nt find the right channel !
for i in 0..<music.count {
outputBuffer?.floatChannelData!.pointee[i] = music[i]
}
case 32:
for i in 0..<music.count {
outputBuffer?.floatChannelData!.pointee[i] = music[i]
}
default:
return
}
outputBuffer?.frameLength = AVAudioFrameCount( frameCount )
do{
try audioFile?.write(from: outputBuffer!)
} catch let error as NSError {
print("error:", error.localizedDescription)
return
}
}
Thanks by advance if someone have an idea of how to handle this !
Representing a 24 bit int in C isn't fun so in Swift I'm sure it's downright painful, and none of the API's support it anyway. Your best bet is to convert to a more convenient format for processing.
AVAudioFile has two formats and an internal converter to convert between them. Its fileFormat represents the format of the file on disk, while its processingformat represents the format of the lpcm data when it is read from, and the format of the lpcm data that it will accept when being written to.
The typical workflow is choose a standard processingFormat, do all of your processing using this format, and let AVAudioFile convert to and from the file format for reading and writing to disk. All of the Audio Unit APIs accept non-interleaved formats, so I tend to use non interleaved for all of my processing formats.
Here's an example that copies the first half of an audio file. It doesn't address your existing code, but illustrates a more common approach:
func halfCopy(src: URL, dst: URL) throws {
let srcFile = try AVAudioFile(forReading: src) //This opens the file for reading using the standard format (deinterleaved floating point).
let dstFile = try AVAudioFile(forWriting: dst,
settings: srcFile.fileFormat.settings,
commonFormat: srcFile.processingFormat.commonFormat,
interleaved: srcFile.processingFormat.isInterleaved) //AVAudioFile(forReading: src) always returns a non-interleaved processing format, this will be false
let frameCount = AVAudioFrameCount(srcFile.length) / 2 // Copying first half of file
guard let buffer = AVAudioPCMBuffer(pcmFormat: srcFile.processingFormat,
frameCapacity: frameCount) else {
fatalError("Derp")
}
try srcFile.read(into: buffer, frameCount: frameCount)
try dstFile.write(from: buffer)
}