How to convert Data of Int16 audio samples to array of float audio samples - swift

Question

I'm currently working with audio samples.
I get them from AVAssetReader and have a CMSampleBuffer with something like this:
guard let sampleBuffer = readerOutput.copyNextSampleBuffer() else {
guard reader.status == .completed else { return nil }
// Completed
// samples is an array of Int16
let samples = sampleData.withUnsafeBytes {
Array(UnsafeBufferPointer<Int16>(
start: $0, count: sampleData.count / MemoryLayout<Int16>.size))
}
// The only way I found to convert [Int16] -> [Float]...
return samples.map { Float($0) / Float(Int16.max)}
}
guard let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) else {
return nil
}
let length = CMBlockBufferGetDataLength(blockBuffer)
let sampleBytes = UnsafeMutablePointer<UInt8>.allocate(capacity: length)
CMBlockBufferCopyDataBytes(blockBuffer, 0, length, sampleBytes)
sampleData.append(sampleBytes, count: length)
}
As you can see the only I found to convert [Int16] -> [Float] issamples.map { Float($0) / Float(Int16.max) but by doing this my processing time is increasing. Does it exist an other way to cast a pointer of Int16 to a pointer of Float?

Answer 1

"Casting" or "rebinding" a pointer only changes the way how memory is
interpreted. You want to compute floating point values from integers,
the new values have a different memory representation (and also a different
size).
Therefore you somehow have to iterate over all input values
and compute the new values. What you can do is to omit the Array
creation:
let samples = sampleData.withUnsafeBytes {
UnsafeBufferPointer<Int16>(start: $0, count: sampleData.count / MemoryLayout<Int16>.size)
}
return samples.map { Float($0) / Float(Int16.max) }
Another option would be to use the vDSP functions from the
Accelerate framework:
import Accelerate
// ...
let numSamples = sampleData.count / MemoryLayout<Int16>.size
var factor = Float(Int16.max)
var floats: [Float] = Array(repeating: 0.0, count: numSamples)
// Int16 array to Float array:
sampleData.withUnsafeBytes {
vDSP_vflt16($0, 1, &floats, 1, vDSP_Length(numSamples))
}
// Scaling:
vDSP_vsdiv(&floats, 1, &factor, &floats, 1, vDSP_Length(numSamples))
I don't know if that is faster, you'll have to check.
(Update: It is faster, as ColGraff demonstrated in his answer.)
An explicit loop is also much faster than using map:
let factor = Float(Int16.max)
let samples = sampleData.withUnsafeBytes {
UnsafeBufferPointer<Int16>(start: $0, count: sampleData.count / MemoryLayout<Int16>.size)
}
var floats: [Float] = Array(repeating: 0.0, count: samples.count)
for i in 0..<samples.count {
floats[i] = Float(samples[i]) / factor
}
return floats
An additional option in your case might be to use CMBlockBufferGetDataPointer() instead of CMBlockBufferCopyDataBytes()
into allocated memory.

Answer 2

You can do considerably better if you use the Accelerate Framework for the conversion:
import Accelerate
// Set up random [Int]
var randomInt = [Int16]()
randomInt.reserveCapacity(10000)
for _ in 0..<randomInt.capacity {
let value = Int16(Int32(arc4random_uniform(UInt32(UInt16.max))) - Int32(UInt16.max / 2))
randomInt.append(value)
}
// Time elapsed helper: https://stackoverflow.com/a/25022722/887210
func printTimeElapsedWhenRunningCode(title:String, operation:()->()) {
let startTime = CFAbsoluteTimeGetCurrent()
operation()
let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
print("Time elapsed for \(title): \(timeElapsed) s.")
}
// Testing
printTimeElapsedWhenRunningCode(title: "vDSP") {
var randomFloat = [Float](repeating: 0, count: randomInt.capacity)
vDSP_vflt16(randomInt, 1, &randomFloat, 1, vDSP_Length(randomInt.capacity))
}
printTimeElapsedWhenRunningCode(title: "map") {
randomInt.map { Float($0) }
}
// Results
//
// Time elapsed for vDSP : 0.000429034233093262 s.
// Time elapsed for flatMap: 0.00233501195907593 s.
It's an improvement of about 5 times faster.
(Edit: Added some changes suggested by Martin R)

Answer 3

#MartinR and #ColGraff gave really good answers, and thank you for everybody and the fast replies.
however I found an easier way to do that without any computation. AVAssetReaderAudioMixOutput requires an audio settings dictionary. Inside we can set the key AVLinearPCMIsFloatKey: true. This way I will read my data like this
let samples = sampleData.withUnsafeBytes {
UnsafeBufferPointer<Float>(start: $0,
count: sampleData.count / MemoryLayout<Float>.size)
}

Answer 4

for: Xcode 8.3.3 • Swift 3.1
extension Collection where Iterator.Element == Int16 {
var floatArray: [Float] {
return flatMap{ Float($0) }
}
}
usage:
let int16Array: [Int16] = [1, 2, 3 ,4]
let floatArray = int16Array.floatArray