I launch code in this instruction: https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_swift
Everything works, but I don't understand how to get the prediction values. I tried: print(outputTensor but got:
Tensor(name: "Identity", dataType: TensorFlowLite.Tensor.DataType.float32, shape: TensorFlowLite.Tensor.Shape(rank: 2, dimensions: [1, 3]), data: 12 bytes, quantizationParameters: nil)
Did you try this part from the example guide
// Copy output to `Data` to process the inference results.
let outputSize = outputTensor.shape.dimensions.reduce(1, {x, y in x * y})
let outputData =
UnsafeMutableBufferPointer<Float32>.allocate(capacity: outputSize)
outputTensor.data.copyBytes(to: outputData)
Then just print outputData buffer
Related
I am writing some Linear Algebra algorithms using Apples Swift / Accelerate framework. All works and the solved Ax = b equations produce the right results (this code is from the apple examples).
I would like to be able to extract the LLT factorisation from the
SparseOpaqueFactorization_Double
object. But there doesn't seem to be any way to extract (to print) the factorisation. Does anyone know of a way of extracting the factorised matrix from the SparseOpaqueFactorization_Double object?
import Foundation
import Accelerate
print("Hello, World!")
// Example of a symmetric sparse matrix, empty cells represent zeros.
var rowIndices: [Int32] = [0, 1, 3, // Column 0
1, 2, 3, // Column 1
2, // col 2
3] // Col 3
// note that the Matrix representation is the upper triangular
// here. Since the matrix is symmetric, no need to store the lower
// triangular.
var values: [Double] = [10.0, 1.0 , 2.5, // Column 0
12.0, -0.3, 1.1, // Column 1
9.5, // Col 2
6.0 ] // Column 3
var columnStarts = [0, // Column 0
3, // Column 1
6, 7, // Column 2
8] // col 3
var attributes = SparseAttributes_t()
attributes.triangle = SparseLowerTriangle
attributes.kind = SparseSymmetric
let structure = SparseMatrixStructure(rowCount: 4,
columnCount: 4,
columnStarts: &columnStarts,
rowIndices: &rowIndices,
attributes: attributes,
blockSize: 1)
let llt: SparseOpaqueFactorization_Double = values.withUnsafeMutableBufferPointer { valuesPtr in
let a = SparseMatrix_Double(
structure: structure,
data: valuesPtr.baseAddress!
)
return SparseFactor(SparseFactorizationCholesky, a)
}
var bValues = [ 2.20, 2.85, 2.79, 2.87 ]
var xValues = [ 0.00, 0.00, 0.00, 0.00 ]
bValues.withUnsafeMutableBufferPointer { bPtr in
xValues.withUnsafeMutableBufferPointer { xPtr in
let b = DenseVector_Double(
count: 4,
data: bPtr.baseAddress!
)
let x = DenseVector_Double(
count: 4,
data: xPtr.baseAddress!
)
SparseSolve(llt, b, x)
}
}
for val in xValues {
print("x = " + String(format: "%.2f", val), terminator: " ")
}
print("")
print("Success")
OK so after much sleuthing around the apple swift headers, I have solved this problem.
There is an Accelerate API call called
public func SparseCreateSubfactor(_ subfactor: SparseSubfactor_t, _ Factor: SparseOpaqueFactorization_Double) -> SparseOpaqueSubfactor_Double
which returns this SparceOpaqueSubfactor_ type. This can be used in a matrix multiplication to produce a "transparent" result (i.e. a matrix you can use/print/see). So I multiplied the SubFactor for the Lower triangular part of the Cholesky factorisation by the Identity matrix to extract the factors. Works a treat!
let subfactors = SparseCreateSubfactor(SparseSubfactorL, llt)
var identValues = generateIdentity(n)
ppm(identValues)
let sparseAs = SparseAttributes_t(transpose: false,
triangle: SparseUpperTriangle,
kind: SparseOrdinary,
_reserved: 0,
_allocatedBySparse: false)
let identity_m = DenseMatrix_Double(rowCount: Int32(n),
columnCount: Int32(n),
columnStride: Int32(n),
attributes: sparseAs,
data: &identValues)
SparseMultiply(subfactors, identity_m) // Output is in identity_m after the call
I wrote a small function to generate an identity matrix which I've used in the code above:
func generateIdentity(_ dimension: Int) -> [Double] {
var iden = Array<Double>()
for i in 0...dimension - 1 {
for j in 0...dimension - 1 {
if i == j {
iden.append(1.0)
} else {
iden.append(0.0)
}
}
}
return iden
}
I converted my mlmodel from tf.keras. The goal is to recognize handwritten text from the image
When I run it using this code:
func performCoreMLImageRecognition(_ image: UIImage) {
let model = try! HTRModel()
// process input image
let scale = image.scaledImage(200)
let sized = scale?.resize(size: CGSize(width: 200, height: 50))
let gray = sized?.rgb2GrayScale()
guard let pixelBuffer = sized?.pixelBufferGray(width: 200, height: 50) else { fatalError("Cannot convert image to pixelBufferGray")}
UIImageWriteToSavedPhotosAlbum(gray! ,
self,
#selector(self.didFinishSavingImage(_:didFinishSavingWithError:contextInfo:)),
nil)
let mlArray = try! MLMultiArray(shape: [1, 1], dataType: MLMultiArrayDataType.float32)
let htrinput = HTRInput(image: pixelBuffer, label: mlArray)
if let prediction = try? model.prediction(input: htrinput) {
print(prediction)
}
}
I get the following error:
[espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Invalid argument": generic_reshape_kernel: Invalid bottom shape (64 12 1 1 1) for reshape to (768 50 -1 1 1) status=-6
2021-01-21 20:23:50.712585+0900 Guided Camera[7575:1794819] [coreml] Error computing NN outputs -6
2021-01-21 20:23:50.712611+0900 Guided Camera[7575:1794819]
[coreml] Failure in -executePlan:error:.
Here is the model configuration
The model ran perfectly fine. Where am I going wrong in this. I am not well versed with swift and need help.
What does this error mean and How do I resolve this error?
Sometimes during the conversion from Keras (or whatever) to Core ML, the converter doesn't understand how to handle certain operations, which results in a model that doesn't work.
In your case, there is a layer that outputs a tensor with shape (64, 12, 1, 1, 1) while there is a reshape layer that expects something that can be reshaped to (768, 50, -1, 1, 1).
You'll need to find out which layer does this reshape and then examine the Core ML model why it gets an input tensor that is not the correct size. Just because it works OK in Keras does not mean the conversion to Core ML was flawless.
You can examine the Core ML model with Netron, an open source model viewer.
(Note that 64x12 = 768, so the issue appears to be with the 50 in that tensor.)
I need to perform simple math operation on Data that contains RGB pixels data. Currently Im doing this like so:
let imageMean: Float = 127.5
let imageStd: Float = 127.5
let rgbData: Data // Some data containing RGB pixels
let floats = (0..<rgbData.count).map {
(Float(rgbData[$0]) - imageMean) / imageStd
}
return Data(bytes: floats, count: floats.count * MemoryLayout<Float>.size)
This works, but it's too slow. I was hoping I could use the Accelerate framework to calculate this faster, but have no idea how to do this. I reserved some space so that it's not allocated every time this function starts, like so:
inputBufferDataNormalized = malloc(width * height * 3) // 3 channels RGB
I tried few functions, like vDSP_vasm, but I couldn't make it work. Can someone direct me to how to use it? Basically I need to replace this map function, because it takes too long time. And probably it would be great to use pre-allocated space all the time.
Following up on my comment on your other related question. You can use SIMD to parallelize the operation, but you'd need to split the original array into chunks.
This is a simplified example that assumes that the array is exactly divisible by 64, for example, an array of 1024 elements:
let arr: [Float] = (0 ..< 1024).map { _ in Float.random(in: 0...1) }
let imageMean: Float = 127.5
let imageStd: Float = 127.5
var chunks = [SIMD64<Float>]()
chunks.reserveCapacity(arr.count / 64)
for i in stride(from: 0, to: arr.count, by: 64) {
let v = SIMD64.init(arr[i ..< i+64])
chunks.append((v - imageMean) / imageStd) // same calculation using SIMD
}
You can now access each chunk with a subscript:
var results: [Float] = []
results.reserveCapacity(arr.count)
for chunk in chunks {
for i in chunk.indices {
results.append(chunk[i])
}
}
Of course, you'd need to deal with a remainder if the array isn't exactly divisible by 64.
I have found a way to do this using Accelerate. First I reserve space for converted buffer like so
var inputBufferDataRawFloat = [Float](repeating: 0, count: width * height * 3)
Then I can use it like so:
let rawBytes = [UInt8](rgbData)
vDSP_vfltu8(rawBytes, 1, &inputBufferDataRawFloat, 1, vDSP_Length(rawBytes.count))
vDSP.add(inputBufferDataRawScalars.mean, inputBufferDataRawFloat, result: &inputBufferDataRawFloat)
vDSP.multiply(inputBufferDataRawScalars.std, inputBufferDataRawFloat, result: &inputBufferDataRawFloat)
return Data(bytes: inputBufferDataRawFloat, count: inputBufferDataRawFloat.count * MemoryLayout<Float>.size)
Works very fast. Maybe there is better function in Accelerate, if anyone know of it, please let me know. It need to perform function (A[n] + B) * C (or to be exact (A[n] - B) / C but the first one could be converted to this).
I'm using vDSP_conv to perform autocorrelation. Mostly it works just fine but every so often it's filling the output array with NaNs.
The code:
func corr_test() {
var pass = 0
var x = [Float]()
for i in 0..<2000 {
x.append(Float(i))
}
while true {
print("pass \(pass)")
let corr = autocorr(x)
if corr[1].isNaN {
print("!!!")
}
pass += 1
}
}
func autocorr(a: [Float]) -> [Float] {
let resultLen = a.count * 2 + 1
let padding = [Float].init(count: a.count, repeatedValue: 0.0)
let a_pad = padding + a + padding
var result = [Float].init(count: resultLen, repeatedValue: 0.0)
vDSP_conv(a_pad, 1, a_pad, 1, &result, 1, UInt(resultLen), UInt(a_pad.count))
return result
}
The output:
pass ...
pass 169
pass 170
pass 171
(lldb) p corr
([Float]) $R0 = 4001 values {
[0] = 2.66466637E+9
[1] = NaN
[2] = NaN
[3] = NaN
[4] = NaN
...
I'm not sure what's going on here. I think I'm handling the 0 padding correctly since if I weren't I don't think I'd be getting correct results 99% of the time.
Ideas? Gracias.
Figured it out. The key was this comment from https://developer.apple.com/library/mac/samplecode/vDSPExamples/Listings/DemonstrateConvolution_c.html :
// “The signal length is padded a bit. This length is not actually passed to the vDSP_conv routine; it is the number of elements
// that the signal array must contain. The SignalLength defined below is used to allocate space, and it is the filter length
// rounded up to a multiple of four elements and added to the result length. The extra elements give the vDSP_conv routine
// leeway to perform vector-load instructions, which load multiple elements even if they are not all used. If the caller did not
// guarantee that memory beyond the values used in the signal array were accessible, a memory access violation might result.”
“Padded a bit.” Thanks for being so specific. Anyway here's the final working product:
func autocorr(a: [Float]) -> [Float] {
let filterLen = a.count
let resultLen = filterLen * 2 - 1
let signalLen = ((filterLen + 3) & 0xFFFFFFFC) + resultLen
let padding1 = [Float].init(count: a.count - 1, repeatedValue: 0.0)
let padding2 = [Float].init(count: (signalLen - padding1.count - a.count), repeatedValue: 0.0)
let signal = padding1 + a + padding2
var result = [Float].init(count: resultLen, repeatedValue: 0.0)
vDSP_conv(signal, 1, a, 1, &result, 1, UInt(resultLen), UInt(filterLen))
// Remove the first n-1 values which are just mirrored from the end so that [0] always has the autocorrelation.
result.removeFirst(filterLen - 1)
return result
}
Note that the results here aren't normalized.
My first question here at Stackoverflow... hope my question is specific enough.
I have an array in Swift with measurements at certain dates. Like:
var myArray:[(day: Int, mW: Double)] = []
myArray.append(day:0, mW: 31.98)
myArray.append(day:1, mW: 31.89)
myArray.append(day:2, mW: 31.77)
myArray.append(day:4, mW: 31.58)
myArray.append(day:6, mW: 31.46)
Some days are missing, I just didn't take a measurement... All measurements should be on a line, more or less. So I thought about linear regression. I found the Accelerate framework, but the documentation is missing and I can't find examples.
For the missing measurements I would like to have a function, with as input a missing day and as output a best guess, based on the other measurements.
func bG(day: Int) -> Double {
return // return best guess for measurement
}
Thanks for helping out.
Jan
My answer doesn't specifically talk about the Accelerate Framework, however I thought the question was interesting and thought I'd give it a stab. From what I gather you're basically looking to create a line of best fit and interpolate or extrapolate more values of mW from that. To do that I used the Least Square Method, detailed here: http://hotmath.com/hotmath_help/topics/line-of-best-fit.html and implemented this in Playgrounds using Swift:
// The typealias allows us to use '$X.day' and '$X.mW',
// instead of '$X.0' and '$X.1' in the following closures.
typealias PointTuple = (day: Double, mW: Double)
// The days are the values on the x-axis.
// mW is the value on the y-axis.
let points: [PointTuple] = [(0.0, 31.98),
(1.0, 31.89),
(2.0, 31.77),
(4.0, 31.58),
(6.0, 31.46)]
// When using reduce, $0 is the current total.
let meanDays = points.reduce(0) { $0 + $1.day } / Double(points.count)
let meanMW = points.reduce(0) { $0 + $1.mW } / Double(points.count)
let a = points.reduce(0) { $0 + ($1.day - meanDays) * ($1.mW - meanMW) }
let b = points.reduce(0) { $0 + pow($1.day - meanDays, 2) }
// The equation of a straight line is: y = mx + c
// Where m is the gradient and c is the y intercept.
let m = a / b
let c = meanMW - m * meanDays
In the code above a and b refer to the following formula from the website:
a:
b:
Now you can create the function which uses the line of best fit to interpolate/extrapolate mW:
func bG(day: Double) -> Double {
return m * day + c
}
And use it like so:
bG(3) // 31.70
bG(5) // 31.52
bG(7) // 31.35
If you want to do fast linear regressions in Swift, I suggest using the Upsurge framework. It provides a number of simple functions that wrap the Accelerate library and so you get the benefits of SIMD on either iOS or OSX
without having to worry about the complexity of vDSP calls.
To do a linear regression with base Upsurge functions is simply:
let meanx = mean(x)
let meany = mean(y)
let meanxy = mean(x * y)
let meanx_sqr = measq(x)
let slope = (meanx * meany - meanxy) / (meanx * meanx - meanx_sqr)
let intercept = meany - slope * meanx
This is essentially what is implemented in the linregress function.
You can use it with an array of [Double], other classes such as RealArray (comes with Upsurge) or your own objects if they can expose contiguous memory.
So a script to meet your needs would look like:
#!/usr/bin/env cato
import Upsurge
typealias PointTuple = (day: Double, mW:Double)
var myArray:[PointTuple] = []
myArray.append((0, 31.98))
myArray.append((1, 31.89))
myArray.append((2, 31.77))
myArray.append((4, 31.58))
myArray.append((6, 31.46))
let x = myArray.map { $0.day }
let y = myArray.map { $0.mW }
let (slope, intercept) = Upsurge.linregress(x, y)
func bG(day: Double) -> Double {
return slope * day + intercept
}
(I left in the appends rather than using literals as you are likely programmatically adding to your array if it is of significant length)
and full disclaimer: I contributed the linregress code. I hope to also add the co-efficient of determination at some point in the future.
To estimate the values between different points, you can also use SKKeyframeSequence from SpriteKit
https://developer.apple.com/documentation/spritekit/skinterpolationmode/spline
import SpriteKit
let sequence = SKKeyframeSequence(keyframeValues: [0, 20, 40, 60, 80, 100], times: [64, 128, 256, 512, 1024, 2048])
sequence.interpolationMode = .spline // .linear, .step
let estimatedValue = sequence.sample(atTime: CGFloat(1500)) as! Double // 1500 is the value you want to estimate
print(estimatedValue)