I'm currently working on a iOS-application, which should be able to detect the localization. I've created an tflite which comprises some CNN layers. In order to use the tflite in XCode/Swift I've created a helper class in which the tflite calculates the output. Whenever I run the predict-function once, it works. But apparently the predict function doesn't work in real-time camera-thread.
After about 7 seconds, XCode is throwing the following error:
Thread 1: EXC_BAD_ACCESS (code=1, address=0x123424001). This error must be evoken by looping through the image. Since I need each pixel value I'm using the solution suggested by Firebase.But apparently this solution is not waterproofed. Can anybody help to resolve this memory issue?
func creatInputForCNN(resizedImage: UIImage?) -> Data{
// In this section of the code I loop through the image (150,200,3)
// in order to fetch each pixel value (RGB).
let image: CGImage = resizedImage.cgImage!
guard let context = CGContext(
data: nil,
width: image.width, height: image.height,
bitsPerComponent: 8, bytesPerRow: image.width * 4,
space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue
) else {return nil}
context.draw(image, in: CGRect(x: 0, y: 0, width: image.width, height: image.height))
guard let imageData = context.data else {return nil}
let size_w = 150
let size_H = 200
var inputData:Data?
inputData = Data()
for row in 0 ..< size_H{
for col in 0 ..< size_w {
let offset = 4 * (row * context.width + col)
// (Ignore offset 0, the unused alpha channel)
let red = imageData.load(fromByteOffset: offset+1, as: UInt8.self)
let green = imageData.load(fromByteOffset: offset+2, as: UInt8.self)
let blue = imageData.load(fromByteOffset: offset+3, as: UInt8.self)
// Normalize channel values to [0.0, 1.0]. This requirement varies
// by model. For example, some models might require values to be
// normalized to the range [-1.0, 1.0] instead, and others might
// require fixed-point values or the original bytes.
var normalizedRed:Float32 = Float32(red) / 255
var normalizedGreen:Float32 = (Float32(green) / 255
var normalizedBlue:Float32 = Float32(blue) / 255
// Append normalized values to Data object in RGB order.
let elementSize = MemoryLayout.size(ofValue: normalizedRed)
var bytes = [UInt8](repeating: 0, count: elementSize)
memcpy(&bytes, &normalizedRed, elementSize)
inputData!.append(&bytes, count: elementSize)
memcpy(&bytes, &normalizedGreen, elementSize)
inputData!.append(&bytes, count: elementSize)
memcpy(&bytes, &normalizedBlue, elementSize)
inputData!.append(&bytes, count: elementSize)
return inputData
}
This is the code
I'm trying to render a point cloud using SceneKit (ultimately for ARKit). However, I'm finding the pointSize (and minimumPointScreenSpaceRadius) properties are ignored when trying to modify the SCNGeometryElement. There was this solution to call on the underlying OpenGL property, but this seems to work only in simulation.
No matter what pointSize I put in, the points are the same size. This is roughly how my code is laid out. I'm wondering if I'm doing something wrong with my GeometryElement setup?
var index: Int = 0
for x_pos in stride(from: start , to: end, by: increment) {
for y_pos in stride(from: start , to: end, by: increment) {
for z_pos in stride(from: start , to: end, by: increment) {
let pos:SCNVector3 = SCNVector3(x: x_pos, y: y_pos, z: z_pos)
positions.append(pos)
normals.append(normal)
indices.append(index)
index = index + 1
}
}
}
let pt_positions : SCNGeometrySource = SCNGeometrySource.init(vertices: positions)
let n_positions: SCNGeometrySource = SCNGeometrySource.init(normals: normals)
let pointer = UnsafeRawPointer(indices)
let indexData = NSData(bytes: pointer, length: MemoryLayout<Int32>.size * indices.count)
let elements = SCNGeometryElement(data: indexData as Data, primitiveType: .point, primitiveCount: positions.count, bytesPerIndex: MemoryLayout<Int>.size)
elements.pointSize = 10.0 //being ignored
elements.minimumPointScreenSpaceRadius = 10 //also being ignored
let pointCloud = SCNGeometry (sources: [pt_positions, n_positions], elements: [elements])
pointCloud.firstMaterial?.lightingModel = SCNMaterial.LightingModel.constant
let ptNode = SCNNode(geometry: pointCloud)
scene.rootNode.addChildNode(ptNode)
you should also specify both element.minimumPointScreenSpaceRadius and element.maximumPointScreenSpaceRadius. if so your point size work! :)
I'm trying to get the SceneKit Physics Vehicle demonstration from WWDC working using Swift 2 and also the Scene Editor and .scnassets. I'm adapting this port of the demo to Swift 1.
I've also been experimenting with some of the Model I/O functions, such as SkyBox and normal map generation.
Everything works fine, except that the wheels are rotated 180 degrees around their steering axes, so that the hubcaps point in towards the centre of the car:
Hubcaps pointing inwards
The wheels spin in the right direction, and the car can be driven and steered fine, but it just looks very weird (and the left-right wheelbase is narrower than it should be).
If I comment out the SCNPhysicsVehicle and SCNPhysicsVehicleWheel code, so that it's just geometry with a physics body for the chassis, the geometry displays correctly, with the wheel hubs pointing outward:
Hubcaps pointing outwards
I've tried rotating the wheelnodes using wheelnode0.rotation before defining the SCNPhysicsVehicleWheel, but it has no effect.
I tried reversing the wheel axles with wheel0.axle = SCNVector3(x: 1,y: 0,z: 0) and that makes the wheels face the right way, hubcap outwards, (and they still spin the right way), but the car moves backwards (ie the opposite direction to how the wheels are spinning).
It feels like something weird is going on with how axes are being translated, but I can't figure what it is. I've tried it with the original .dae model from the Apple demo, as well as converted to an .scn file. I've tried it inside and outside an .scnassets folder (in case it was something to do with scnassets auto correction of up axis). I tried inverting the wheel steering axes, but that just resulted in the wheels pointing up in the air.
Here's my vehicle creation function, with all the stuff I've tried commented out. Full repo is here: https://github.com/Utsira/SCNPhysicsVehicle-test. Can anyone see what's going wrong here? Thanks in advance.
func setupCar() -> SCNNode {
let carScene = SCNScene(named: "art.scnassets/rc_car.scn") //rc_car.dae
let chassisNode = carScene!.rootNode.childNodeWithName("rccarBody", recursively: true)!
chassisNode.position = SCNVector3Make(0, 10, 30)
//chassisNode.rotation = SCNVector4(0, 1, 0, CGFloat(M_PI))
let body = SCNPhysicsBody.dynamicBody()
body.allowsResting = false
body.mass = 80
body.restitution = 0.1
body.friction = 0.5
body.rollingFriction = 0
chassisNode.physicsBody = body
scnScene.rootNode.addChildNode(chassisNode)
//getNode("rccarBody", fromDaePath: "rc_car.dae")
let wheelnode0 = chassisNode
.childNodeWithName("wheelLocator_FL", recursively: true)!
let wheelnode1 = chassisNode
.childNodeWithName("wheelLocator_FR", recursively: true)!
let wheelnode2 = chassisNode
.childNodeWithName("wheelLocator_RL", recursively: true)!
let wheelnode3 = chassisNode
.childNodeWithName("wheelLocator_RR", recursively: true)!
//wheelnode0.geometry!.firstMaterial!.emission.contents = UIColor.blueColor()
// SCNTransaction.begin()
// wheelnode0.rotation = SCNVector4(x: 0, y: 1, z: 0, w: Float(M_PI)) //CGFloat(M_PI)
// wheelnode1.rotation = SCNVector4(x: 0, y: 1, z: 0, w: Float(M_PI))
// wheelnode2.rotation = SCNVector4(x: 0, y: 1, z: 0, w: Float(M_PI))
// wheelnode3.rotation = SCNVector4(x: 0, y: 1, z: 0, w: Float(M_PI))
// SCNTransaction.commit()
// wheelnode0.eulerAngles = SCNVector3Make(0, Float(M_PI), 0 )
let wheel0 = SCNPhysicsVehicleWheel(node: wheelnode0)
let wheel1 = SCNPhysicsVehicleWheel(node: wheelnode1)
let wheel2 = SCNPhysicsVehicleWheel(node: wheelnode2)
let wheel3 = SCNPhysicsVehicleWheel(node: wheelnode3)
// wheel0.steeringAxis = SCNVector3Make(0, -1, 1) //wheels point up in the air with 0,1,0
// wheel1.steeringAxis = SCNVector3Make(0, -1, 1)
// wheel2.steeringAxis = SCNVector3Make(0, -1, 1)
// wheel3.steeringAxis = SCNVector3Make(0, -1, 1)
//
// wheel0.axle = SCNVector3(x: 1,y: 0,z: 0) //wheels face and spin the right way, but car moves in opposite direction with 1,0,0
// wheel1.axle = SCNVector3(x: 1,y: 0,z: 0)
// wheel2.axle = SCNVector3(x: 1,y: 0,z: 0)
// wheel3.axle = SCNVector3(x: 1,y: 0,z: 0)
// wheel0.steeringAxis = SCNVector3()
// var min = SCNVector3(x: 0, y: 0, z: 0)
// var max = SCNVector3(x: 0, y: 0, z: 0)
// wheelnode0.getBoundingBoxMin(&min, max: &max)
// let wheelHalfWidth = Float(0.5 * (max.x - min.x))
// var w0 = wheelnode0.convertPosition(SCNVector3Zero, toNode: chassisNode)
// w0 = w0 + SCNVector3Make(wheelHalfWidth, 0, 0)
// wheel0.connectionPosition = w0
// var w1 = wheelnode1.convertPosition(SCNVector3Zero, toNode: chassisNode)
// w1 = w1 - SCNVector3Make(wheelHalfWidth, 0, 0)
// wheel1.connectionPosition = w1
// var w2 = wheelnode2.convertPosition(SCNVector3Zero, toNode: chassisNode)
// w2 = w2 + SCNVector3Make(wheelHalfWidth, 0, 0)
// wheel2.connectionPosition = w2
// var w3 = wheelnode3.convertPosition(SCNVector3Zero, toNode: chassisNode)
// w3 = w3 - SCNVector3Make(wheelHalfWidth, 0, 0)
// wheel3.connectionPosition = w3
vehicle = SCNPhysicsVehicle(chassisBody: chassisNode.physicsBody!,
wheels: [wheel0, wheel1, wheel2, wheel3])
scnScene.physicsWorld.addBehavior(vehicle)
return chassisNode
}
I Found A Solution:
(but not the solution I hoped for)
My fix was to adjust the connectionPosition of "FrontLeftWheel", so much to the right, that it took the role of "FrontRightWheel".
And "FrontRightWheel", so much to the left, that it took the role of "FrontLeftWheel".
I also did this for the back.
Basically, where you have:
SCNVector3Make(wheelHalfWidth, 0, 0)
I replaced it with:
SCNVector3Make(wheelHalfWidth * 5.2, 0, 0)
However, you may have to fool around with the number "5.2" to match up with your COLLADA (.dae).
A Note:
Do Not forget to replace:
vehicle = SCNPhysicsVehicle(chassisBody: chassisNode.physicsBody!,
wheels: [wheel0, wheel1, wheel2, wheel3])
With:
vehicle = SCNPhysicsVehicle(chassisBody: chassisNode.physicsBody!,
wheels: [wheel1, wheel0, wheel3, wheel2])
This way, when you use car.wheels[0].You get the wheel you are expecting
I think your problem is that you are not setting the connection positions for each wheel. Apple's default ObjectiveC code is tricky to translate to swift.
wheel0.connectionPosition = SCNVector3FromFloat3(SCNVector3ToFloat3([wheel0Node convertPosition:SCNVector3Zero toNode:chassisNode]) + (vector_float3){wheelHalfWidth, 0.0, 0.0});
wheel1.connectionPosition = SCNVector3FromFloat3(SCNVector3ToFloat3([wheel1Node convertPosition:SCNVector3Zero toNode:chassisNode]) - (vector_float3){wheelHalfWidth, 0.0, 0.0});
wheel2.connectionPosition = SCNVector3FromFloat3(SCNVector3ToFloat3([wheel2Node convertPosition:SCNVector3Zero toNode:chassisNode]) + (vector_float3){wheelHalfWidth, 0.0, 0.0});
wheel3.connectionPosition = SCNVector3FromFloat3(SCNVector3ToFloat3([wheel3Node convertPosition:SCNVector3Zero toNode:chassisNode]) - (vector_float3){wheelHalfWidth, 0.0, 0.0});
These four lines translate (ignoring the repetition)
let wheel0Position = wheel0Node.convertPosition(SCNVector3Zero, to: chassisNode)
let vector0Float3 = vector_float3(wheel0Position.x, wheel0Position.y, wheel0Position.z) + vector_float3(wheelHalfWidth, 0,0)
wheel0.connectionPosition = SCNVector3(vector0Float3.x, vector0Float3.y, vector0Float3.z)
let wheel1Position = wheel1Node.convertPosition(SCNVector3Zero, to: chassisNode)
let vector1Float3 = vector_float3(wheel1Position.x, wheel1Position.y, wheel1Position.z) - vector_float3(wheelHalfWidth, 0,0)
wheel1.connectionPosition = SCNVector3(vector1Float3.x, vector1Float3.y, vector1Float3.z)
let wheel2Position = wheel2Node.convertPosition(SCNVector3Zero, to: chassisNode)
let vector2Float3 = vector_float3(wheel2Position.x, wheel2Position.y, wheel2Position.z) + vector_float3(wheelHalfWidth, 0,0)
wheel2.connectionPosition = SCNVector3(vector2Float3.x, vector2Float3.y, vector2Float3.z)
let wheel3Position = wheel3Node.convertPosition(SCNVector3Zero, to: chassisNode)
let vector3Float3 = vector_float3(wheel3Position.x, wheel3Position.y, wheel3Position.z) - vector_float3(wheelHalfWidth, 0,0)
wheel3.connectionPosition = SCNVector3(vector3Float3.x, vector3Float3.y, vector3Float3.z)
By adding or substracting vector_float3, the wheels are positioned in the right place.
Also, bear in mind that you can't scale the node in the scene or weird things may happen.
Encountered the same issue, my solution was to rotate all wheel nodes so the Y axis pointed down. I figured gravity is a downward force.
I'm new to swift and am trying to use the Accelerate framework to multiply two matrices.
However I cannot get this to work. Any help would be appreciated. Code is below:
import Accelerate
let firstMatrix :[[Float]] = [[1,2],[2,3]]
let secondMatrix : [[Float]] = [[3,2],[1,4]]
var answerMatrix :[[Float]] = [[0,0],[0,0]]
cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans, 2, 2, 2, 1.0, firstMatrix, 2, secondMatrix, 2, 0.0, &answerMatrix, 2)
print(answerMatrix)
There are two problems:
cblas_dgemm() is for "double precision matrices", i.e. the elements have the Double type. For Float (single-precision) matrices use cblas_sgemm().
Each matrix must be passed to the function as a pointer to a one-dimensional array.
Example:
let firstMatrix : [Double] = [1,2 , 2,3]
let secondMatrix : [Double] = [3,2 , 1,4]
var answerMatrix : [Double] = [0,0 , 0,0]
cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans, 2, 2, 2, 1.0,
firstMatrix, 2, secondMatrix, 2, 0.0, &answerMatrix, 2)
print(answerMatrix)
// [5.0, 10.0, 9.0, 16.0]
Martin R's answer addresses your actual direct question. I want to note, though, that if you're really just working with 2x2 matrices (and not arbitrarily large matrices), you may find it easier to use the simd module:
import simd
let firstMatrix = float2x2(rows: [[1,2],[2,3]])
let secondMatrix = float2x2(rows: [[3,2],[1,4]])
let answerMatrix = firstMatrix * secondMatrix
I'm trying figure out what size to send "realloc" when I call it through Swift. It seems that I have to add an extra byte, but I don't understand why.
typealias Floats = UnsafeMutablePointer<Float>
let FLOAT_SIZE = sizeof( Float )
func floats_realloc( floats:Floats, qty_of_floats:Int ) -> Floats {
let KLUDGE = 1 // Why?
let qty_of_bytes = ( qty_of_floats * FLOAT_SIZE ) + KLUDGE
let realloced_floats = Floats(
realloc( floats, UInt( qty_of_bytes ) )
)
return realloced_floats
}
If I set KLUDGE to 0 here, this is what happens when I attempt to make room for one new member in a three member array:
In: [0.9, 0.9, 0.9]
Out: [0.0, 0.0, 0.9, 0.0]
What I would expect is:
Out: [0.9, 0.9, 0.9, 0.0]
The arrays I'm sending it are created within Swift, using
var foo_floats = Floats.alloc(QTY_OF_FLOATS)
What's wrong with my call to realloc?
I've discussed this on Apple's Developer Forum. It turns out that using Swift's alloc allots space for a Swift array, not a C array. So if you want to use MutableUnsafePointer for a C array and "realloc" (bridged from C) you need to stick with C functions, like "malloc" (bridged from C).
By adding the following function, and using it when I initially set up my Floats array, the "realloc" bug went away:
func floats_alloc( qty_of_floats:Int ) -> Floats {
// return Floats.alloc( qty_of_floats )
let qty_of_bytes = ( qty_of_floats * FLOAT_SIZE )
let alloced_floats = Floats(
malloc( UInt( qty_of_bytes ) )
)
return alloced_floats
}
I've tested for a couple weeks now, and all is well.