How can I get the points in face mesh like eye ,eyebrow , lip, mouth ,nose using ARFaceTracking in Swift 4.2? - arkit

Currently i am getting the left and right eye points, How can i get the other parts points using ARFaceTracking or other framework in swift 4 in ios.
please give the feedback above the questions?

You can use the ARFaceGeometry vertices. It’s a magic number. The ARFaceGeometry has 1220 vertices in it and index 9 is on the nose. This works.
let vertices = [anchor.geometry.vertices[9]] // nose
// You can use Features Indexes with array
let features = ["nose", "leftEye", "rightEye", "mouth", "hat"]
let featureIndices = [[9], [1064], [42], [24, 25], [20]]
Here features is an array of the node names you gave to each feature and featureIndices are the vertex indexes in the ARFaceGeometry that correspond to those features. ARFaceAnchor property is used.

Here's some magic numbers for you:
let mouthTopLeft = Array(250...256)
let mouthTopCenter = [24]
let mouthTopRight = Array(685...691).reversed()
let mouthRight = [684]
let mouthBottomRight = [682, 683,700,709,710,725]
let mouthBottomCenter = [25]
let mouthBottomLeft = [265,274,290,275,247,248]
let mouthLeft = [249]
let mouthClockwise : [Int] = mouthLeft +
mouthTopLeft + mouthTopCenter +
mouthTopRight + mouthRight +
mouthBottomRight + mouthBottomCenter +
mouthBottomLeft
let eyeTopLeft = Array(1090...1101)
let eyeBottomLeft = Array(1102...1108) + Array(1085...1089)
let eyeTopRight = Array(1069...1080)
let eyeBottomRight = Array(1081...1084) + Array(1061...1068)

Related

How to combine two ClosedRange in Swift?

I am learning Quick Sort in swift and need to compose a complicated array.
Here is the code:
var arrayOne = 1...500
var arrayTwo = 501...1000
var array_one = arrayOne.reversed()
var array_two = arrayTwo.reversed()
var array = arrayOne + arrayTwo
I want to combine arrayOne + arrayTwo to array.
I can not use the + operator, Xcode tips me
Binary operator '+' cannot be applied to two
'CountableClosedRange' operands
I know how to get it by using for loops.
Elegant way is really needed. Such as Higher order function.
1...500 is a range and (1...500).reversed() is a collection. Both are sequences so that you can append them to an array:
let rangeOne = 1...500
let rangeTwo = 501...1000
let array = [] + rangeOne.reversed() + rangeTwo.reversed()
// [500, 499, ..., 2, 1, 1000, 999, ..., 502, 501]
Alternative solutions are:
let array = Array(rangeOne.reversed()) + rangeTwo.reversed()
let array = Array([rangeOne.reversed(), rangeTwo.reversed()].joined())
let array = Array(rangeOne.reversed()) + Array(rangeTwo.reversed())
let array = [rangeOne.reversed(), rangeTwo.reversed()].flatMap { $0 }

How to use SceneKit vortex field to create a tornato effect

In the SceneKit WWDC 2014, they have an example of a vortex field with this effect:
The particle system looks much like a tornato, as it spins inward with a hollow center.
However, the documentation for vortex fields have no information on how to achieve this effect. Right now, I have this:
// create the particle system
let exp = SCNParticleSystem()
exp.loops = true
exp.particleMass = 5
exp.birthRate = 10000
exp.emissionDuration = 10
exp.emitterShape = SCNTorus(ringRadius: 5, pipeRadius: 1)
exp.particleLifeSpan = 15
exp.particleVelocity = 2
exp.particleColor = UIColor.white
exp.isAffectedByPhysicsFields = true
scene.addParticleSystem(exp, transform: SCNMatrix4MakeRotation(0, 0, 0, 0))
// create the field
let field = SCNPhysicsField.vortex()
field.strength = -5
field.direction = SCNVector3(x: 0, y: 1, z: 0)
let fieldNode = SCNNode()
fieldNode.physicsField = field
scene.rootNode.addChildNode(fieldNode)
This creates this effect:
Where I am looking down at the particles rotating clockwise with a really big radius outwards. It looks nothing like a tornato effect. How can I create this effect?
You say tornato, I say tornado, let’s call the whole thing off...
The SceneKit WWDC 2014 demo/slides is a sample code project, so you can see for yourself how they made any of the effects you see therein. In this case, it looks like the “vortex” demo isn’t actually using the vortexField API, but instead the custom field API that lets you supply your own math in an evaluator block. (See the link for the code in that block.)
You might be able to get similar behavior without a custom field by combining a vortex (causes rotation only) with radial gravity (attracts inward) with linear gravity (attracts downward), or some other combination (possibly something involving electric charge). But you’d probably have to experiment with tweaking the parameters quite a bit.
If anyone is still interested in this topic - here is a Swift 5 implementation of that legendary tornado effect.
Here is an example function that will create your tornado.
func addTornadoPhysicsField() {
// Tornado Particles Field Example
guard let tornadoSystem = SCNParticleSystem(named: "tornado.scnp", inDirectory: nil) else { return }
let emitterGeometry = SCNTorus(ringRadius: 1.0, pipeRadius: 0.2)
emitterGeometry.firstMaterial?.transparency = 0.0
let fieldAndParticleNode = SCNNode(geometry: emitterGeometry)
fieldAndParticleNode.position = SCNVector3(0.0, 0.0, -20.0)
tornadoSystem.emitterShape = emitterGeometry
fieldAndParticleNode.addParticleSystem(tornadoSystem)
yourScene.rootNode.addChildNode(fieldAndParticleNode)
// Tornado
let worldOrigin = SCNVector3Make(fieldAndParticleNode.worldTransform.m41,
fieldAndParticleNode.worldTransform.m42,
fieldAndParticleNode.worldTransform.m43)
let worldAxis = simd_float3(0.0, 1.0, 0.0) // i.Ex. the Y axis
// Custom Field (Tornado)
let customVortexField = SCNPhysicsField.customField(evaluationBlock: { position, velocity, mass, charge, time in
let l = simd_float3(worldOrigin.x - position.x, 1.0, worldOrigin.z - position.z)
let t = simd_cross(worldAxis, l)
let d2: Float = l.x * l.x + l.z * l.z
let vs: Float = 27 / sqrt(d2) // diameter, the bigger the value the wider it becomes (Apple Default = 20)
let fy: Float = 1.0 - Float((min(1.0, (position.y / 240.0)))) // rotations, a higher value means more turn arounds (more screwed, Apple Default = 15.0))
return SCNVector3Make(t.x * vs + l.x * 10 * fy, 0, t.z * vs + l.z * 10 * fy)
})
customVortexField.halfExtent = SCNVector3Make(100, 100, 100)
fieldAndParticleNode.physicsField = customVortexField // Attach the Field
}
Additional Configuration Options:
Finally all this can result in something like that:
Note: if you would like to move your static tornado almost like a real tornado, you will have to find a way to re-apply the physics field for each rendererd frame. If you don't, the world origin used in the evaluation block will not move and it will distort your tornado.
Note: You can also split the particle/field node into two different nodes that moves independently from each other. Constrain the field node to the position of the particle node and play around with the influence factor (still need to re-apply the field each frame)
For more information on Custom Fields check out here.

pointSize property in SceneKit being Ignored

I'm trying to render a point cloud using SceneKit (ultimately for ARKit). However, I'm finding the pointSize (and minimumPointScreenSpaceRadius) properties are ignored when trying to modify the SCNGeometryElement. There was this solution to call on the underlying OpenGL property, but this seems to work only in simulation.
No matter what pointSize I put in, the points are the same size. This is roughly how my code is laid out. I'm wondering if I'm doing something wrong with my GeometryElement setup?
var index: Int = 0
for x_pos in stride(from: start , to: end, by: increment) {
for y_pos in stride(from: start , to: end, by: increment) {
for z_pos in stride(from: start , to: end, by: increment) {
let pos:SCNVector3 = SCNVector3(x: x_pos, y: y_pos, z: z_pos)
positions.append(pos)
normals.append(normal)
indices.append(index)
index = index + 1
}
}
}
let pt_positions : SCNGeometrySource = SCNGeometrySource.init(vertices: positions)
let n_positions: SCNGeometrySource = SCNGeometrySource.init(normals: normals)
let pointer = UnsafeRawPointer(indices)
let indexData = NSData(bytes: pointer, length: MemoryLayout<Int32>.size * indices.count)
let elements = SCNGeometryElement(data: indexData as Data, primitiveType: .point, primitiveCount: positions.count, bytesPerIndex: MemoryLayout<Int>.size)
elements.pointSize = 10.0 //being ignored
elements.minimumPointScreenSpaceRadius = 10 //also being ignored
let pointCloud = SCNGeometry (sources: [pt_positions, n_positions], elements: [elements])
pointCloud.firstMaterial?.lightingModel = SCNMaterial.LightingModel.constant
let ptNode = SCNNode(geometry: pointCloud)
scene.rootNode.addChildNode(ptNode)
you should also specify both element.minimumPointScreenSpaceRadius and element.maximumPointScreenSpaceRadius. if so your point size work! :)

Detect overlaping if enumerating nodes

I would like to know how should I detect overlapping nodes while enumerating them? Or how should I make that every random generated position in Y axis is at least some points higher or lower.
This is what I do:
1 - Generate random number between -400 and 400
2 - Add those into array
3 - Enumerate and add nodes to scene with generated positions like this:
var leftPositions = [CGPoint]()
for _ in 0..<randRange(lower: 1, upper: 5){
leftPositions.append(CGPoint(x: -295, y: Helper().randomBetweenTwoNumbers(firstNumber: leftSparkMinimumY, secondNumber: leftSparkMaximumY)))
}
leftPositions.enumerated().forEach { (index, point) in
let leftSparkNode = SKNode()
leftSparkNode.position = point
leftSparkNode.name = "LeftSparks"
let leftSparkTexture = SKTexture(imageNamed: "LeftSpark")
LeftSpark = SKSpriteNode(texture: leftSparkTexture)
LeftSpark.name = "LeftSparks"
LeftSpark.physicsBody = SKPhysicsBody(texture: leftSparkTexture, size: LeftSpark.size)
LeftSpark.physicsBody?.categoryBitMask = PhysicsCatagory.LeftSpark
LeftSpark.physicsBody?.collisionBitMask = PhysicsCatagory.Bird
LeftSpark.physicsBody?.contactTestBitMask = PhysicsCatagory.Bird
LeftSpark.physicsBody?.isDynamic = false
LeftSpark.physicsBody?.affectedByGravity = false
leftSparkNode.addChild(LeftSpark)
addChild(leftSparkNode)
}
But like this sometimes they overlap each other because the generated CGPoint is too close to the previous one.
I am trying to add some amount of triangles to the wall and those triangles are rotated by 90°
To describe in image what I want to achieve:
And I want to avoid thing like this:
Your approach to this is not the best, i would suggest only storing the Y values in your position array and check against those values to make sure your nodes will not overlap. The following will insure no two sparks are within 100 points of each other. You may want to change that value depending on your node's actual height or use case.
Now, obviously if you end up adding too many sparks within an 800 point range, this just will not work and cause an endless loop.
var leftPositions = [Int]()
var yWouldOverlap = false
for _ in 0..<randRange(lower: 1, upper: 5){
//Moved the random number generator to a function
var newY = Int(randY())
//Start a loop based on the yWouldOverlap Bool
repeat{
yWouldOverlap = false
//Nested loop to range from +- 100 from the randomly generated Y
for p in newY - 100...newY + 100{
//If array already contains one of those values
if leftPosition.contains(p){
//Set the loop Bool to true, get a new random value, and break the nested for.
yWouldOverlap = true
newY = Int(randY())
break
}
}
}while(yWouldOverlap)
//If we're here, the array does not contain the new value +- 100, so add it and move on.
leftPositions.append(newY)
}
func randY() -> CGFloat{
return Helper().randomBetweenTwoNumbers(firstNumber: leftSparkMinimumY, secondNumber: leftSparkMaximumY)
}
And here is a different version of your following code.
for (index,y) in leftPositions.enumerated() {
let leftSparkNode = SKNode()
leftSparkNode.position = CGPoint(x:-295,y:CGFloat(y))
leftSparkNode.name = "LeftSparks\(index)" //All node names should be unique
let leftSparkTexture = SKTexture(imageNamed: "LeftSpark")
LeftSpark = SKSpriteNode(texture: leftSparkTexture)
LeftSpark.name = "LeftSparks"
LeftSpark.physicsBody = SKPhysicsBody(texture: leftSparkTexture, size: LeftSpark.size)
LeftSpark.physicsBody?.categoryBitMask = PhysicsCatagory.LeftSpark
LeftSpark.physicsBody?.collisionBitMask = PhysicsCatagory.Bird
LeftSpark.physicsBody?.contactTestBitMask = PhysicsCatagory.Bird
LeftSpark.physicsBody?.isDynamic = false
LeftSpark.physicsBody?.affectedByGravity = false
leftSparkNode.addChild(LeftSpark)
addChild(leftSparkNode)
}

Linear regression - accelerate framework in Swift

My first question here at Stackoverflow... hope my question is specific enough.
I have an array in Swift with measurements at certain dates. Like:
var myArray:[(day: Int, mW: Double)] = []
myArray.append(day:0, mW: 31.98)
myArray.append(day:1, mW: 31.89)
myArray.append(day:2, mW: 31.77)
myArray.append(day:4, mW: 31.58)
myArray.append(day:6, mW: 31.46)
Some days are missing, I just didn't take a measurement... All measurements should be on a line, more or less. So I thought about linear regression. I found the Accelerate framework, but the documentation is missing and I can't find examples.
For the missing measurements I would like to have a function, with as input a missing day and as output a best guess, based on the other measurements.
func bG(day: Int) -> Double {
return // return best guess for measurement
}
Thanks for helping out.
Jan
My answer doesn't specifically talk about the Accelerate Framework, however I thought the question was interesting and thought I'd give it a stab. From what I gather you're basically looking to create a line of best fit and interpolate or extrapolate more values of mW from that. To do that I used the Least Square Method, detailed here: http://hotmath.com/hotmath_help/topics/line-of-best-fit.html and implemented this in Playgrounds using Swift:
// The typealias allows us to use '$X.day' and '$X.mW',
// instead of '$X.0' and '$X.1' in the following closures.
typealias PointTuple = (day: Double, mW: Double)
// The days are the values on the x-axis.
// mW is the value on the y-axis.
let points: [PointTuple] = [(0.0, 31.98),
(1.0, 31.89),
(2.0, 31.77),
(4.0, 31.58),
(6.0, 31.46)]
// When using reduce, $0 is the current total.
let meanDays = points.reduce(0) { $0 + $1.day } / Double(points.count)
let meanMW = points.reduce(0) { $0 + $1.mW } / Double(points.count)
let a = points.reduce(0) { $0 + ($1.day - meanDays) * ($1.mW - meanMW) }
let b = points.reduce(0) { $0 + pow($1.day - meanDays, 2) }
// The equation of a straight line is: y = mx + c
// Where m is the gradient and c is the y intercept.
let m = a / b
let c = meanMW - m * meanDays
In the code above a and b refer to the following formula from the website:
a:
b:
Now you can create the function which uses the line of best fit to interpolate/extrapolate mW:
func bG(day: Double) -> Double {
return m * day + c
}
And use it like so:
bG(3) // 31.70
bG(5) // 31.52
bG(7) // 31.35
If you want to do fast linear regressions in Swift, I suggest using the Upsurge framework. It provides a number of simple functions that wrap the Accelerate library and so you get the benefits of SIMD on either iOS or OSX
without having to worry about the complexity of vDSP calls.
To do a linear regression with base Upsurge functions is simply:
let meanx = mean(x)
let meany = mean(y)
let meanxy = mean(x * y)
let meanx_sqr = measq(x)
let slope = (meanx * meany - meanxy) / (meanx * meanx - meanx_sqr)
let intercept = meany - slope * meanx
This is essentially what is implemented in the linregress function.
You can use it with an array of [Double], other classes such as RealArray (comes with Upsurge) or your own objects if they can expose contiguous memory.
So a script to meet your needs would look like:
#!/usr/bin/env cato
import Upsurge
typealias PointTuple = (day: Double, mW:Double)
var myArray:[PointTuple] = []
myArray.append((0, 31.98))
myArray.append((1, 31.89))
myArray.append((2, 31.77))
myArray.append((4, 31.58))
myArray.append((6, 31.46))
let x = myArray.map { $0.day }
let y = myArray.map { $0.mW }
let (slope, intercept) = Upsurge.linregress(x, y)
func bG(day: Double) -> Double {
return slope * day + intercept
}
(I left in the appends rather than using literals as you are likely programmatically adding to your array if it is of significant length)
and full disclaimer: I contributed the linregress code. I hope to also add the co-efficient of determination at some point in the future.
To estimate the values between different points, you can also use SKKeyframeSequence from SpriteKit
https://developer.apple.com/documentation/spritekit/skinterpolationmode/spline
import SpriteKit
let sequence = SKKeyframeSequence(keyframeValues: [0, 20, 40, 60, 80, 100], times: [64, 128, 256, 512, 1024, 2048])
sequence.interpolationMode = .spline // .linear, .step
let estimatedValue = sequence.sample(atTime: CGFloat(1500)) as! Double // 1500 is the value you want to estimate
print(estimatedValue)