I am trying to detect ears and neck to put jewelry using ARKit. Currently, I am using hardcode value to do this but this is very inaccurate for different - different face.
SCNNode *contentNode = [SCNNode node];
ARSCNFaceGeometry *faceGeometry = [ARSCNFaceGeometry faceGeometryWithDevice:_sceneView.device];
SCNNode *faceNode = [SCNNode nodeWithGeometry:faceGeometry];
contentNode = [SCNNode nodeWithGeometry:faceGeometry];
earringsPlane.firstMaterial.doubleSided = true;
[earringsNode setPosition:SCNVector3Make(-0.09, -0.04 , -0.04)];
earringsNode.geometry = earringsPlane;
[earringsNode2 setPosition:SCNVector3Make(0.09, -0.04 , -0.04)];
earringsNode2.geometry = earringsPlane2;
Related
I am trying to attach an object in front of the camera, but the issue is that it is always in relation to the initial camera direction. How can I adjust/get the SCNVector3 position to place the object in front, even if the direction of the camera is up or down?
This is how I do it now:
let ballShape = SCNSphere(radius: 0.03)
let ballNode = SCNNode(geometry: ballShape)
let viewPosition = sceneView.pointOfView!.position
ballNode.position = SCNVector3Make(viewPosition.x, viewPosition.y, viewPosition.z - 0.4)
sceneView.scene.rootNode.addChildNode(ballNode)
Edited to better answer the question now that it's clarified in a comment
New Answer:
You are using only the position of the camera, so if the camera is rotated, it doesn't affect the ball.
What you can do is get the transform matrix of the ball and multiply it by the transform matrix of the camera, that way the ball position will be relative to the full transformation of the camera, including rotation.
e.g.
let ballShape = SCNSphere(radius: 0.03)
let ballNode = SCNNode(geometry: ballShape)
ballNode.position = SCNVector3Make(0.0, 0.0, -0.4)
let ballMatrix = ballNode.transform
let cameraMatrix = sceneView.pointOfView!.transform
let newBallMatrix = SCNMatrix4Mult(ballMatrix, cameraMatrix)
ballNode.transform = newBallMatrix
sceneView.scene.rootNode.addChildNode(ballNode)
Or if you only want the SCNVector3 position, to answer exactly to your question (this way the ball will not rotate):
...
let newBallMatrix = SCNMatrix4Mult(ballMatrix, cameraMatrix)
let newBallPosition = SCNVector3Make(newBallMatrix.m41, newBallMatrix.m42, newBallMatrix.m43)
ballNode.position = newBallPosition
sceneView.scene.rootNode.addChildNode(ballNode)
Old Answer:
You are using only the position of the camera, so when the camera rotates, it doesn't affect the ball.
SceneKit uses a hierarchy of nodes, so when a node is "child" of another node, it follows the position, rotation and scale of its "parent". The proper way of attaching an object to another object, in this case the camera, is to make it "child" of the camera.
Then, when you set the position, rotation or any other aspect of the transform of the "child" node, you are setting it relative to its parent. So if you set the position to SCNVector3Make(0.0, 0.0, -0.4), it's translated -0.4 units in Z on top of its "parent" translation.
So to make what you want, it should be:
let ballShape = SCNSphere(radius: 0.03)
let ballNode = SCNNode(geometry: ballShape)
ballNode.position = SCNVector3Make(0.0, 0.0, -0.4)
let cameraNode = sceneView.pointOfView
cameraNode?.addChildNode(ballNode)
This way, when the camera rotates, the ball follows exactly its rotation, but separated -0.4 units from the camera.
I have a 32x32 .png image that I want to repeat over a SCNPlane. The code I've got (See below) results in the image being stretched to fit the size of the plane, rather than repeated.
CODE:
let planeGeo = SCNPlane(width: 15, height: 15)
let imageMaterial = SCNMaterial()
imageMaterial.diffuse.contents = UIImage(named: "art.scnassets/grid.png")
planeGeo.firstMaterial = imageMaterial
let plane = SCNNode(geometry: planeGeo)
plane.geometry?.firstMaterial?.diffuse.wrapS = SCNWrapMode.repeat
plane.geometry?.firstMaterial?.diffuse.wrapT = SCNWrapMode.repeat
I fixed it. It seems like the image was zoomed in. If I do imageMaterial.diffuse.contentsTransform = SCNMatrix4MakeScale(32, 32, 0), the image repeats.
I faced an identical issue when implementing plane visualisation in ARKit. I wanted to visualise the detected plane as a checkerboard pattern. I fixed it by creating a custom SCNNode called a "PlaneNode" with a correctly configured SCNMaterial. The material uses wrapS, wrapT = .repeat and calculates the scale correctly based on the size of the plane itself.
Looks like this:
Have a look at the code below, the inline comments contain the explanation.
class PlaneNode : SCNNode {
init(planeAnchor: ARPlaneAnchor) {
super.init()
// Create the 3D plane geometry with the dimensions reported
// by ARKit in the ARPlaneAnchor instance
let planeGeometry = SCNPlane(width:CGFloat(planeAnchor.extent.x), height:CGFloat(planeAnchor.extent.z))
// Instead of just visualizing the grid as a gray plane, we will render
// it in some Tron style colours.
let material = SCNMaterial()
material.diffuse.contents = PaintCode.imageOfViewARPlane
//the scale gives the number of times the image is repeated
//ARKit givest the width and height in meters, in this case we want to repeat
//the pattern each 2cm = 0.02m so we divide the width/height to find the number of patterns
//we then round this so that we always have a clean repeat and not a truncated one
let scaleX = (Float(planeGeometry.width) / 0.02).rounded()
let scaleY = (Float(planeGeometry.height) / 0.02).rounded()
//we then apply the scaling
material.diffuse.contentsTransform = SCNMatrix4MakeScale(scaleX, scaleY, 0)
//set repeat mode in both direction otherwise the patern is stretched!
material.diffuse.wrapS = .repeat
material.diffuse.wrapT = .repeat
//apply material
planeGeometry.materials = [material];
//make a node for it
self.geometry = planeGeometry
// Move the plane to the position reported by ARKit
position.x = planeAnchor.center.x
position.y = 0
position.z = planeAnchor.center.z
// Planes in SceneKit are vertical by default so we need to rotate
// 90 degrees to match planes in ARKit
transform = SCNMatrix4MakeRotation(-Float.pi / 2.0, 1.0, 0.0, 0.0);
}
required init?(coder aDecoder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
func update(planeAnchor: ARPlaneAnchor) {
guard let planeGeometry = geometry as? SCNPlane else {
fatalError("update(planeAnchor: ARPlaneAnchor) called on node that has no SCNPlane geometry")
}
//update the size
planeGeometry.width = CGFloat(planeAnchor.extent.x)
planeGeometry.height = CGFloat(planeAnchor.extent.z)
//and material properties
let scaleX = (Float(planeGeometry.width) / 0.02).rounded()
let scaleY = (Float(planeGeometry.height) / 0.02).rounded()
planeGeometry.firstMaterial?.diffuse.contentsTransform = SCNMatrix4MakeScale(scaleX, scaleY, 0)
// Move the plane to the position reported by ARKit
position.x = planeAnchor.center.x
position.y = 0
position.z = planeAnchor.center.z
}
}
To do this in the SceneKit editor, select your plane (add one if needed) in the scene and then select the "Material Inspector" tab on the top right. Then, under "Properties" and where it says "Diffuse", select your texture. Now, expand the diffuse section by clicking the carat to the left of "Diffuse" and go down to where it says "Scale". Here, you can increase the scaling so that the texture can look repeated rather than stretched. For this question, the OP would have to set the scaling to 32x32.
You can learn it from Scene kit viewer Suppose You have SCNplane in your scene kit
Create scene file drag a plane
Which size is 12 inches in meter it is 0.3048
and select image in diffuse
now You have image with 4 Grid as shown in image
we want each box to be show in each inches so for 12 Inches we need 12 box * 12 box as we have 12 inches box
to calculate it. First we need convert 0.3048 meter to inches
which is meters / 0.0254 answer is 12.
but we need each grid to show in each inch so we also need to divide 12 / 4 = 3
now goto show material inspector and change scale value to 3
you can see 12 boxes for 12 inch plane.
Hope it is helpful
Based on the image below (I used different colours for circle and flat surface so they can be seen, but in the end the colours will be the same), using Swift and Spritekit, I am trying to create the effect of a circular object entering a thick substance (not necessarily sticky) and separating from the thick substance. Basically, when the circular object is separating, it will pull away from the flat surface as it forms into a circle.
I wanted to use image animation frames, but since the objects are SKSpriteNodes with physics bodies this will make timing the collision of objects with animation quite difficult. Another approach would be using CAAnimation, but I don't know how this can be combined with SKSpriteNodes with physics bodies. How can I create this separation effect using any of the above stated approaches or a different one?
UPDATE
The image below shows the change in the surface of the thick substance as the circular object enters the thick substance till it's submerged.
You are looking for a fluid simulator
This is indeed possible with modern hardware.
Let's have a look at what we are going to build here.
Components
In order to achieve that we'll need to
create several molecules having a physics body and a blurred image
use a Shader to apply a common color to every pixel having alpha > 0
Molecule
Here's the Molecule class
import SpriteKit
class Molecule: SKSpriteNode {
init() {
let texture = SKTexture(imageNamed: "molecule")
super.init(texture: texture, color: .clear, size: texture.size())
let physicsBody = SKPhysicsBody(circleOfRadius: 8)
physicsBody.restitution = 0.2
physicsBody.affectedByGravity = true
physicsBody.friction = 0
physicsBody.linearDamping = 0.01
physicsBody.angularDamping = 0.01
physicsBody.density = 0.13
self.physicsBody = physicsBody
}
required init?(coder aDecoder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
}
Shader
Next we need a Fragment Shader, let's create a file with name Water.fsh
void main() {
vec4 current_color = texture2D(u_texture, v_tex_coord);
if (current_color.a > 0) {
current_color.r = 0.0;
current_color.g = 0.57;
current_color.b = 0.95;
current_color.a = 1.0;
} else {
current_color.a = 0.0;
}
gl_FragColor = current_color;
}
Scene
And finally we can define the scene
import SpriteKit
class GameScene: SKScene {
lazy var label: SKLabelNode = {
return childNode(withName: "label") as! SKLabelNode
}()
let effectNode = SKEffectNode()
override func didMove(to view: SKView) {
physicsBody = SKPhysicsBody(edgeLoopFrom: frame)
effectNode.shouldEnableEffects = true
effectNode.shader = SKShader(fileNamed: "Water")
addChild(effectNode)
}
var touchLocation: CGPoint?
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
guard let touch = touches.first else { return }
let touchLocation = touch.location(in: self)
if label.contains(touchLocation) {
addRedCircle(location: touchLocation)
} else {
self.touchLocation = touchLocation
}
}
override func touchesEnded(_ touches: Set<UITouch>, with event: UIEvent?) {
touchLocation = nil
}
override func update(_ currentTime: TimeInterval) {
if let touchLocation = touchLocation {
let randomizeX = CGFloat(arc4random_uniform(20)) - 10
let randomizedLocation = CGPoint(x: touchLocation.x + randomizeX, y: touchLocation.y)
addMolecule(location: randomizedLocation)
}
}
private func addMolecule(location: CGPoint) {
let molecule = Molecule()
molecule.position = location
effectNode.addChild(molecule)
}
private func addRedCircle(location: CGPoint) {
let texture = SKTexture(imageNamed: "circle")
let sprite = SKSpriteNode(texture: texture)
let physicsBody = SKPhysicsBody(circleOfRadius: texture.size().width / 2)
physicsBody.restitution = 0.2
physicsBody.affectedByGravity = true
physicsBody.friction = 0.1
physicsBody.linearDamping = 0.1
physicsBody.angularDamping = 0.1
physicsBody.density = 1
sprite.physicsBody = physicsBody
sprite.position = location
addChild(sprite)
}
}
The full project is available on my GitHub account https://github.com/lucaangeletti/SpriteKitAqua
From a high level of understanding there are two ways to do this.
THE BAD WAY (but works better when fluid has textures): Create the sprite sheet in advance, then overlay an additional child of the SKSpriteNode object. The frame in the animation sprite will be a function of distance from the ball to the surface when distance between them is less than some amount. The desired distance range (range) will have to be mapped to the sprites frame number (frameIndex). f(range) = frameIndex. Linear interpolation will help out here. More on interpolation later.
THE RIGHT WAY: Make the fluid a curve object, then animate the points on the curve with linear interpolation between the initial, intermediate and final states. This will require three curves each one with the same number of points. Let the initial fluid state be F1. Model F1 points as the static fluid. Let the fluid state be F2 when the ball is halfway submerged. Model F2 points to look like the ball submerged at its maximum width. Let the fluid state be F3 when the ball is 75% submerged. Notice that when the ball is fully submerged the fluid looks unchanged. This is why when the ball is 75% submerged it has the maximum surface tension grabbing the ball. As far as SpriteKit goes, you may use these objects:
CGMutablePathRef path = CGPathCreateMutable();
CGPathMoveToPoint(path, NULL, 0, 0);
CGPathAddQuadCurveToPoint(path, NULL, 50, 100, 100, 0);
CGPathAddLineToPoint(path, NULL, 50, -100);
CGPathCloseSubpath(path);
SKShapeNode *shape = [[SKShapeNode alloc]init];
shape.path = path;
Then detect when the ball is on the outside of the fluid by using the vector cross product with 3D vectors, even if your project is in 2D.
Ball Vector (Vb)
^
|
(V) O---> Closest Fluid Surface Vector (Vs)
V = Vb x Vs
Then look at the Z component of V called Vz.
If (Vz < 0), the ball is outside of the fluid:
Create a variable t:
t = distOfBall/radiusOfBall
Then for every ordered point in your fluid shapes do the following:
newFluidPointX = F1pointX*(t-1) + F2pointX*t
newFluidPointY = F1pointY*(t-1) + F2pointY*t
If Vz > 0), the ball is inside of the fluid:
t = -(((distOfBall/radiusOfBall) + 0.5)^2) *4 + 1
newFluidPointX = F2pointX*(t-1) + F3pointX*t
newFluidPointY = F2pointY*(t-1) + F3pointY*t
This works because any two shapes can be blended together using interpolation.
The parameter "t" acts as a percentage to blend between two shapes.
You can actually create seamless blends between any two shapes, so long as the number of points are the same. This is how a man morphs into a wolf in Hollywood movies, or how a man can morph into a liquid puddle. The only principle in play for those effects is interpolation. Interpolation is a very powerful tool. It is defined as:
L = A*(t-1) + B*t
where t is in between 0.0 and 1.0
and A and B is what you are morphing from and to.
For more on interpolation see:
Wiki Article
For further study. If you are considering animating any dynamic shapes, I would consider understanding Bezier curves. Pomax has a wonderful article on the topic. Although many frameworks have curves in them, having a general understanding of how they work will allow you to manipulate them extensivly or roll your own features where the framework is lacking. Her is Pomax's article:
A Primer on Curves
Good luck on your progress:)
I have a SceneKit SCNBox Cube that should follow all the movements of my iphone. So If my iphone rotate 90 or 360 degrees sideways should the cube also rotate 90 or 360 degrees and if I spin my phone forward the cube should spin like the phone.
I have tried with startDeviceMotionUpdatesUsingReferenceFrame, but it doesn't match the movements of my phone :(
motionManager.deviceMotionUpdateInterval = 1.0 / 60.0
motionManager.startDeviceMotionUpdatesUsingReferenceFrame(
CMAttitudeReferenceFrame.XArbitraryZVertical,
toQueue: NSOperationQueue.mainQueue(),
withHandler: { (motion: CMDeviceMotion!, error: NSError!) -> Void in
let currentAttitude = motion.attitude
let roll = Float(currentAttitude.roll)
let pitch = Float(currentAttitude.pitch)
let yaw = Float(currentAttitude.yaw)
self.arrRoll.append(roll)
self.arrPitch.append(pitch)
self.arrYaw.append(yaw)
cubeNode.eulerAngles = SCNVector3Make(roll, 0.0, 0.0)
cubeNode.eulerAngles = SCNVector3Make(0.0, 0.0, pitch)
cubeNode.eulerAngles = SCNVector3Make(0.0, yaw, 0.0)
})
Your code snippet is a little out of context, but one thing that strikes me off the bat is that you are (most likely unintentionally) overwriting the eulerAngles property of your node.
Why do you have 3 separate assignments (only the last of which will have an effect), instead of:
cubeNode.eulerAngles = SCNVector3Make(roll, yaw, pitch)
?
I am wondering if it is possible to include a SpriteKit scene in a SceneKit scene, and if it is, how to do that ?
Yes there is!
You can assign a Sprite Kit scene (SKScene) as the contents of a material property (SCNMaterialProperty) in Scene Kit.
I've come across a few questions about this exact topic and there don't seem to be a whole lot of actual code examples, so here's another one. Hopefully this is helpful:
SKSpriteNode *someSKSpriteNode;
// initialize your SKSpriteNode and set it up..
SKScene *tvSKScene = [SKScene sceneWithSize:CGSizeMake(100, 100)];
tvSKScene.anchorPoint = CGPointMake(0.5, 0.5);
[tvSKScene addChild:someSKSpriteNode];
// use spritekit scene as plane's material
SCNMaterial *materialProperty = [SCNMaterial material];
materialProperty.diffuse.contents = tvSKScene;
// this will likely change to whereever you want to show this scene.
SCNVector3 tvLocationCoordinates = SCNVector3Make(0, 0, 0);
SCNPlane *scnPlane = [SCNPlane planeWithWidth:100.0 height:100.0];
SCNNode *scnNode = [SCNNode nodeWithGeometry:scnPlane];
scnNode.geometry.firstMaterial = materialProperty;
scnNode.position = tvLocationCoordinates;
// Assume we have a SCNCamera and SCNNode set up already.
SCNLookAtConstraint *constraint = [SCNLookAtConstraint lookAtConstraintWithTarget:cameraNode];
constraint.gimbalLockEnabled = NO;
scnNode.constraints = #[constraint];
// Assume we have a SCNView *sceneView set up already.
[sceneView.scene.rootNode addChildNode:scnNode];