ARKit Adding node causes frame drop even after using `prepare` - swift

I am adding a 3D model containing animations to the scene that I previously download from the internet. Before adding this node I use prepare function on it because I wan't to avoid frame drop. But still I get a very short frame drop to about 47 fps. This is caused by executing this prepare function. I also tried using prepare(_:, shouldAbortBlock:) on other dispatch queue, but this still didn't help. Can someone help me resolve this or tell me why there is this happening?
arView.sceneView.prepare([mediaNode]) { [mediaNode, weak self] (success) in
guard let `self` = self else { return }
guard
let currentMediaNode = self.mediaNode as? SCNNode,
currentMediaNode === mediaNode,
!self.mainNode.childNodes.contains(mediaNode)
else { return }
self.mainNode.addChildNode(mediaNode)
}
By the way this is a list of files I'm using to load this model:
https://www.dropbox.com/s/7968fe5wfdcxbyu/Serah-iOS.dae?dl=1
https://www.dropbox.com/s/zqb6b6rxynnvc5e/0001.png?dl=1
https://www.dropbox.com/s/hy9y8qyazkcnvef/0002.tga?dl=1
https://www.dropbox.com/s/fll9jbjud7zjlsq/0004.tga?dl=1
https://www.dropbox.com/s/4niq12mezlvi5oz/0005.png?dl=1
https://www.dropbox.com/s/wikqgd46643327i/0007.png?dl=1
https://www.dropbox.com/s/fioj9bqt90vq70c/0008.tga?dl=1
https://www.dropbox.com/s/4a5jtmccyx413j7/0010.png?dl=1
DAE file is already compiled by Xcode tools so that it can be loaded after being downloaded from the internet. And this is the code I use to load it after it's downloaded:
class func loadModel(fromURL url: URL) -> SCNNode? {
let options = [SCNSceneSource.LoadingOption.animationImportPolicy : SCNSceneSource.AnimationImportPolicy.playRepeatedly]
let sceneSource = SCNSceneSource(url: url, options: options)
let node = sceneSource?.entryWithIdentifier("MDL_Obj", withClass: SCNNode.self)
return node
}

I was experiencing the same issue. My nodes were all taking advantage of physically-based rendering (PBR) and the first time I added a node to the scene, the frame rate dropped significantly, but was fine after that. I could add as many other nodes without a frame rate drop.
I figured out a work around to this issue. What I do is after I create my ARConfiguration and before I call session.run(configuration) I add a test node with PBR to the scene. In order for that node to not appear, I set the node's material's colorBufferWriteMask to an empty array (see this answer: ARKit hide objects behind walls) Then before I add my content I remove that node. Adding and removing this test node does the trick for me.
Here is an example:
var pbrTestNode: SCNNode!
func addPBRTestNode() {
let testGeometrie = SCNBox(width: 0.5, height: 0.5, length: 0.5, chamferRadius: 0)
testGeometrie.materials.first?.diffuse.contents = UIColor.blue
testGeometrie.materials.first?.colorBufferWriteMask = []
testGeometrie.materials.first?.lightingModel = .physicallyBased
pbrTestNode = SCNNode(geometry: testGeometrie)
scene.rootNode.addChildNode(pbrTestNode)
}
func removePBRTestNode() {
pbrTestNode.removeFromParentNode()
}
func startSessionWithPlaneDetection() {
// Create a session configuration
let configuration = ARWorldTrackingConfiguration()
if #available(iOS 11.3, *) {
configuration.planeDetection = [.horizontal, .vertical]
} else {
configuration.planeDetection = .horizontal
}
configuration.isLightEstimationEnabled = true
// this prevents the delay when adding any nodes with PBR later
sceneController.addPBRTestNode()
// Run the view's session
sceneView.session.run(configuration)
}
Call removePBRTestNode() when you add your content to the scene.

Firstly
Get 3D model for AR app with no more than 10K polygons and a texture of 1K x 1K. The best result can be accomplished with 5K...7K polygons per each model. Totally, SceneKit's scene may contain not more than 100K polygons. This recommendation helps you considerably improve rendering performance and, I suppose, you'll have a minimal drop frame.
Secondly
The simplest way to get rid of drop frame in ARKit/SceneKit/AVKit is to use Metal framework. Just imagine: a simple image filter can be more than a hundred times faster to perform on the GPU than an equivalent CPU-based filter. The same things I could say about realtime AV-video and 3D animation – they perform much better on GPU.
For instance, you can read this useful post about using Metal rendering for AVCaptureSession. There's awesome workflow how to use Metal.
P.S. Check your animated object/scene in 3D authoring tool (if it's OK) before writing a code.

Related

Animating SCN Objects with CAKeyframeAnimation in Swift SceneKit

I'm writing an application that displays chemical reactions and molecules in 3D. I read in all the values and positions of each atom from a text file and I am creating each atom shape with SCNSpheres. I have all the other values I need read in properly, but I can't figure out how to add keyframe animations to each node object in my scene.
I set up the molecules like this in ViewController.swift
func makeAtom(atomName: String, coords: [Double], scene: SCNScene) {
guard let radius = atomRadii[atomName]?.atomicRadius else { return }
atoms.append(Atom(name: atomName, x: coords[0], y: coords[1], z: coords[2], radius: radius, positions: []))
let atomGeometry = SCNSphere(radius: CGFloat(radius))
let atomNode = SCNNode(geometry: atomGeometry)
atomNode.position = SCNVector3(coords[0], coords[1], coords[2])
scene.rootNode.addChildNode(atomNode)
atomNodes.append(atomNode)
}
I know that the CAKeyframeAnimations are supposed to be set up like this
let animation = CAKeyframeAnimation()
animation.keyPath = "position.y"
animation.values = [0, 300, 0]
animation.keyTimes = [0, 0.5, 1]
animation.duration = 2
animation.isAdditive = true
vw.layer.add(animation, forKey: "move")
I just don't know where I should be declaring these animations and how the layers factor into all this. What layer should I be adding the animations to? And how can I trigger them to play? I've been searching all over the internet for help with this but I can't find anything that just shows a simple implementation.
I can provide more code if need be, I'm pretty new to StackOverflow and want to make sure I'm doing this right.
You can do it different ways, but I like this method: 58001288 (my answer here) as you can pre-build some animations using scenekit and then run them as a sequence.
Per the comment.. needed more room.
A sequence is a fixed thing. You can start, repeat, and stop it. However, it's hard to interact with it during phases.
If you really need to do that, then one way is to break up your sequence into its parts and call the next one yourself after a completion handler of the current one. I keep an array and a counter so that I know where I am. So basically it's just a queue of actions that I manage - if I'm on a certain step and the button is pressed, then I can cancel all current actions, set the desired effect, and restart it.
Edit:
The completion handler calls itself at the end of the function and advances its own array count so that the next one in the list can be called. This is obviously a bit dangerous, so I would use sparingly, but that's how I did it. I started mine on a timer, then don't forget to clean it up. I had a global GAME_ACTIVE switch and within the code I checked for it before calling myself again.
Edit2: This is actually a moveTo, but it's still just a custom set of SCNActions that calls itself when complete based on duration so that it immediately goes to the next one without a delay.
func moveTo()
{
let vPanelName = moves[moveCount]
let vLaneNode = grid.gridPanels[vPanelName]!.laneNodes[lane]
let vAction = SCNAction.move(to: vLaneNode.presentation.position, duration: TimeInterval(data.getAttackSpeed(vGameType: gameType)))
node.runAction(vAction, completionHandler:
{
self.moveCount += 1
if(self.moveCount >= self.moves.count - 1)
{
self.killMe(vRealKill: false)
return
}
else
{
self.moveTo()
}
})
}

PhysicsBody: Could not create physics body

I have an entire working game that has been up on the App Store, but since iOS 13 it simply does not work. I've installed the game onto my device via Xcode and I'm getting a lot of errors coming up saying:
PhysicsBody: Could not create physics body.
I've been creating my SKSpriteNodes like this:
let bird = SKSpriteNode(texture: SKTextureAtlas(named:"player").textureNamed("bird0001"))
bird.physicsBody = SKPhysicsBody(texture: bird.texture!,
size: bird.size)
Based on some research, this is possibly an ongoing bug with iOS and Xcode. Could someone please confirm if this is the case, as this seems to be like a major problem for Games on the app store that create their SKSpriteNodes using textures?
Is there a fix to this where textures are needed?
OK, here's a test of different approaches for avoiding this bug as of iOS 13.3 (edit also now tried on 13.3.1) and Xcode version 11.3. Full source of the test at this link:
https://github.com/bg2b/bugtest
Relevant code:
func addShip(_ texture: SKTexture, how: String) {
let sprite = SKSpriteNode(texture: texture)
sprite.position = CGPoint(x: x, y: 0)
sprite.zRotation = .pi / 4
x += 100
sprite.physicsBody = SKPhysicsBody(texture: texture, size: sprite.size)
if sprite.physicsBody == nil {
print("\(how) failed")
} else {
print("\(how) worked")
}
addChild(sprite)
}
override func didMove(to view: SKView) {
// The atlas version of a texture
addShip(SKTexture(imageNamed: "ship_blue"), how: "simple atlas reference")
// From an atlas, but call size() to force loading
let texture = SKTexture(imageNamed: "ship_blue")
_ = texture.size()
addShip(texture, how: "atlas force load")
// Reconstruct via CGImage (size would be wrong because of 2x)
let cgTexture = SKTexture(cgImage: texture.cgImage())
addShip(cgTexture, how: "reconstruct via cgImage")
// Re-render using view
let renderedTexture = view.texture(from: SKSpriteNode(texture: texture))!
addShip(renderedTexture, how: "re-render using view")
// Non-atlas texture
addShip(SKTexture(imageNamed: "nonatlas_ship_blue"), how: "not in atlas")
}
Summary:
Simply referencing the texture from an atlas and making the physics body may fail
Force-loading the texture by calling size() before making the body fails
Trying to make a new texture by going through cgImage() fails (the image itself is broken, probably related to the same bug)
Rendering to a texture using a view and then making the physics body from that new texture works
Making the physics body from a non-atlas copy of the texture works
Console output from the test program showing what works and what does not:
2020-02-01 06:23:51.872975-0500 bugtest[14399:9898087] PhysicsBody: Could not create physics body.
simple atlas reference failed
2020-02-01 06:23:51.886387-0500 bugtest[14399:9898087] PhysicsBody: Could not create physics body.
atlas force load failed
2020-02-01 06:23:51.913927-0500 bugtest[14399:9898087] PhysicsBody: Could not create physics body.
reconstruct via cgImage failed
re-render using view worked
not in atlas worked
Here's a screen shot showing the effect of the different approaches. You have to look a bit closely, but only the last two have valid physics bodies.

SceneKit: Rotate node but define target angles

EDIT: I have solved the problem and will post the solution in the next couple of days.
I'm building 3D achievements similar to Apple's Activity app.
I've already loaded my 3D model (a scene with a single node), can show it, and can tap on it to apply a rotational force:
#objc func objectTapped(_ gesture: UITapGestureRecognizer) {
let tapLocation = gesture.location(in: scnView)
let hitResults = scnView.hitTest(tapLocation, options: [:])
if let tappedNode = (hitResults.first { $0.node === badgeNode })?.node {
let pos = Float(tapLocation.x) - tappedNode.boundingBox.max.x
let tappedVector = SCNVector4(x: 0, y: pos, z: 0, w: 0.1)
tappedNode.physicsBody?.applyTorque(tappedVector,
asImpulse: true)
}
}
This works fine. Now to the tricky part:
I want the node to rotate until it either shows its front or backside (like in the Activity app), where it then should stop. It should stop naturally, which means it can overshoot a bit and then return.
To describe it with pictures - here I am holding the node in this position...
...and if I let go of the node, it will rotate to show the front side, which includes a little bit of overshooting. This is the ending position:
Since I'm quite new to SceneKit, I have troubles figuring out how to achieve this effect. It seems like I can achieve that by using SceneKit objects like gravity fields, without having to calculate a whole lot of stuff by myself, or at least that's what I'm hoping for.
I don't necessarily ask for a full solution, I basically just need a point in the right direction. Thanks in advance!

SKTextureAtlas gets overridden by declaration of second SKTextureAtlas

Whenever I create and define two variables as type SKTextureAtlas they rewrite each other. Here is my initial creation of my two SKTextureAtlas
class GameScene: SKScene {
var idle = true
var TextureAtlas = SKTextureAtlas()
var TextureAtlasIdle = SKTextureAtlas()
Later on in my code I assign a folder of images to each atlas. If I comment out the second atlas my animation works, but when I define the two as shown below my animation overlaps and plays frames from cat_walk even though it is told to only play cat_idle
TextureAtlasIdle = SKTextureAtlas(named: "cat_idle")
for i in 1...TextureAtlasIdle.textureNames.count{
let Name = "\(i).png"
TextureArrayIdle.append(SKTexture (imageNamed: Name))
}
TextureAtlas = SKTextureAtlas(named: "cat_walk")
This is how I start my cat_idle animation. I don't start the cat_walk animation
aN.run(SKAction.repeatForever(SKAction.animate(with:self.TextureArrayIdle, timePerFrame: 0.1)))
I'm trying to find out why this line of code is causing my two different animations to overlap.
TextureAtlas = SKTextureAtlas(named: "cat_walk")
Texture Atlas is kind of a sprite sheet for your game. Your approach is not right for accessing atlas images for the game. modify your for loop like below:
for i in 0..< 8 {
let texture:SKTexture = TextureAtlasIdle.textureNamed(String(format: "%i", i+1))
TextureArrayIdle.insert(texture, at:i)
}
Every time you access your atlas folder, use this for loop procedure. i have used static 8 number in for loop, it would be your sprites count. There is a simple game called Desert Run in github. please check this out for more clarification.
NB: your cat's images naming must start with 1.png
if you want to run the for loop

Copying SCNParticleSystem doesn't seem to work well

I'm trying to use an SCNParticleSystem as a "template" for others. I basically want the exact same properties except for the color animation for the particles. Here's what I've got so far:
if let node = self.findNodeWithName(nodeName),
let copiedParticleSystem: SCNParticleSystem = particleSystemToCopy.copy() as? SCNParticleSystem,
let colorController = copiedParticleSystem.propertyControllers?[SCNParticleSystem.ParticleProperty.color],
let animation: CAKeyframeAnimation = colorController.animation as? CAKeyframeAnimation {
guard animation.values?.count == animationColors.count else {
return nil
}
// Need to copy both the animations and the controllers
let copiedAnimation: CAKeyframeAnimation = animation.copy() as! CAKeyframeAnimation
copiedAnimation.values = animationColors
let copiedController: SCNParticlePropertyController = colorController.copy() as! SCNParticlePropertyController
copiedController.animation = copiedAnimation
// Finally set the new copied controller
copiedParticleSystem.propertyControllers?[SCNParticleSystem.ParticleProperty.color] = copiedController
// Add the particle system to the desired node
node.addParticleSystem(copiedParticleSystem)
// Some other work ...
}
I copy not only the SCNParticleSystem, but also SCNParticlePropertyController and CAKeyframeAnimation just to be safe. I've found that I've had to manually do these "deep" copies "manually" since a .copy() on SCNParticleSystem doesn't copy the animation, etc.
When I turn on the copied particle system on the node it was added to (by setting the birthRate to a positive number), nothing happens.
I don't think the problem is with the node that I've added it to, since I've tried adding the particleSystemToCopy to that node and turning that on, and the original particle system becomes visible in that case. This seems to indicate to me that the the node I've added the copied particle system to is OK in terms of its geometry, rendering order, etc.
Something else perhaps worth mentioning: the scene is loaded from a .scn file and not created programmatically in code. In theory that shouldn't affect anything, but who knows ...
Any ideas on why this copied particle system doesn't do anything when I turn it on?
Do not use copy() method for particle systems !
copy() method does not allow copy particles' color (copied particles will be default white).
You can test it with the following code:
let particleSystem01 = SCNParticleSystem()
particleSystem01.birthRate = 2
particleSystem01.particleSize = 0.5
particleSystem01.particleColor = .systemIndigo // INDIGO
particleSystem01.emitterShape = .some(SCNSphere(radius: 2.0))
let particlesNode01 = SCNNode()
particlesNode01.addParticleSystem(particleSystem01)
particlesNode01.position.y = -3
sceneView.scene.rootNode.addChildNode(particlesNode01)
let particleSystem02 = particleSystem01.copy() // WHITE
let particlesNode02 = SCNNode()
particlesNode02.addParticleSystem(particleSystem02 as! SCNParticleSystem)
particlesNode02.position.y = 3
sceneView.scene.rootNode.addChildNode(particlesNode02)
Use clone() method for nodes instead !
clone() method works more consistently for 3d objects and particle systems and it can help you save particles' color but, of course, doesn't allow save a position for each individual particle.
let particlesNode02 = particlesNode01.clone() // INDIGO
particlesNode02.position.y = 3
sceneView.scene.rootNode.addChildNode(particlesNode02)