SceneKit SCNSceneRendererDelegate - renderer function not called - swift

I recently asked a question which had a pretty obvious answer. I'm still working on the same project and running into another problem. I need to implement per frame logic and the SCNSceneRendererDelegate protocol worked perfectly fine on iOS, but on OSX, the renderer function is not firing. I have created a little example project to illustrate my problem. It consists of a Scene Kit View in storyboard and following code in the ViewController class:
import Cocoa
import SceneKit
class ViewController: NSViewController, SCNSceneRendererDelegate {
#IBOutlet weak var sceneView: SCNView!
let cubeNode = SCNNode()
override func viewDidLoad() {
super.viewDidLoad()
let scene = SCNScene()
let sphere = SCNSphere(radius: 0.1)
sphere.firstMaterial!.diffuse.contents = NSColor.yellowColor()
let sphereNode = SCNNode(geometry: sphere)
scene.rootNode.addChildNode(sphereNode)
let cube = SCNBox(width: 0.2, height: 0.2, length: 0.2, chamferRadius: 0)
cube.firstMaterial!.diffuse.contents = NSColor.greenColor()
cubeNode.geometry = cube
cubeNode.position = SCNVector3(1,0,0)
scene.rootNode.addChildNode(cubeNode)
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(2, 1, 2)
let constraint = SCNLookAtConstraint(target: cubeNode)
cameraNode.constraints = [constraint]
scene.rootNode.addChildNode(cameraNode)
sceneView.scene = scene
sceneView.backgroundColor = NSColor(red: 0.5, green: 0, blue: 0.3, alpha: 1)
sceneView.allowsCameraControl = true
sceneView.delegate = self
sceneView.playing = true
}
func renderer(renderer: SCNSceneRenderer, updateAtTime time: NSTimeInterval) {
cubeNode.position.x += 0.1
}
}
All I want is to basically move the cube with every frame. But nothing happens. What is weird is that when I set sceneView.allowsCameraControl to true, the renderer function is called whenever I click or drag on the screen (which makes sense because it needs to update the view based on camera angles). But I would want it to be called every frame.
Is there an error I don't see or is this a bug in my Xcode?
Edit:
I have tried following the instructions in the answer below and now have the following code for the ViewController:
import Cocoa
import SceneKit
class ViewController: NSViewController {
#IBOutlet weak var sceneView: SCNView!
let scene = MyScene(create: true)
override func viewDidLoad() {
super.viewDidLoad()
sceneView.scene = scene
sceneView.backgroundColor = NSColor(red: 0.5, green: 0, blue: 0.3, alpha: 1)
sceneView.allowsCameraControl = true
sceneView.delegate = scene
sceneView.playing = true
}
}
And a MyScene class:
import Foundation
import SceneKit
final class MyScene: SCNScene, SCNSceneRendererDelegate {
let cubeNode = SCNNode()
convenience init(create: Bool) {
self.init()
let sphere = SCNSphere(radius: 0.1)
sphere.firstMaterial!.diffuse.contents = NSColor.yellowColor()
let sphereNode = SCNNode(geometry: sphere)
rootNode.addChildNode(sphereNode)
let cube = SCNBox(width: 0.2, height: 0.2, length: 0.2, chamferRadius: 0)
cube.firstMaterial!.diffuse.contents = NSColor.greenColor()
cubeNode.geometry = cube
cubeNode.position = SCNVector3(1,0,0)
rootNode.addChildNode(cubeNode)
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(2, 1, 2)
let constraint = SCNLookAtConstraint(target: cubeNode)
cameraNode.constraints = [constraint]
rootNode.addChildNode(cameraNode)
}
#objc func renderer(aRenderer: SCNSceneRenderer, updateAtTime time: NSTimeInterval) {
cubeNode.position.x += 0.01
}
}
However, it is still not working. What am I doing wrong?
Edit: setting sceneView.loops = true fixes the described problem

I don't understand what's causing the problem, but I was able to replicate it. I got it to work, though, by adding a meaningless SCNAction:
let dummyAction = SCNAction.scaleBy(1.0, duration: 1.0)
let repeatAction = SCNAction.repeatActionForever(dummyAction)
cubeNode.runAction(repeatAction)
The render loop fires only if the scene is "playing" (see SKScene becomes unresponsive while being idle). I expect that setting
sceneView.isPlaying = true
(as you're already doing) would be enough to trigger the render callbacks.
The code I have above is not a solution. It's a nasty hack to work around your problem and allow you to get on with life.

For anyone still having problems, setting the delegate and playing variables will work.
sceneView.delegate = self
sceneView.isPlaying = true

I suspect the answer hinges on what Querent means by "every frame". Querent should probably clarify this, but I'll try to answer anyway because I'm like that.
The simplest interpretation is probably "every frame that would render anyway", but that seems unlikely to be what is desired unless the cube is intended as a kind of activity monitor for the renderer, which doesn't seem likely either; there are much better approaches to that.
What Querent may want is to render repeatedly while the view's playing property is YES. If that's the case, then perhaps the answer is as simple as setting the view's loops property to YES. This recently solved a problem for me in which I wanted rendering to occur while a keyboard key was held down. (I had noticed that setting playing to YES would induce a single call to my delegate.)

In addition to several helpful hints in this chain, the final one for me to get delegate called was the following: If you use the pre Swift 4 methods for the SCNSceneRendererDelegate class, it compiles fine with no errors or warnings, but the delegate is never called.
Thus the obsolete pre-Swift 4 definition:
func renderer(aRenderer:SCNSceneRenderer, updateAtTime time:TimeInterval) {...}
(which I got from a snippet on the web) compiled just fine and was never called, while the correct definition
func renderer(_ renderer:SCNSceneRenderer, updateAtTimet time:TimeInterval) {...}
compiles and gets called!
Since SCNSceneRendererDelegate is a protocol, the normal Swift protections afforded by override are not appropriate. Since SCNSceneRendererDelegate defines its methods as optional (which I like), it is not caught that way either.

I've been struggling with some similar bugs using SCNRenderer rather than SCNView, and this was the first hit on Google, so I just wanted to add a note with my solution in case it helps anyone.
I was using
.render(withViewport:commandBuffer:passDescriptor:)
but this does not call the delegate render method. Instead, use
.render(atTime:viewport:commandBuffer:passDescriptor:)
even if you are not using the time interval parameter. Then the delegate method renderer(_:updateAtTime:) will be called properly.

Try this…put your code in a scene class instead – keep the view controller clean.
final class MySCNScene:SCNScene, SCNSceneRendererDelegate
{
#objc func renderer(aRenderer:SCNSceneRenderer, updateAtTime time:NSTimeInterval)
{
}
}
Also set the view's delegate to your scene:
mySCNView!.delegate = mySCNScene

Related

SceneKit. How to achieve default scene light behavior without autoenablesDefaultLighting?

I tried to set autoenablesDefaultLighting=true for my SCNView and it looks good. However i want to achieve the same behavior without autoenablesDefaultLighting with setting light and adjust it a little bit.
I tried omni light with this code:
let lightNode = SCNNode()
lightNode.light = SCNLight()
lightNode.light?.castsShadow = true
lightNode.light?.type = .omni
lightNode.light?.intensity = 10000
lightNode.position = SCNVector3(x: 0, y: 0, z: 100)
scene.rootNode.addChildNode(lightNode)
And got this:
And with autoenablesDefaultLighting=true I got this:
Custom Default Lighting
I believe, in SceneKit, the default scene lighting is a Directional Light without any shadows, attached directly to the default camera node (i.e. pointOfView node). To simulate the same lighting conditions as when the .autoenablesDefaultLighting property is true, use the following code:
Delegate's renderer method – light's position orientation will be updated 60 times per second:
import SceneKit
extension GameViewController: SCNSceneRendererDelegate {
func renderer(_ renderer: SCNSceneRenderer,
updateAtTime time: TimeInterval) {
sunNode.transform = (sceneView?.pointOfView?.worldTransform)!
let cameraAngles = (self.sceneView?.pointOfView?.eulerAngles)!
let lightAngles = self.sunNode.eulerAngles
print("Camera: " + String(format: "%.2f, %.2f, %.2f", cameraAngles.x,
cameraAngles.y,
cameraAngles.z))
print("Light: " + String(format: "%.2f, %.2f, %.2f", lightAngles.x,
lightAngles.y,
lightAngles.z))
}
}
Here's GameViewController class:
class GameViewController: NSViewController {
var sceneView: SCNView? = nil
let sunNode = SCNNode()
override func viewDidLoad() {
super.viewDidLoad()
sceneView = self.view as? SCNView
sceneView?.delegate = self
let scene = SCNScene(named: "ship.scn")!
sceneView?.scene = scene
sceneView?.scene?.lightingEnvironment.contents = .none
sceneView?.scene?.background.contents = .none
sceneView?.backgroundColor = .black
sceneView?.allowsCameraControl = true
// sceneView?.autoenablesDefaultLighting = true
sunNode.light = SCNLight()
sunNode.light?.type = .directional
sceneView?.scene?.rootNode.addChildNode(sunNode)
}
}
Explanations
I'd like to add that if there is no light in the scene at all (including the autoenablesDefaultLighting parameter), then the only uncontrollable source of light in the scene will be the non-switchable Ambient Light.
In addition to the above, the Physically Based shader always requires additional Ambient Light fixture (otherwise the physically based surface will be black). The location and orientation of this light source does not matter.
If Directional Light illuminates the surface perpendicularly, then the surface is illuminated with 100% intensity (default intensity is 1000 lumens), but if the rays of the light source are parallel to the surface, then the surface is not illuminated by this source.
As you can see, the first and last images have identical lighting environment.

Orienting a directional light and adding to scene in ARKit

I have found a few great examples how to add a directional light to my code, but not how to change the orientation as well as add it to my scene. How do I do this with my code? Here is my light class:
class Lighting: Entity, HasDirectionalLight {
required init() {
super.init()
self.light = DirectionalLightComponent(color: .white,
intensity: 100000,
isRealWorldProxy: true)
}
}
And here is the function that calls it:
func addTableToPlane(arView: ARView) {
let tableAnchor = AnchorEntity(plane: .horizontal)
let table = try! Entity.load(named: "Table_1500")
tableAnchor.addChild(table)
let dirLight = Lighting().light
let shadow = Lighting().shadow
tableAnchor.components.set(shadow!)
tableAnchor.components.set(dirLight)
}
I'm a pretty new to ARKit, so I haven't figured out how to edit the orientation of the directional light as I have it.
Another unsuccessful method that I tried was to create a lighting function, but I haven't been able to figure out how to add it to the scene:
func addLights(arView: ARView) {
// 1
let directionalLight = SCNLight()
directionalLight.type = .directional
directionalLight.intensity = 500
// 2
directionalLight.castsShadow = true
directionalLight.shadowMode = .deferred
// 3
directionalLight.shadowColor = UIColor(red: 0, green: 0, blue: 0, alpha: 0.5)
// 4
let directionalLightNode = SCNNode()
directionalLightNode.light = directionalLight
directionalLightNode.rotation = SCNVector4Make(1, 0, 0, -Float.pi / 3)
sceneView.scene.rootNode.addChildNode(directionalLightNode)
}
I then added addLights(arView: uiView) to the addTableToPlane function. I tried to add the light with:
arView.scene.rootNode.addChildNode(ambientLightNode)
but this gives the error that I don't have a childNode and so on. I guess that I'm spoiled with decent docs for Python that supply examples interspersed to help figure out problems, unlike the overly concise docs for Xcode, such as, what the heck I do with "Use the light’s look(at:from:upVector:relativeTo:) method to aim the light". Where do I put this? Where might I find answers to these simple questions?
Chasing my tail for the past couple days just to rotate a light is frustrating.
Use the following code to control orientation of directional light:
Take into consideration that position of Directional Light is not important!
import ARKit
import RealityKit
class Lighting: Entity, HasDirectionalLight, HasAnchoring {
required init() {
super.init()
self.light = DirectionalLightComponent(color: .green,
intensity: 1000,
isRealWorldProxy: true)
}
}
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewWillAppear(animated: Bool) {
super.viewWillAppear(animated)
let light = Lighting()
light.orientation = simd_quatf(angle: .pi/8,
axis: [0, 1, 0])
let boxAnchor = try! Experience.loadBox()
let directLightAnchor = AnchorEntity()
directLightAnchor.addChild(light)
boxAnchor.addChild(directLightAnchor)
boxAnchor.steelBox!.scale = [30,30,30]
boxAnchor.steelBox!.position.z = -3
arView.scene.anchors.append(boxAnchor)
}
}
If you want to know how implement directional light's orientation in SceneKit, read this post.

RealityKit - Animate opacity of a ModelEntity?

By setting the color of a material on the model property of a ModelEntity, I can alter the opacity/alpha of an object. But how do you animate this? My goal is to animate objects with full opacity, then have them fade to a set opacity, such as 50%.
With SCNAction.fadeOpacity on a SCNNode in SceneKit, this was particularly easy.
let fade = SCNAction.fadeOpacity(by: 0.5, duration: 0.5)
node.runAction(fade)
An Entity conforms to HasTransform, but that will only allow you to animate scale, position, and orientation. Nothing to do with animation of the material for something like fading it in or out. The effect is in RealityComposer if you create a behavior for animating hide or showing, but there doesn't seem to be something similar to HasTransform to provide functionality for animating opacity.
I've been all around the documentation looking for something, my next idea is essentially creating a custom animation to replace this behavior, but it seems like it should be available and I am just not finding it.
I tested it using different techniques and came to the sad conclusion: you can't animate a material's opacity in RealityKit framework because RealityKit materials don't support animation at runtime (for now I hope). Let's wait for RealityKit's major update.
Here's a code you can use for test
(arView.alpha property just works):
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
arView.alpha = 1.0
opacityAnimation()
}
func opacityAnimation() {
UIView.animate(withDuration: 5.0,
animations: {
self.arView.alpha = 0.0
})
}
}
And use this code snippet in order to make sure that animation doesn't work properly
(there's no animation process, just value assignment):
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
let tetheringAnchor = AnchorEntity(world: [0,0,0])
var material = SimpleMaterial()
let mesh: MeshResource = .generateSphere(radius: 0.5)
var sphereComponent: ModelComponent? = nil
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
material.metallic = .float(1.0)
material.roughness = .float(0.0)
material.baseColor = .color(.red)
sphereComponent = ModelComponent(mesh: mesh,
materials: [material])
tetheringAnchor.components.set(sphereComponent!)
arView.scene.anchors.append(tetheringAnchor)
opacityAnimation()
}
func opacityAnimation() {
UIView.animate(withDuration: 5.0,
animations: {
self.material.metallic = .float(1.0)
self.material.roughness = .float(0.0)
self.material.baseColor = .color(.green)
self.sphereComponent = ModelComponent(mesh: self.mesh,
materials: [self.material])
self.tetheringAnchor.components.set(self.sphereComponent!)
self.arView.scene.anchors.append(self.tetheringAnchor)
})
}
}
As #AndyFedo says there is currently no way to animate the opacity nor alpha of an Entity.
Even changing a SimpleMaterial at run time currently results in flickering.
Having said this I was able to animate the Alpha of a SimpleMaterials Color, however based on testing it is in no way optimal or recommended for that matter.
But just in case you wanted to try to further experiment with this avenue please see an attached example which assumes that you only have a single SimpleMaterial:
class CustomBox: Entity, HasModel, HasAnchoring {
var timer: Timer?
var baseColour: UIColor!
//MARK:- Initialization
/// Initializes The Box With The Desired Colour
/// - Parameter color: UIColor
required init(color: UIColor) {
self.baseColour = color
super.init()
self.components[ModelComponent] = ModelComponent(mesh: .generateBox(size: [0.2, 0.2, 0.2]),
materials: [SimpleMaterial (color: baseColour, isMetallic: false)]
)
}
required init() { super.init() }
//MARK:- Example Fading
/// Fades The Colour Of The Entities Current Material
func fadeOut() {
var alpha: CGFloat = 1.0
timer = Timer.scheduledTimer(withTimeInterval: 0.05, repeats: true) { timer in
if alpha == 0 {
timer.invalidate()
return
}
var material = SimpleMaterial()
alpha -= 0.01
material.baseColor = MaterialColorParameter.color(self.baseColour.withAlphaComponent(alpha))
material.metallic = .float(Float(alpha))
material.roughness = .float(Float(alpha))
DispatchQueue.main.async {
self.model?.materials = [material]
}
}
}
}
As such just to test you can create and then call the function like so:
let box = CustomBox(color: .green)
box.position = [0,0,-0.5]
arView.scene.anchors.append(box)
box.fadeOut()
Also I would politely ask, that this answer not get downvoted as I am simply iterating the fact that (a) it isn't possible with any current built in methods, and (b) that it can in part be achieved albeit to a very limited extent (and thus currently; in a way which one would see fit for production).
I don't know if it suits with your use case. But you should consider video material.
As you can see in this WWDC session (2min45). An entity with complex pulsating opacity.
https://developer.apple.com/videos/play/wwdc2020/10612/
you can also create the fade in experience in Reality Composer and trigger the .rcproject file in Xcode. Have not tested other interactions with .rcproject but I know at least this can load a model to fade in into the scene.

Getting the SCNRenderer from an instantiated SCNScene

I would like to know how I am supposed to extract the SCNRenderer from an instantiated SceneKit scene. I am trying to get the AVAudioEngine which lies in the SCNRenderer so that I can apply audio filters to my nodes.
Here is the override didFinishLaunching part reduced to relevant code:
override func awakeFromNib() {
// create a new scene
let scene = SCNScene()
// create and add a camera to the scene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
scene.rootNode.addChildNode(cameraNode)
// place the camera
cameraNode.position = SCNVector3(x: 0, y: 0, z: 15)
// set the scene to the view
self.gameView!.scene = scene
gameView.delegate = self
}
Please if someone can give me a pointer, I would really appreciate it as I was able to run sounds in positional but now am stuck with using the AVEngine graph to do stuff like:
AVAudioInput > AVAudioUnitDistortion > AVAudioOutput and start doing some fun mixing.
Edit:
This is what I had in mind for the engine:
distortion = AVAudioUnitDistortion()
let URL = NSURL(fileURLWithPath: dataPath+"/welcome.aiff")
if(NSFileManager.defaultManager().fileExistsAtPath(dataPath+"/welcome.aiff")){
let source = SCNAudioSource(URL: URL)!
source.volume = 30.0
source.reverbBlend = 50.0
source.rate = 0.9
let clip = SCNAudioPlayer(source: source)
engine = clip.audioNode!.engine
distortion.loadFactoryPreset(AVAudioUnitDistortionPreset.SpeechRadioTower)
engine.attachNode(distortion)
engine.connect(clip.audioNode!, to: distortion, format: nil)
engine.connect(self.distortion, to: engine.outputNode, format: nil)
return clip
But I am now having a null pointer exception over the distortion AVAudioUnitDistortion instance.
Where am I going wrong ?
audioEngine is a property on the SCNSceneRenderer protocol, not on the SCNRenderer class. Since SCNScene conforms to SCNSceneRenderer, scene.audioEngine will work.
edit:
Since SCNView conforms to SCNSceneRenderer, gameView.audioEngine will work.

SpriteKit scene transition good practices

I am writing a game using SpriteKit with Swift and have run into a memory concern.
The layout of my game is such that the GameViewController (UIViewController) presents the first SKScene (levelChooserScene) in the viewDidLoad Screen. This scene does nothing more than display a bunch of buttons. When the user selects a button the scene then transitions to the correct scene using skView.presentScene, and when the level is complete, that scene then transitions back to the levelChooserScene and the game is ready for the user to select the next level.
The problem is that when the transition back to the levelChooserScene occurs the memory allocated for the game play scene is not deallocated, so after selecting only a few levels I start receiving memory errors.
Is my design correct in transitioning from SKScene to SKScene, or should I instead return to the GameViewController each time and then transition to the next SKScene from there?
I have found a few posts on here that say I should call skView.presentScene(nil) between scenes, but I am confused on how or where to implement that.
I simply want to transition from one SKScene to another and have the memory used from the outgoing scene to be returned to the system.
This is an example of how I have implemented the SKScene:
class Level3: SKScene
{
var explodingRockTimer = NSTimer()
var blowingUpTheRocks = SKAction()
override func didMoveToView(view: SKView)
{
NSTimer.scheduledTimerWithTimeInterval(5.0, target: self, selector: "dismissTheScene:", userInfo: nil, repeats: false)
var wait = SKAction.waitForDuration(0.5)
var run = SKAction.runBlock{
// your code here ...
self.explodeSomeRocks()
}
let runIt = SKAction.sequence([wait,run])
self.runAction(SKAction.repeatActionForever(runIt), withKey: "blowingUpRocks")
var dismissalWait = SKAction.waitForDuration(5.0)
var dismissalRun = SKAction.runBlock{
self.removeActionForKey("blowingUpRocks")
self.dismissTheScene()
}
self.runAction(SKAction.sequence([dismissalWait,dismissalRun]))
}
func explodeSomeRocks()
{
println("Timer fired")
}
//MARK: - Dismiss back to the level selector
func dismissTheScene()
{
let skView = self.view as SKView?
var nextScene = SKScene()
nextScene = LevelChooserScene()
nextScene.size = skView!.bounds.size
nextScene.scaleMode = .AspectFill
var sceneTransition = SKTransition.fadeWithColor(UIColor.blackColor(), duration: 1.5) //WithDuration(2.0)
//var sceneTransition = SKTransition.pushWithDirection(SKTransitionDirection.Down, duration: 0.75) //WithDuration(2.0)
//var sceneTransition = SKTransition.crossFadeWithDuration(1.0)
//var sceneTransition = SKTransition.doorwayWithDuration(1.0)
sceneTransition.pausesOutgoingScene = true
skView!.presentScene(nextScene, transition: sceneTransition)
}
}
Well the thing that was causing my trouble was inserting particle emitters every half second for 5 seconds using SKAction.repeatActionForever() to call the emitter insert function.
This repeatAction apparently was not killed by transitioning to another scene, and was causing the memory for the whole scene to be retained. I switched to SKAction.repeatAction() instead and specify how many time it should fire. The scene now returns all of its memory when I transition to the new scene.
I am not sure I understand this behavior though.
SpriteKit it's not strongly documented when it comes to create complex games. I personally had a problem like this for days until I managed to figure it out.
Some objects retain the reference, so it doesn't deinit. (SKActions, Timers, etc)
Before presenting a new scene I call a prepare_deinit() function where I manually remove the strong references which are usually not deallocated by swift.
func prepare_deinit()
{
game_timer.invalidate() // for Timer()
removeAction(forKey: "blowingUpRocks") // for SKAction in your case
// I usually add the specific actions to an object and then remove
object.removeAllActions()
// If you create your own object/class that doesn't deinit, remove all object
//actions and the object itself
custom_object.removeAllActions()
custom_object.removeFromParent()
}
deinit
{
print("GameScene deinited")
}
The last problem I encountered was that the new scene was presented much faster than my prepare_deinit() so I had to present the new scene a little later, giving the prepare_deinit() enough time to deallocate all objects.
let new_scene =
{
let transition = SKTransition.flipVertical(withDuration: 1.0)
let next_scene = FinishScene(fileNamed: "FinishScene")
next_scene?.scaleMode = self.scaleMode
next_scene?.name = "finish"
self.view?.presentScene(next_scene!, transition: transition)
}
run(SKAction.sequence([SKAction.run(prepare_deinit), SKAction.wait(forDuration: 0.25), SKAction.run(exit_to_finish)]))