I put some objects in AR space using ARKit and SceneKit. That works well. Now I'd like to add an additional camera (SCNCamera) that is placed elsewhere in the scene attached and positioned by a common SCNNode. It is oriented to show me the current scene from an other (fixed) perspective.
Now I'd like to show this additional SCNCamera feed on i.Ex. a SCNPlane (as the diffuse first material) - Like a TV screen. Of course I am aware that it will only display the SceneKit content which stays in the camera focus and not rest of the ARKit image (which is only possible by the main camera of course). A simple colored background then would be fine.
I have seen tutorials that describes, how to play a video file on a virtual display in ARSpace, but I need a realtime camera feed from my own current scene.
I defined this objects:
let camera = SCNCamera()
let cameraNode = SCNNode()
Then in viewDidLoad I do this:
camera.usesOrthographicProjection = true
camera.orthographicScale = 9
camera.zNear = 0
camera.zFar = 100
cameraNode.camera = camera
sceneView.scene.rootNode.addChildNode(cameraNode)
Then I call my setup function to place the virtual Display next to all my AR stuff, position the cameraNode as well (pointing in the direction where objects stay in the scene)
cameraNode.position = SCNVector3(initialStartPosition.x, initialStartPosition.y + 0.5, initialStartPosition.z)
let cameraPlane = SCNNode(geometry: SCNPlane(width: 0.5, height: 0.3))
cameraPlane.geometry?.firstMaterial?.diffuse.contents = cameraNode.camera
cameraPlane.position = SCNVector3(initialStartPosition.x - 1.0, initialStartPosition.y + 0.5, initialStartPosition.z)
sceneView.scene.rootNode.addChildNode(cameraPlane)
Everything compiles and loads... The display shows up at the given position, but it stays entirely gray. Nothing is displayed at all from the SCNCamera I put in the scene. Everything else in the AR scene works well, I just don't get any feed from that camera.
Hay anyone an approach to get this scenario working?
To even better visualize, I add some more print screens.
The following shows the Image trough the SCNCamera according to ARGeo's input. But it takes the whole screen, instead of displaying its contents on a SCNPlane, like I need.
The next Print screen actually shows the current ARView result as I got it using my posted code. As you can see, the gray Display-Plane remains gray - it shows nothing.
The last print screen is a photomontage, showing the expected result, as I'd like to get.
How could this be realized? Am I missing something fundamental here?
After some research and sleep, I came to the following, working solution (including some inexplainable obstacles):
Currently, the additional SCNCamera feed is not linked to a SCNMaterial on a SCNPlane, as it was the initial idea, but I will use an additional SCNView (for the moment)
In the definitions I add an other view like so:
let overlayView = SCNView() // (also tested with ARSCNView(), no difference)
let camera = SCNCamera()
let cameraNode = SCNNode()
then, in viewDidLoad, I setup the stuff like so...
camera.automaticallyAdjustsZRange = true
camera.usesOrthographicProjection = false
cameraNode.camera = camera
cameraNode.camera?.focalLength = 50
sceneView.scene.rootNode.addChildNode(cameraNode) // add the node to the default scene
overlayView.scene = scene // the same scene as sceneView
overlayView.allowsCameraControl = false
overlayView.isUserInteractionEnabled = false
overlayView.pointOfView = cameraNode // this links the new SCNView to the created SCNCamera
self.view.addSubview(overlayView) // don't forget to add as subview
// Size and place the view on the bottom
overlayView.frame = CGRect(x: 0, y: 0, width: self.view.bounds.width * 0.8, height: self.view.bounds.height * 0.25)
overlayView.center = CGPoint(x: self.view.bounds.width * 0.5, y: self.view.bounds.height - 175)
then, in some other function, I place the node containing the SCNCamera to my desired position and angle.
// (exemplary)
cameraNode.position = initialStartPosition + SCNVector3(x: -0.5, y: 0.5, z: -(Float(shiftCurrentDistance * 2.0 - 2.0)))
cameraNode.eulerAngles = SCNVector3(-15.0.degreesToRadians, -15.0.degreesToRadians, 0.0)
The result, is a kind of window (the new SCNView) at the bottom of the screen, displaying the same SceneKit content as in the main sceneView, viewed trough the perspective of the SCNCamera plus its node position, and that very nicely.
In a common iOS/Swift/ARKit project, this construct generates some side effects, that one may struggle into.
1) Mainly, the new SCNView shows SceneKit content from the desired perspective, but the background is always the actual physical camera feed. I could not figure out, how to make the background a static color, by still displaying all the SceneKit content. Changing the new scene's background property affects also the whole main scene, what is actually NOT desired.
2) It might sound confusing, but as soon as the following code get's included (which is essential to make it work):
overlayView.scene = scene
the animation speed of the entire scenes (both) DOUBLES! (Why?)
I got this corrected by adding/changing the following property, which restores the animation speed behavour almost like normal (default):
// add or change this in the scene setup
scene.physicsWorld.speed = 0.5
3) If there are actions like SCNAction.playAudio in the project, all the effects will no longer play - as long as I don't do this:
overlayView.scene = nil
Of course, the additional SCNView stops working but everything else gets gets back to its normal.
Use this code (as a starting point) to find out how to setup a virtual camera.
Just create a default ARKit project in Xcode and copy-paste my code:
import UIKit
import SceneKit
import ARKit
class ViewController: UIViewController, ARSCNViewDelegate {
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
sceneView.delegate = self
sceneView.showsStatistics = true
let scene = SCNScene(named: "art.scnassets/ship.scn")!
sceneView.scene = scene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(0, 0, 1)
cameraNode.camera?.focalLength = 70
cameraNode.camera?.categoryBitMask = 1
scene.rootNode.addChildNode(cameraNode)
sceneView.pointOfView = cameraNode
sceneView.allowsCameraControl = true
sceneView.backgroundColor = UIColor.darkGray
let plane = SCNNode(geometry: SCNPlane(width: 0.8, height: 0.45))
plane.position = SCNVector3(0, 0, -1.5)
// ASSIGN A VIDEO STREAM FROM SCENEKIT-RECORDER TO YOUR MATERIAL
plane.geometry?.materials.first?.diffuse.contents = capturedVideoFromSceneKitRecorder
scene.rootNode.addChildNode(plane)
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration()
sceneView.session.run(configuration)
}
}
UPDATED:
Here's a SceneKit Recorder App that you can tailor to your needs (you don't need to write a video to disk, just use a CVPixelBuffer stream and assign it as a texture for a diffuse material).
Hope this helps.
I'm a little late to the party, but I've had a similar issue recently.
As far as I can tell, you cannot directly connect a camera to a node's material. You can, however, use a scene's layer as a texture for a node.
The code below is not verified, but should be more or less ok:
class MyViewController: UIViewController {
override func loadView() {
let projectedScene = createProjectedScene()
let receivingScene = createReceivingScene()
let projectionPlane = receivingScene.scene?.rootNode.childNode(withName: "ProjectionPlane", recursively: true)!
// Here's the important part:
// You can't directly connect a camera to a material's diffuse texture.
// But you can connect a scene's layer as a texture.
projectionPlane.geometry?.firstMaterial?.diffuse.contents = projectedScene.layer
projectedScene.layer.contentsScale = 1
// Note how we only need to connect the receiving view to the controller.
// The projected view is not directly connected as a subview,
// but updates in projectedScene will still be reflected in receivingScene.
self.view = receivingScene
}
func createProjectedScene() -> SCNView {
let view = SCNView()
// ... set up scene ...
return view
}
func createReceivingScene() -> SCNView {
let view = SCNView()
// ... set up scene ...
let projectionPlane = SCNNode(geometry: SCNPlane(width: 2, height: 2)
projectionPlane.name = "ProjectionPlane"
view.scene.rootNode.addChildNode(projectionPlane)
return view
}
}
I am working with a simple 2d game and I am trying to switch from level 1 (scene one) to level 2 (scene two). For the sake of testing, both levels contain the same content. This is my code for transitioning:
let transition = SKTransition.push(with: SKTransitionDirection.left, duration: 1)
let nextScene = Scene(size: levelBuilder.scene.size)
nextScene.scaleMode = .aspectFill
nextScene.backgroundColor = UIColor(red:0.17, green:0.24, blue:0.31, alpha:1.0)
levelBuilder.scene.view?.presentScene(nextScene, transition: transition)
nextScene.physicsWorld.gravity = CGVector(dx: 0, dy: 0)
nextScene.physicsWorld.contactDelegate = levelBuilder.scene.physicsWorld.contactDelegate
loadLevelContents(level: 2, world: 1, borderPercentage: 30)
This works perfectly, but the issue is not the transitioning but the loading of nodes into scene two which happens here:
loadLevelContents(level: 2, world: 1, borderPercentage: 30)
The positioning goes fine, but the problem is that everything is based according to the center point, or relative (0,0), being in the middle of the screen.
Scene 2 has the center point at the lower left corner
While I need it to be like scene 1, where it is in the middle of the screen:
How can I change this relative point or center point, not sure what it is called, to the middle of the screen for all SKScenes
The point that you are looking for is called anchorPoint. In Spritekit lower left is 0,0 and upper right is 1, 1. So to put it in the center you would have to set and achorPoint of 0.5,0.5 in the didMove func
self.anchorPoint = CGPoint(x: 0.5, y: 0.5)
For SKScenes the default value is (0,0), but it's not just SKScenes that have anchorPoints, SKSpriteNodes have it as well.
I am trying to create game in SpriteKit with parallax effect, but when I implement my method, my hero lags (quickly shakes) from time to time (no pattern observed)...
I wrote this simple code that illustrates the method and also contains the same problem.
The hero should stay in the center of the screen and only his position relative to his parent (foreground) should change. But it lags (quickly shakes for a really short moment) every now and then. I also noticed that when I run the game, the hero sinks down very, very slowly a few pixels (maybe 5-10) before it starts holding its position after ~15 sec.
I can't figure out what is causing this behavior. Any help would be appreciated. I am testing on IPhone 6.
import SpriteKit
class GameScene: SKScene, SKPhysicsContactDelegate {
let hero = SKSpriteNode(imageNamed: "hero")
let foreground = SKNode()
required init?(coder aDecoder: NSCoder) {
super.init(coder: aDecoder)
}
override init(size: CGSize) {
super.init(size: size)
physicsWorld.gravity = CGVector(dx: 0.0, dy: -0.5)
physicsWorld.contactDelegate = self
// anchor point of scene is at (x: 0.5, y: 0.5)
hero.position = CGPoint(x: 0, y: 0)
hero.physicsBody = SKPhysicsBody(rectangleOfSize: hero.size)
hero.physicsBody?.dynamic = true
hero.physicsBody?.allowsRotation = false
addChild(foreground)
foreground.addChild(hero)
}
override func update(currentTime: NSTimeInterval) {
foreground.position = CGPoint(x: 0, y: -(hero.position.y))
}
}
// I need the foreground to move in opposite direction so I can add stars,
// monsters, etc while keeping the player in the view... Update method does
// this: hero is pulled down by gravity so his y position would be -1 pixel
// relative to foreground but I assign foreground the opposite y-position of
// hero (+1) and since hero's position relative to foreground is -1, he should
// be positioned at y-position = 0 which, in this code example, should create
// an illusion that he is positioned at the center and doesn't move
Im making a game with swift and spriteKit with xcode 6.1.
My problem is that, when I run the game on the iPhone 6 or iPhone 6+ the scene/images dont resize properly
I have this on my GameViewController
// Configure the view.
let skView = view as SKView
skView.multipleTouchEnabled = false
// Create and configure the scene.
scene = GameScene(size: skView.bounds.size)
scene.scaleMode = .AspectFill
// Present the scene.
skView.presentScene(scene)
And this on my GameScene
override init(size: CGSize) {
super.init(size: size)
anchorPoint = CGPoint(x: 0.5, y: 0.5)
let background = SKSpriteNode(imageNamed: "Background")
addChild(background)
}
I have tried all the scale modes but they dont seem to work. I hope someone can tell me what is going on
The size of the scene is being set to the size of the view in
scene = GameScene(size: skView.bounds.size)
so it is only the images in the scene that are too small to fill the view.
Make these changes to your GameViewController:
if you set the size of the scene to the size of the background image:
scene = GameScene(size: CGSize(width: backgroundImageWidth, height: backgroundImageHeight))
(replace backgroundImageWidth and backgroundImageHeight with actual values)
then set the scene's scale mode to fill the view:
scene.scaleMode = .Fill
(this may make the scene look stretched, you could use other scale modes)
I've been getting stuck doing what should be really simple stuff in Xcode, I am trying to add a sprite to the scene and for it to sit in the bottom left corner:
var groundTexture : SKTexture = SKTexture(imageNamed: "dirt-block")
var ground : SKSpriteNode = SKSpriteNode(texture: groundTexture)
ground.position = CGPointMake(ground.size.width/2, ground.size.height/2)
self.addChild(ground)
This doesn't show anything, so I took a look at what the scene dimensions are and self.frame.size.width and self.frame.size.height are returning a width of 1024 and a height of 768, regardless of orientation.
I want this fixed in landscape mode if that makes any difference to the answer?
I'm obviously missing a very fundamental idea to do with setting up the scene, if anybody could shed any light it would be much appreciated.
Thanks.
You have to edit the GameViewController:
Cut everything from viewDidLoad() except super.viewDidLoad()
Overwrite viewWillLayoutSubviews
override func viewWillLayoutSubviews() {
super.viewWillLayoutSubviews()
if let scene = GameScene.unarchiveFromFile("GameScene") as? GameScene {
// Configure the view.
var skView = self.view as SKView
skView.showsFPS = true
skView.showsNodeCount = true
/* Sprite Kit applies additional optimizations to improve rendering performance */
skView.ignoresSiblingOrder = true
/* Set the scale mode to scale to fit the window */
scene.size = skView.bounds.size
scene.scaleMode = .AspectFill
skView.presentScene(scene)
}
}
Also you should set the scene size as you can see
scene.size = skView.bounds.size
Hope this helps
There is also a great Tutorial where this is explained better:
http://www.raywenderlich.com/49721/how-to-create-a-breakout-game-using-spritekit
(Chapter 1 or 2)