I put some objects in AR space using ARKit and SceneKit. That works well. Now I'd like to add an additional camera (SCNCamera) that is placed elsewhere in the scene attached and positioned by a common SCNNode. It is oriented to show me the current scene from an other (fixed) perspective.
Now I'd like to show this additional SCNCamera feed on i.Ex. a SCNPlane (as the diffuse first material) - Like a TV screen. Of course I am aware that it will only display the SceneKit content which stays in the camera focus and not rest of the ARKit image (which is only possible by the main camera of course). A simple colored background then would be fine.
I have seen tutorials that describes, how to play a video file on a virtual display in ARSpace, but I need a realtime camera feed from my own current scene.
I defined this objects:
let camera = SCNCamera()
let cameraNode = SCNNode()
Then in viewDidLoad I do this:
camera.usesOrthographicProjection = true
camera.orthographicScale = 9
camera.zNear = 0
camera.zFar = 100
cameraNode.camera = camera
sceneView.scene.rootNode.addChildNode(cameraNode)
Then I call my setup function to place the virtual Display next to all my AR stuff, position the cameraNode as well (pointing in the direction where objects stay in the scene)
cameraNode.position = SCNVector3(initialStartPosition.x, initialStartPosition.y + 0.5, initialStartPosition.z)
let cameraPlane = SCNNode(geometry: SCNPlane(width: 0.5, height: 0.3))
cameraPlane.geometry?.firstMaterial?.diffuse.contents = cameraNode.camera
cameraPlane.position = SCNVector3(initialStartPosition.x - 1.0, initialStartPosition.y + 0.5, initialStartPosition.z)
sceneView.scene.rootNode.addChildNode(cameraPlane)
Everything compiles and loads... The display shows up at the given position, but it stays entirely gray. Nothing is displayed at all from the SCNCamera I put in the scene. Everything else in the AR scene works well, I just don't get any feed from that camera.
Hay anyone an approach to get this scenario working?
To even better visualize, I add some more print screens.
The following shows the Image trough the SCNCamera according to ARGeo's input. But it takes the whole screen, instead of displaying its contents on a SCNPlane, like I need.
The next Print screen actually shows the current ARView result as I got it using my posted code. As you can see, the gray Display-Plane remains gray - it shows nothing.
The last print screen is a photomontage, showing the expected result, as I'd like to get.
How could this be realized? Am I missing something fundamental here?
After some research and sleep, I came to the following, working solution (including some inexplainable obstacles):
Currently, the additional SCNCamera feed is not linked to a SCNMaterial on a SCNPlane, as it was the initial idea, but I will use an additional SCNView (for the moment)
In the definitions I add an other view like so:
let overlayView = SCNView() // (also tested with ARSCNView(), no difference)
let camera = SCNCamera()
let cameraNode = SCNNode()
then, in viewDidLoad, I setup the stuff like so...
camera.automaticallyAdjustsZRange = true
camera.usesOrthographicProjection = false
cameraNode.camera = camera
cameraNode.camera?.focalLength = 50
sceneView.scene.rootNode.addChildNode(cameraNode) // add the node to the default scene
overlayView.scene = scene // the same scene as sceneView
overlayView.allowsCameraControl = false
overlayView.isUserInteractionEnabled = false
overlayView.pointOfView = cameraNode // this links the new SCNView to the created SCNCamera
self.view.addSubview(overlayView) // don't forget to add as subview
// Size and place the view on the bottom
overlayView.frame = CGRect(x: 0, y: 0, width: self.view.bounds.width * 0.8, height: self.view.bounds.height * 0.25)
overlayView.center = CGPoint(x: self.view.bounds.width * 0.5, y: self.view.bounds.height - 175)
then, in some other function, I place the node containing the SCNCamera to my desired position and angle.
// (exemplary)
cameraNode.position = initialStartPosition + SCNVector3(x: -0.5, y: 0.5, z: -(Float(shiftCurrentDistance * 2.0 - 2.0)))
cameraNode.eulerAngles = SCNVector3(-15.0.degreesToRadians, -15.0.degreesToRadians, 0.0)
The result, is a kind of window (the new SCNView) at the bottom of the screen, displaying the same SceneKit content as in the main sceneView, viewed trough the perspective of the SCNCamera plus its node position, and that very nicely.
In a common iOS/Swift/ARKit project, this construct generates some side effects, that one may struggle into.
1) Mainly, the new SCNView shows SceneKit content from the desired perspective, but the background is always the actual physical camera feed. I could not figure out, how to make the background a static color, by still displaying all the SceneKit content. Changing the new scene's background property affects also the whole main scene, what is actually NOT desired.
2) It might sound confusing, but as soon as the following code get's included (which is essential to make it work):
overlayView.scene = scene
the animation speed of the entire scenes (both) DOUBLES! (Why?)
I got this corrected by adding/changing the following property, which restores the animation speed behavour almost like normal (default):
// add or change this in the scene setup
scene.physicsWorld.speed = 0.5
3) If there are actions like SCNAction.playAudio in the project, all the effects will no longer play - as long as I don't do this:
overlayView.scene = nil
Of course, the additional SCNView stops working but everything else gets gets back to its normal.
Use this code (as a starting point) to find out how to setup a virtual camera.
Just create a default ARKit project in Xcode and copy-paste my code:
import UIKit
import SceneKit
import ARKit
class ViewController: UIViewController, ARSCNViewDelegate {
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
sceneView.delegate = self
sceneView.showsStatistics = true
let scene = SCNScene(named: "art.scnassets/ship.scn")!
sceneView.scene = scene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(0, 0, 1)
cameraNode.camera?.focalLength = 70
cameraNode.camera?.categoryBitMask = 1
scene.rootNode.addChildNode(cameraNode)
sceneView.pointOfView = cameraNode
sceneView.allowsCameraControl = true
sceneView.backgroundColor = UIColor.darkGray
let plane = SCNNode(geometry: SCNPlane(width: 0.8, height: 0.45))
plane.position = SCNVector3(0, 0, -1.5)
// ASSIGN A VIDEO STREAM FROM SCENEKIT-RECORDER TO YOUR MATERIAL
plane.geometry?.materials.first?.diffuse.contents = capturedVideoFromSceneKitRecorder
scene.rootNode.addChildNode(plane)
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration()
sceneView.session.run(configuration)
}
}
UPDATED:
Here's a SceneKit Recorder App that you can tailor to your needs (you don't need to write a video to disk, just use a CVPixelBuffer stream and assign it as a texture for a diffuse material).
Hope this helps.
I'm a little late to the party, but I've had a similar issue recently.
As far as I can tell, you cannot directly connect a camera to a node's material. You can, however, use a scene's layer as a texture for a node.
The code below is not verified, but should be more or less ok:
class MyViewController: UIViewController {
override func loadView() {
let projectedScene = createProjectedScene()
let receivingScene = createReceivingScene()
let projectionPlane = receivingScene.scene?.rootNode.childNode(withName: "ProjectionPlane", recursively: true)!
// Here's the important part:
// You can't directly connect a camera to a material's diffuse texture.
// But you can connect a scene's layer as a texture.
projectionPlane.geometry?.firstMaterial?.diffuse.contents = projectedScene.layer
projectedScene.layer.contentsScale = 1
// Note how we only need to connect the receiving view to the controller.
// The projected view is not directly connected as a subview,
// but updates in projectedScene will still be reflected in receivingScene.
self.view = receivingScene
}
func createProjectedScene() -> SCNView {
let view = SCNView()
// ... set up scene ...
return view
}
func createReceivingScene() -> SCNView {
let view = SCNView()
// ... set up scene ...
let projectionPlane = SCNNode(geometry: SCNPlane(width: 2, height: 2)
projectionPlane.name = "ProjectionPlane"
view.scene.rootNode.addChildNode(projectionPlane)
return view
}
}
I am trying to add sprites to my SKScene, but the coordinate system seems to be weird. The [0, 0] point is in the middle of the screen and not in the lower left corner like it should be. I have no idea how this could have happened as I have tried correcting using a variety of methods I have found in similar questions.
Here is my GameViewController class:
class GameViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
if let view = self.view as! SKView? {
if let scene = MenuScene(fileNamed: "MenuScene"){
scene.scaleMode = .aspectFill
view.presentScene(scene)
}
view.ignoresSiblingOrder = true
view.showsFPS = true
view.showsNodeCount = true
}
}
I have tried changed the scaleMode which only changed the scale of sprites and didn't fix the issue. I have also tried changing MenuScene(fileNamed: ) to MenuScene(size: view.bounds.size) which gave me errors. If I am missing any details, please ask.
That is happening because by default, SKScenes have an anchorPoint of (0.5, 0.5). This places the point (0,0) in the centre. If you're looking to change the anchorPoint to the bottom-left, add this to the beginning of didMoveToView:
self.anchorPoint = CGPoint(x: 0, y: 0)
Or initialize the scene with that anchorPoint before presenting it (from GameViewController):
scene.anchorPoint = CGPoint(x: 0, y: 0)
Or change it in the scene editor.
FYI: CGPoint(x: 0, y: 0) is the same as CGPoint.zero
Personally, I find it easier to build the scene outwards from the centre because it makes symmetry easier and laying out HUD easier, instead of from a corner, which results in building the scene from the corner, potentially creating difficulties.
I'm trying to animate transform from '+' button to 'X' button by rotating it on click.
Everything works great when I'm using standard button on UIView. I'm using following code:
UIView.animateWithDuration(2.0, delay: 0.0, options: .CurveEaseOut, animations: {
let transform = CGAffineTransformMakeRotation(CGFloat(M_PI_4))
self.navigationPlusButton.transform = transform
}, completion: nil)
The problem occurs when I try to achieve the same with button in navigation bar.
Looks like button height is resized to 0 before the rotation and I don't know how to fix navigation button size since there are no constraints definitions. Note storyboard
and for constraints
Note: Second approach using CABasicAnimation rotated button nicely but it returned it to initial state immediately after animation ends. I would like to stay 'X' until I click again.
let rotateAnimation = CABasicAnimation(keyPath: "transform.rotation")
rotateAnimation.fromValue = 0.0
rotateAnimation.toValue = CGFloat(M_PI_4)
rotateAnimation.duration = 2.0
self.navigationPlusButton.layer.addAnimation(rotateAnimation, forKey: nil)
How it works so easily outside the navigation bar and it is so hard to do it on navigation bar? I can't find the solution. Please help
try following lines to your code, it will solve your problem.
rotateAnimation.fillMode = kCAFillModeForwards
rotateAnimation.removedOnCompletion = false
Try this line on the CABasicAnimation:
rotateAnimation.removedOnCompletion = false
It may solve your problem.
I am using a curl up animation when swiping up, which makes a card fly away to the top - it works fine. I'd like to use the same animation, but to swipe down, so that the card flies away down. I guess I'd have to somehow turn the swipe up animation 180 degrees. Is that possible?
let views = (frontView: self.frontView, backView: self.frontView)
// set a transition style
let transitionOptions = UIViewAnimationOptions.TransitionCurlUp
UIView.transitionWithView(self.flashCardView, duration: 0.5, options: transitionOptions, animations: {
views.frontView.removeFromSuperview()
self.flashCardView.addSubview(views.frontView)
}, completion: { finished in
// any code entered here will be applied
})
The UIViewAnimationOptions struct has another option that will solve this for you:
let transitionOptions = UIViewAnimationOptions.TransitionCurlDown
I'm new to Swift and Spritekit and having trouble with implementing a button or imagebutton that will take you to the main menu from the GameScene. All I want to do is to make a button that will display a dialog that will tell you if you want to return to the main menu.
For example is there a way to make a dialog that will transition you to other scenes with the code below?
let skview = self.view as! GameSKView!
skview.score = score
let scene = MainMenuScene(size: self.size)
let tr = SKTransition.revealWithDirection(SKTransitionDirection.Down, duration: 1)
scene.scaleMode = SKSceneScaleMode.AspectFill
skview.presentScene(scene,transition: tr)
try this
func changeScene() {
let newScene = MainMenuScene(size: self.size)
let theTransition = SKTransition.revealWithDirection(SKTransitionDirection.Down, duration: 1)
newScene.scaleMode = .AspectFill
scene?.view?.presentScene(newScene, transition: theTransition)
}
Currently developing a button that works like a uibutton using spritekit. Here is what I have, it may be slightly overkill for what you want but here it is.https://github.com/marcJV/SKAToolKit/tree/swift?files=1
It will be pulled over to SKAToolKit when as we are porting it to swift. There is also an example project showing off how to use it. Also there is a lack of a proper readme at this point, but the example should have everything you need. This is using swift 2.0