I'm working on creating an SCNNode instance that consists of an SCNPlane - type geometry, with a video playing on the surface of the node. This is what I have so far:
let plane = SCNPlane(width: 5, height: 5)
node = SCNNode(geometry: plane)
GlobalScene.shared.rootNode.addChildNode(node!)
let urlStr = Bundle.main.path(forResource: "test", ofType: "mov")
videoURL = NSURL(fileURLWithPath: urlStr!)
player = AVPlayer(playerItem: AVPlayerItem(url: videoURL! as URL))
let videoNode = SKVideoNode(avPlayer: player!)
player?.actionAtItemEnd = .none
let spritescene = SKScene(size: CGSize(width: 122, height: 431))
videoNode.size.width=spritescene.size.width
videoNode.size.height=spritescene.size.height
spritescene.addChild(videoNode)
plane.firstMaterial?.diffuse.contents = spritescene
The overall functionality so far works great! What I'm trying to figure out is, how to I get the node's ENTIRE surface to be made up of the video ? So far, I've only gotten it to appear in the single corner shown below:
EDIT: it looks like the issue is that I'm only setting the first material of the plane node (rather than all of them...) which is why I'm seeing the other 3 "quadrants" as blank.
EDIT2: That first conclusion may not be correct - if I set:
plane.firstMaterial?.diffuse.contents = NSColor.green
I get the following result:
So...why won't that work when applying the contents of a SpriteKit scene?
Using a SKScene and a SKVideoNode is not necessary. You can directly set the AVPlayer as the contents of a SCNMaterialProperty instance. This will allow for better performance and will avoid having to deal with scaling and positioning the SpriteKit elements.
With the help of this gist, I was able to solve my issue; which ended up involving both scaling the video node properly, as well as setting the position properly. My working code is below:
GlobalScene.shared.rootNode.addChildNode(node!)
let urlStr = Bundle.main.path(forResource: "test", ofType: "mov")
videoURL = NSURL(fileURLWithPath: urlStr!)
player = AVPlayer(playerItem: AVPlayerItem(url: videoURL! as URL))
videoSpriteKitNode = SKVideoNode(avPlayer: player!)
player?.actionAtItemEnd = .none
NotificationCenter.default.addObserver(
self,
selector: #selector(self.playerItemDidReachEnd),
name: NSNotification.Name.AVPlayerItemDidPlayToEndTime,
object: player?.currentItem)
let spriteKitScene = SKScene(size: CGSize(width: 1276.0 / 2.0, height: 712.0 / 2.0))
videoSpriteKitNode?.size.width=spriteKitScene.size.width
videoSpriteKitNode?.size.height=spriteKitScene.size.height
node?.geometry?.firstMaterial?.diffuse.contentsTransform = SCNMatrix4Translate(SCNMatrix4MakeScale(1, -1, 1), 0, 1, 0)
videoSpriteKitNode?.position = CGPoint(x: spriteKitScene.size.width / 2.0, y: spriteKitScene.size.height / 2.0)
videoSpriteKitNode?.size = spriteKitScene.size
videoSpriteKitNode?.play()
spriteKitScene.addChild(videoSpriteKitNode!)
plane?.firstMaterial?.diffuse.contents = spriteKitScene
Related
on my study project using ARKit and sceneKit I'm facing a strange behaviour when using .dae file.
i have created 2 object a cube and a plane with the following methods (both have physics):
func loadBox() {
let ballGeometry = SCNBox(width: 0.2, height: 0.2, length: 0.2, chamferRadius: 0.02)
let mat = SCNMaterial()
mat.diffuse.contents = UIImage(named: "ball")
ballGeometry.materials = [mat]
//Physic
let physSchape = SCNPhysicsShape(geometry: ballGeometry, options: nil)
let ballPhysic = SCNPhysicsBody(type: .dynamic, shape: physSchape)
ballPhysic.mass = 100
ballPhysic.friction = 0.8
ballPhysic.isAffectedByGravity = true
let nodeBall = SCNNode(geometry: ballGeometry)
nodeBall.name = "ball"
nodeBall.physicsBody = ballPhysic
box = nodeBall
}
func loadPlane(){
let geometry = SCNPlane(width: 1, height: 1)
let material = SCNMaterial()
material.diffuse.contents = UIImage(named: "texture")
geometry.materials = [material]
// physics
let physSchape = SCNPhysicsShape(geometry: geometry, options: nil)
let planePhysic = SCNPhysicsBody(type: .static, shape: physSchape)
planePhysic.restitution = 0.0
planePhysic.friction = 1.0
planePhysic.isAffectedByGravity = true
let nodo = SCNNode(geometry: geometry)
nodo.eulerAngles.x = -.pi / 2
nodo.physicsBody = planePhysic
plane = nodo
}
when I place the box over the plane in the scene they are correctly positioned and touching each other as you can see from the following picture:
Here the problem:
if i use a model downloaded from internet as .dae file of the plane, when insert it in the scene it create a strange space between the box and the plane (see picture)
method I use to load the scn file.
// load the elicopterBase
func loadElicopterBase(){
guard let scene = SCNScene(named: "Model.scnassets/base.scn") else {fatalError()}
guard let nodo = scene.rootNode.childNode(withName: "Plane", recursively: true) else {fatalError()}
// physics
let physSchape = SCNPhysicsShape(node: nodo, options: nil)
let planePhysic = SCNPhysicsBody(type: .static, shape: physSchape)
planePhysic.restitution = 0.0
planePhysic.friction = 1.0
planePhysic.isAffectedByGravity = true
nodo.physicsBody = planePhysic
nodo.name = "base"
DispatchQueue.main.async {
self.eliporto = nodo
}
}
Look like the .dae file have some sort of invisible boundingBox around it.
What or where I can look to find a solution to this.
here some picture of the .dae file in Xcode
Make your floor plane of type "concavePolyhedron" like so:
let body = SCNPhysicsBody(type: .static, shape: SCNPhysicsShape(geometry: yourGeometry, options: [SCNPhysicsShape.Option.type: SCNPhysicsShape.ShapeType.concavePolyhedron]))
I have a UIView which Is supposed to play a video using AVPlayerLayer and AVPlayer.
I set the player in this way:
fileprivate func setUpPlayer(){
let urlPathString = Bundle.main.path(forResource: "dance", ofType: "mp4")
if let videoURL = urlPathString{
let url = URL(fileURLWithPath: videoURL)
player = AVPlayer(url: url)
playerLayer = AVPlayerLayer(player: player)
playerLayer.videoGravity = .resizeAspectFill
self.playerView.layer.addSublayer(playerLayer)
self.playerView.layer.masksToBounds = true
}
}
override func layoutSubviews() {
self.layer.cornerRadius = 15
playerLayer.frame = self.bounds
self.setupShadow(opacity: 0.6, radius: 6, offset: CGSize.init(width: 0, height: 0), color: .darkGray)
}
The problem is that no matter how many subviews I set over the PlayerView, the video actually goes over everything hiding all the subviews I set before.
Do you know how I can put buttons or actually other UIViews over AVPlayerLayer without it actually hiding them?
Change
self.playerView.layer.addSublayer(playerLayer)
to
self.playerView.layer.insertSublayer(playerLayer, at: 0)
Currently, when I build to my phone I see the SCNPlane pop over an image the camera is detecting. The SCNPlane does not have an issue adjusting to the size of the image it detects. What I am having trouble with is taking the SCNPlane and replacing it with a SKVideoNode and its SKScene that can also auto adjust its size to the image.
Any Ideas?
Thank You!
if let imageAnchor = anchor as? ARImageAnchor
{
//get plane
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
plane.firstMaterial?.diffuse.contents = UIColor(white: 1, alpha: 0.8)
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles.x = -.pi / 2
// get container
guard let container = ARSceneView.scene.rootNode.childNode(withName: "Container", recursively: false) else {return}
container.removeFromParentNode()
container.position.y = planeNode.position.y + Float(CGFloat (0.25))
container.position.z = planeNode.position.z + Float(CGFloat (0.1))
planeNode.addChildNode(container)
node.addChildNode(planeNode)
container.isHidden = false
//VIDEO SCENE
let videoURL = Bundle.main.url(forResource: "video", withExtension: "mp4")!
let videoPlayer = AVPlayer(url:videoURL)
let videoPlane = SKScene(size: CGSize(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height))
//videoPlane = SKScene(size: CGSize(width: 720.0, height: 1280.0))
let videoNode = SKVideoNode(avPlayer: videoPlayer)
videoNode.play()
videoPlane.addChild(videoNode)
guard let video = container.childNode(withName: "Video", recursively: true) else {return}
video.geometry?.firstMaterial?.diffuse.contents = videoPlane
can someone please advise.
I am trying to add a text overlay (title) to a video I am composing using AVFoundation. I found a few online resources (see http://stackoverflow.com/questions/21684549/add-a-text-overlay-with-avmutablevideocomposition-to-a-specific-timerange)
However all these resources are in Objective-C.
My project is in Swift and I cannot find any related resources in Swift.
I am not able to get the text to overlay properly is seems distorted as if the frame in which is gets rendered is skewed...
See picture Distorted text in AVPlayer
I have attempted to convert the Objective-C code I found to Swift but obviously I am missing something.
Below is the code I am using.
(I used some code for the player and the video file from:www.raywenderlich.com/90488/calayer-in-ios-with-swift-10-examples
func MergeUnWeldedVideoByUserPref(showInBounds: CGRect) -> (AVMutableComposition, AVMutableVideoComposition)
{
let fps: Int32 = 30
// 1 - Create AVMutableComposition object. This object will hold your AVMutableCompositionTrack instances.
let mixComposition = AVMutableComposition()
// 2 - Create a video track for each of the video assests. Add your media data to the appropriate tracks
//let url = NSBundle.mainBundle().URLForResource("colorfulStreak", withExtension: "m4v")!
let url = NSBundle.mainBundle().URLForResource("colorfulStreak", withExtension: "m4v")!
let avAsset = AVAsset(URL: url)
let track = mixComposition.addMutableTrackWithMediaType(AVMediaTypeVideo, preferredTrackID: Int32(kCMPersistentTrackID_Invalid))
let segmentInMovie = CMTimeRangeMake(kCMTimeZero, avAsset.duration)
let videoTrack = avAsset.tracksWithMediaType(AVMediaTypeVideo)[0]
do
{
try track.insertTimeRange(segmentInMovie, ofTrack: videoTrack, atTime: kCMTimeZero)
} catch{
print("Failed to load track")
}
let mainInstruction = AVMutableVideoCompositionInstruction()
mainInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, avAsset.duration)
let instruction = videoCompositionInstructionForTrack(showInBounds, track: track, asset: avAsset)
mainInstruction.layerInstructions.append(instruction)
let mainComposition = AVMutableVideoComposition()
mainComposition.instructions = [mainInstruction]
mainComposition.frameDuration = CMTimeMake(1, fps)
mainComposition.renderSize = CGSize(width: showInBounds.width, height: showInBounds.height)
let textLayer = CATextLayer()
textLayer.backgroundColor = UIColor.clearColor().CGColor
textLayer.foregroundColor = UIColor.whiteColor().CGColor
textLayer.string = "T E S T"
textLayer.font = UIFont(name: "Arial", size: 18)
textLayer.shadowOpacity = 0.5
textLayer.alignmentMode = kCAAlignmentCenter
textLayer.frame = CGRectMake(5, 5, 100, 50)
textLayer.shouldRasterize = true
textLayer.rasterizationScale = showInBounds.width / videoTrack.naturalSize.width
let parentLayer = CALayer()
let videoLayer = CALayer()
parentLayer.frame = CGRectMake(0, 0, showInBounds.width, showInBounds.height);
videoLayer.frame = CGRectMake(0, 0, showInBounds.width, showInBounds.height);
parentLayer.addSublayer(videoLayer)
parentLayer.addSublayer(textLayer)
mainComposition.animationTool = AVVideoCompositionCoreAnimationTool(postProcessingAsVideoLayer: videoLayer, inLayer: parentLayer)
return (mixComposition, mainComposition)
}
There is nothing wrong with your Swift interpretation and is rather an issue with the rendering engine of the simulator. I tried your code on the simulator and it indeed looked skewed and distorted but when compiling to the device it worked beautifully.
If you can help me add this animation to the scene or if you know how to add an animation to the scene, it would be much appreciated.
Code:
let BallScene = SCNScene(named: "art.scnassets/Ball.dae")!
let ManScene = SCNScene(named: "art.scnassets/Genric_Simplifiyed9.dae")!
let Man: SCNNode = ManScene.rootNode.childNodeWithName("Bob_001", recursively: true)!
Man.position = SCNVector3(x: 0, y: 0, z: 5)
BallScene.rootNode.addChildNode(Man)
let Animation: CAAnimation
let AnimationUrl: NSURL = NSBundle.mainBundle().URLForResource("art.scnassets/Genric_Simplifiyed9.dae", withExtension: "dae")!
let sceneSource = SCNSceneSource(URL: AnimationUrl, options: [SCNSceneSourceAnimationImportPolicyKey : SCNSceneSourceAnimationImportPolicyDoNotPlay])
sceneSource.addAnimation(attackAnimation, forKey: "attack”)
If you have a DAE/COLLADA file with animations and you can just click play button to play the animation in your storyboard.
Then you can just
let subScene = SCNScene(named: "art.scnassets/test_ball_2011.dae")!
let childNodes = subScene.rootNode.childNodes
for child in childNodes {
//child.position = SCNVector3(0, 0, -100)
rootScene.rootNode.addChildNode(child)
}
The animation will play in your rootScene
and you can use
for child in childNodes {
let keys = child.animationKeys
child.position = SCNVector3(0, 0, -100)
for key in keys {
let animation = child.animationForKey(key)!
animation.repeatCount = 5
child.removeAnimationForKey(key)
child.addAnimation(animation, forKey: key)
}
rootScene.rootNode.addChildNode(child)
to set the position or the repeatCount...
Another way like your code:
you can use
let url = NSBundle.mainBundle().URLForResource(/* your DAE file */)
let sceneSource = SCNSceneSource(URL: url, options: [
SCNSceneSourceAnimationImportPolicyKey : SCNSceneSourceAnimationImportPolicyPlayRepeatedly
])!
to get the source and in the options you can set many things, just check API
and use
let node = sceneSource.entryWithIdentifier("/* the node name */", withClass: SCNNode.self)
let animation = sceneSource.entryWithIdentifier("/* the animation name */", withClass: CAAnimation.self)
to get the node and the animation. If the two files is in different file, just make 2 sources to get :)
the last thing is node.addAnimation(animation, forKey: nil) // check the API