Is there a way to programmatically change the material of an Entity that was created in Reality Composer? - swift

I want to change the color of an entity programmatically after it was created in Reality Composer.
As Reality Composer does not create a ModelEntity (it creates a generic Entity), it does not appear that I have access to change its color. When I typecast to a ModelEntity, I now have access to the ModelComponent materials. However, when I try to add that to the scene I get a Thread 1: signal SIGABART error. Could not cast value of type 'RealityKit.Entity' (0x1fcebe6e8) to 'RealityKit.ModelEntity' (0x1fceba970). Sample code below.
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
// Load the "Box" scene from the "Experience" Reality File
let boxAnchor = try! Experience.loadBox()
// Typecast Steelbox as ModelEntity to change its color
let boxModelEntity = boxAnchor.steelBox as! ModelEntity
// Remove materials and create new material
boxModelEntity.model?.materials.removeAll()
let blueMaterial = SimpleMaterial(color: .blue, isMetallic: false)
boxModelEntity.model?.materials.append(blueMaterial)
// Add the box anchor to the scene
arView.scene.anchors.append(boxAnchor)
}
}

Model entity is stored deeper in RealityKit's hierarchy, and as you said, it's Entity, not ModelEntity. So use downcasting to access mesh and materials:
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
let boxScene = try! Experience.loadBox()
print(boxScene)
let modelEntity = boxScene.steelBox?.children[0] as! ModelEntity
let material = SimpleMaterial(color: .green, isMetallic: false)
modelEntity.model?.materials = [material]
let anchor = AnchorEntity()
anchor.scale = [5,5,5]
modelEntity.setParent(anchor)
arView.scene.anchors.append(anchor)
}
}

Related

Can't play USDZ animation with RealityKit

I've followed the answer in this SO question regarding playing the animation of a USDZ file with the following code:
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
arView = ARView(frame: self.view.frame)
self.view.addSubview(arView)
arView.cameraMode = .nonAR
let newAnchor = AnchorEntity(world: .zero)
let newEnt = try! Entity.load(named: "Vintage Toy Robot")
newAnchor.addChild(newEnt)
arView.scene.addAnchor(newAnchor)
print(newEnt.availableAnimations)
for anim in newEnt.availableAnimations {
newEnt.playAnimation(anim.repeat(duration: .infinity),
transitionDuration: 1.25, startsPaused: false)
}
}
However, the animation does not play, the USDZ file is just static.
Download robot model with animation from AR Quick Look page. Your code is working. Do not use not-animated robot from Reality Composer library.
let entity = try! Entity.load(named: "toy_robot_vintage.usdz")
And delete these two lines of code (you've got 2 ARViews, it's blocking the animation):
arView = ARView(frame: self.view.frame)
self.view.addSubview(arView)

Create border or outline on ModelEntity?

How can I create a border/outline on a ModelEntity in RealityKit?
Something like this blue border in Reality Composer:
You can achieve similar effect in two ways: either using Metal framework's features, or natively, in RealityKit (but sometimes with some visual artifacts). In RealityKit, such an outline could be rendered with faceCulling property for cloned model:
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
let scene = try! Experience2.loadScene()
let scene2 = scene.clone(recursive: true)
let outline = scene2.findEntity(named: "simpBld_root") as! ModelEntity
outline.scale *= 1.02
var material = PhysicallyBasedMaterial()
material.emissiveColor.color = .white
material.emissiveIntensity = 0.5
// an outer surface doesn't contribute to the final image
material.faceCulling = .front
outline.model?.materials[0] = material
arView.scene.anchors.append(scene)
arView.scene.anchors.append(scene2)
}
}
P. S.
In your case, the name of a rook is:
.findEntity(named: "chess_rook_white_base_iconic_lod0")

Anchoring Multiple Scenes in RealityKit

While loading multiple scenes (from reality composer) into arView, the scenes is not anchored in the same space.
In this example, scene1 is loaded when the app starts. After the button is pressed, the scene2 is added into the scene. In both the scenes, the models are placed at the origin and are expected to overlap with scene2 is added into the view. However, the position of scene1 and scene2 is different when they are added into the arView.
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
#IBOutlet weak var button: UIButton!
var scene1: Experience.Scene1!
var scene2: Experience.Scene2!
override func viewDidLoad() {
super.viewDidLoad()
// Load the "Box" scene from the "Experience" Reality File
scene1 = try! Experience.loadScene1()
scene2 = try! Experience.loadScene2()
// Add the box anchor to the scene
arView.scene.addAnchor(scene1)
}
#IBAction func buttonPressed(_ sender: Any) {
arView.scene.addAnchor(scene2)
}
}
Note: This issues does not happen when both the scenes are added simultaneously.
How to make sure that both the scenes are anchored at the same ARAnchor?
Use the following approach:
let scene01 = try! Cube.loadCube()
let scene02 = try! Ball.loadSphere()
let cubeEntity: Entity = scene01.steelCube!.children[0]
let ballEntity: Entity = scene02.glassBall!.children[0]
// var cubeComponent: ModelComponent = cubeEntity.components[ModelComponent].self!
// var ballComponent: ModelComponent = ballEntity.components[ModelComponent].self!
let anchor = AnchorEntity()
anchor.addChild(cubeEntity)
anchor.addChild(ballEntity)
// scene01.steelCube!.components.set(cubeComponent)
// scene02.glassBall!.components.set(ballComponent)
arView.scene.anchors.append(anchor)

Reality Composer - Custom Collision Between Entities of Different Scenes

I'm pretty new to RealityKit and ARKit. I have two scenes in Reality Composer, one with a book image anchor and one with a horizontal plane anchor. The first scene with an image anchor has a cube attached to the top of it and the second scene built on a horizontal plane has two rings. All objects have a fixed collision. I'd like to run an animation when the rings and the cube touch. I couldn't find a way to do this in Reality Composer, so I made two attempts within the code to no avail. (I'm printing "collision started" just to test the collision code without the animation) Unfortunately, it didn't work. Would appreciate help on this.
Attempt #1:
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
let componentBreakdownAnchor = try! CC.loadComponentBreakdown()
arView.scene.anchors.append(componentBreakdownAnchor)
let bookAnchor = try! CC.loadBook()
arView.scene.anchors.append(bookAnchor)
let ringsAnchor = try! CC.loadRings()
arView.scene.anchors.append(ringsAnchor)
// Add the componentBreakdown anchor to the scene
arView.scene.anchors.append(componentBreakdownAnchor)
let bookAnchor = try! CC.loadBook()
arView.scene.anchors.append(bookAnchor)
let ringsAnchor = try! CC.loadRings()
arView.scene.anchors.append(ringsAnchor)
let _ = ringsAnchor.scene?.subscribe(
to: CollisionEvents.Began.self,
on: bookAnchor
) { event in
print("collision started")
}
return arView
}
Attempt #2
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
let componentBreakdownAnchor = try! CC.loadComponentBreakdown()
arView.scene.anchors.append(componentBreakdownAnchor)
let bookAnchor = try! CC.loadBook()
arView.scene.anchors.append(bookAnchor)
let ringsAnchor = try! CC.loadRings()
arView.scene.anchors.append(ringsAnchor)
// Add the componentBreakdown anchor to the scene
arView.scene.anchors.append(componentBreakdownAnchor)
let bookAnchor = try! CC.loadBook()
arView.scene.anchors.append(bookAnchor)
let ringsAnchor = try! CC.loadRings()
arView.scene.anchors.append(ringsAnchor)
arView.scene.subscribe(
to: CollisionEvents.Began.self,
on: bookAnchor
) { event in
print("collision started")
}
return arView
}
RealityKit scene
If you want to use models' collisions made in RealityKit's scene from scratch, at first you need to implement a HasCollision protocol.
Let's see what a developer documentation says about it:
HasCollision protocol is an interface used for ray casting and collision detection.
Here's how your implementation should look like if you generate models in RealityKit:
import Cocoa
import RealityKit
class CustomCollision: Entity, HasModel, HasCollision {
let color: NSColor = .gray
let collider: ShapeResource = .generateSphere(radius: 0.5)
let sphere: MeshResource = .generateSphere(radius: 0.5)
required init() {
super.init()
let material = SimpleMaterial(color: color,
isMetallic: true)
self.components[ModelComponent] = ModelComponent(mesh: sphere,
materials: [material])
self.components[CollisionComponent] = CollisionComponent(shapes: [collider],
mode: .trigger,
filter: .default)
}
}
Reality Composer scene
And here's how your code should look like if you use models from Reality Composer:
import UIKit
import RealityKit
import Combine
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
var subscriptions: [Cancellable] = []
override func viewDidLoad() {
super.viewDidLoad()
let groundSphere = try! Experience.loadStaticSphere()
let upperSphere = try! Experience.loadDynamicSphere()
let gsEntity = groundSphere.children[0].children[0].children[0]
let usEntity = upperSphere.children[0].children[0].children[0]
// CollisionComponent exists in case you turn on
// "Participates" property in Reality Composer app
print(gsEntity)
let gsComp: CollisionComponent = gsEntity.components[CollisionComponent]!.self
let usComp: CollisionComponent = usEntity.components[CollisionComponent]!.self
gsComp.shapes = [.generateBox(size: [0.05, 0.07, 0.05])]
usComp.shapes = [.generateBox(size: [0.05, 0.05, 0.05])]
gsEntity.components.set(gsComp)
usEntity.components.set(usComp)
let subscription = self.arView.scene.subscribe(to: CollisionEvents.Began.self,
on: gsEntity) { event in
print("Balls' collision occured!")
}
self.subscriptions.append(subscription)
arView.scene.anchors.append(upperSphere)
arView.scene.anchors.append(groundSphere)
}
}

How do I make an entity a physics entity in RealityKit?

I am not able to figure out how to make the "ball" entity a physics entity/body and apply a force to it.
// I'm using UIKit for the user interface and RealityKit +
// the models made in Reality Composer for the Augmented reality and Code
import RealityKit
import ARKit
class ViewController: UIViewController {
var ball: (Entity & HasPhysics)? {
try? Entity.load(named: "golfball") as? Entity & HasPhysics
}
#IBOutlet var arView: ARView!
// referencing the play now button on the home screen
#IBAction func playNow(_ sender: Any) { }
// referencing the slider in the AR View - this slider will be used to
// control the power of the swing. The slider values range from 10% to
// 100% of swing power with a default value of 55%. The user will have
// to gain experience in the game to know how much power to use.
#IBAction func slider(_ sender: Any) { }
//The following code will fire when the view loads
override func viewDidLoad() {
super.viewDidLoad()
// defining the Anchor - it looks for a flat surface .3 by .3
// meters so about a foot by a foot - on this surface, it anchors
// the golf course and ball when you tap
let anchor = AnchorEntity(plane: .horizontal, minimumBounds: [0.3, 0.3])
// placing the anchor in the scene
arView.scene.addAnchor(anchor)
// defining my golf course entity - using modelentity so it
// participates in the physics of the scene
let entity = try? ModelEntity.load(named: "golfarnew")
// defining the ball entity - again using modelentity so it
// participates in the physics of the scene
let ball = try? ModelEntity.load(named: "golfball")
// loading my golf course entity
anchor.addChild(entity!)
// loading the golf ball
anchor.addChild(ball!)
// applying a force to the ball at the balls position and the
// force is relative to the ball
ball.physicsBody(SIMD3(1.0, 1.0, 1.0), at: ball.position, relativeTo: ball)
// sounds, add physics body to ball, iPad for shot direction,
// connect slider to impulse force
}
}
Use the following code to find out how to implement a RealityKit's physics.
Pay particular attention: Participates in Physics is ON in Reality Composer.
import ARKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let boxScene = try! Experience.loadBox()
let secondBoxAnchor = try! Experience.loadBox()
let boxEntity = boxScene.steelBox as! (Entity & HasPhysics)
let kinematics: PhysicsBodyComponent = .init(massProperties: .default,
material: nil,
mode: .kinematic)
let motion: PhysicsMotionComponent = .init(linearVelocity: [0.1 ,0, 0],
angularVelocity: [3, 3, 3])
boxEntity.components.set(kinematics)
boxEntity.components.set(motion)
let anchor = AnchorEntity()
anchor.addChild(boxEntity)
arView.scene.addAnchor(anchor)
arView.scene.addAnchor(secondBoxAnchor)
print(boxEntity.isActive) // Entity must be active!
}
}
Also, look at THIS POST to find out how to implement RealityKit's physics with a custom class.