RealityKit – Rotating an Entity affects its Scale - swift

I am loading an entity using the USDZ file. I want after loading the entity, I want to rotate is forever. I am using the following code.
cancellable = ModelEntity.loadAsync(named: "toy_drummer").sink { [weak self] completion in
if case let .failure(error) = completion {
print("Unable to load model \(error)")
}
self?.cancellable?.cancel()
} receiveValue: { entity in
anchor.addChild(entity)
arView.scene.addAnchor(anchor)
let rotation = Transform(pitch: 0, yaw: .pi, roll: 0)
entity.move(to: rotation,
relativeTo: nil,
duration: 15.0,
timingFunction: .linear)
}
Instead of rotating correctly, the entity is scaling and getting bigger and bigger. Any ideas?

You need a starting transform "point" and ending transform "point". If a value of referenceEntity (relativeTo) argument equal to nil it means relative to world space. Since the same 4x4 matrix slots are used for rotation values ​​as for scaling, when the model is rotated, its scale also changes at the same time, if there is a difference in scale.
For perpetual transform animation use some of RealityKit 2.0 tricks.
And, of course, there is a Trigonometry that was really conceived for perpetual orbiting.
Here's a correct version of your code:
import UIKit
import RealityKit
import Combine
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
var cancellable: Cancellable? = nil
let anchor = AnchorEntity()
override func viewDidLoad() {
super.viewDidLoad()
cancellable = ModelEntity.loadAsync(named: "drummer.usdz").sink { _ in
self.cancellable?.cancel()
} receiveValue: { entity in
self.anchor.addChild(entity)
self.arView.scene.addAnchor(self.anchor)
let rotation = Transform(pitch: 0, yaw: .pi, roll: 0)
entity.move(to: rotation,
relativeTo: entity,
duration: 5.0,
timingFunction: .linear)
}
}
}

I made a swift package a couple of years ago, RealityUI, which does include animations like a continuous rotation:
https://github.com/maxxfrazer/RealityUI/wiki/Animations#spin
You'd just need to include the package, and call:
entity.ruiSpin(by: [0, 1, 0], period: 1)
docs here:
https://maxxfrazer.github.io/RealityUI/Extensions/Entity.html#/s:10RealityKit6EntityC0A2UIE7ruiSpin2by6period5times10completionys5SIMD3VySfG_SdSiyycSgtF

Related

RealityKit – Which Entity is intersecting with other Entity

let height: Float = 1
let width: Float = 0.5
let box = MeshResource.generateBox(width: 0.02, height: height, depth: width)
This box will have a real-time position same as the current camera position, In AR World I would have multiple boxes with different shapes, I want to identify which object is intersecting with the current real-time box.
I can not do this with position matching (The nearest one). I literally want to know the object which is touching/intersecting the real-time box.
Thanks in advance.
You can easily do that using subscribe() method. The following code is a reference:
(physics for both objects was enabled in Reality Composer)
import UIKit
import RealityKit
import Combine
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
var subscriptions: [Cancellable] = []
override func viewDidLoad() {
super.viewDidLoad()
let boxScene = try! Experience.loadBox()
arView.scene.anchors.append(boxScene)
let floorEntity = boxScene.children[0].children[1]
let subscribe = arView.scene.subscribe(to: CollisionEvents.Began.self,
on: floorEntity) { (event) in
print("Collision Occured")
print(event.entityA.name)
print(event.entityB.name)
}
self.subscriptions += [subscribe]
}
}

RealityKit - Animate opacity of a ModelEntity?

By setting the color of a material on the model property of a ModelEntity, I can alter the opacity/alpha of an object. But how do you animate this? My goal is to animate objects with full opacity, then have them fade to a set opacity, such as 50%.
With SCNAction.fadeOpacity on a SCNNode in SceneKit, this was particularly easy.
let fade = SCNAction.fadeOpacity(by: 0.5, duration: 0.5)
node.runAction(fade)
An Entity conforms to HasTransform, but that will only allow you to animate scale, position, and orientation. Nothing to do with animation of the material for something like fading it in or out. The effect is in RealityComposer if you create a behavior for animating hide or showing, but there doesn't seem to be something similar to HasTransform to provide functionality for animating opacity.
I've been all around the documentation looking for something, my next idea is essentially creating a custom animation to replace this behavior, but it seems like it should be available and I am just not finding it.
I tested it using different techniques and came to the sad conclusion: you can't animate a material's opacity in RealityKit framework because RealityKit materials don't support animation at runtime (for now I hope). Let's wait for RealityKit's major update.
Here's a code you can use for test
(arView.alpha property just works):
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
arView.alpha = 1.0
opacityAnimation()
}
func opacityAnimation() {
UIView.animate(withDuration: 5.0,
animations: {
self.arView.alpha = 0.0
})
}
}
And use this code snippet in order to make sure that animation doesn't work properly
(there's no animation process, just value assignment):
import UIKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
let tetheringAnchor = AnchorEntity(world: [0,0,0])
var material = SimpleMaterial()
let mesh: MeshResource = .generateSphere(radius: 0.5)
var sphereComponent: ModelComponent? = nil
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
material.metallic = .float(1.0)
material.roughness = .float(0.0)
material.baseColor = .color(.red)
sphereComponent = ModelComponent(mesh: mesh,
materials: [material])
tetheringAnchor.components.set(sphereComponent!)
arView.scene.anchors.append(tetheringAnchor)
opacityAnimation()
}
func opacityAnimation() {
UIView.animate(withDuration: 5.0,
animations: {
self.material.metallic = .float(1.0)
self.material.roughness = .float(0.0)
self.material.baseColor = .color(.green)
self.sphereComponent = ModelComponent(mesh: self.mesh,
materials: [self.material])
self.tetheringAnchor.components.set(self.sphereComponent!)
self.arView.scene.anchors.append(self.tetheringAnchor)
})
}
}
As #AndyFedo says there is currently no way to animate the opacity nor alpha of an Entity.
Even changing a SimpleMaterial at run time currently results in flickering.
Having said this I was able to animate the Alpha of a SimpleMaterials Color, however based on testing it is in no way optimal or recommended for that matter.
But just in case you wanted to try to further experiment with this avenue please see an attached example which assumes that you only have a single SimpleMaterial:
class CustomBox: Entity, HasModel, HasAnchoring {
var timer: Timer?
var baseColour: UIColor!
//MARK:- Initialization
/// Initializes The Box With The Desired Colour
/// - Parameter color: UIColor
required init(color: UIColor) {
self.baseColour = color
super.init()
self.components[ModelComponent] = ModelComponent(mesh: .generateBox(size: [0.2, 0.2, 0.2]),
materials: [SimpleMaterial (color: baseColour, isMetallic: false)]
)
}
required init() { super.init() }
//MARK:- Example Fading
/// Fades The Colour Of The Entities Current Material
func fadeOut() {
var alpha: CGFloat = 1.0
timer = Timer.scheduledTimer(withTimeInterval: 0.05, repeats: true) { timer in
if alpha == 0 {
timer.invalidate()
return
}
var material = SimpleMaterial()
alpha -= 0.01
material.baseColor = MaterialColorParameter.color(self.baseColour.withAlphaComponent(alpha))
material.metallic = .float(Float(alpha))
material.roughness = .float(Float(alpha))
DispatchQueue.main.async {
self.model?.materials = [material]
}
}
}
}
As such just to test you can create and then call the function like so:
let box = CustomBox(color: .green)
box.position = [0,0,-0.5]
arView.scene.anchors.append(box)
box.fadeOut()
Also I would politely ask, that this answer not get downvoted as I am simply iterating the fact that (a) it isn't possible with any current built in methods, and (b) that it can in part be achieved albeit to a very limited extent (and thus currently; in a way which one would see fit for production).
I don't know if it suits with your use case. But you should consider video material.
As you can see in this WWDC session (2min45). An entity with complex pulsating opacity.
https://developer.apple.com/videos/play/wwdc2020/10612/
you can also create the fade in experience in Reality Composer and trigger the .rcproject file in Xcode. Have not tested other interactions with .rcproject but I know at least this can load a model to fade in into the scene.

How to animate a model's rotation in RealityKit?

In my previous question I already found how to put a rotation transform over only one axis on an object, now I want this to be animated.
Is there a way to do this in RealityKit?
1. Transform Animation
You can move, rotate and scale a model in RealityKit using .move(...) instance method. For a faster compiling I used SwiftUI macOS app – although, you can use this code in iOS app as well.
import SwiftUI
import RealityKit
struct ContentView: View {
var body: some View {
ARInterface().ignoresSafeArea()
}
}
struct ARInterface: NSViewRepresentable {
let arView = ARView(frame: .zero)
func makeNSView(context: Context) -> ARView {
let scene = try! Experience.loadBox()
scene.steelBox?.scale = [10, 10, 10]
let transform = Transform(pitch: 0, yaw: 0, roll: .pi)
scene.steelBox?.orientation = transform.rotation
arView.scene.anchors.append(scene)
scene.steelBox?.move(to: transform,
relativeTo: scene.steelBox,
duration: 5.0,
timingFunction: .linear)
return arView
}
func updateNSView(_ uiView: ARView, context: Context) { }
}
2. Transform Animation using Matrices
For those who prefer to use a matrix math, I recommend reading this post:
Change a rotation of AnchorEntity in RealityKit
3. Transform Animation using Physics
For those who like to use dynamics, I give a link to this post:
How to move a model and generate its collision shape at the same time?
4. Asset Animation
To play an asset animation (whether it's a skeletal character animation or a set of transform animations, including a rotation about mesh's pivot point) made in 3D apps, like Maya or Houdini, use an animationPlaybackController:
import Cocoa
import RealityKit
class ViewController: NSViewController {
#IBOutlet var arView: ARView!
override func awakeFromNib() {
do {
let robot = try ModelEntity.load(named: "drummer")
let anchor = AnchorEntity(world: [0, -0.7, 0])
anchor.transform.rotation = simd_quatf(angle: .pi/4,
axis: [0, 1, 0])
arView.scene.anchors.append(anchor)
robot.scale = [1, 1, 1] * 0.1
anchor.children.append(robot)
robot.playAnimation(robot.availableAnimations[0].repeat(),
transitionDuration: 0.5,
startsPaused: false)
} catch {
fatalError("Cannot load USDZ asset.")
}
}
}
5. Transform Animation in Reality Composer
For those who prefer UI, there's a "perpetual" spin behavior in Reality Composer:
Reality Composer - How to rotate an object forever?
6. Transform Animation using Python bindings for USDZ
USDZ schemas become more and more popular in everyday Python scripting for Pixar's format.
Augmented Reality 911 — USDZ Schemas
float xformOp:rotateY:spin.timeSamples = { 1: 0, 300: 1800 }
uniform token[] xformOpOrder = ["xformOp:rotateY:spin"]
7. Eternal Orbiting using Trigonometry
Trigonometric functions sin() and cos(), Timer and counter objects will allow you to orbit a model around any axis.
Hovering an entity in front of ARCamera
Rotation with animation:
copy the box's current transform
var rotationTransform = boxAnchor.steelBox?.transform
set the box to rotate 90 degrees over z-axis
rotationTransform?.rotation = simd_quatf(angle: .pi/2, axis: [0,0,1])
move the box to the new transform over 10s
boxAnchor.steelBox?.move(to: rotationTransform!, relativeTo: boxAnchor.steelBox?.parent, duration: 10, timingFunction: .easeInOut)
Translation with animation:
var translationTransform = boxAnchor.steelBox?.transform
translationTransform?.translation = SIMD3<Float>(x: 5, y: 0, z: 0)
boxAnchor.steelBox?.move(to: translationTransform!, relativeTo: boxAnchor.steelBox?.parent, duration: 10, timingFunction: .easeInOut)
Scale with animation:
var scaleTransform = boxAnchor.steelBox?.transform
scaleTransform?.scale = SIMD3<Float>(x: 1, y: 1, z: 1)
boxAnchor.steelBox?.move(to: scaleTransform!, relativeTo: boxAnchor.steelBox?.parent, duration: 10, timingFunction: .easeInOut)

How do I make an entity a physics entity in RealityKit?

I am not able to figure out how to make the "ball" entity a physics entity/body and apply a force to it.
// I'm using UIKit for the user interface and RealityKit +
// the models made in Reality Composer for the Augmented reality and Code
import RealityKit
import ARKit
class ViewController: UIViewController {
var ball: (Entity & HasPhysics)? {
try? Entity.load(named: "golfball") as? Entity & HasPhysics
}
#IBOutlet var arView: ARView!
// referencing the play now button on the home screen
#IBAction func playNow(_ sender: Any) { }
// referencing the slider in the AR View - this slider will be used to
// control the power of the swing. The slider values range from 10% to
// 100% of swing power with a default value of 55%. The user will have
// to gain experience in the game to know how much power to use.
#IBAction func slider(_ sender: Any) { }
//The following code will fire when the view loads
override func viewDidLoad() {
super.viewDidLoad()
// defining the Anchor - it looks for a flat surface .3 by .3
// meters so about a foot by a foot - on this surface, it anchors
// the golf course and ball when you tap
let anchor = AnchorEntity(plane: .horizontal, minimumBounds: [0.3, 0.3])
// placing the anchor in the scene
arView.scene.addAnchor(anchor)
// defining my golf course entity - using modelentity so it
// participates in the physics of the scene
let entity = try? ModelEntity.load(named: "golfarnew")
// defining the ball entity - again using modelentity so it
// participates in the physics of the scene
let ball = try? ModelEntity.load(named: "golfball")
// loading my golf course entity
anchor.addChild(entity!)
// loading the golf ball
anchor.addChild(ball!)
// applying a force to the ball at the balls position and the
// force is relative to the ball
ball.physicsBody(SIMD3(1.0, 1.0, 1.0), at: ball.position, relativeTo: ball)
// sounds, add physics body to ball, iPad for shot direction,
// connect slider to impulse force
}
}
Use the following code to find out how to implement a RealityKit's physics.
Pay particular attention: Participates in Physics is ON in Reality Composer.
import ARKit
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let boxScene = try! Experience.loadBox()
let secondBoxAnchor = try! Experience.loadBox()
let boxEntity = boxScene.steelBox as! (Entity & HasPhysics)
let kinematics: PhysicsBodyComponent = .init(massProperties: .default,
material: nil,
mode: .kinematic)
let motion: PhysicsMotionComponent = .init(linearVelocity: [0.1 ,0, 0],
angularVelocity: [3, 3, 3])
boxEntity.components.set(kinematics)
boxEntity.components.set(motion)
let anchor = AnchorEntity()
anchor.addChild(boxEntity)
arView.scene.addAnchor(anchor)
arView.scene.addAnchor(secondBoxAnchor)
print(boxEntity.isActive) // Entity must be active!
}
}
Also, look at THIS POST to find out how to implement RealityKit's physics with a custom class.

How to set a known position and orientation as a starting point of ARKit

I am starting to use ARKit and I have a use case where I want to know the motion from a known position to another one.
So I was wondering if it is possible (like every tracking solution) to set a known position and orientation a starting point of the tracking in ARKit?
Regards
There are at least six approaches allowing you set a starting point for a model. But using no ARAnchors at all in your ARScene is considered as bad AR experience (although Apple's Augmented Reality app template has no any ARAnchors in a code).
First approach
This is the approach that Apple engineers propose us in Augmented Reality app template in Xcode. This approach doesn't use anchoring, so all you need to do is to accommodate a model in air with coordinates like (x: 0, y: 0, z: -0.5) or in other words your model will be 50 cm away from camera.
override func viewDidLoad() {
super.viewDidLoad()
sceneView.scene = SCNScene(named: "art.scnassets/ship.scn")!
let model = sceneView.scene.rootNode.childNode(withName: "ship",
recursively: true)
model?.position.z = -0.5
sceneView.session.run(ARWorldTrackingConfiguration())
}
Second approach
Second approach is almost the same as the first one, except it uses ARKit's anchor:
guard let sceneView = self.view as? ARSCNView
else { return }
if let currentFrame = sceneView.session.currentFrame {
var translation = matrix_identity_float4x4
translation.columns.3.z = -0.5
let transform = simd_mul(currentFrame.camera.transform, translation)
let anchor = ARAnchor(transform: transform)
sceneView.session.add(anchor: anchor)
}
Third approach
You can also create a pre-defined model's position pinned with ARAnchor using third approach, where you need to import RealityKit module as well:
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
let model = ModelEntity(mesh: MeshResource.generateSphere(radius: 1.0))
// ARKit's anchor
let anchor = ARAnchor(transform: simd_float4x4(diagonal: [1,1,1]))
// RealityKit's anchor based on position of ARAnchor
let anchorEntity = AnchorEntity(anchor: anchor)
anchorEntity.addChild(model)
arView.scene.anchors.append(anchorEntity)
}
Fourth approach
If you turned on a plane detection feature you can use Ray-casting or Hit-testing methods. As a target object you can use a little sphere (located at 0, 0, 0) that will be ray-casted.
let query = arView.raycastQuery(from: screenCenter,
allowing: .estimatedPlane,
alignment: .any)
let raycast = session.trackedRaycast(query) { results in
if let result = results.first {
object.transform = result.transform
}
}
Fifth approach
This approach is focused to save and share ARKit's worldMaps.
func writeWorldMap(_ worldMap: ARWorldMap, to url: URL) throws {
let data = try NSKeyedArchiver.archivedData(withRootObject: worldMap,
requiringSecureCoding: true)
try data.write(to: url)
}
func loadWorldMap(from url: URL) throws -> ARWorldMap {
let mapData = try Data(contentsOf: url)
guard let worldMap = try NSKeyedUnarchiver.unarchivedObject(ofClass: ARWorldMap.self,
from: mapData)
else {
throw ARError(.invalidWorldMap)
}
return worldMap
}
Sixth approach
In ARKit 4.0 a new ARGeoTrackingConfiguration is implemented with the help of MapKit module. So now you can use a pre-defined GPS data.
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
for geoAnchor in anchors.compactMap({ $0 as? ARGeoAnchor }) {
arView.scene.addAnchor(Entity.placemarkEntity(for: geoAnchor)
}
}