Dynamically change text of RealityKit entity - swift

Question

I have created a very simple scene ("SpeechScene") using Reality Composer, with a single speech callout object ("Speech Bubble") anchored to a Face anchor.
I have loaded this scene into code via the following:
let speechAnchor = try! Experience.loadSpeechScene()
arView.scene.anchors.append(speechAnchor)
let bubble = (arView.scene as? Experience.SpeechScene)?.speechBubble
It renders as expected. However, I would like to dynamically change the text of this existing entity.
I found a similar question here, but it's unclear to me how to refer to the meshResource property of a vanilla RealityKit.Entity object.
Is this possible? Thank you!

Answer 1

First Approach
At first you need to find out what's an hierarchy in Reality Composer's scene containing Bubble Speech object. For that I used simple print() command:
print(textAnchor.swift!.children[0].components.self) /* Bubble Plate */
print(textAnchor.swift!.children[1].components.self) /* Text Object */
Now I can extract a text entity object:
let textEntity: Entity = textAnchor.swift!.children[1].children[0].children[0]
And bubble plate entity object:
let bubbleEntity: Entity = textAnchor.swift!.children[0]
Here's a final code version that you can adapt for your needs:
import RealityKit
class GameViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
let textAnchor = try! SomeText.loadTextScene()
let textEntity: Entity = textAnchor.swift!.children[1].children[0].children[0]
textAnchor.swift!.parent!.scale = [4,4,4] // Scale for both objects
var textModelComp: ModelComponent = (textEntity.components[ModelComponent])!
var material = SimpleMaterial()
material.baseColor = .color(.red)
textModelComp.materials[0] = material
textModelComp.mesh = .generateText("Obj-C",
extrusionDepth: 0.01,
font: .systemFont(ofSize: 0.08),
containerFrame: CGRect(),
alignment: .left,
lineBreakMode: .byCharWrapping)
textEntity.position = [-0.1,-0.05, 0.01]
textAnchor.swift!.children[1].children[0].children[0].components.set(textModelComp)
arView.scene.anchors.append(textAnchor)
}
}
Second Approach
And you can always use a simpler approach for this case – to create several scenes in Reality Composer, each one must contain different speech-object.
Consider, this code isn't for tracking, it's just a test for dynamically switching two objects using Tap Gesture. Then you need to adapt this code for tracking faces.
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
var counter = 0
var bonjourObject: FaceExperience.Bonjour? = nil
var holaObject: FaceExperience.Hola? = nil
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
// Reality Composer Scene named "Bonjour"
// Model name – "french"
bonjourObject = try! FaceExperience.loadBonjour()
bonjourObject?.french?.scale = SIMD3(x: 2, y: 2, z: 2)
bonjourObject?.french?.position.y = 0.25
// Reality Composer Scene named "Hola"
// Model name – "spanish"
holaObject = try! FaceExperience.loadHola()
holaObject?.spanish?.scale = SIMD3(x: 2, y: 2, z: 2)
holaObject?.spanish?.position.z = 0.3
}
#IBAction func tapped(_ sender: UITapGestureRecognizer) {
if (counter % 2) == 0 {
arView.scene.anchors.removeAll()
arView.scene.anchors.append(holaObject!)
} else {
arView.scene.anchors.removeAll()
arView.scene.anchors.append(bonjourObject!)
}
counter += 1
}
}
If you want a text portion to be on the same place – just copy-paste object from one scene to another.

Answer 2

#maxxfrazer is correct in his assertion that currently the only way to change text dynamically is to replace the ModelComponentof the Entity assuming of course it adheres to the HasModel Protocol.
I have written a simple extension which can help with this:
//-------------------------
//MARK: - Entity Extensions
//-------------------------
extension Entity{
/// Changes The Text Of An Entity
/// - Parameters:
/// - content: String
func setText(_ content: String){ self.components[ModelComponent] = self.generatedModelComponent(text: content) }
/// Generates A Model Component With The Specified Text
/// - Parameter text: String
func generatedModelComponent(text: String) -> ModelComponent{
let modelComponent: ModelComponent = ModelComponent(
mesh: .generateText(text, extrusionDepth: TextElements().extrusionDepth, font: TextElements().font,
containerFrame: .zero, alignment: .center, lineBreakMode: .byTruncatingTail),
materials: [SimpleMaterial(color: TextElements().colour, isMetallic: true)]
)
return modelComponent
}
}
//--------------------
//MARK:- Text Elements
//--------------------
/// The Base Setup Of The MeshResource
struct TextElements{
let initialText = "Cube"
let extrusionDepth: Float = 0.01
let font: MeshResource.Font = MeshResource.Font.systemFont(ofSize: 0.05, weight: .bold)
let colour: UIColor = .white
}
In order to use it lets say you create an Entity called textEntity:
var textEntity = Entity()
You can then set the dynamically change the Text via replacing the ModelComponent and setting the MeshResource at any time by simply calling the following method:
textEntity.setText("Stack Overflow")
Of course in regard to centering or aligning the text you will need to do some simple calculations (which I have ommited here).
Hope it helps.

Answer 3

Find your model entity (maybe by putting a breakpoint and looking through the children initially), find the Entity that conforms to the HasModel protocol, then replace its model with a different one using generatetext:
https://developer.apple.com/documentation/realitykit/meshresource/3244422-generatetext