How to Add Material to ModelEntity programatically in RealityKit? - swift

Question

The docs for RealityKit include the structs: OcclusionMaterial, SimpleMaterial, and UnlitMaterial for adding materials to a ModelEntity.
Alternatively you can load in a model with a material attached to it.
I want to add a custom material/texture to a ModelEntity programmatically. How can I achieve this on the fly without adding the material to a model in Reality Composer or some other 3D Software?

Answer 1

Updated: January 26, 2023
RealityKit materials
There are 6 types of materials in RealityKit 2.0 and RealityFoundation at the moment:
SimpleMaterial
UnlitMaterial
OcclusionMaterial (read this post to find out how to setup SceneKit occlusion shader)
VideoMaterial (look at this post to find out how to setup it)
PhysicallyBasedMaterial
CustomMaterial (Medium story)
SwiftUI version
Here I used two macOS implementations (SwiftUI and Cocoa) to demonstrate how to programmatically assign RealityKit materials.
import SwiftUI
import RealityKit
struct VRContainer : NSViewRepresentable {
let arView = ARView(frame: .zero)
let anchor = AnchorEntity()
func makeNSView(context: Context) -> ARView {
var smpl = SimpleMaterial()
smpl.color.tint = .blue
smpl.metallic = 0.7
smpl.roughness = 0.2
var pbr = PhysicallyBasedMaterial()
pbr.baseColor.tint = .green
let mesh: MeshResource = .generateBox(width: 0.5,
height: 0.5,
depth: 0.5,
cornerRadius: 0.02,
splitFaces: true)
let box = ModelEntity(mesh: mesh, materials: [smpl, pbr])
box.orientation = Transform(pitch: .pi/4,
yaw: .pi/4, roll: 0.0).rotation
anchor.addChild(box)
arView.scene.anchors.append(anchor)
arView.environment.background = .color(.black)
return arView
}
func updateNSView(_ view: ARView, context: Context) { }
}
struct ContentView: View {
var body: some View {
VRContainer().ignoresSafeArea()
}
}
Cocoa version
import Cocoa
import RealityKit
class ViewController: NSViewController {
#IBOutlet var arView: ARView!
override func awakeFromNib() {
let box = try! Experience.loadBox()
var simpleMat = SimpleMaterial()
simpleMat.color = .init(tint: .blue, texture: nil)
simpleMat.metallic = .init(floatLiteral: 0.7)
simpleMat.roughness = .init(floatLiteral: 0.2)
var pbr = PhysicallyBasedMaterial()
pbr.baseColor = .init(tint: .green, texture: nil)
let mesh: MeshResource = .generateBox(width: 0.5,
height: 0.5,
depth: 0.5,
cornerRadius: 0.02,
splitFaces: true)
let boxComponent = ModelComponent(mesh: mesh,
materials: [simpleMat, pbr])
box.steelBox?.children[0].components.set(boxComponent)
box.steelBox?.orientation = Transform(pitch: .pi/4,
yaw: .pi/4,
roll: 0).rotation
arView.scene.anchors.append(box)
}
}
Read this post to find out how to load a texture for RealityKit's shaders.
RealityKit shaders vs SceneKit shaders
We know that in SceneKit there are 5 different shading models, so we can use RealityKit's SimpleMaterial, PhysicallyBasedMaterial and UnlitMaterial to generate all these five shaders that we've been accustomed to.
Let's see how it looks like:
SCNMaterial.LightingModel.blinn – SimpleMaterial(color: . gray,
roughness: .float(0.5),
isMetallic: false)
SCNMaterial.LightingModel.lambert – SimpleMaterial(color: . gray,
roughness: .float(1.0),
isMetallic: false)
SCNMaterial.LightingModel.phong – SimpleMaterial(color: . gray,
roughness: .float(0.0),
isMetallic: false)
SCNMaterial.LightingModel.physicallyBased – PhysicallyBasedMaterial()
// all three shaders (`.constant`, `UnlitMaterial` and `VideoMaterial `)
// don't depend on lighting
SCNMaterial.LightingModel.constant – UnlitMaterial(color: .gray)
– VideoMaterial(avPlayer: avPlayer)