RealityKit – Material's Alpha transparency - swift

Is it possible to have alpha transparency with textures?
I have png file that contains 8 bit RGBA, but for some reason, the supposed-to-be-transparent parts are simply black.
I assign the material like this:
private func setupLightMeshes(_ scene: Entity) {
let lightEntity = scene.findEntity(named: "LightWindow_Plane")!
var lightMaterial = UnlitMaterial()
lightMaterial.baseColor = try! MaterialColorParameter.texture(
TextureResource.load(named: "light.png")) // this is 8bpc RGBA
var modelComponent = lightEntity.components[ModelComponent] as! ModelComponent
modelComponent = ModelComponent(mesh: modelComponent.mesh, materials: [lightMaterial])
lightEntity.components.set(modelComponent)
}

RealityKit 1.0
.tintColor is a multiplier for .baseColor
If you have a .png file with a premultiplied alpha (RGB*A). all you need to do is to additionally use a tintColor instance property with alpha equal to 0.9999.
material.tintColor = UIColor(white: 1.0, alpha: 0.9999)
Here's how it looks like in a real code:
fileprivate func material() -> UnlitMaterial {
var material = UnlitMaterial()
material.baseColor = try! .texture(.load(named: "transparent.png"))
material.tintColor = UIColor(white: 1.0, alpha: 0.9999)
return material
}
override func viewDidLoad() {
super.viewDidLoad()
let sphere: MeshResource = .generateSphere(radius: 0.5)
let entity = ModelEntity(mesh: sphere,
materials: [material()])
let anchor = AnchorEntity()
anchor.orientation = simd_quatf(angle: .pi, axis: [0, 1, 0])
anchor.addChild(entity)
arView.scene.anchors.append(anchor)
}
P.S.
For me, it seems like a bug in RealityKit 1.0. I have no clues why method .load(named: "file.png") doesn't work as expected.
RealityKit 2.0
The same story about partially transparent textures is in RealityKit 2.0:
var material = SimpleMaterial()
material.color = try! .init(tint: .white.withAlphaComponent(0.9999),
texture: .init(.load(named: "semi.png", in: nil)))
tint parameter is a multiplier for texture as well.

Related

RealityKit won't show more than one DirectionalLight

I'm trying to create a simple 3D scene in RealityKit with two lights lighting a mesh from opposite sides. Everything seems to be working but both lights won't work at once. If I comment out light01, then light02 shows up fine.
Obviously from the type of project, you can tell I'm pretty new at this. What have I missed?
func makeUIView(context: Context) -> ARView {
//Configure ARView
let arView = ARView(frame: .zero, cameraMode: .nonAR,
automaticallyConfigureSession: true)
//Set background color
arView.environment.background = .color(.black)
let light01 = DirectionalLight()
light01.light.color = .red
light01.light.intensity = 30000
light01.light.isRealWorldProxy = true
light01.shadow?.maximumDistance = 10.0
light01.shadow?.depthBias = 5.0
light01.orientation = simd_quatf(angle: -.pi/1.5, axis: [0,1,0])
let light01Anchor = AnchorEntity(world: [0, 20, 0])
light01Anchor.addChild(light01)
arView.scene.addAnchor(light01Anchor)
//NOT WORKING
let light02 = DirectionalLight()
light02.light.color = .green
light02.light.intensity = 20000
light02.light.isRealWorldProxy = true
light02.shadow?.maximumDistance = 10.0
light02.shadow?.depthBias = 5.0
light02.orientation = simd_quatf(angle: .pi/1.5, axis: [0,1,0])
let light02Anchor = AnchorEntity(world: [0, 40, 0])
light02Anchor.addChild(light02)
arView.scene.addAnchor(light02Anchor)
//Create plane for floor
let floorMesh = MeshResource.generatePlane(width: 10, depth: 10)
let floorMaterial = SimpleMaterial(color: .white, isMetallic: false)
let floorEntity = ModelEntity(mesh: floorMesh,
materials: [floorMaterial])
let floorAnchor = AnchorEntity(world: [0, 0, 0])
floorAnchor.addChild(floorEntity)
arView.scene.addAnchor(floorAnchor)
let sphereMesh = MeshResource.generateSphere(radius: 1.5)
let sphereMaterial = SimpleMaterial(color: .white,
roughness: 0.9,
isMetallic: false)
let sphereEntity = ModelEntity(mesh: sphereMesh,
materials: [sphereMaterial])
let sphereAnchor = AnchorEntity(world: [0, 1.5, -4])
sphereAnchor.addChild(sphereEntity)
arView.scene.addAnchor(sphereAnchor)
//Camera
let camera = PerspectiveCamera()
let cameraAnchor = AnchorEntity(world: [0, 1, 1])
cameraAnchor.addChild(camera)
arView.scene.addAnchor(cameraAnchor)
return arView
}
About DirectionalLight in RealityKit 2.0
RealityKit scene can contain up to nine lights. Eight of them could be dynamic lights of different types – PointLights, SpotLights, and a DirectionalLight, and one of them is image-based light (IBL). However, RealityKit supports just ONE DirectionalLight (a.k.a. virtual Sun) per scene.
In addition to the above, it makes sense to note that the position of the DirectionalLight in the RealityKit scene doesn't matter and DirectionalLight has .isRealWorldProxy instance property which, if set to true, cast shadows on virtual content without illuminating anything in the scene. You can use it to create shadows on occlusion materials that accept dynamic lighting.

How to show metallic and transparent textures in SceneKit?

I have trouble showing some textures in SceneKit, this is the model I would like to use:
Model in Sketchfab : https://skfb.ly/6QVTQ
The model should appear in these colors and textures in the AR environment using Scene Kit. But the golden tips appear black and the transparent lenses do not appear at all. Are there any suggestions to solve this problem?
The model is .scn format. here is the the model materials properties:
https://drive.google.com/file/d/1HIHEsyONLXyL95dcSy9xWMGX89udMPND/view?usp=sharing
https://drive.google.com/file/d/1ndrZfcjqIQ4d2OfG6ZNvwyfDXnihJbnH/view?usp=sharing
If you need any additional information please let me know.
Thank you in advance.
There's no photorealistic glass with true index of refraction (IoR) in SceneKit but you can easily create a fake one using phong shader. Phong shader also has three important properties of a glass – specularity, reflectivity, and fresnelExponent.
For metallic material use physicallyBased shading model.
Here's a code:
import SceneKit
class ViewController: NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
let sceneView = self.view as! SCNView
sceneView.scene = SCNScene(named: "glasses.scn")!
sceneView.allowsCameraControl = true
sceneView.pointOfView?.position.z = 20
let glassesFrame = sceneView.scene?.rootNode.childNode(withName: "glassesFrame",
recursively: true)
glassesFrame?.geometry?.firstMaterial?.lightingModel = .physicallyBased
glassesFrame?.geometry?.firstMaterial?.metalness.intensity = 1
glassesFrame?.geometry?.firstMaterial?.diffuse.contents = NSColor.systemBrown
let lens1 = sceneView.scene?.rootNode.childNode(withName: "rightLens",
recursively: true)
let lens2 = sceneView.scene?.rootNode.childNode(withName: "leftLens",
recursively: true)
let material = SCNMaterial()
material.lightingModel = .phong
material.diffuse.contents = NSColor(white: 0.2,
alpha: 1)
material.diffuse.intensity = 0.9
material.specular.contents = NSColor(white: 1,
alpha: 1)
material.specular.intensity = 1.0
material.reflective.contents = NSImage.Name("art.scnassets/texture.png")
material.reflective.intensity = 2.0
material.transparencyMode = .dualLayer
material.fresnelExponent = 2.2
material.isDoubleSided = true
material.blendMode = .alpha
material.shininess = 100
material.transparency.native = 0.7
material.cullMode = .back
lens1?.geometry?.firstMaterial = material
lens2?.geometry?.firstMaterial = material
}
}

RealityKit – How to set a ModelEntity's transparency?

In SceneKit, there are lots of options such as
Use alpha channel of UIColor via SCNMaterial.(diffuse|emission|ambient|...).contents
Use SCNMaterial.transparency (a CGFloat from 0.0 to 1.0)
Use SCNMaterial.transparent (another SCNMaterialProperty)
Use SCNNode.opacity (a CGFloat from 0.0 (fully transparent) to 1.0
(fully opaque))
I wonder if there is a way to set transparency/opacity/alpha for ModelEntity in RealityKit?
RealityKit 1.0
There's one solution in RealityKit 1.0 allowing you to control object's transparency. You can do it using baseColor or tintColor instance properties of SimpleMaterial():
var tintColor: NSColor { get set }
var baseColor: NSColor { get set }
var tintColor: UIColor { get set }
var baseColor: UIColor { get set }
It perfectly works in iOS even with color parameter:
import UIKit
import RealityKit
class GameViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
arView.backgroundColor = .black
var material = SimpleMaterial()
material.tintColor = UIColor.init(red: 1.0,
green: 1.0,
blue: 1.0,
alpha: 0.025)
material.baseColor = MaterialColorParameter.color(UIColor.red)
let mesh: MeshResource = .generateSphere(radius: 0.7)
let modelEntity = ModelEntity(mesh: mesh,
materials: [material])
let anchor = AnchorEntity()
anchor.addChild(modelEntity)
arView.scene.anchors.append(anchor)
}
}
macOS solution (texture example for RealityKit 1.0):
var material = SimpleMaterial()
// CYAN TINT and SEMI-TRANSPARENT ALPHA
material.tintColor = NSColor.init(red: 0.0, green: 1.0, blue: 1.0, alpha: 0.5)
material.baseColor = try! MaterialColorParameter.texture(TextureResource.load(contentsOf: url))
material.roughness = MaterialScalarParameter(floatLiteral: 0.0)
material.metallic = MaterialScalarParameter(floatLiteral: 1.0)
// CUBE WAS MADE IN REALITY COMPOSER
cubeComponent.materials = [material]
// SPHERE IS MADE PROGRAMMATICALLY
let mesh: MeshResource = .generateSphere(radius: 0.7)
let sphereComponent = ModelComponent(mesh: mesh,
materials: [material])
anchor.steelBox!.components.set(cubeComponent)
anchor.components.set(sphereComponent)
arView.scene.anchors.append(anchor)
Or if you do not need any texture on a model (just the color with opacity), you can control transparency via baseColor instance property:
material.baseColor = MaterialColorParameter.color(.init(red: 0.0,
green: 1.0,
blue: 1.0,
alpha: 0.5))
If your scene contains both types of objects – that made in Reality Composer and made programmatically in Xcode and you assign the same material to both objects – a compiled app is presenting some rendering artefacts (look at the picture below).
It's due to unstable work of RealityKit (because framework is too young at the moment). I think that in next version of RealityKit such bugs as missing texture on Reality Composer model and weird reflection left from sphere will be eliminated.
RealityKit 2.0
In RealityKit 2.0 engineers of AR team gave us a .color property instead of .baseColor and .tintColor. These two mentioned are deprecated in iOS 15.
iOS solution (color example for RealityKit 2.0)
var material = SimpleMaterial()
material.color = .init(tint: .red.withAlphaComponent(0.05), texture: nil)
material.baseColor // deprecated in iOS 15
material.tintColor // deprecated in iOS 15
iOS solution (texture example for RealityKit 2.0)
Texture can be applied using the same initializer:
material.color = try! .init(tint: .white.withAlphaComponent(0.9999),
texture: .init(.load(named: "mat.png", in: nil)))
Pay particular attention to tint multiplier – you must use 0.9999 value in case your texture has transparent parts.
And HERE you can find how to setup transparency of PhysicallyBasedMaterial.
I've found a several ways to do that.
Without animation and the easiest is to use OcclusionMaterial():
let plane = ModelEntity(
mesh: .generatePlane(width: 0.1, depth: 0.1),
materials: [OcculusionMaterial()]
)
change existing Entity's opacity:
plane.model?.materials = [OcclusionMaterial()]
With animation (you can tweak these snippets for your needs):
var planeColor = UIColor.blue
func fadeOut() {
runTimer(duration: 0.25) { (percentage) in
let color = self.planeColor.withAlphaComponent(1 - percentage)
var material: Material = SimpleMaterial(color: color, isMetallic: false)
if percentage >= 0.9 {
material = OcclusionMaterial()
}
self.plane.model?.materials = [material]
}
}
func fadeIn() {
runTimer(duration: 0.25) { (percentage) in
let color = self.planeColor.withAlphaComponent(percentage)
let material: Material = SimpleMaterial(color: color, isMetallic: false)
self.plane.model?.materials = [material]
}
}
func runTimer(duration: Double, completion: #escaping (_ percentage: CGFloat) -> Void) {
let startTime = Date().timeIntervalSince1970
let endTime = duration + startTime
Timer.scheduledTimer(withTimeInterval: 1 / 60, repeats: true) { (timer) in
let now = Date().timeIntervalSince1970
if now > endTime {
timer.invalidate()
return
}
let percentage = CGFloat((now - startTime) / duration)
completion(percentage)
}
}
hope this helped someone )

How to Add Material to ModelEntity programatically in RealityKit?

The docs for RealityKit include the structs: OcclusionMaterial, SimpleMaterial, and UnlitMaterial for adding materials to a ModelEntity.
Alternatively you can load in a model with a material attached to it.
I want to add a custom material/texture to a ModelEntity programmatically. How can I achieve this on the fly without adding the material to a model in Reality Composer or some other 3D Software?
Updated: January 26, 2023
RealityKit materials
There are 6 types of materials in RealityKit 2.0 and RealityFoundation at the moment:
SimpleMaterial
UnlitMaterial
OcclusionMaterial (read this post to find out how to setup SceneKit occlusion shader)
VideoMaterial (look at this post to find out how to setup it)
PhysicallyBasedMaterial
CustomMaterial (Medium story)
SwiftUI version
Here I used two macOS implementations (SwiftUI and Cocoa) to demonstrate how to programmatically assign RealityKit materials.
import SwiftUI
import RealityKit
struct VRContainer : NSViewRepresentable {
let arView = ARView(frame: .zero)
let anchor = AnchorEntity()
func makeNSView(context: Context) -> ARView {
var smpl = SimpleMaterial()
smpl.color.tint = .blue
smpl.metallic = 0.7
smpl.roughness = 0.2
var pbr = PhysicallyBasedMaterial()
pbr.baseColor.tint = .green
let mesh: MeshResource = .generateBox(width: 0.5,
height: 0.5,
depth: 0.5,
cornerRadius: 0.02,
splitFaces: true)
let box = ModelEntity(mesh: mesh, materials: [smpl, pbr])
box.orientation = Transform(pitch: .pi/4,
yaw: .pi/4, roll: 0.0).rotation
anchor.addChild(box)
arView.scene.anchors.append(anchor)
arView.environment.background = .color(.black)
return arView
}
func updateNSView(_ view: ARView, context: Context) { }
}
struct ContentView: View {
var body: some View {
VRContainer().ignoresSafeArea()
}
}
Cocoa version
import Cocoa
import RealityKit
class ViewController: NSViewController {
#IBOutlet var arView: ARView!
override func awakeFromNib() {
let box = try! Experience.loadBox()
var simpleMat = SimpleMaterial()
simpleMat.color = .init(tint: .blue, texture: nil)
simpleMat.metallic = .init(floatLiteral: 0.7)
simpleMat.roughness = .init(floatLiteral: 0.2)
var pbr = PhysicallyBasedMaterial()
pbr.baseColor = .init(tint: .green, texture: nil)
let mesh: MeshResource = .generateBox(width: 0.5,
height: 0.5,
depth: 0.5,
cornerRadius: 0.02,
splitFaces: true)
let boxComponent = ModelComponent(mesh: mesh,
materials: [simpleMat, pbr])
box.steelBox?.children[0].components.set(boxComponent)
box.steelBox?.orientation = Transform(pitch: .pi/4,
yaw: .pi/4,
roll: 0).rotation
arView.scene.anchors.append(box)
}
}
Read this post to find out how to load a texture for RealityKit's shaders.
RealityKit shaders vs SceneKit shaders
We know that in SceneKit there are 5 different shading models, so we can use RealityKit's SimpleMaterial, PhysicallyBasedMaterial and UnlitMaterial to generate all these five shaders that we've been accustomed to.
Let's see how it looks like:
SCNMaterial.LightingModel.blinn – SimpleMaterial(color: . gray,
roughness: .float(0.5),
isMetallic: false)
SCNMaterial.LightingModel.lambert – SimpleMaterial(color: . gray,
roughness: .float(1.0),
isMetallic: false)
SCNMaterial.LightingModel.phong – SimpleMaterial(color: . gray,
roughness: .float(0.0),
isMetallic: false)
SCNMaterial.LightingModel.physicallyBased – PhysicallyBasedMaterial()
// all three shaders (`.constant`, `UnlitMaterial` and `VideoMaterial `)
// don't depend on lighting
SCNMaterial.LightingModel.constant – UnlitMaterial(color: .gray)
– VideoMaterial(avPlayer: avPlayer)

How to apply Color with Metallic Shade on 3D object's Node in SceneKit?

This is the example of 3d object's node Yellow color with metallic shader.
I want to apply color to metallic shader for a 3d object in SceneKit via Swift.
How can I do it?
To accomplish what you need – just use the following code:
override func viewDidLoad() {
super.viewDidLoad()
let scene = SCNScene(named: "art.scnassets/myScene.scn")!
let sphereNode = SCNNode(geometry: SCNSphere(radius: 5))
let goldenMaterial = SCNMaterial()
goldenMaterial.lightingModel = .physicallyBased
goldenMaterial.metalness.contents = 1.0
goldenMaterial.roughness.contents = 0.0
goldenMaterial.diffuse.contents = UIColor(red: 0.95,
green: 0.75,
blue: 0.2,
alpha: 1.0)
sphereNode.geometry?.materials = [goldenMaterial]
scene.rootNode.addChildNode(sphereNode)
//......................................................
let scnView = self.view as! SCNView
scnView.scene = scene
scnView.autoenablesDefaultLighting = true
}