Using a MTLTexture as the environment map of a SCNScene - swift

Question

I want to set a MTLTexture object as the environment map of a scene, as it seems to be possible according to the documentation. I can set the environment map to be a UIImage with the following code:
let roomImage = UIImage(named: "room")
scene.lightingEnvironment.contents = roomImage
This works and I see the reflection of the image on my metallic objects. I tried converting the image to a MTLTexture and setting it as the environment map with the following code:
let roomImage = UIImage(named: "room")
let loader = MTKTextureLoader(device: MTLCreateSystemDefaultDevice()!)
let envMap = try? loader.newTexture(cgImage: (roomImage?.cgImage)!, options: nil)
scene.lightingEnvironment.contents = envMap
However this does not work and I end up with a blank environment map with no reflection on my objects.
Also, instead of setting the options as nil, I tried setting the MTKTextureLoader.Option.textureUsage key with every possible value it can get, but that didn't work either.
Edit: You can have a look at the example project in this repo and use it to reproduce this use case.

Answer 1

Lighting SCN Environment with an MTK texture
Using Xcode 13.3.1 on macOS 12.3.1 for iOS 15.4 app.
The trick is, the environment lighting requires a cube texture, not a flat image.
Create 6 square images for MetalKit cube texture
in Xcode Assets folder create Cube Texture Set
place textures to their corresponding slots
mirror images horizontally and vertically, if needed
Paste the code:
import ARKit
import MetalKit
class ViewController: UIViewController {
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
let scene = SCNScene()
let imageName = "CubeTextureSet"
let textureLoader = MTKTextureLoader(device: sceneView.device!)
let environmentMap = try! textureLoader.newTexture(name: imageName,
scaleFactor: 2,
bundle: .main,
options: nil)
let daeScene = SCNScene(named: "art.scnassets/testCube.dae")!
let model = daeScene.rootNode.childNode(withName: "polyCube",
recursively: true)!
scene.lightingEnvironment.contents = environmentMap
scene.lightingEnvironment.intensity = 2.5
scene.background.contents = environmentMap
sceneView.scene = scene
sceneView.allowsCameraControl = true
scene.rootNode.addChildNode(model)
}
}
Apply metallic materials to models. Now MTL environment lighting is On.
If you need a procedural skybox texture – use MDLSkyCubeTexture class.
Also, this post may be useful for you.