I’m having no luck trying to selectively extract certain elements of a scene file. I’m using SCNSceneSource class. I’m running this swift code inside the Playgrounds App on my ipad pro. I’ve tried multiple dae, scn files using different ways. From the Resources directory.
I’ve also tried different class types, SCNScene, SCNNode, SCNGeometry, etc
Any help would be greatly appreciated.
guard let url = Bundle.main.url(forResource: "file", withExtension: "scn") else {
fatalError("Failed to find model file")
}
let sceneSource = SCNSceneSource(url: url as URL, options: nil)!
//List identifiers
let identifiers = sceneSource.identifiersOfEntries(withClass: SCNGeometry.self) as [String]
for identifier in identifiers {
print(identifiers)
}
Edit:
Btw, I’m using SketchUp to create the original Collada (DAE) files which I import into Xcode. SketchUp supports Collada 1.4 specification and schema. In a past SO answer, it was recommended to avoid Collada version 1.5 as its not supported by Scenekit.
When I review the Collada dae files in Xcode using scene graph (a hierarchy of nodes with attached geometries, lights, cameras etc. ) The attributes window corresponding to each node/geometry is showing “no identifier” when clicking on the different geometry/nodes in the scene graph. I was able to add some names by manually editing the xml schema in the dae file using a text editor (or clicking on the scene graph parts and renaming, on the relevant nodes/geometry... but even though each geometry element has an id=“geometry1” etc.... in the orignal Collada dae file schema.. that is still not showing up as the identifier in the attributes menu of Xcode.
Does anyone have a sample validated DAE file they can provide or link to, in others words, sceneSource.identifiersOfEntries(withClass: SCNGeometry.self) will return the SCNGeometry identifiers... this will help me troubleshoot any problems with SketchUp Collada dae files not being formatted properly by the SketchUp exporter.
Apple’s own scenekit documentation does not seemed to go into any details about how each scn class reconciles with the Collada schema.
NOTE: I’M not having a problem loading or using dae files in an iOS application. Or converting to an SCN file format. This problem only relates to extracting id’s from the sceneSource via “identfiersOfEntries”
Related
In our workflow, we have 3D objects with the type of gltf provided to the iOS via API endpoints. Using a different types of files is not an option here.
Inside the iOS, the glb file is converted to a scn file using a third-party framework. The nodes in the converted scn file look as expected and similar to the original glb file.
In the second step, the scn file should be converted to a USDZ file to use in RealityKit. As far as I know, there is only one way to convert scn to USDZ inside the iOS app. which is using an undocumented approach ( assign a usdz format to URL and using write(to:options:delegate:progressHandler:))
let path = FileManager.default.urls(for: .documentDirectory,
in: .userDomainMask)[0]
.appendingPathComponent("model.usdz")
sceneView.scene.write(to: path)
In the converted USDZ file, all the one-sided materials (transparent) materials are changed to a not transparent material. We tested on a car object, and all the window glasses became not transparent after converting.
On the other hand, if we use the Reality converter app in macOS to convert glb file directly to USDZ file, every node converts as expected, and it works so fine. Is there any workaround to fix that issue?
There is one similar method to covert OBJ to USDZ, first I could't find a way to convert scn to OBJ, and then I am not sure that approach has the same issue or not
Thanks
I'm trying to set up a custom video source for a video stream in Agora for Unity, following the instructions from Agora's developer center here (and particularly, the example code at the bottom):
https://docs.agora.io/en/Video/custom_video_unity?platform=Unity
THIS CODE WORKS. I can successfully send a video stream and watch it on another device and it looks correct.
However, the Unity console is reporting an error on every single frame, saying:
D3D11 unsupported ReadPixels destination texture format (14)
Unity's documentation for Texture2D.ReadPixels says that it works on RGBA32, ARGB32 and RGB24 texture formats, but Agora's example is using a texture in BGRA32 format.
If I alter the example to set the texture to RGBA32 format instead, then the program still works, except the colors are wrong--red and blue are swapped (unsurprisingly).
I tried to adjust the expected texture on Agora's end by modifying this line of the example:
externalVideoFrame.format = ExternalVideoFrame.VIDEO_PIXEL_FORMAT.VIDEO_PIXEL_BGRA;
But...there is no corresponding define for VIDEO_PIXEL_RGBA. The available options are VIDEO_PIXEL_UNKNOWN, VIDEO_PIXEL_I420, VIDEO_PIXEL_BGRA, VIDEO_PIXEL_NV12, VIDEO_PIXEL_I422
So....my app is functioning correctly, but I'm drowning in error messages of dubious significance, which seems like it's going to cause headaches for development and debugging down the road.
What can I do?
For the inverted color issue, make sure you have the same encoding format on the receiver side. If you are using the SDK script VideoSurface.cs, change the line where it instantiates the Texture (about line 172), where it should be like:
nativeTexture = new Texture2D((int)defWidth, (int)defHeight, TextureFormat.BGRA32, false);
(It was RGBA32 in the stock SDK code).
Update: This format issue has been resolved in version 3.0.1. If it hasn't been released in Asset Store, you may come grab the beta to try out. Check with slack channel here: https://agoraiodev.slack.com/messages/unity-help-me
I have an ARKit app that uses image recognition to trigger SceneKit/3D Object files, specifically for art exhibitions.
I've recently began implementing Firebase Storage in order to reduce the overall download size and rather download the 3D files on demand, per which exhibition the user is using!
I've successfully set up the app to download the SceneKit files to the mobile storage, but where I am stuck now is figuring out how to read the file from the specific downloaded location and continue working it in the image recognition/AR process accordingly.
Before, when the Scenekit files were included in the initial download via Scenekit Catalog in the app folder, I would read them like so:
let ShipScene = SCNScene(named: "art.scnassets/ship.scn")
Now, I've updated it to read the same string as the download URL, but the image recognition is not working.
I assume my problem is that it is not reading the right location, or I may be using the wrong function. I've put the app through to my phone through TestFlight and still no luck.
// Downloading from firebase to device URL
let shipURL = documentsURL.appendingPathComponent("file:///var/mobile/Containers/Data/Application/QZ_Gallery/SceneKitFiles/ship.scn", isDirectory: true)
let shipDownload = shipRef.write(toFile: shipURL)
// Attempting to pull the file from the downloaded location
let ShipScene = SCNScene(named:"file:///var/mobile/Containers/Data/Application/QZ_Gallery/SceneKitFiles/ship.scn" )
*** My question is in the final line of code I included. I am trying to make sure I am searching in the right place to retrieve the files that were downloaded, or if the function I am using is even proper for retrieving files stored locally on a mobile device.
Solved by first adding a function to find the scenepath of the container which has all my scenes, and then using the scene path in the fetching of the individual .scn file. Scene downloads from Firebase Storage (No use of Realtime Database) and integrates properly with the image recognition.
let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let scenePath = documentsURL.appendingPathComponent("Scenes")
if(!FileManager.default.fileExists(atPath: scenePath.absoluteString)) {
do {
try FileManager.default.createDirectory(atPath: scenePath.absoluteString, withIntermediateDirectories: true, attributes: nil)
} catch {
print(error.localizedDescription);
}
}
// Start Download, writing to a file
let sceneURL = scenePath.appendingPathComponent("Ship.scn", isDirectory: true)
I added an USDZ file to my Xcode project. The USDZ file is showing as normal in the project in Xcode open on my mac, however, when I try to load it via code using modelIO library it loads the mesh but it doesn't load the texture.
In the snapshots below there is a bee model that I try to load. In the second there is a bee and aircraft, bee textures are not rendered whereas aircraft textures are loaded without problem. I also convert the USDZ file to the scn file and it worked properly. I think the problem is not about the file but the code I am using can't load the USDZ file properly. I am trying to load a USDZ file via code and which library I am using not much important. If you have other suggestions I am open to listen them.
I use the following code to load my model:
let fileName = “bee”
let ext = “usdz”
let asset = MDLAsset(url: Bundle.main.url(forResource:fileName, withExtension: ext)!)
let loadedScene = SCNScene(mdlAsset: asset)
for child in loadedScene.rootNode.childNodes {
child.geometry?.firstMaterial?.lightingModel = .physicallyBased
}
Here are some snapshots to demonstrate results I get:
- snapshot from Xcode viewer on my mac, successfully showing the bee with texture:
snapshot from the app on mobile phone, bee and aircraft, bee textures are not visible:
on the Xcode console I get following error logs:
What am I missing here to load the model file entirely, including textures, to my scene?
I found the solution. There is a method in MDLAsset as MDLAsset.loadTextures() to load the textures. When you call it then it loads the textures.
I’m trying to access files in the art.scnassets folder. I’m using swift playgrounds so I put this folder in a Contents/Chapter.playgroundchapter/PublicResources/art.scnassets/ship.scn
I want to use this 3d model for ARKit but I can’t acess it using
let ship = SCNScene(name: “ship.scn”)
I also tried let ship = SCNScene(name: “PublicResources/art.scnassets/ship.scn”)