I’m trying to access files in the art.scnassets folder. I’m using swift playgrounds so I put this folder in a Contents/Chapter.playgroundchapter/PublicResources/art.scnassets/ship.scn
I want to use this 3d model for ARKit but I can’t acess it using
let ship = SCNScene(name: “ship.scn”)
I also tried let ship = SCNScene(name: “PublicResources/art.scnassets/ship.scn”)
Related
In our workflow, we have 3D objects with the type of gltf provided to the iOS via API endpoints. Using a different types of files is not an option here.
Inside the iOS, the glb file is converted to a scn file using a third-party framework. The nodes in the converted scn file look as expected and similar to the original glb file.
In the second step, the scn file should be converted to a USDZ file to use in RealityKit. As far as I know, there is only one way to convert scn to USDZ inside the iOS app. which is using an undocumented approach ( assign a usdz format to URL and using write(to:options:delegate:progressHandler:))
let path = FileManager.default.urls(for: .documentDirectory,
in: .userDomainMask)[0]
.appendingPathComponent("model.usdz")
sceneView.scene.write(to: path)
In the converted USDZ file, all the one-sided materials (transparent) materials are changed to a not transparent material. We tested on a car object, and all the window glasses became not transparent after converting.
On the other hand, if we use the Reality converter app in macOS to convert glb file directly to USDZ file, every node converts as expected, and it works so fine. Is there any workaround to fix that issue?
There is one similar method to covert OBJ to USDZ, first I could't find a way to convert scn to OBJ, and then I am not sure that approach has the same issue or not
Thanks
I have an ARKit app that uses image recognition to trigger SceneKit/3D Object files, specifically for art exhibitions.
I've recently began implementing Firebase Storage in order to reduce the overall download size and rather download the 3D files on demand, per which exhibition the user is using!
I've successfully set up the app to download the SceneKit files to the mobile storage, but where I am stuck now is figuring out how to read the file from the specific downloaded location and continue working it in the image recognition/AR process accordingly.
Before, when the Scenekit files were included in the initial download via Scenekit Catalog in the app folder, I would read them like so:
let ShipScene = SCNScene(named: "art.scnassets/ship.scn")
Now, I've updated it to read the same string as the download URL, but the image recognition is not working.
I assume my problem is that it is not reading the right location, or I may be using the wrong function. I've put the app through to my phone through TestFlight and still no luck.
// Downloading from firebase to device URL
let shipURL = documentsURL.appendingPathComponent("file:///var/mobile/Containers/Data/Application/QZ_Gallery/SceneKitFiles/ship.scn", isDirectory: true)
let shipDownload = shipRef.write(toFile: shipURL)
// Attempting to pull the file from the downloaded location
let ShipScene = SCNScene(named:"file:///var/mobile/Containers/Data/Application/QZ_Gallery/SceneKitFiles/ship.scn" )
*** My question is in the final line of code I included. I am trying to make sure I am searching in the right place to retrieve the files that were downloaded, or if the function I am using is even proper for retrieving files stored locally on a mobile device.
Solved by first adding a function to find the scenepath of the container which has all my scenes, and then using the scene path in the fetching of the individual .scn file. Scene downloads from Firebase Storage (No use of Realtime Database) and integrates properly with the image recognition.
let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let scenePath = documentsURL.appendingPathComponent("Scenes")
if(!FileManager.default.fileExists(atPath: scenePath.absoluteString)) {
do {
try FileManager.default.createDirectory(atPath: scenePath.absoluteString, withIntermediateDirectories: true, attributes: nil)
} catch {
print(error.localizedDescription);
}
}
// Start Download, writing to a file
let sceneURL = scenePath.appendingPathComponent("Ship.scn", isDirectory: true)
I added an USDZ file to my Xcode project. The USDZ file is showing as normal in the project in Xcode open on my mac, however, when I try to load it via code using modelIO library it loads the mesh but it doesn't load the texture.
In the snapshots below there is a bee model that I try to load. In the second there is a bee and aircraft, bee textures are not rendered whereas aircraft textures are loaded without problem. I also convert the USDZ file to the scn file and it worked properly. I think the problem is not about the file but the code I am using can't load the USDZ file properly. I am trying to load a USDZ file via code and which library I am using not much important. If you have other suggestions I am open to listen them.
I use the following code to load my model:
let fileName = “bee”
let ext = “usdz”
let asset = MDLAsset(url: Bundle.main.url(forResource:fileName, withExtension: ext)!)
let loadedScene = SCNScene(mdlAsset: asset)
for child in loadedScene.rootNode.childNodes {
child.geometry?.firstMaterial?.lightingModel = .physicallyBased
}
Here are some snapshots to demonstrate results I get:
- snapshot from Xcode viewer on my mac, successfully showing the bee with texture:
snapshot from the app on mobile phone, bee and aircraft, bee textures are not visible:
on the Xcode console I get following error logs:
What am I missing here to load the model file entirely, including textures, to my scene?
I found the solution. There is a method in MDLAsset as MDLAsset.loadTextures() to load the textures. When you call it then it loads the textures.
I have created a prototype application in unity using vuforia where I upload an image to myserver the server then sends the image (and associated assetbundle's link in metadata) to vuforia cloud to add it to the image target database. then in unity when camera tracks the image target I download the asset bundle to augment it.
public void OnNewSearchResult(TargetFinder.TargetSearchResult targetSearchResult)
{
TargetFinder.TargetSearchResult cloudRecoSearchResult =
(TargetFinder.TargetSearchResult)targetSearchResult;
mTargetMetadata = cloudRecoSearchResult.MetaData;
Debug.Log(mTargetMetadata);
mCloudRecoBehaviour.CloudRecoEnabled = false;
// Build augmentation based on target
if (ImageTargetTemplate)
{
Debug.Log("Image target activated");
// enable the new result with the same ImageTargetBehaviour:
ObjectTracker tracker = TrackerManager.Instance.GetTracker<ObjectTracker>();
ImageTargetBehaviour imageTargetBehaviour =
(ImageTargetBehaviour)tracker.TargetFinder.EnableTracking(
targetSearchResult, ImageTargetTemplate.gameObject);
JsonData jd = JsonMapper.ToObject(mTargetMetadata);
string url = jd["content-url"].ToString();
Debug.Log("video url :"+ "http://192.168.2.92/arads/" + url);
vidPlayer.url = "http://192.168.2.92/arads/"+url;
vidPlayer.Prepare();
if(!vidPlayer.isPlaying)
vidPlayer.Play();
}
}
the above code is to get associated video from the server. Can I get similar functionality with arcore or arfoundation, I read that arcore's refrence image database can have 1000 images,
what if the image I am tracking is not in the current database, can I switch to different db in that case ?
do I have to download and add the image to database in the applicatoin whenever I upload a new image on the server ?
can these images in arcore have meta data like in vuforia ?
The difference between ARCore and Vuforia is in ARCore you can add images to database in run time so you do not have to use any server.
You can switch to a different database by modifying Session config using this: GoogleARCore.ARCoreSessionConfig.AugmentedImageDatabase
As i said you can add images to database in run time so as long as you have the image in your project hierarchy you can add images to database.
I do not think having a meta data is possible only information you can get is database index of the image.
Good Luck!
I’m having no luck trying to selectively extract certain elements of a scene file. I’m using SCNSceneSource class. I’m running this swift code inside the Playgrounds App on my ipad pro. I’ve tried multiple dae, scn files using different ways. From the Resources directory.
I’ve also tried different class types, SCNScene, SCNNode, SCNGeometry, etc
Any help would be greatly appreciated.
guard let url = Bundle.main.url(forResource: "file", withExtension: "scn") else {
fatalError("Failed to find model file")
}
let sceneSource = SCNSceneSource(url: url as URL, options: nil)!
//List identifiers
let identifiers = sceneSource.identifiersOfEntries(withClass: SCNGeometry.self) as [String]
for identifier in identifiers {
print(identifiers)
}
Edit:
Btw, I’m using SketchUp to create the original Collada (DAE) files which I import into Xcode. SketchUp supports Collada 1.4 specification and schema. In a past SO answer, it was recommended to avoid Collada version 1.5 as its not supported by Scenekit.
When I review the Collada dae files in Xcode using scene graph (a hierarchy of nodes with attached geometries, lights, cameras etc. ) The attributes window corresponding to each node/geometry is showing “no identifier” when clicking on the different geometry/nodes in the scene graph. I was able to add some names by manually editing the xml schema in the dae file using a text editor (or clicking on the scene graph parts and renaming, on the relevant nodes/geometry... but even though each geometry element has an id=“geometry1” etc.... in the orignal Collada dae file schema.. that is still not showing up as the identifier in the attributes menu of Xcode.
Does anyone have a sample validated DAE file they can provide or link to, in others words, sceneSource.identifiersOfEntries(withClass: SCNGeometry.self) will return the SCNGeometry identifiers... this will help me troubleshoot any problems with SketchUp Collada dae files not being formatted properly by the SketchUp exporter.
Apple’s own scenekit documentation does not seemed to go into any details about how each scn class reconciles with the Collada schema.
NOTE: I’M not having a problem loading or using dae files in an iOS application. Or converting to an SCN file format. This problem only relates to extracting id’s from the sceneSource via “identfiersOfEntries”