I am loading a 300mb 3d model of extension .dae, converted into .scn, with 4.4 million vertices, 1.5 million polygons, which is a 3d model of a building, created in 3DS MAX by an artist, like so:
let sceneToLoad = SCNScene(named: "art.scnassets/building1.scn")!
(It is loaded in a SCNView default viewer in the app so that the user can view, rotate it etc., by SCNView.allowsCameraControl = true)
Xcode will immediately crash when it reads that line, with only compiler info "unexpectedly found nil while unwrapping an optional value".
The memory does not go up at all when it runs to that line - suggesting it refuses to read it , and crashes instead. The 3d model is perfectly loaded and vieweable, rotateable etc in the XCODE Scenekit editor graphical viewer. When I replace it to point to another file name of a smaller 3d model it works fine, and even when I remove the model SCNNode in the same file (in the same "building1.scn" file) and replace with a smaller SCNNode of another random object, then miraculously it also works and loads fine.
I have not found anything similar on SO - in other similar answers iOS tries to load the model even if it's huge, but in none it crashes immediately finding a nil value.
Have tried all workarounds, remove/delete file and add again, load it as .dae in its original form, load the scene without unwrapping and unwrap later when searching for a node - nothing works, always crashes in the same way. The same thing happens when I try to load it in an ARKIT scene - it crashes at the above line that tries to just load the file.
Has anyone come across this, or knows of any workaround?
Many thanks
When load a 3D model with 1.5M polygons into SceneKit/ARKit, into RealityKit, or into AR Quick Look you'll always fail. That's because a robust number of polygons per 3D model must be not greater that 10K (with UV-texture having max resolution 2Kx2K, or with a regular texture rez 1Kx1K), and a maximum number of polygons per 3D scene must be not greater that 100K. You have exceeded the "unspoken" AR limit in 15 times.
Game engines and AR frameworks, like SceneKit, RealityKit and AR Quick Look, are incapable of rendering such a huge number of polygons using 60 fps framerate on iOS device (even most desktop computers fail to do this). The best solution for an ARKit/RealityKit applications is to use an optimized low-poly models. The most preferred format for working with AR on mobile platform is Pixar USDZ. A USDZ file is a no compression, unencrypted zip archive of USD file.
Look at this low-poly model from Turbosquid. It has just 5K polygons and it looks fine, doesn't it?
P.S.
You can convert obj, fbx or abc in usdz using command line tools. Read about it HERE.
Related
I'm trying to figure how can I load models and textures in the most efficient way with Unity3D at run-time, so far I've read about glTF formats, Draco API, and currently I've implemented a simple procedure which works really slow. I'm using BestHTTP and for textures I'm doing this:
var texFoundPath = Directory.GetFiles(outFolder, texFileName, SearchOption.AllDirectories).Single();
Debug.Log("Loading texture from " + texFoundPath);
var texBytes = File.ReadAllBytes(texFoundPath);
Texture2D tex = new Texture2D(2,2);
tex.LoadImage(texBytes);
And using ObjImporter for importing objects.
var gameObject = ObjImporter.Import(objStr, mtlStr, textureHashtable);
So at run-time, models that are 3-5mb takes up to 3-5 seconds to load. Which is very slow and not suitable for anything. Doing this locally will result in a 200-300mb apk if I have around 100 models.
So currently I'm looking for a way to do this efficiently, and would love your help. I think it'll be the best to load the models with Draco API. but my I'm not sure how to create the plugin that will communicate with that API.
1 MB per second isn't a bad download speed. Really the only option without using prediction is to lower the resolution of your textures and make them smaller, compress your texture files into "chunks" and download each "chunk" as needed, uncompress, and apply to models, but that would probably still take a good amount of time.
I think what you really want is to have some prediction logic that loads "bundles" of assets before they are actually needed, and store them in memory for later use.
For instance, if the player is 200 meters away from a house that needs to be downloaded before viewing, and the players view distance is 100 meters, then start downloading the house. By the time the house is within viewing distance of the player, it will be downloaded and ready to use.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
I want to create an hologram that is exported via the kinect to an hololense. But it's very slow.
I use this tutorial to collect point cloud data, and this library to export my data as a 3D object in .obj format. The library that exports obj doesnt accept points so I had to draw little triangles. I save the files .obj .png and .mtl on my local xampp.
Next, I download the files with a unity script and WWW object. I also use Runtime OBJ Importer from unity's asset store to create a 3D object at runtime.
The last part is to export the unity app on a hololense. (I will do it next).
But before that,
The process is working but is very slow. I want the hologram to be fluid. A lot of time is wasted :
take depth and rgb data of the kinect
export data to an obj png and mtl file
download the files on unity as frequent as possible
render the files
I think of streaming but does unity need a complet obj file to render ? If I compress .png to .jpg will a gain some time ?
Do you have some pointers to help me ?
Currently the way your question is phrased is confusing: it's unclear wether you want to record point clouds that you later load and render in Unity or you want to somehow stream the point cloud with the aligned RGB texture in close to realtime to Unity.
You initial attempts are using Processing.
In terms of recording data, I recommend using the SimpleOpenNI library which can record both depth and RGB data to an .oni file (see the RecorderPlay example).
Once you have a recording, you can loop through each frame and for each frame store the vertices to a file.
In terms of saving to .obj you'll need to convert the point cloud to a mesh (triangulate the vertices in 3D).
Another option would be to store the point cloud to a format like .ply.
You can find more info on writing to a .ply file in Processing in this answer
In terms of streaming the data, this will be complicated:
if you stream all the vertices it that's a lot of data: up to 921600 floats ( (640 x 480 = 307200) * 3)
if you stream both depth (11bit 640x480) and RGB (8bit 640x480) images that will be even more data.
One option might be to only send the vertices that have a depth and overall skipping points (e.g. send every 3rd point). In terms of sending the data you can try OSC
Once you get the points in Unity you should be able to render a point cloud in Unity
What would be ideal in terms of network performance is a codec (compressor/decompressor) for the depth data. I haven't used one thus far, but doing a quick I see there are options like this one(very date).
You'll need to do a bit of research and see what kinect v1 depth streaming libraries are out there already, test and see what works best for your scenario.
Ideally, if the library is written in C# there's a chance you'll be able to use it to decode the received in Unity.
How can I scale up the size of my world/level to include more gameobjects without causing lag for the player?
I am creating an asset for the asset store. It is a random procedural world generator. There is only one major problem: world size.
I can't figure out how to scale up the worlds to have more objects/tiles.
I have generated worlds up to 2000x500 tiles, but it lags very badly.
The maximum sized world that will not affect the speed of the game is
around 500x200 tiles.
I have generated worlds of the same size with smaller blocks: 1/4th the size (it doesn't affect how many tiles you can spawn)
I would like to create a world at least the size of 4200x1200 blocks without lag spikes.
I have looked at object pooling (it doesn't seem like it can help me
that much)
I have looked at LoadLevelAsync (don't really know how to use this,
and rumor is that you need Unity Pro which I do not have)
I have tried setting chunks Active or Deactive based on player
position (This caused more lag than just leaving the blocks alone).
Additional Information:
The terrain is split up into chunks. It is 2d, and I have box colliders on all solid tiles/blocks. Players can dig/place blocks. I am not worried about the amount of time it takes for the level to load initially, but rather about the smoothness of the game while playing it -no lag spikes while playing.
question on Unity Forums
If you're storing each tile as an individual GameObject, don't. Use a texture atlas and 'tile data' to generate the look of each chunk whenever it is dug into or a tile placed on it.
Also make sure to disable, potentially even delete any chunks not within the visible range of the player. Object pooling will help significantly here if you can work out the maximum number of chunks that will ever be needed at once, and just recycle chunks as they go off the screen.
DETAILS:
There is a lot to talk about for the optimal generation, so I'm going to post this link (http://studentgamedev.blogspot.co.uk/2013/08/unity-voxel-tutorial-part-1-generating.html) It shows you how to do it in a 3D space, but the principales are essentially the same if not a little easier for 2D space. The following is just a rough outline of what might be involved, and going down this path will result in huge benefits, but will require a lot of work to get there. I've included all the benefits at the bottom of the answer.
Each tile can be made to be a simple struct with fields like int id, vector2d texturePos, bool visible in it's simplest form. You can then store these tiles in a 2 dimensional array within each chunk, though to make them even more memory efficient you could store the texturePos once elsewhere in the program and write a method to get a texturePos by id.
When you make changes to this 2 dimensional array which represents either the addition or removal of a tile, you update the chunk, which is the actual GameObject used to represent the tiles. By iterating over the tile data stored in the chunk, it will be possible to generate a mesh of vertices based on the position of each tile in the 2 dimensional array. If visible is false, simply don't generate any vertices for it.
This mesh alone could be used as a collider, but won't look like anything. It will also be necessary to generate UV co-ords which happen to be the texturePos. When Unity then displays the mesh, it will display specific points of the texture atlas as defined by the UV co-ords of the mesh.
This has the benefit of resulting in significantly fewer GameObjects, better texture batching for Unity, less memory usage, faster random access for any tile as it's not got any MonoBehaviour overhead, and a genuine plethora of additional benefits.
Abstract
What I require is a technique, given a single, but layered Flash animation, to export the position and rotation of each key movie clip in every frame as XML.
Code to read in this information into a cocos2d-ready format would save a lot of time but isn't necessary as I know how to achieve this.
Our artists often draw vector using Flash and have wonderful and impressive animations. Our technique in the past to put this art into our games is to export the separate animations as a sequence of pngs, stick them in a sprite sheet, and turn them into CCAnimations.
This technique works well and we can get quite a lot of art into the 20MB over the air download limit thanks to spritesheets and pvrtc. As we grow, however, we are looking to make bigger and better games, and this would mean more art!
What I'd like to achieve now is a mass reduction in the amount of art by using a keyframing technique to mimic Flash behaviour, removing duplicate pieces of art.
i.e. Using the current technique, one character would take:
A walk sequence with 10 frames (say 100 * 300 resolution, 30k pixels) would have
10, full sized humans in each stance. (for a total of 1000 * 300
resolution, 300k pixels for one animation)
Say 5 similar animations for a total of 1.5 million pixels
The desired outcome for the same character:
1 right leg: (40 * 30, 1200 pixels)
1 left leg: (40 * 30, 1200 pixels)
1 torso (50 * 50, 2500 pixels)
1 left arm (40 * 30, 1200 pixels)
1 right arm (40 * 30, 1200 pixels)
1 head (30 * 30, 900 pixels)
Total (8200 pixels) for all animations
And an xml file to explain the transform of each part for each frame.
My knowledge of Flash is limited. I have exported a LOT of art, so that's not a problem, but I don't have much experience in Actionscript or in exporting other information. I have noticed the Export Motion as XML option but it doesn't provide the information I need (It doesn't traverse to the children of movie clips to get their transform).
The Objective-c side of things is less of a problem, I know how to parse XML, though if somebody has already written this too, I'd be very grateful if you'd like to share.
I never did find out about any existing tools and so ended up learning a bit of ActionScript and writing the solution myself.
Unfortunately due to corporate policies I won't be able to share my code but I will outline the steps I needed to take
ActionScript / Flash
Step through each frame
Get the children on the stage on that frame and note their transform
Go through each of their children and note their transform with relation to their parent
Do this recursively until the children have no children or you have reached a DrawingObject
Save the information as XML
Art
Export each symbol you need from the library and add to a sprite sheet. (I did this using JSFL)
Obj-C / Cocos2d
The XML should contain a set of frames with a list of children for each frame and, recursively, each of their children. Each child node contains transform information.
Read in this XML, adding children to frames of an animation as you find them and children to their respective parents. Apply children's transforms.
Hope this helps somebody else.
I assume your question is about the "mass reduction" part. You should definitely have a look at TexturePacker. With TexturePacker you can create texture atlases that contains only a single image of multiple, identical animation frames while retaining your ability to address that frame with its original filenames.
TexturePacker has a cocos2d and flash exporter, and can import from the usual file formats. If there's something not quite as you need it you can contact the author, Andreas Löw. He is very responsive and committed to providing the best texture packing tool possible.
I wrote my own lightweight solution.
A simple library to export and animate Flash symbols in iOS / cocos2d, with included example.
Workflow:
Create your animations in Flash as MovieClips with animated layers containing Graphic symbols.
Export animations as JSON + frame PNGs using included JSFL script.
Convert PNGs into sprite sheets (use your own tools, I use - http://www.codeandweb.com/texturepacker)
Include animation JSON file(s) and sprite sheets in app bundle.
Load textures, run complex multilayer animations on sprites using simple commands.
Check it out on github:
FxAnimation for Cocos2d 2.x
FxAnimation for Cocos2d V3 RC1
This is my first github submission and I'm curious how useful people find it, so let me know.
I'm working on an iPhone App that relies heavily on OpenGL. Right now it runs a bit slow on the iPhone 3G, but looks snappy on the new 32G iPod Touch. I assume this is hardware related. Anyway, I want to get the iPhone performance to resemble the iPod Touch performance. I believe I'm doing a lot of things sub-optimally in OpenGL and I'd like advice on what improvements will give me the most bang for the buck.
My scene rendering goes something like this:
Repeat 35 times
glPushMatrix
glLoadIdentity
glTranslate
Repeat 7 times
glBindTexture
glVertexPointer
glNormalPointer
glTexCoordPointer
glDrawArrays(GL_TRIANGLES, ...)
glPopMatrix
My Vertex, Normal and Texture Coords are already interleaved.
So, what steps should I take to speed this up? What step would you try first?
My first thought is to eliminate all those glBindTexture() calls by using a Texture Atlas.
What about some more efficient matrix operations? I understand the gl*() versions aren't too efficient.
What about VBOs?
Update
There are 8260 triangles.
Texture sizes are 64x64 pngs. There are 58 different textures.
I have not run instruments.
Update 2
After running the OpenGL ES Instrument on the iPhone 3G I found that my Tiler Utilization is in the 90-100% range, and my Render Utilization is in the 30% range.
Update 3
Texture Atlasing had no noticeable affect on the problem. Utilization ranges are still as noted above.
Update 4
Converting my Vertex and Normal pointers to GL_SHORT seemed to improve FPS, but the Tiler Utilization is still in the 90% range a lot of the time. I'm still using GL_FLOAT for my texture coordinates. I suppose I could knock those down to GL_SHORT and save four more bytes per vertex.
Update 5
Converting my texture coordinates to GL_SHORT yielded another performance increase. I'm now consistently getting >30 FPS. Tiler Utilization is still around 90%, but frequently drops down in the the 70-80% range. The Renderer Utilization is hovering around 50%. I suppose this might have something to do with scaling the texture coordinates from GL_TEXTURE Matrix Mode.
I'm still seeking additional improvements. I'd like to get closer to 40 FPS, as that's what my iPod Touch gets and it's silky smooth there. If anyone is still paying attention, what other low-hanging fruit can I pick?
With a tiler utilization still above 90%, you’re likely still vertex throughput-bound. Your renderer utilization is higher because the GPU is rendering more frames. If your primary focus is improving performance on older devices, then the key is still to cut down on the amount of vertex data needed per triangle. There are two sides to this:
Reducing the amount of data per vertex: Now that all of your vertex attributes are already GL_SHORTs, the next thing to pursue is finding a way to do what you want using fewer attributes or components. For example, if you can live without specular highlights, using DOT3 lighting instead of OpenGL ES fixed-function lighting would replace your 3 shorts (+ 1 short of padding) for normals with 2 shorts for an extra texture coordinate. As an additional bonus, you’d be able to light your models per-pixel.
Reducing the number of vertices needed per triangle: When drawing with indexed triangles, you should make sure that your indices are sorted for maximum reuse. Running your geometry through Imagination Technologies’ PVRTTriStrip tool would probably be your best bet here.
If you only have 58 different 64x64 textures, a texture atlas seems like a good idea, since they'd all fit in a single 512x512 texture... if you don't rely on texture wrap modes, I'd certainly at least try this.
What format are your textures in? You might try using a compressed PVRTC texture; I think that's less load on the Tiler, and I've been pleasantly surprised by the image quality even for 2-bit-per-pixel textures. (Good for natural images, not good if you're doing something that looks like an 8-bit video game)
The first thing I would do is run Instruments profiling on the hardware device that is slow. It should show you pretty quickly where the bottlenecks are for your particular case.
Update after instruments results:
This question has a similar result in Instruments to you, perhaps the advice is also applicable in your case (basically reducing number vertex data)
The biggest win in graphics programming comes down to this:
Batch, Batch, Batch
TextureAtlasing will make a bigger difference than most anything else you can do. Switching textures is like stopping a speeding train to let on new passengers every time.
Combine all those textures into an atlas and cut your draw calls down a lot.
This web-based tool may be helpful: http://zwoptex.zwopple.com/
Have you looked over the "OpenGL ES Programming Guide for iPhone OS" in the dev center? There are sections on Best Practices for Vertex Data and Texture Data.
Is your data formatted to be able to use triangle strips?
In terms of least effort, the modification sequence for you would probably be:
Reducing vertex attribute size
VBOs
Note that when you do these, you need to make sure that components are aligned on their native alignment, i.e. the floats or full ints are on 4-byte boundaries, the shorts are on 2-byte boundaries. If you don't do this it will tank your performance. It might be helpful to mentally map it by typing out your attribute ordering as a struct definition so you can sanity check your layout and alignment.
making sure your data is stripped to share vertices
using a texture atlas to reduce texture swaps
To try converting your textures to 16-bit RGB565 format, see this code in Apple's venerable Texture2D.m, search for kTexture2DPixelFormat_RGB565
http://code.google.com/p/cocos2d-iphone/source/browse/branches/branch-0.1/OpenGLSupport/Texture2D.m
(this code loads PNGs and converts them to RGB565 at texture creation time; I don't know if there's an RGB565 file format as such)
For more information on PVRTC compressed textures (which looked way better than I expected when I used them, even at 2 bits per pixel) see Apple's PVRTextureLoader sample:
http://developer.apple.com/iPhone/library/samplecode/PVRTextureLoader/index.html
it has both the code for loading PVRTC textures in your app and also instructions for using the texturetool to convert your .png files into .pvr files.