In learning 3d graphics programming for games I decided to start off simple by using the Scene Kit 3D API. My first gaming goal was to build a very simplified mimic of MineCraft. A game of just cubes - how hard can it be.
Below is a loop I wrote to place a ride of 100 x 100 cubes (10,000) and the FPS performance was abysmal (~20 FPS). Is my initial gaming goal too much for Scene Kit or is there a better way to approach this?
I have read other topics on StackExchange but don't feel they answer my question. Converting the exposed surface blocks to a single mesh won't work as the SCNGeometry is immutable.
func createBoxArray(scene : SCNScene, lengthCount: Int, depthCount: Int) {
let startX : CGFloat = -(CGFloat(lengthCount) * CUBE_SIZE) + (CGFloat(lengthCount) * CUBE_MARGIN) / 2.0
let startY : CGFloat = 0.0
let startZ : CGFloat = -(CGFloat(lengthCount) * CUBE_SIZE) + (CGFloat(lengthCount) * CUBE_MARGIN) / 2.0
var currentZ : CGFloat = startZ
for z in 0 ..< depthCount {
currentZ += CUBE_SIZE + CUBE_MARGIN
var currentX = startX
for x in 0 ..< lengthCount {
currentX += CUBE_SIZE + CUBE_MARGIN
createBox(scene, x: currentX, y: startY, z: currentZ)
}
}
}
func createBox(scene : SCNScene, x: CGFloat, y: CGFloat, z: CGFloat) {
var box = SCNBox(width: CUBE_SIZE, height: CUBE_SIZE, length: CUBE_SIZE, chamferRadius: 0.0)
box.firstMaterial?.diffuse.contents = NSColor.purpleColor()
var boxNode = SCNNode(geometry: box)
boxNode.position = SCNVector3Make(x, y, z)
scene.rootNode.addChildNode(boxNode)
}
UPDATE 12-30-2014:
I modified the code so the SCNBoxNode is created once and then each additional box in the array of 100 x 100 is created via:
var newBoxNode = firstBoxNode.clone()
newBoxNode.position = SCNVector3Make(x, y, z)
This change appears to have increased FPS to ~30fps. The other statistics are as follows (from the statistics displayed in the SCNView):
10K (I assume this is draw calls?)
120K (I assume this is faces)
360K (Assuming this is the vertex count)
The bulk of the run loop is in Rendering (I'm guesstimating 98%). The total loop time is 26.7ms (ouch). I'm running on a Mac Pro Late 2013 (6-core w/Dual D500 GPU).
Given that a MineCraft style game has a landscape that constantly changes based on the players actions I don't see how I can optimize this within the confines of Scene Kit. A big disappointment as I really like the framework. I'd love to hear someone's ideas on how I can address this issue - without that, I'm forced to go with OpenGL.
UPDATE 12-30-2014 # 2:00pm ET:
I am seeing a significant performance improvement when using flattenedClone(). The FPS is now a solid 60fps even with more boxes and TWO drawing calls. However, accommodating a dynamic environment (as MineCraft supports) is still proving problematic - see below.
Since the array would change composition over time I added a keyDown handler to add an even larger box array to the existing and timed the difference between adding the array of boxes resulting in far more calls versus adding as a flattenedClone. Here's what I found:
On keyDown I add another array of 120 x 120 boxes (14,400 boxes)
// This took .0070333 milliseconds
scene?.rootNode.addChildNode(boxArrayNode)
// This took .02896785 milliseconds
scene?.rootNode.addChildNode(boxArrayNode.flattenedClone())
Calling flattenedClone() again is 4x slower than adding the array.
This results in two drawing calls having 293K faces and 878K vertices. I'm still playing with this and will update if I find anything new. Bottom line, with my additional testing I still feel Scene Kit's immutable geometric constraints mean I can't leverage the framework.
As you mentionned Minecraft, I think it's worth looking at how it works.
I have no technical details or code sample for you, but everything should be pretty straightfoward:
Have you ever played minecraft online, and the terrain is not loading allowing you to see through? That's because there is no geometry inside.
let's assume I have a 2x2x2 array of cubes. That makes 2*2*2*6*2 = 96 triangles.
However, if you test and draw only the polygons on the visible from the camera point of view, maybe by testing the normals (easy since it's cubes), this number goes down to 48 triangles.
If you find a way to see which faces are occluded by other ones (which shouldn't be too hard either considering you're working with flat, quared, grid based faces) you can only draw these. that way, we're drawing between 8 and 24 triangleS. That's up to 90% optimisation.
If you want to get really deep, you can even combine faces, to make a single N-gon out of the visible, flat faces. You can do that if you create a new way to generate the geometry on the fly that combines the two previous methods and test for adgacent visible faces on the same plane.
If you succeed, we're talking 2 to 6 polygons instead of 96, to render 8 cubes.
Note that the last method only works if your blocks are touching each other.
There is probably a ton of Minecraft-like renderer papers, a few googles will help you figure it out!
Why does drop-frame occur?
September 04, 2022
Almost 8 years passed since you asked this question, but not much has changed...
1. Polygons' count
The number of polygons in SceneKit or RealityKit scene must not exceed 100,000 triangular polygons. An ideal SceneKit's scene, that is capable of rendering all the models faster, should contain less than 50,000 polygons. Your scene contains 120,000 polygons. Do not forget that SceneKit renders models using single thread (unlike multi-threaded RealityKit renderer).
2. Shaders
In Xcode 14.0+, SceneKit's default .lightingModel of any 3D library's primitive set in Material Inspector (UI version) is .physicallyBased material. This is the most computationally intensive shader. Programmatic version of the .lightingModel for any SCN procedural geometry is .blinn shading model. The least computationally intensive shader is .constant (it doesn't depend on lighting).
3. What's inside a frustum
If all 10,000 cubes are inside the SceneKit camera frustum, then the frame rate will be 20-30 fps. But if you dollied in the cubes' matrix and see no more than a ninth part of it, then the frame rate will be 60 fps. Thus, SceneKit does not render those objects that are outside the frustum's bounds.
4. Number of meshes in SCNScene
Each model mesh results in a draw call. To achieve 60 fps each draw call should be 16 milliseconds or less. For best performance, Apple engineers advise to limit the number of meshes in a .usdz file to around 50. Unfortunately, I did not find a value for .scn files in the official documentation.
5. Lighting and shadows
Lighting and shadowing (especially shadowing) are very computationally intensive tasks. The general advice is the following – avoid using .forward shadows and hi-rez textures with fake shadows.
Look at this post for details.
SwiftUI code for testing
Xcode 14.0+, SwiftUI 4.0+, Swift 5.7
import SwiftUI
import SceneKit
struct ContentView: View {
var scene = SCNScene()
var options: SceneView.Options = [.allowsCameraControl]
var body: some View {
ZStack {
ForEach(-50...49, id: \.self) { x in
ForEach(-50...49, id: \.self) { z in
let _ = DispatchQueue.global().async {
scene.rootNode.addChildNode(createCube(x, 0, z))
}
}
}
SceneView(scene: scene, options: options)
.ignoresSafeArea()
let _ = scene.background.contents = UIColor.black
}
}
func createCube(_ posX: Int, _ posY: Int, _ posZ: Int) -> SCNNode {
let geo = SCNBox(width: 0.5, height: 0.5, length: 0.5,
chamferRadius: 0.0)
geo.firstMaterial?.lightingModel = .constant
let boxNode = SCNNode(geometry: geo)
boxNode.position = SCNVector3(posX, posY, posZ)
return boxNode
}
}
Here, all cubes are within the viewing frustum, so there are obvious reasons for a drop-frame.
And here, just a part of a scene is within the viewing frustum, so there is no drop-frame.
Related
I am making a scientific visualization app of the Galaxy. Among other things, it displays where certain deep sky objects (stars, star clusters, nebulae, etc) are located in and around the Galaxy.
There are 6 or 7 classes of object types (stars, star clusters, nebulae, globular clusters, etc). Each object within a class looks the same (i.e. using the same image).
I've tried creating GameObjects for each deep sky object, but the system can get bogged down with many objects (~10,000). So instead I create a particle system for each class of deep sky object, setting the specific image to display for each class.
Each particle (i.e. deep sky object) is created at the appropriate location and then I do a SetParticles() to add them to that class's particle system. This works really well and I can have 100,000 objects (particles) with decent performance.
However, I need to allow the user to click/tap on an object to select it. I have not found any examples of how to do hit testing on individual particles in the particle system. Is this possible in Unity?
Thanks,
Bill
You'll have to do the raycasting yourself.
Just implement a custom raycasting algorithm using a simple line rectangle intersection. Simply assume a small rectangle at each particle's position. Since you do not rely on Unity's built in methods you can do this async. For performance optimization you can also cluster the possible targets at simulation start, allowing the elimination of whole clusters when their bounding box is not hit by your ray.
Note: Imho you should choose a completely different approach for your data rendering.
Take a look at unity's entity component system. This allows for large amounts of data, but comes with some disadvantages (e.g. when using Unity's physics engine) (which will not be of relevance for your case I suppose).
I ended up rolling my own solution.
In Update(), upon detecting a click, I iterate through all the particles. For each particle, I calculate its size on the screen based on the particle's size and its distance from the camera.
Then I take the particle's position and translate that into screen coordinates. I use the screen size to generate a bounding rectangle and then test to see if the mouse point is inside it.
As I iterate through the particles I keep track of which is the closest hit. At the end, that is my answer.
if (Input.GetMouseButtonDown(0))
{
Particle? closestHitParticle = null;
var closestHitDist = float.MaxValue;
foreach (var particle in gcParticles)
{
var pos = particle.position;
var size = particle.GetCurrentSize(gcParticleSystem);
var distance = Vector3.Distance(pos, camera.transform.position);
var screenSize = Utility.angularSizeOnScreen(size, distance, camera);
var screenPos = camera.WorldToScreenPoint(pos);
var screenRect = new Rect(screenPos.x - screenSize / 2, screenPos.y - screenSize / 2, screenSize, screenSize);
if (screenRect.Contains(Input.mousePosition) && distance < closestHitDist)
{
closestHitParticle = particle;
closestHitDist = distance;
}
}
if (closestHitDist < float.MaxValue)
{
Debug.Log($"Hit particle at {closestHitParticle?.position}");
}
Here is the angularSizeOnScreen method:
public static float angularSizeOnScreen (float diam, float dist, Camera cam)
{
var aSize = (diam / dist) * Mathf.Rad2Deg;
var pSize = ((aSize * Screen.height) / cam.fieldOfView);
return pSize;
}
Is it normal to have very low FPS (~7fps to ~10fps) with Sprite Kit using the code below?
Use case:
I'm drawing just lines from bottom to top (1024 * 64 lines). I have some delta value that determines the positions of a single line for every frame. These lines represent my CGPath, which is assigned to the SKShapeNode every frame. Nothing else. I'm wondering about the performance of SpriteKit (or maybe of Swift).
Do you have any suggestions to improve the performance?
Screen:
Code:
import UIKit
import SpriteKit
class SKViewController: UIViewController {
#IBOutlet weak var skView: SKView!
var scene: SKScene!
var lines: SKShapeNode!
let N: Int = 1024 * 64
var delta: Int = 0
override func viewDidLoad() {
super.viewDidLoad()
scene = SKScene(size: skView.bounds.size)
scene.delegate = self
skView.showsFPS = true
skView.showsDrawCount = true
skView.presentScene(scene)
lines = SKShapeNode()
lines.lineWidth = 1
lines.strokeColor = .white
scene.addChild(lines)
}
}
extension SKViewController: SKSceneDelegate {
func update(_ currentTime: TimeInterval, for scene: SKScene) {
let w: CGFloat = scene.size.width
let offset: CGFloat = w / CGFloat(N)
let path = UIBezierPath()
for i in 0 ..< N { // N -> 1024 * 64 -> 65536
let x1: CGFloat = CGFloat(i) * offset
let x2: CGFloat = x1
let y1: CGFloat = 0
let y2: CGFloat = CGFloat(delta)
path.move(to: CGPoint(x: x1, y: y1))
path.addLine(to: CGPoint(x: x2, y: y2))
}
lines.path = path.cgPath
// Updating delta to simulate the changes
//
if delta > 100 {
delta = 0
}
delta += 1
}
}
Thanks and Best regards,
Aiba ^_^
CPU
65536 is a rather large number. Telling the CPU to comprehend this many loops will always result in slowness. For example, even if I make a test Command Line project that only measures the time it takes to run an empty loop:
while true {
let date = Date().timeIntervalSince1970
for _ in 1...65536 {}
let date2 = Date().timeIntervalSince1970
print(1 / (date2 - date))
}
It will result in ~17 fps. I haven't even applied the CGPath, and it's already appreciably slow.
Dispatch Queue
If you want to keep your game at 60fps, but your rendering of specifically your CGPath may be still slow, you can use a DispatchQueue.
var rendering: Bool = false // remember to make this one an instance value
while true {
let date = Date().timeIntervalSince1970
if !rendering {
rendering = true
let foo = DispatchQueue(label: "Run The Loop")
foo.async {
for _ in 1...65536 {}
let date2 = Date().timeIntervalSince1970
print("Render", 1 / (date2 - date))
}
rendering = false
}
}
This retains a natural 60fps experience, and you can update other objects, however, the rendering of your SKShapeNode object is still quite slow.
GPU
If you'd like to speed up the rendering, I would recommend looking into running it on the GPU instead of the CPU. The GPU (Graphics Processing Unit) is much better fitted for this, and can handle huge loops without disturbing gameplay. This may require you to program it as an SKShader, in which there are tutorials for.
Check the number of subdivisions
No iOS device has a screen width over 3000 pixels or 1500 points (retina screens have logical points and physical pixels where a point is equivalent to 2 or 3 pixels depending on the scale factor; iOS works with points, but you have to also remember pixels), and the ones that even come close are those with the biggest screens (iPad Pro 12.9 and iPhone Pro Max) in landscape mode.
A typical device in portrait orientation will be less than 500 points and 1500 pixels wide.
You are dividing this width into 65536 parts, and will end up with pixel (not even point) coordinates like 0.00, 0.05, 0.10, 0.15, ..., 0.85, which will actually refer to the same pixel twenty times (my result, rounded up, in an iPhone simulator).
Your code draws twenty to sixty lines in the exact same physical position, on top of each other! Why do that? If you set N to w and use 1.0 for offset, you'll have the same visible result at 60 FPS.
Reconsider the approach
The implementation will still have some drawbacks, though, even if you greatly reduce the amount of work to be done per frame. It's not recommended to advance animation frames in update(_:) since you get no guarantees on the FPS, and you usually want your animation to follow a set schedule, i.e. complete in 1 second rather than 60 frames. (Should the FPS drop to, say, 10, a 60-frame animation would complete in 6 seconds, whereas a 1-second animation would still finish in 1 second, but at a lower frame rate, i.e. skipping frames.)
Visibly, what your animation does is draw a rectangle on the screen whose width fills the screen, and whose height increases from 0 to 100 points. I'd say, a more "standard" way of achieving this would be something like this:
let sprite = SKSpriteNode(color: .white, size: CGSize(width: scene.size.width, height: 100.0))
sprite.yScale = 0.0
scene.addChild(sprite)
sprite.run(SKAction.repeatForever(SKAction.sequence([
SKAction.scaleY(to: 1.0, duration: 2),
SKAction.scaleY(to: 0.0, duration: 0.0)
])))
Note that I used SKSpriteNode because SKShapeNode is said to suffer from bugs and performance issues, although people have reported some improvements in the past few years.
But if you do insist on redrawing the entire texture of your sprite every frame due to some specific need, that may indeed be something for custom shaders… But those require learning a whole new approach, not to mention a new programming language.
Your shader would be executed on the GPU for each pixel. I repeat: the code would be executed for each single pixel – a radical departure from the world of SpriteKit.
The shader would access a bunch of values to work with, such a normalized set of coordinates (between (0.0,0.0) and (1.0,1.0) in a variable called v_tex_coord) and a system time (seconds elapsed since the shader has been running) in u_time, and it would need to determine what color value the pixel in question would need to be – and set it by storing the value in the variable gl_FragColor.
It could be something like this:
void main() {
// default to a black color, or a three-dimensional vector v3(0.0, 0.0, 0.0):
vec3 color = vec3(0.0);
// take the fraction part of the time in seconds;
// this value will go from 0.0 to 0.9999… every second, then drop back to 0.0.
// use this to determine the relative height of the area we want to paint white:
float height = fract(u_time);
// check if the current pixel is below the height of the white area:
if (v_tex_coord.y < height) {
// if so, set it to white (a three-dimensional vector v3(1.0, 1.0, 1.0)):
color = vec3(1.0);
}
gl_FragColor = vec4(color,1.0); // the fourth dimension is the alpha
}
Put this in a file called shader.fsh, create a full-screen sprite mySprite, and assign the shader to it:
mySprite.shader = SKShader.init(fileNamed: "shader.fsh")
Once you display the sprite, its shader will take care of all of the rendering. Note, however, that your sprite will lose some SpriteKit functionalities as a result.
I have used the particle emitter to create a background with stars. It looks ok, but I would like them to blink, or flicker. The closest I get is when I change the birthrate and lifetime variables so that particles disappear and appear at different places. I would like the particles to remain in the same place and fade in and out, randomly, though. Any ideas on how to do this? This is what I've got so far:
I don't think you can do much directly in the editor. If you're comfortable working with code for adjusting the emitter, you have a couple of possibilities: setting a particle action to animate color or alpha or scale or texture, or a custom shader to do whatever sort of animation. (I'm assuming based on your picture with a basically infinite lifetime that you don't want things to move or disappear. That may rule out keyframing, but perhaps having the keyframe sequence set to repeat mode with the frames spaced by really tiny values would work.)
Another possibility since positions are static would be to just make some fixed sprites scattered around at random and have them run actions to animate them. We've used this approach before with ~100 animated sprites against a backdrop that has a bunch of dimmer stars, and it looked pretty good. Something along these lines:
let twinklePeriod = 8.0
let twinkleDuration = 0.5
let bright = CGFloat(0.3)
let dim = CGFloat(0.1)
let brighten = SKAction.fadeAlpha(to: bright, duration: 0.5 * twinkleDuration)
brighten.timingMode = .easeIn
let fade = SKAction.fadeAlpha(to: dim, duration: 0.5 * twinkleDuration)
fade.timingMode = .easeOut
let twinkle = SKAction.repeatForever(.sequence([brighten, fade, .wait(forDuration: twinklePeriod - twinkleDuration)]))
for _ in 0 ..< 100 {
let star = SKSpriteNode(imageNamed: "star")
star.position = CGPoint(x: .random(in: minX ... maxX), y: .random(in: minY ... maxY))
star.alpha = dim
star.speed = .random(in: 0.5 ... 1.5)
star.run(.sequence([.wait(forDuration: .random(in: 0 ... twinklePeriod)), twinkle]))
addChild(star)
}
That's cut-and-pasted from various bits and simplified some, so there may be typos, but it should give the idea. If you keep the emitter, you can try something like the twinkle above as the particle action. I don't see how you can change the relative periods of particles though like you could with separate sprites, and the only offsets would come from differences in the birth time of the particles.
Blocks just crumble apart.
How can this problem be solved?
Initializing blocks:
var boxNode = SCNNode(geometry: SCNBox(width: 0.75, height: 0.15, length: 0.25, chamferRadius: 0))
boxNode.position = SCNVector3(x: x1, y: y, z: z1)
boxNode.geometry?.firstMaterial = SCNMaterial()
boxNode.geometry?.firstMaterial?.diffuse.contents = UIImage(named: "wood.jpg")
boxNode.physicsBody = SCNPhysicsBody(type: .dynamic, shape: nil)
boxNode.eulerAngles.y = Float(Double.pi / 2) * rotation
boxNode.physicsBody?.friction = 1
boxNode.physicsBody?.mass = 0.5
boxNode.physicsBody?.angularDamping = 1.0
boxNode.physicsBody?.damping = 1
picture
video
full code
I won't be able to tell you how to fix it as I have the exact same problem which I wasn't able to solve. However, as I played around I figured out a couple of things (which you may find useful):
The same problem hasn't happened to me in pure SceneKit, hence I think it's a bug in ARKit
Node with physics has to be added to the rootNode of the scene, otherwise odd stuff happens (elements passing through each other, gravity behaving in an inconsistent way)
If you pass nil as shape parameter, SceneKit will figure bounding box based on the geometry of the node. This hasn't worked properly for me so what I've done (using SceneKit editor) was to duplicate the geometry and then set it as a custom shape for the bounding box (have a look at the attached image)
Overall I've found physics simulation in SceneKit when used with ARKit to be extremely buggy and I spent a lot of time "tricking" it into working more-or-less how I wanted it to work.
Apologies in advance as I'm not sure exactly what the right question is. The problems that I'm ultimately trying to address are:
1) Game gets laggy at times
2) CPU % can get high, as much as 50-60% at times, but is also sometimes relatively low (<20%)
3) Device (iPhone 6s) can get slightly warm
I believe what's driving the lagginess is that I'm constantly creating and removing circles in the SKScene. It's pretty much unavoidable because the circles are a critical element to the game and I have to constantly change their size and physicsBody properties so there's not much I can do in terms of reusing nodes. Additionally, I'm moving another node almost constantly.
func addCircle() {
let attributes = getTargetAttributes() //sets size, position, and color of the circle
let target = /*SKShapeNode()*/SKShapeNode(circleOfRadius: attributes.size.width)
let outerPathRect = CGRect(x: 0, y: 0, width: attributes.size.width * 2, height: attributes.size.width * 2)
target.position = attributes.position
target.fillColor = attributes.color
target.strokeColor = attributes.stroke
target.lineWidth = 8 * attributes.size.width / 35
target.physicsBody = SKPhysicsBody(circleOfRadius: attributes.size.width)
addStandardProperties(node: target, name: "circle", z: 5, contactTest: ContactCategory, category: CircleCategory) //Sets physicsBody properties
addChild(target)
}
The getAttributes() function is not too costly. It does have a while loop to set the circle position, but it doesn't usually get used when the function is called. Otherwise, it's simple math.
Some other details:
1) The app runs at a constant 120 fps. I've tried setting the scene/view lower by adding view.preferredFramesPerSecond = 60 in GameScene.swift and gameScene.preferredFramesPerSecond = 60 in GameViewController. Neither one of these does anything to change the fps. Normally when I've had performance issues in other apps, the fps dipped, however, that isn't happening here.
2) I’ve tried switching the SKShapeNode initializer to use a path versus circleOfRadius and then resetting the path. I’ve also tried images, however, because I have to reset the physicsBody, there doesn’t appear to be a performance gain.
3) I tried changing the physicsWorld speed, but this also had little effect.
4) I've also used Instruments to try to identify the issue. There are big chunks of resources being used by SKRenderer, however, I can't find much information on this.
Creating SKShapeNodes are inefficient, try to use it as few times as you can. instead, create a template shape, and convert it to an SKSpriteNode.
If you need to change the size, then use xScale and yScale, if you need to change the color, then use color with colorBlendFactor of 1
If you need to have a varying color stroke, then change the below code to have 2 SKSpriteNodes, 1 SKSpriteNode that handles only the fill, and 1 SKSpriteNode that handles only the stroke. Have the stroke sprite be a child of the fill sprite with a zPosition of 0 and set the stroke color to white. You can then apply the color and colorBlendFactor to the child node of the circle to change the color.
lazy var circle =
{
let target = SKShapeNode(circleOfRadius: 1000)
target.fillColor = .white
//target.strokeColor = .black //if stroke is anything other than black, you may need to do 2 SKSpriteNodes that layer each other
target.lineWidth = 8 * 1000 / 35
let texture = SKView().texture(from:target)
let spr = SKSpriteNode(texture:texture)
spr.physicsBody = SKPhysicsBody(circleOfRadius: 1000)
addStandardProperties(node: spr, name: "circle", z: 5, contactTest:ContactCategory, category: CircleCategory) //Sets physicsBody properties
return spr
}()
func createCircle(of radius:CGFloat,color:UIColor) -> SKSpriteNode
{
let spr = circle.copy()
let scale = radius/1000.0
spr.xScale = scale
spr.yScale = scale
spr.color = color
spr.colorBlendFactor = 1.0
return spr
}
func addCircle() {
let attributes = getTargetAttributes() //sets size, position, and color of the circle
let spr = createCircle(of:attribute.width,color:attributes.color)
spr.position = attributes.position
addChild(str)
}