Memory leak, despite no strong references? - swift

I'm doing a performance test to try to measure the rendering performance of an important NSOutlineView in my Mac app. In the process, I'm looping several times, creating the view, embedding it in a dummy window, and rendering it to an image. I'm generalizing a bit, but this is roughly what it looks like:
// Intentionally de-indented these for easier reading in this narrow page
class MyPerformanceTest: XCTestCase { reading
func test() {
measure() {
// autoreleasepool {
let window: NSWindow = {
let w = NSWindow(
contentRect: NSRect.init(x: 100, y: 100, width: 800, height: 1200),
styleMask: [.titled, .resizable, .closable, .miniaturizable],
backing: .buffered,
defer: false
)
w.tabbingMode = .disallowed
w.cascadeTopLeft(from: NSPoint(x: 200, y: 200))
w.makeKeyAndOrderFront(nil)
w.contentView = testContentView // The thing I'm performance testing
return w
}()
let bitmap = self.bitmapImageRepForCachingDisplay(in: self.frame)
.map { bitmap in
self.cacheDisplay(in: self.frame, to: bitmap)
return bitmap
}
let data = bitmap.representation(using: .png, properties: [:])!
saveToDesktop(data, name: "image1.png") // Helper function around Data.write(to:). Boring.
window.isReleasedWhenClosed = false // Defaults to true, but crashes if true.
window.close()
// }
}
}
}
I noticed that this was building up memory usage. Each window allocated in each loop of my measure(_:) block was sticking around. This makes sense, because I don't have the main run loop running so the Thread's auto-release pool is never drained. I wrapped my entire measure block in a call to autoreleasepool block, and this was resolved. Using the memory graph debugger, I confirmed that there was only ever 1 window max, which would be the one from the current iteration. Great.
However, I found the my NSOutlineViews, their rows, and their row models were still sticking around. There were thousands of them, so it was really blowing up the memory usage.
I profiled it with the Leaks instrument in Instruments: no leaks.
Then I inspected the objects in the memory graph debugger. There were no obvious strong reference cycles, and all of the objects had cases similar to this example. It's an NSOutlineView (well, a dynamic NSKVONotifying_* subclass, but that doesn't matter), with only one strong reference from an ObjC block. But that block is only referenced, weakly, by one reference (the black line). Shouldn't this whole thing have been deallocated?
How can I troubleshoot why this is being kept alive?

How can I troubleshoot why this is being kept alive?
Use Instruments.
Configure Instruments for the Allocations template. Before you start recording, under File > Recording Options, configure the Allocations template options to Record Reference Counts.
Record and pause. Select a region of the track to study. Find the type of object you want to study and hit the little right-arrow to reveal all objects of that type. Select one in the list. Next to the address, hit the little right-arrow.
You will now see a history of retains and releases along with a running reference count. Selecting a retain/release shows the calls stack on the right. Thus you can deduce the memory management history of this object.

Related

MTKView refresh issue

I am compositing an array of UIImages via an MTKView, and I am seeing refresh issues that only manifest themselves during the composite phase, but which go away as soon as I interact with the app. In other words, the composites are working as expected, but their appearance on-screen looks glitchy until I force a refresh by zooming in/translating, etc.
I posted two videos that show the problem in action: Glitch1, Glitch2
The composite approach I've chosen is that I convert each UIImage into an MTLTexture which I submit to a render buffer set to ".load" which renders a poly with this texture on it, and I repeat the process for each image in the UIImage array.
The composites work, but the screen feedback, as you can see from the videos is very glitchy.
Any ideas as to what might be happening? Any suggestions would be appreciated
Some pertinent code:
for strokeDataCurrent in strokeDataArray {
let strokeImage = UIImage(data: strokeDataCurrent.image)
let strokeBbox = strokeDataCurrent.bbox
let strokeType = strokeDataCurrent.strokeType
self.brushStrokeMetal.drawStrokeImage(paintingViewMetal: self.canvasMetalViewPainting, strokeImage: strokeImage!, strokeBbox: strokeBbox, strokeType: strokeType)
} // end of for strokeDataCurrent in strokeDataArray
...
func drawStrokeUIImage (strokeUIImage: UIImage, strokeBbox: CGRect, strokeType: brushTypeMode) {
// set up proper compositing mode fragmentFunction
self.updateRenderPipeline(stampCompStyle: drawStampCompMode)
let stampTexture = UIImageToMTLTexture(strokeUIImage: strokeUIImage)
let stampColor = UIColor.white
let stampCorners = self.stampSetVerticesFromBbox(bbox: strokeBbox)
self.stampAppendToVertexBuffer(stampUse: stampUseMode.strokeBezier, stampCorners: stampCorners, stampColor: stampColor)
self.renderStampSingle(stampTexture: stampTexture)
} // end of func drawStrokeUIImage (strokeUIImage: UIImage, strokeBbox: CGRect)
func renderStampSingle(stampTexture: MTLTexture) {
// this routine is designed to update metalDrawableTextureComposite one stroke at a time, taking into account
// whatever compMode the stroke requires. Note that we copy the contents of metalDrawableTextureComposite to
// self.currentDrawable!.texture because the goal will be to eventually display a resulting composite
let renderPassDescriptorSingleStamp: MTLRenderPassDescriptor? = self.currentRenderPassDescriptor
renderPassDescriptorSingleStamp?.colorAttachments[0].loadAction = .load
renderPassDescriptorSingleStamp?.colorAttachments[0].clearColor = MTLClearColorMake(0, 0, 0, 0)
renderPassDescriptorSingleStamp?.colorAttachments[0].texture = metalDrawableTextureComposite
// Create a new command buffer for each tessellation pass
let commandBuffer: MTLCommandBuffer? = commandQueue.makeCommandBuffer()
let renderCommandEncoder: MTLRenderCommandEncoder? = commandBuffer?.makeRenderCommandEncoder(descriptor: renderPassDescriptorSingleStamp!)
renderCommandEncoder?.label = "Render Command Encoder"
renderCommandEncoder?.setTriangleFillMode(.fill)
defineCommandEncoder(
renderCommandEncoder: renderCommandEncoder,
vertexArrayStamps: vertexArrayStrokeStamps,
metalTexture: stampTexture) // foreground sub-curve chunk
renderCommandEncoder?.endEncoding() // finalize renderEncoder set up
//begin presentsWithTransaction approach (needed to better synchronize with Core Image scheduling
copyTexture(buffer: commandBuffer!, from: metalDrawableTextureComposite, to: self.currentDrawable!.texture)
commandBuffer?.commit() // commit and send task to gpu
commandBuffer?.waitUntilScheduled()
self.currentDrawable!.present()
// end presentsWithTransaction approach
self.initializeStampArray(stampUse: stampUseMode.strokeBezier) // clears out the stamp array in preparation of next draw call
} // end of func renderStampSingle(stampTexture: MTLTexture)
First of all, the domain Metal is very deep, and it's use within the MTKView construct is sparsely documented, especially for any applications that fall outside the more traditional gaming paradigm. This is where I have found myself in the limited experience I have accumulated with Metal with the help from folks like #warrenm, #ken-thomases, and #modj, whose contributions have been so valuable to me, and to the Swift/Metal community at large. So a deep thank you to all of you.
Secondly, to anyone troubleshooting metal, please take note of the following: If you are getting the message:
[CAMetalLayerDrawable present] should not be called after already presenting this drawable. Get a nextDrawable instead
please don't ignore it. It mays seem harmless enough, especially if it only gets reported once. But beware that this is a sign that a part of your implementation is flawed, and must be addressed before you can troubleshoot any other Metal-related aspect of your app. At least this was the case for me. As you can see from the video posts, the symptoms of having this problem were pretty severe and caused unpredictable behavior that I was having a difficult time pinpointing the source of. The thing that was especially difficult for me to see was that I only got this message ONCE early on in the app cycle, but that single instance was enough to throw everything else graphically out of whack in ways that I thought were attributable to CoreImage and/or other totally unrelated design choices I had made.
So, how did I get rid of this warning? Well, in my case, I assumed that having the settings:
self.enableSetNeedsDisplay = true // needed so we can call setNeedsDisplay() to force a display update as soon as metal deems possible
self.isPaused = true // needed so the draw() loop does not get called once/fps
self.presentsWithTransaction = true // for better synchronization with CoreImage (such as simultaneously turning on a layer while also clearing MTKView)
meant that I could pretty much call currentDrawable!.present() or commandBuffer.presentDrawable(view.currentDrawable) directly whenever I wanted to refresh the screen. Well, this is not the case AT ALL. It turns out these calls should only be made within the draw() loop and only accessed via a setNeedsDisplay() call. Once I made this change, I was well on my way to solving my refresh riddle.
Furthermore, I found that the MTKView setting self.isPaused = true (so that I could make setNeedsDisplay() calls directly) still resulted in some unexpected behavior. So, instead, I settled for:
self.enableSetNeedsDisplay = false // needed so we can call setNeedsDisplay() to force a display update as soon as metal deems possible
self.isPaused = false // draw() loop gets called once/fps
self.presentsWithTransaction = true // for better synchronization with CoreImage
as well as modifying my draw() loop to drive what kind of update to carry out once I set a metalDrawableDriver flag AND call setNeedsDisplay():
override func draw(_ rect: CGRect) {
autoreleasepool(invoking: { () -> () in
switch metalDrawableDriver {
case stampRenderMode.canvasRenderNoVisualUpdates:
return
case stampRenderMode.canvasRenderClearAll:
renderClearCanvas()
case stampRenderMode.canvasRenderPreComputedComposite:
renderPreComputedComposite()
case stampRenderMode.canvasRenderStampArraySubCurve:
renderSubCurveArray()
} // end of switch metalDrawableDriver
}) // end of autoreleasepool
} // end of draw()
This may seem round-about, but it was the only mechanism I found to get consistent user-driven display updates.
It is my hope that this post describes an error-free and viable solution that Metal developers may find useful in the future.

sceneDidLoad Running Twice

When I run my program. The code I put into "override func sceneDidLoad()" runs two times.
E.g.
Note: I have no idea why this picture is not uploading, but it shows "spawn" happening twice.
This code should only run once when "sceneDidLoad()" is called.
Here is the code for the "sceneDidLoad" function, and for the "testSpawn()" function (which is the specific one that gave the duplicated printout).
class GameScene: SKScene {
var mapTerrain: SKTileMapNode!
override func sceneDidLoad() {
cam = SKCameraNode()
cam.xScale = 1
cam.yScale = 1
//do zoom by change in scale in pinch. (E.g. if they start out 5 units apart and end up 15 units apart, zoom by a factor of 3
self.camera = cam
self.addChild(cam)
cam.position = CGPoint(x: 100, y: 100)
setupLayers()
loadSceneNodes()
setUpUI()
testSpawn()
//print("\(self.frame.width), \(self.frame.height)")
}
func testSpawn(){
let RedLegion = legion(texture: textureRedLegion, moveTo: nil, tag: 1, health: 2)
RedLegion.position = mapTerrain.centerOfTile(atColumn: 0, row: 0)
RedLegion.team = "Red"
unitsLayer.addChild(RedLegion)
legionList.append(RedLegion)
print("spawn")
}
}
Note: Not all of the code is here (like "setUpLayers()"), if needed I can supply it, I just do not think it is neccessary.
Search your whole document for "print("spawn")" just to make sure that is the only time you call the function. Also check for "testSpawn()" to make sure it is only called once. Additionally, instead of relying on this print to count how many times the sceneDidLoad runs, place a print directly within your sceneDidLoad. Finally, check to make sure you are not creating the scene twice.
I've also seen this and submitted a bug report but apple responded saying that it is intended behavior. Apple said that it creates a dummy scene and then creates the actual scene. Before it runs the second time it gets rid of anything done the first time so you shouldn't get any errors from it. The bug is really hard to reproduce, one of my friends working off the same repository that I was but did not experience the bug.
I changed sceneDidLoad to didMoveToView:(SKView *)view if you are looking for a solution to this. Make sure you xcode is up to date.

Accessing properties of multiple SKShapeNodes

In my program I have a method called addObstacle, which creates a rectangular SKShapeNode with an SKPhysicsBody, and a leftward velocity.
func addObstacle(bottom: CGFloat, top: CGFloat, width: CGFloat){
let obstacleRect = CGRectMake(self.size.width + 100, bottom, width, (top - bottom))
let obstacle = SKShapeNode(rect: obstacleRect)
obstacle.name = "obstacleNode"
obstacle.fillColor = UIColor.grayColor()
obstacle.physicsBody = SKPhysicsBody(edgeLoopFromPath: obstacle.path!)
obstacle.physicsBody?.dynamic = false
obstacle.physicsBody?.affectedByGravity = false
obstacle.physicsBody?.contactTestBitMask = PhysicsCatagory.Ball
obstacle.physicsBody?.categoryBitMask = PhysicsCatagory.Obstacle
obstacle.physicsBody?.usesPreciseCollisionDetection = true
self.addChild(obstacle)
obstacle.runAction(SKAction.moveBy(obstacleVector, duration: obstacleSpeed))
}
In a separate method, called endGame, I want to fade out all the obstacles currently in existence on the screen. All the obstacle objects are private, which makes accessing their properties difficult. If there is only one on the screen, I can usually access it by its name. However, when I say childNodeWithName("obstacleNode")?.runAction(SKAction.fadeAlphaBy(-1.0, duration: 1.0)), only one of the "obstacles" fades away; the rest remain completely opaque. Is there a good way of doing this? Thanks in advance (:
You could probably go with:
self.enumerateChildNodesWithName("obstacleNode", usingBlock: {
node, stop in
//do your stuff
})
More about this method can be found here.
In this example I assumed that you've added obstacles to the scene. If not, then instead of scene, run this method on obstacle's parent node.
And one side note...SKShapeNode is not performant solution in many cases because it requires at least one draw pass to be rendered by the scene (it can't be drawn in batches like SKSpriteNode). If using a SKShapeNode is not "a must" in your app, and you can switch them with SKSpriteNode, I would warmly suggest you to do that because of performance.
SpriteKit can render hundreds of nodes in a single draw pass if you are using same atlas and same blending mode for all sprites. This is not the case with SKShapeNodes. More about this here. Search SO about this topic, there are some useful posts about all this.

vImageAlphaBlend crashes

I'm trying to alpha blend some layers: [CGImageRef] in the drawLayer(thisLayer: CALayer!, inContext ctx: CGContext!) routine of my custom NSView. Until now I used CGContextDrawImage() for drawing those layers into the drawLayer context. While profiling I noticed CGContextDrawImage() needs 70% of the CPU time so I decided to try the Accelerate framework. I changed the code but it just crashes and I have no clue what the reason could be.
I'm creating those layers like this:
func addLayer() {
let colorSpace: CGColorSpaceRef = CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB)
let bitmapInfo = CGBitmapInfo(CGImageAlphaInfo.PremultipliedFirst.rawValue)
var layerContext = CGBitmapContextCreate(nil, UInt(canvasSize.width), UInt(canvasSize.height), 8, UInt(canvasSize.width * 4), colorSpace, bitmapInfo)
var newLayer = CGBitmapContextCreateImage(layerContext)
layers.append( newLayer )
}
My drawLayers routine looks like this:
override func drawLayer(thisLayer: CALayer!, inContext ctx: CGContext!)
{
var ctxImageBuffer = vImage_Buffer(data:CGBitmapContextGetData(ctx),
height:CGBitmapContextGetHeight(ctx),
width:CGBitmapContextGetWidth(ctx),
rowBytes:CGBitmapContextGetBytesPerRow(ctx))
for imageLayer in layers
{
//CGContextDrawImage(ctx, CGRect(origin: frameOffset, size: canvasSize), imageLayer)
var inProvider:CGDataProviderRef = CGImageGetDataProvider(imageLayer)
var inBitmapData:CFDataRef = CGDataProviderCopyData(inProvider)
var buffer:vImage_Buffer = vImage_Buffer(data: &inBitmapData, height:
CGImageGetHeight(imageLayer), width: CGImageGetWidth(imageLayer), rowBytes:
CGImageGetBytesPerRow(imageLayer))
vImageAlphaBlend_ARGB8888(&buffer, &ctxImageBuffer, &ctxImageBuffer, 0)
}
}
the canvasSize is allways the same and also all the layers have the same size, so I don't understand why the last line crashes.
Also I don't see how to use the new convenience functions to create vImageBuffers directly from CGLayerRefs. That's why I do it the complicated way.
Any help appreciated.
EDIT
inBitmapData indeed holds pixel data that reflect the background color I set. However the debugger can not po &inBitmapData and fails with this message:
error: reference to 'CFData' not used to initialize a inout parameter &inBitmapData
So I looked for a way to get the pointer to inBitmapData. That is what I came up with:
var bitmapPtr: UnsafeMutablePointer<CFDataRef> = UnsafeMutablePointer<CFDataRef>.alloc(1)
bitmapPtr.initialize(inBitmapData)
I also had to change the way to point at my data for both buffers that i need for the alpha blend input. Now it's not crashing anymore and luckily the speed boost is inspectable with a profiler (vImageAlphaBlend only takes about a third of CGContextDrawImage), but unfortunately the image results in a transparent image with pixel failures instead of the white image background.
So far I don't get any runtime errors anymore but since the result is not as expected I fear that I still don't use the alpha blend function correctly.
vImage_Buffer.data should point to the CFData data (pixel data), not the CFDataRef.
Also, not all images store their data as four channel, 8-bit per channel data. If it turns out to be three channel or RGBA or monochrome, you may get more crashing or funny colors. Also, you have assumed that the raw image data is not premultiplied, which may not be a safe assumption.
You are better off using vImageBuffer_initWithCGImage so that you can guarantee the format and colorspace of the raw image data. A more specific question about that function might help us resolve your confusion about it.
Some CG calls fall back on vImage to do the work. Rewriting your code in this way might be unprofitable in such cases. Usually the right thing to do first is to look carefully at the backtraces in the CG call to try to understand why you are causing so much work for it. Often the answer is colorspace conversion. I would look carefully at the CGBitmapInfo and colorspace of the drawing surface and your images and see if there wasn't something I could do to get those to match up a bit better.
IIRC, CALayerRefs usually have their data in non cacheable storage for better GPU access. That could cause problems for the CPU. If the data is in a CALayerRef I would use CA to do the compositing. Also, I thought that CALayers are nearly always BGRA 8-bit premultiplied. If you are not going to use CA to do the compositing, then the right vImage function is probably vImagePremultipliedAlphaBlend_RGBA/BGRA8888.

How do I remove a group of Nodes in Swift?

I'm new to objective c and swift and I created a small app where small circles are rendered and once the player collides with a circle, the game ends. I managed to get everything to work, but how do I remove the nodes after they collide. I tried removeAllChildren(), but none of them disappear. When I use removeFromParent(), only 1 disappears. I want a way to remove all 3 nodes that will be rendered in the code below
//addEvilGuys() is called first
func addEvilGuys()
{
addEvilGuy(named: "paul", speed: 1.3, xPos: CGFloat(self.size.height/3))
addEvilGuy(named: "boris", speed: 1.7, xPos: frame.size.width/4 + 50)
addEvilGuy(named: "natasha", speed: 1.5, xPos: frame.size.width/4 + 150)
}
func addEvilGuy(#named:String, speed:Float, xPos: CGFloat)
{
evilGuyNode = SKSpriteNode(imageNamed: named)
evilGuyNode.zPosition = 10
evilGuyNode.physicsBody = SKPhysicsBody(circleOfRadius: 16)
evilGuyNode.physicsBody!.affectedByGravity = false
evilGuyNode.physicsBody!.categoryBitMask = ColliderType.BadGuy.rawValue
evilGuyNode.physicsBody!.contactTestBitMask = ColliderType.Hero.rawValue
evilGuyNode.physicsBody!.collisionBitMask = ColliderType.Hero.rawValue
evilGuyNodeCount++
var evilGuy = EvilGuy(speed: speed, eGuy: evilGuyNode)
evilGuys.append(evilGuy)
resetEvilGuy(evilGuyNode, xPos: xPos)
evilGuy.xPos = evilGuyNode.position.x
addChild(evilGuyNode)
}
func resetEvilGuy(evilGuyNode:SKSpriteNode, xPos:CGFloat)
{
evilGuyNode.position.y = endOfScreenBottom
evilGuyNode.position.x = xPos
}
It looks like in addEvilGuy you are recreating a stored property (i.e. that is visible for the entire class + whatever the access level allows) to create the SKSpriteNode that you're adding. This means that you are orphaning the previously created EvilGuy.
In addEvilGuy, replace
evilGuyNode = SKSpriteNode(imageNamed: named)
with
let evilGuyNode = SKSpriteNode(imageNamed: named)
and remove the property from your class (it doesn't seem like you have a need for in in a larger scope).
It also looks like you're creating EvilGuys and storing them in an array, which is good. So when you can remove all of them from the screen with a function like:
func removeAllEvilGuys(evilGuys: [EvilGuy]) {
for evilGuy in evilGuys {
evilGuy.eGuy.removeFromParent()
}
}
As a best practice advice, since you mentioned you're a beginner:
I'd recommend defining the characteristics of the evil guys in a .plist and then use the file to create an array of evil guys. This way you can easily make changes to the evil guys in that file without having to change anything in your code.
The code that creates an EvilGuy object should be separated from the one that adds the evil guy to the screen. As long as you are storing the SKNode of each one, you'll be able to add/remove without unnecessarily recreating the entire object.