Drawing directly in a NSView without using the draw(_ updateRect: NSRect) function - swift

I would like to draw CGImage pictures directly to a View and with the normal method using the draw func I only get 7 pictures in a second on a new Mac Book Pro. So I decided to use the updateLayer func instead. I have defined wantsUpdateLayer = true and my new updateLayer func is called as expected. But then starts my problem. When using the draw func, I get the current CGContext with "NSGraphicsContext.current?.cgContext" but in my updateLayer func the "NSGraphicsContext.current?.cgContext" is nil. So I do not know where to put my CGImage, that it will be displayed on my screen. Also "self.view?.window?.graphicsContext?.cgContext" and "self.window?.graphicsContext?.cgContext" are nil, too. There are no buttons or other elements in this view and in the window of the view, only one picture, filling the complete window. And this picture must change 30 times in a second. Generating the pictures is done by a separate thread and needs about 1 millisecond for a picture. I think that from "outside" the NSView class it is not possible to write the picture but my updateLayer func is inside the class.
Here is what the func looks like actually:
override func updateLayer ()
{
let updateRect: NSRect = NSRect(x: 0.0, y: 0.0, width: 1120.0, height: 768.0)
let context1 = self.view?.window?.graphicsContext?.cgContext
let context2 = self.window?.graphicsContext?.cgContext
let context3 = NSGraphicsContext.current?.cgContext
}
And all three contexts are nil in the time the function is called automatically after I set the needsDisplay flag.
Any ideas where to draw my CGImages?
The updateLayer func is called automatically by the user interface. I do not call it manually. It is called by the view. My problem is where inside this method to put my picture to be shown on the screen. Perhaps I have to add a layer or use a default layer of the view but I do not know how to do this.

Meanwhile I have found the solution with some tipps from a good friend:
override var wantsUpdateLayer: Bool
{
return (true)
}
override func updateLayer ()
{
let cgimage: CGImage? = picture // Here comes the picture
if cgimage != nil
{
let nsimage: NSImage? = NSImage(cgImage: cgimage!, size: NSZeroSize)
if nsimage != nil
{
let desiredScaleFactor: CGFloat? = self.window?.backingScaleFactor
if desiredScaleFactor != nil
{
let actualScaleFactor: CGFloat? = nsimage!.recommendedLayerContentsScale(desiredScaleFactor!)
if actualScaleFactor != nil
{
self.layer!.contents = nsimage!.layerContents(forContentsScale: actualScaleFactor!)
self.layer!.contentsScale = actualScaleFactor!
}
}
}
}
}
This is the way to directly write into the layer. Depending on the format (CGImage or NSImage) you first must convert it. As soon as the func wantsUpdateLayer returns a true, the func updateLayer() is used instead of the func draw(). Thats all.
For all who want to see my "Normal" draw function:
override func draw (_ updateRect: NSRect)
{
let cgimage: CGImage? = picture // Here comes the picture
if cgimage != nil
{
if #available(macOS 10.10, *)
{
NSGraphicsContext.current?.cgContext.draw(cgimage!, in: updateRect)
}
}
else
{
super.draw(updateRect)
}
}
The additional speed is 2 times or more, depending on what hardware you use. On a modern Mac Pro there is only a little bit more speed but on a modern Mac Book Pro you will get 10 times or more speed. This works with Mojave 10.14.6 and Catalina 10.15.6. I did not test it with older macOS versions. The "Normal" draw function works with 10.10.6 to 10.15.6.

Related

How to get target UITraitCollection as well as target rect

I have a tool pane which in compact H mode, will be at the bottom spanning the full screen, but in compact V mode (or non compact H mode), it will be on the right as a floating pane. How do I get the target UITraitCollection + the target size? They seem to be in 2 different methods:
override func willTransition(to newCollection: UITraitCollection, with coordinator: UIViewControllerTransitionCoordinator) {
// need size rect
}
override func viewWillTransition(to size: CGSize, with coordinator: UIViewControllerTransitionCoordinator) {
super.viewWillTransition(to: size, with: coordinator)
// need traits
}
I need both infos for animating things properly! Thanks so much!
You can in fact get both values, the size and the trait collection in both methods, by operating on the UIViewControllerTransitionCoordinator.
let didQueueAnimation = coordinator.animate(alongsideTransition: { context in
guard let view = context.view(forKey: UITransitionContextViewKey.to) else {
return
}
let newScreenSize = view.frame.size
guard let viewController = context.viewController(forKey: UITransitionContextViewKey.to) else {
return
}
let newTraitCollection = viewController.traitCollection // <-- New Trait Collection
// You can also get the new view size from the VC:
// let newTraitCollection = viewController.view.frame.size
// Perform animation if trait collection and size match.
}, completion: { context in
// Perform animation cleanup / other pending tasks.
})
if (didQueueAnimation) {
print("Animation was queued.")
}
Apple's idea here is to simplify the call site with one context parameter that can be queried for multiple properties, and also execute asynchronously so that the values for the final transitioned view or VC can be fetched accurately in one place even before the update occurs.
While you can perform animations in either of the WillTransition methods, I would use the coordinator.animate(alongsideTransition:completion:) method if possible rather than coordinator.notifyWhenInteractionChanges(_:) since it synchronizes the system animation alongside your own custom animation and you can still query the context for the new traitCollection or frame.size using the techniques above.

How, exactly, do I render Metal on a background thread?

This problem is caused by user interface interactions such as showing the titlebar while in fullsreen. That question's answer provides a solution, but not how to implement that solution.
The solution is to render on a background thread. The issue is, the code provided in Apple's is made to cover a lot of content so most of it will extraneous code, so even if I could understand it, it isn't feasible to use Apple's code. And I can't understand it so it just plain isn't an option. How would I make a simple Swift Metal game use a background thread being as concise as possible?
Take this, for example:
class ViewController: NSViewController {
var MetalView: MTKView {
return view as! MTKView
}
var Device: MTLDevice = MTLCreateSystemDefaultDevice()!
override func viewDidLoad() {
super.viewDidLoad()
MetalView.delegate = self
MetalView.device = Device
MetalView.colorPixelFormat = .bgra8Unorm_srgb
Device = MetalView.device
//setup code
}
}
extension ViewController: MTKViewDelegate {
func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
}
func draw(in view: MTKView) {
//drawing code
}
}
That is the start of a basic Metal game. What would that code look like, if it were rendering on a background thread?
To fix that bug when showing the titlebar in Metal, I need to render it on a background thread. Well, how do I render it on a background thread?
I've noticed this answer suggests to manually redraw it 60 times a second. Presumably using a loop that is on a background thread? But that seems... not a clean way to fix it. Is there a cleaner way?
The main trick in getting this to work seems to be setting up the CVDisplayLink. This is awkward in Swift, but doable. After some work I was able to modify the "Game" template in Xcode to use a custom view backed by CAMetalLayer instead of MTKView, and a CVDisplayLink to render in the background, as suggested in the sample code you linked — see below.
Edit Oct 22:
The approach mentioned in this thread seems to work just fine: still using an MTKView, but drawing it manually from the display link callback. Specifically I was able to follow these steps:
Create a new macOS Game project in Xcode.
Modify GameViewController to add a CVDisplayLink, similar to below (see this question for more on using CVDisplayLink from Swift). Start the display link in viewWillAppear and stop it in viewWillDisappear.
Set mtkView.isPaused = true in viewDidLoad to disable automatic rendering, and instead explicitly call mtkView.draw() from the display link callback.
The full content of my modified GameViewController.swift is available here.
I didn't review the Renderer class for thread safety, so I can't be sure no more changes are required, but this should get you up and running.
Older implementation with CAMetalLayer instead of MTKView:
This is just a proof of concept and I can't guarantee it's the best way to do everything. You might find these articles helpful too:
I didn't try this idea, but given how much convenience MTKView generally provides over CAMetalLayer, it might be worth giving it a shot:
https://developer.apple.com/forums/thread/89241?answerId=268384022#268384022
Is drawing to an MTKView or CAMetalLayer required to take place on the main thread? and https://developer.apple.com/documentation/quartzcore/cametallayer/1478157-presentswithtransaction
class MyMetalView: NSView {
var displayLink: CVDisplayLink?
var metalLayer: CAMetalLayer!
override init(frame frameRect: NSRect) {
super.init(frame: frameRect)
setupMetalLayer()
}
required init?(coder: NSCoder) {
super.init(coder: coder)
setupMetalLayer()
}
override func makeBackingLayer() -> CALayer {
return CAMetalLayer()
}
func setupMetalLayer() {
wantsLayer = true
metalLayer = layer as! CAMetalLayer?
metalLayer.device = MTLCreateSystemDefaultDevice()!
// ...other configuration of the metalLayer...
}
// handle display link callback at 60fps
static let _outputCallback: CVDisplayLinkOutputCallback = { (displayLink, inNow, inOutputTime, flagsIn, flagsOut, context) -> CVReturn in
// convert opaque context pointer back into a reference to our view
let view = Unmanaged<MyMetalView>.fromOpaque(context!).takeUnretainedValue()
/*** render something into view.metalLayer here! ***/
return kCVReturnSuccess
}
override func viewDidMoveToWindow() {
super.viewDidMoveToWindow()
guard CVDisplayLinkCreateWithActiveCGDisplays(&displayLink) == kCVReturnSuccess,
let displayLink = displayLink
else {
fatalError("unable to create display link")
}
// pass a reference to this view as an opaque pointer
guard CVDisplayLinkSetOutputCallback(displayLink, MyMetalView._outputCallback, Unmanaged<MyMetalView>.passUnretained(self).toOpaque()) == kCVReturnSuccess else {
fatalError("unable to configure output callback")
}
guard CVDisplayLinkStart(displayLink) == kCVReturnSuccess else {
fatalError("unable to start display link")
}
}
deinit {
if let displayLink = displayLink {
CVDisplayLinkStop(displayLink)
}
}
}

SwiftUI - Saving Image to Share Sheet causes image to save blurry/low res

I have a bit of code in my app that generates a QR Code and scales it up (code reference I used from this link from Hackng with Swift. Now, I'm using the share sheet to allow the user to save the qr code to their camera roll and, it is working, but saving the image low res, and it saves to the camera roll blurry (and i assume if it is shared via other methods it will also be blurry)
Here is the code of my share sheet function:
struct ActivityView: UIViewControllerRepresentable {
let activityItems: [Any]
let applicationActivities: [UIActivity]?
func makeUIViewController(context: UIViewControllerRepresentableContext<ActivityView>) -> UIActivityViewController {
return UIActivityViewController(activityItems: activityItems, applicationActivities: applicationActivities)
}
func updateUIViewController(_ uiViewController: UIActivityViewController, context: UIViewControllerRepresentableContext<ActivityView>) {
}
}
and here's the code in my view struct:
.sheet(isPresented: $showShareSheet) {
ShareSheet(activityItems: [self.qrCodeImage])
}
Is there a trick to remove the interpolation on the image when it saves to the share sheet like the .interpolation(.none) on the image view itself?
Your problem is that the QR code image is actually tiny! Like really tiny:
Printing description of image:
<UIImage:0x60000202cc60 anonymous {23, 23}>
When you share this image, the way it will be displayed is dependant on the program or app that will display it, and is out of control of your app as far as I know.
However,
there is a way that you could potentially make it "pretty" in other apps, and this would be to increase the resolution to a larger amount so that when it's rendered it'll appear to have "sharp" pixels.
How would this be accomplished? I think I have an example buried somewhere in old code, I'll dig into it and see if I can find you an example ;)
Edit
I found the code:
extension UIImage {
func resized(toWidth width: CGFloat) -> UIImage? {
let canvasSize = CGSize(width: round(width), height: CGFloat(ceil(width/size.width * size.height)))
UIGraphicsBeginImageContextWithOptions(canvasSize, false, scale)
defer { UIGraphicsEndImageContext() }
let context = UIGraphicsGetCurrentContext();
context?.interpolationQuality = .none
// Set the quality level to use when rescaling
draw(in: CGRect(origin: .zero, size: canvasSize))
let r = UIGraphicsGetImageFromCurrentImageContext()
return r
}
}
The trick is to provide a way to scale the image, but the real magic is on line 7:
context?.interpolationQuality = .none
If you exclude this line, you'll get blurry images, which is what the OS does by default because you don't generally want to see the pixel edges in images.
You could use this extension like so:
.sheet(isPresented: $showShareSheet) {
ShareSheet(activityItems: [self.qrCodeImage.resized(toWidth: 512) ?? UIImage()])
}
However, this may be resizing the image way more often than necessary. Optimally you'd resize it in the same function that you generate it.

How to to determinate in Swift the current width of the app when in Split View?

EDIT: I have a project with a row of buttons on top on it. Usually the buttons are 5 in Compact view and 6 in Regular view. I would like to remove a button when the app runs in 1/3 Split View. How can I determine the width of the app?
I'm using this code to determinate the current width of the app when in Split View (multitasking):
override func viewWillTransitionToSize(size: CGSize, withTransitionCoordinator coordinator: UIViewControllerTransitionCoordinator) {
// works but it's deprecated:
let currentWidth = UIScreen.mainScreen().applicationFrame.size.width
print(currentWidth)
}
It works, but unfortunately applicationFrame is deprecated in iOS 9, so I'm trying to replace it with this:
override func viewWillTransitionToSize(size: CGSize, withTransitionCoordinator coordinator: UIViewControllerTransitionCoordinator) {
// gives you the width of the screen not the width of the app:
let currentWidth = UIScreen.mainScreen().bounds.size.width
print(currentWidth)
}
The problem is that the first statement gives you the effective width of the app and it's fine, instead the second one, gives you the width of the screen, so you can't use it to learn the real width of the app when it is in Split View.
Would someone know what code would be necessary to replace this deprecated statement?
let currentWidth = UIScreen.mainScreen().applicationFrame.size.width // deprecated
#TheValyreanGroup's answer will work if there are no intervening view controllers mucking with sizes. If that possibility exists you should be able to use self.view.window.frame.size.width
You can just get the size of the parent view.
let currentSize = self.view.bounds.width
That will return the width accurately even in split view.
You can do something like this to determine whether to show or hide a button.
let totalButtonWidth: Int
for b in self.collectionView.UIViews{
let totalButtonWidth += b.frame.width + 20 //Where '20' is the gap between your buttons
}
if (currentSize < totalButtonWidth){
self.collectionView.subviews[self.collectionView.subviews.count].removeFromSuperview()
}else{
self.collectionView.addSubview(buttonViewToAdd)
}
Something like that, but i think you can get the idea.
Thanks to the replay of TheValyreanGroup and David Berry on this page I made a solution that can respond to the interface changes without using the deprecate statement UIScreen.mainScreen().applicationFrame.size.width I post it here with its context to made more clear what is the problem and the (surely improvable) solution. Please post any suggestion and comment you think could improve the code.
// trigged when app opens and when other events occur
override func traitCollectionDidChange(previousTraitCollection: UITraitCollection?) {
let a = self.view.bounds.width
adaptInterface(Double(a))
}
// not trigged when app opens or opens in Split View, trigged when other changes occours
override func viewWillTransitionToSize(size: CGSize, withTransitionCoordinator coordinator: UIViewControllerTransitionCoordinator) {
adaptInterface(Double(size.width))
}
func isHorizontalSizeClassCompact () -> Bool {
if (view.traitCollection.horizontalSizeClass == UIUserInterfaceSizeClass.Compact) {
return true // Comapact
} else {
return false // Regular
}
}
func adaptInterface(currentWidth: Double) {
if isHorizontalSizeClassCompact() { // Compact
// do what you need to do when sizeclass is Compact
if currentWidth <= 375 {
// do what you need when the width is the one of iPhone 6 in portrait or the one of the Split View in 1/3 of the screen
} else {
// do what you need when the width is bigger than the one of iPhone 6 in portrait or the one of the Split View in 1/3 of the screen
}
} else { // Regular
// do what you need to do when sizeclass is Regular
}
}

How to control one IB custom view with mouseEntered/-Exited in another IB custom view

I'm sorry for my incompetence, but I'm new to Cocoa, Swift, and object-oriented programming, in general. My primary sources have been Cocoa Programming for OS X (5th ed.), and Apple's jargon- and Objective-C-riddled Developer pages. But I'm here because I haven't seen (or didn't realize that I saw) anything that speaks to this problem.
I want to change the contents of one IB-created custom view, LeftView, by mouseEntered/-Exited actions in another IB-created custom view, RightView. Both are in the same window. I created a toy program to try to figure things out, but to no avail.
Here's the class definition for RightView (which is supposed to change LeftView):
import Cocoa
class RightView: NSView {
override func drawRect(dirtyRect: NSRect) {
// Nothing here, for now.
}
override func viewDidMoveToWindow() {
window?.acceptsMouseMovedEvents = true
let options: NSTrackingAreaOptions =
[.MouseEnteredAndExited, .ActiveAlways, .InVisibleRect]
let trackingArea = NSTrackingArea(rect: NSRect(),
options: options,
owner: self,
userInfo: nil)
addTrackingArea(trackingArea)
}
override func mouseEntered(theEvent: NSEvent) {
Swift.print("Mouse entered!")
LeftView().showStuff(true)
}
override func mouseExited(theEvent: NSEvent) {
Swift.print("Mouse exited!")
LeftView().showStuff(false)
}
}
And here's the class definition for LeftView (which is supposed to be changed by RightView):
import Cocoa
class LeftView: NSView {
var value: Bool = false {
didSet {
needsDisplay = true
Swift.print("didSet happened and needsDisplay was \(needsDisplay)")
}
}
override func mouseUp(theEvent: NSEvent) {
showStuff(true)
}
override func drawRect(dirtyRect: NSRect) {
let backgroundColor = NSColor.blackColor()
backgroundColor.set()
NSBezierPath.fillRect(bounds)
Swift.print("drawRect was called when needsDisplay was \(needsDisplay)")
switch value {
case true: NSColor.greenColor().set()
case false: NSColor.redColor().set()
}
NSBezierPath.fillRect(NSRect(x: 40, y: 40,
width: bounds.width - 80, height: bounds.height - 80))
}
func showStuff(showing: Bool) {
Swift.print("Trying to show? \(showing)")
value = showing
}
}
I'm sure I'm missing something "completely obvious," but I'm a little dense. If you could tell me how to fix the code/xib files, I would very much appreciate it. If you could explain things like when talking to a child, I would be even more appreciative. When I take over the world (I'm not incompetent at everything), I will remember your kindness.
I came up with a workaround that is much simpler than my prior strategy. Instead of having mouseEntered/-Exited actions in one custom view try to control what is displayed in another custom view, I simply put the mouseEntered/-Exited code into the view that I wanted to control, and then I altered the location of rect: in NSTrackingArea.
Before moving code over by for that approach, I had tried changing the owner: in NSTrackingArea to LeftView() and just moving over the mouseEntered/-Exited code. That generated lots of scary error messages (naive newbie talking here), so I gave up on it. It would be nice to know, though, how one can correctly assign an owner other than self.
In any case, any further thoughts or insights would be appreciated.