IMessage MSSticker view created from UIView incorrect sizing - swift

Hey I have been struggling with this for a couple of days now and can't seem to find any documentation out side of the standard grid views for MSStickerView sizes
I am working on an app that creates MSStickerViews dynamically - it does this via converting a UIView into an UIImage saving this to disk then passing the URL to MSSticker before creating the MSStickerView the frame of this is then set to the size of the original view.
The problem I have is that when I drag the MSStickerView into the messages window, the MSStickerView shrinks while being dragged - then when dropped in the messages window, changes to a larger size. I have no idea how to control the size when dragged or the final image size
Heres my code to create an image from a view
extension UIView {
func imageFromView() -> UIImage? {
UIGraphicsBeginImageContextWithOptions(self.bounds.size, self.isOpaque, 0.0)
defer { UIGraphicsEndImageContext() }
if let context = UIGraphicsGetCurrentContext() {
self.layer.render(in: context)
let image = UIGraphicsGetImageFromCurrentImageContext()
return image
}
return nil
}
}
And here's the code to save this to disk
extension UIImage {
func savedPath(name: String) -> URL{
let paths = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)
let filePath = "\(paths[0])/name.png"
let url = URL(fileURLWithPath: filePath)
// Save image.
if let data = self.pngData() {
do {
try data.write(to: url)
} catch let error as NSError {
}
}
return url
}
}
finally here is the code that converts the data path to a Sticker
if let stickerImage = backgroundBox.imageFromView() {
let url = stickerImage.savedPath(name: textBox.text ?? "StickerMCSticker")
if let msSticker = try? MSSticker(contentsOfFileURL: url, localizedDescription: "") {
var newFrame = self.backgroundBox.frame
newFrame.size.width = newFrame.size.width
newFrame.size.height = newFrame.size.height
let stickerView = MSStickerView(frame: newFrame, sticker: msSticker)
self.view.addSubview(stickerView)
print("** sticker frame \(stickerView.frame)")
self.sticker = stickerView
}
}
I wondered first off if there was something I need to do regarding retina sizes, but adding #2x in the file just breaks the image - so am stuck on this - the WWDC sessions seem to show stickers being created from file paths and not altering in size in the transition between drag and drop - any help would be appreciated!

I fixed this issue eventually by getting the frame from the view I was copying's frame then calling sizeToFit()-
init(sticker: MSSticker, size: CGSize) {
let stickerFrame = CGRect(x: 0, y: 0, width: size.width, height: size.height)
self.sticker = MSStickerView(frame: stickerFrame, sticker: sticker)
self.sticker.sizeToFit()
super.init(nibName: nil, bund
as the StickerView was not setting the correct size. Essentially the experience I was seeing was that the sticker size on my view was not accurate with the size of the MSSticker - so the moment the drag was initialized, the real sticker size was implemented (which was different to the frame size / autoLayout I was applying in my view)

Related

Full Page screenshot of WKWebView for macOS

This is a macOS app, I'm trying to take a full page screenshot of a webview, but i'm unable to get the screenshot of the full page.
Screenshot function.
func takescreenshot(
_ webView: WKWebView,
didFinish navigation: WKNavigation!) {
let configuration = WKSnapshotConfiguration()
configuration.afterScreenUpdates = true
webView.takeSnapshot(with: configuration) { (image, error) in
if let image = image {
//Save Image
}
}
}
from the answers I've seen here the solution seems to be setting the webview scrollview offset, but this is only available for ios. This is the error i get:
Value of type "WKWebView" has no member "scrollView"
I guess the problem is caused by an internal scroll view that optimizes drawing for scrolling - by redrawing only parts that are soon to be visible.
To overcome this, you can temporarily resize the web view to it's full content extent. I know it's not an ideal solution, but you may derive better one based on this idea.
webView.evaluateJavaScript("[document.body.scrollWidth, document.body.scrollHeight];") { result, error in
if let result = result as? NSArray {
let orig = webView.frame
webView.frame = NSRect(x: 0, y: 0, width: (result[0] as! NSNumber).intValue, height: (result[1] as! NSNumber).intValue)
let configuration = WKSnapshotConfiguration()
webView.takeSnapshot(with: configuration) { image, error in
if let image = image {
NSPasteboard.general.clearContents()
NSPasteboard.general.setData(image.tiffRepresentation, forType: .tiff)
}
}
webView.frame = orig
}
}

How do I update a CALayer with a CVPixelBuffer/IOSurface?

I have an IOSurface-backed CVPixelBuffer that is getting updated from an outside source at 30fps. I want to render a preview of the image data in an NSView -- what's the best way for me to do that?
I can directly set the .contents of a CALayer on the view, but that only updates the first time my view updates (or if, say, I resize the view). I've been poring over the docs but I can't figure out the correct invocation of needsDisplay on the layer or view to let the view infrastructure know to refresh itself, especially when updates are coming from outside the view.
Ideally I'd just bind the IOSurface to my layer and any changes I make to it would be propagated, but I'm not sure if that's possible.
class VideoPreviewController: NSViewController, VideoFeedConsumer {
let customLayer : CALayer = CALayer()
override func viewDidLoad() {
super.viewDidLoad()
// Do view setup here.
print("Loaded our video preview")
view.layer?.addSublayer(customLayer)
customLayer.frame = view.frame
// register our view with the browser service
VideoFeedBrowser.instance.registerConsumer(self)
}
override func viewWillDisappear() {
// deregister our view from the video feed
VideoFeedBrowser.instance.deregisterConsumer(self)
super.viewWillDisappear()
}
// This callback gets called at 30fps whenever the pixelbuffer is updated
#objc func updateFrame(pixelBuffer: CVPixelBuffer) {
guard let surface = CVPixelBufferGetIOSurface(pixelBuffer)?.takeUnretainedValue() else {
print("pixelbuffer isn't IOsurface backed! noooooo!")
return;
}
// Try and tell the view to redraw itself with new contents?
// These methods don't work
//self.view.setNeedsDisplay(self.view.visibleRect)
//self.customLayer.setNeedsDisplay()
self.customLayer.contents = surface
}
}
Here's my attempt of a scaling version that's NSView rather than NSViewController-based, that also doesn't update correctly (or scale correctly for that matter):
class VideoPreviewThumbnail: NSView, VideoFeedConsumer {
required init?(coder decoder: NSCoder) {
super.init(coder: decoder)
self.wantsLayer = true
// register our view with the browser service
VideoFeedBrowser.instance.registerConsumer(self)
}
override init(frame frameRect: NSRect) {
super.init(frame: frameRect)
self.wantsLayer = true
// register our view with the browser service
VideoFeedBrowser.instance.registerConsumer(self)
}
deinit{
VideoFeedBrowser.instance.deregisterConsumer(self)
}
override func updateLayer() {
// Do I need to put something here?
print("update layer")
}
#objc
func updateFrame(pixelBuffer: CVPixelBuffer) {
guard let surface = CVPixelBufferGetIOSurface(pixelBuffer)?.takeUnretainedValue() else {
print("pixelbuffer isn't IOsurface backed! noooooo!")
return;
}
self.layer?.contents = surface
self.layer?.transform = CATransform3DMakeScale(
self.frame.width / CGFloat(CVPixelBufferGetWidth(pixelBuffer)),
self.frame.height / CGFloat(CVPixelBufferGetHeight(pixelBuffer)),
CGFloat(1))
}
}
What am I missing?
Maybe I'm wrong, but I think you are you updating your NSView on a background thread. (I suppose that the callback to updateFrame is on a background thread)
If I'm right, when you want to update the NSView, convert your pixelBuffer to whatever you want (NSImage?), and then dispatch it on the main thread.
Pseudocode (I don't work often with CVPixelBuffer so I'm not sure this is the right way to convert to an NSImage)
let ciImage = CIImage(cvImageBuffer: pixelBuffer)
let context = CIContext(options: nil)
let width = CVPixelBufferGetWidth(pixelBuffer)
let height = CVPixelBufferGetHeight(pixelBuffer)
let cgImage = context.createCGImage(ciImage, from: CGRect(x: 0, y: 0, width: width, height: height))
let nsImage = NSImage(cgImage: cgImage, size: CGSize(width: width, height: height))
DispatchQueue.main.async {
// assign the NSImage to your NSView here
}
Another catch: I did some tests, and it seems that you cannot assign an IOSurface directly to the contents of a CALayer.
I tried with this:
let textureImageWidth = 1024
let textureImageHeight = 1024
let macPixelFormatString = "ARGB"
var macPixelFormat: UInt32 = 0
for c in macPixelFormatString.utf8.reversed() {
macPixelFormat *= 256
macPixelFormat += UInt32(c)
}
let ioSurface = IOSurfaceCreate([kIOSurfaceWidth: textureImageWidth,
kIOSurfaceHeight: textureImageHeight,
kIOSurfaceBytesPerElement: 4,
kIOSurfaceBytesPerRow: textureImageWidth * 4,
kIOSurfaceAllocSize: textureImageWidth * textureImageHeight * 4,
kIOSurfacePixelFormat: macPixelFormat] as CFDictionary)!
IOSurfaceLock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
let test = CIImage(ioSurface: ioSurface)
IOSurfaceUnlock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
v1?.layer?.contents = ioSurface
Where v1 is my view. No effect
Even with a CIImage no effect (just last few lines)
IOSurfaceLock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
let test = CIImage(ioSurface: ioSurface)
IOSurfaceUnlock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
v1?.layer?.contents = test
If I create a CGImage it works
IOSurfaceLock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
let test = CIImage(ioSurface: ioSurface)
IOSurfaceUnlock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
let context = CIContext.init()
let img = context.createCGImage(test, from: test.extent)
v1?.layer?.contents = img
I encountered this problem myself and the solution is to double buffer the IOSurface source: use two IOSurface objects instead of one and render to the current surface, set the surface to the layer contents and then on the next rendering pass use the alternate (back/front) surface and then swap.
It would appear that setting the CALayer.contents twice to the same CVPixelBufferRef has no effect. However, if you alternate between two IOSurfaceRef it works wonderfully.
It maybe also possible to invalidate the layer contents by setting it to nil and then reset. I did not try that case but am using the double buffer technique.
If you have some IBActions that update it then create an observed variable with the didSet block and whenever the IBAction is triggered, change its value. Also remember to write the code you want to run when updated in that block.
I'd suggest making the variable an Int, set its default value to 0 and add 1 to it every time it updates.
And you can cast the NSView into an NSImageView for the part where you ask about showing the IMAGE data on an NSView so that does the job.
You need to convert the pixel buffer to CGImage and convert it to a layer so that you can change the layer of the main view.
Please try this code
#objc
func updateFrame(pixelBuffer: CVPixelBuffer) {
guard let surface = CVPixelBufferGetIOSurface(pixelBuffer)?.takeUnretainedValue() else {
print("pixelbuffer isn't IOsurface backed! noooooo!")
return;
}
void *baseAddr = CVPixelBufferGetBaseAddress(pixelBuffer);
size_t width = CVPixelBufferGetWidth(pixelBuffer);
size_t height = CVPixelBufferGetHeight(pixelBuffer);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef cgContext = CGBitmapContextCreate(baseAddr, width, height, 8, CVPixelBufferGetBytesPerRow(pixelBuffer), colorSpace, kCGImageAlphaNoneSkipLast);
CGImageRef cgImage = CGBitmapContextCreateImage(cgContext);
CGContextRelease(cgContext);
let outputImage = UIImage(cgImage: outputCGImage, scale: 1, orientation: img.imageOrientation)
let newLayer:CGLayer = CGLayer.init(cgImage: outputImage)
self.layer = newLayer
CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);
CVPixelBufferRelease(pixelBuffer);
}

How to name UIImage drawn from UIBezierPath() programmatically in Xcode-Swift?

How do we name an image programmatically. For example, assign a name to the image generated below. A name that we can use to distinguish the image from other images drawn programmatically.
func drawOval (width: CGFloat, height: CGFloat, name: String) -> UIImage {
let renderer = UIGraphicsImageRenderer(size: CGSize(width: width, height: height))
let image = renderer.image { ctx in
let path = UIBezierPath(ovalIn: CGRect(x: 0, y: 0, width: width, height: height))
path.stroke()
}
// TO DO: Assign this image a name, for example "image01"
return image
}
You can use tags on each UIImageView. I’m not aware of a way to add an identifier to a UIImage directly since it is a subclass of NSObject and not UIView. In order to add a tag to an object in Swift, the object must be a view of some kind.
To implement this, you would keep a variable outside of that function that keeps track of the current tag, then increment it in your function. For example:
var currentTag = 0
//Function now returns a UIImageView
func drawOval (width: CGFloat, height: CGFloat, name: String) -> UIImageView {
let renderer = UIGraphicsImageRenderer(size: CGSize(width: width, height: height))
let image = renderer.image { ctx in
let path = UIBezierPath(ovalIn: CGRect(x: 0, y: 0, width: width, height: height))
path.stroke()
}
let imageView = UIImageView(image: image)
imageView.tag = currentTag
currentTag += 1
return imageView
}
Then later in your code:
if (imageView.tag == 0) {
//Do something
}
//You can also use
let taggedImageView = viewWithTag(0)
EDIT: If you want to save the images and load them via one of the available UIImage initializers, you can write them to a cache folder on disk, then retrieve them using UIImage(pathToFile:):
//This will store the images in the caches directory for your app, which
//the system can clear when the device is low on storage. It will not be
//cleared while your app is open, though.
func saveImageToCacheDynamically(image: UIImage, name: String) {
let paths = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)
let localPath = paths[0].appendingPathComponent(“ImageCache”, isDirectory: true)
do {
if FileManager.default.fileExists(atPath: localPath.absoluteString) {
//Write the png data representation of the image to disk in plaintext format
try image.pngData().write(to: localPath.appendingPathComponent("\(name).txt"))
} else {
try FileManager.default.createDirectory(at: localPath, withIntermediateDirectories: true, attributes: nil)
//Write the png data representation of the image to disk in plaintext format
try image.pngData().write(to: localPath.appendingPathComponent("\(name).txt"))
}
} catch {
print("Error locally saving image: \(error.localizedDescription)")
}
//Later in your code...
let paths = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)
let localPath = paths[0].appendingPathComponent("ImageCache/\(imageIdentifier).txt")
if FileManager.default.fileExists(atPath: localPath.absoluteString) {
var fileData: Data!
do {
try fileData = Data(contentsOf: localPath)
} catch {
print("Error reading image file: \(error.localizedDescription)")
}
let image = UIImage(data: fileData)
//Do something with image
} else {
print("Error: image does not exist")
}
I believe what you are looking for is something like NSCache...
You can define a cache by something similar to this:
let imageCache = NSCache<String, UIImage>()
Then you can add objects to the cache like this, where someKeyString is the 'name' you are referring to:
imageCache.setObject(someImage, forKey: someKeyString)
And then finally you can retrieve images from the cache like
imageCache.object(forKey: someKeyString)
I would recommend using extensions or something similar to maintain a reference to your cache everywhere in your app.
** NOTE:
NSCaches are cleared when memory space is short, your app closes, etc. See here
For more permanent storage, I would recommend using UserDefaults, which Apple describes as "An interface to the user’s defaults database, where you store key-value pairs persistently across launches of your app." Use this for things like profile images or things that won't change very often. I would also recommend looking into Core Data

Taking a snapshot of UIImageView instead of main View

I have in my app in my View a Button for taking snapshot with below code
open func takeScreenshot(_ shouldSave: Bool = true) -> UIImage? {
var screenshotImage :UIImage?
let layer = UIApplication.shared.keyWindow!.layer
let scale = UIScreen.main.scale
UIGraphicsBeginImageContextWithOptions(layer.frame.size, false, scale);
guard let context = UIGraphicsGetCurrentContext() else {return nil}
layer.render(in:context)
screenshotImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
if let image = screenshotImage, shouldSave {
UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil)
}
return screenshotImage
}
But in the same View is UIImageView and I want to take a snapshot of this UIImageView. How should I change the code? When I take a snapshot with this code, it will save only black screen with button for play/pause video and with button for taking snapshot.
Instead you using
let layer = UIApplication.shared.keyWindow!.layer
Just use your imageview Layer
let layer = YourImageView.layer

iOS 11 doesn't grab screen from MKMapView

I have an app that displays the locations the user has walked on an MKMapView. When the user leaves the map view the app grabs the screen and saves the image on disk. Up til iOSS 10.3 this method was always successful. With iOS 11.0 the screen grab is a blank image. I get no notification from xcode that there were some changes and that I need to adjust the code.
Interestingly, screen grabs from text pages are still grabbed and saved successfully.
Did anyone encounter the same problem and got the solution?
The code that has always been successful up til now, is:
override func viewWillDisappear(_ animated: Bool) {
//Set the full file name under which the track will be saved.
let fileBaseName = self.imageName.appending(String(describing: (self.display?.trackDate)!))
let fileFullName = fileBaseName.appending(".png")
//Check if the image already has been saved
if !AuxiliaryObjects.shared.getImageFileName(with: fileFullName ) {
//Create the sizes of the capture
let screenRect = self.trackMapView.frame
let screenSize = screenRect.size
let screenScale = UIScreen.main.scale
var grabRect = self.trackMapView.convertRegion(self.mapRegion, toRectTo: self.view)
var heightAdjustment : CGFloat = 0.0
//Grab the image from the screen
UIGraphicsBeginImageContextWithOptions(screenSize, false, screenScale)
self.trackMapView.drawHierarchy(in: screenRect, afterScreenUpdates: true)
let myImage = UIGraphicsGetImageFromCurrentImageContext()
grabRect.origin.x *= (myImage?.scale)!
grabRect.origin.y *= (myImage?.scale)!
grabRect.size.width *= (myImage?.scale)!
grabRect.size.height *= (myImage?.scale)!
let grabImage = (myImage?.cgImage)!.cropping(to: grabRect)
let mapImage = UIImage(cgImage: grabImage!)
UIGraphicsEndImageContext()
AuxiliaryObjects.shared.save(image: mapImage, with: fileFullName, and: self.imageName)
self.display?.displayImage = AuxiliaryObjects.shared.getImage(with: fileFullName, with: self.tableImageRect)!
} else {
self.display?.displayImage = AuxiliaryObjects.shared.getImage(with: fileFullName, with: self.tableImageRect)!
}
}
I submitted a code level support request at Apple to get the answer to the question. Apple does not support the use of drawHierarhy in grabbing a MapKit screen. The way to go is using the MKMapSnapshotter utility to create an MKMapSnapshot and then draw in the lines and annotations by converting all the map coordinates to view coordinates.
Since this gave me some problems with getting the a mirrored image and translating the coordinates properly, I decided to use the layer method render(in: CGContext) this provided me a well functioning very efficient screen grab.
func creatSnapshot(with fileName: String) {
UIGraphicsBeginImageContextWithOptions(self.trackMapView.frame.size, false, UIScreen.main.scale)
let currentContext = UIGraphicsGetCurrentContext()
self.trackMapView.layer.render(in: currentContext!)
let contextImage = (UIGraphicsGetImageFromCurrentImageContext())!
UIGraphicsEndImageContext()
let region = self.trackMapView.region
var cropRect = self.trackMapView.convertRegion(region, toRectTo: self.trackMapView.superview)
cropRect.origin.x *= contextImage.scale
cropRect.origin.y *= contextImage.scale
cropRect.size.height *= contextImage.scale
cropRect.size.width *= contextImage.scale
let cgMapImage = contextImage.cgImage?.cropping(to: cropRect)
let mapImage = UIImage(cgImage: cgMapImage!)
AuxiliaryObjects.shared.save(image: mapImage, with: fileName, and: self.imageName)
self.displayTrack?.displayImage = AuxiliaryObjects.shared.getImage(with: fileName, with: self.tableImageRect)!
NotificationCenter.default.post(name: self.imageSavedNotification, object: self)
}