Drawing a simple line on a JPEG image - swift

I'm stuck again with an apparently simple question.
I loaded a JPEG file into a CGImage. I got the correct values for width and height (in pixels) and was able to show "myImage" in a ImageView Controller. But I wanted to add some graphics on this image and found that I should instead get it into a NSImage. So I did but got different (proportional) values for width and height: 595.08 instead for 1653, and 841.68 instead of 2338, respectively.
I tried to create a NSCGContext from a CGContext 'gc' for drawing (a simple line and a rectangle) which resulted in a "Value of optional type 'CGContext?' not unwrapped, did you mean to use '!' or '?'?"... I'm lost...
// with NSData
//
let imageAsData = try Data(contentsOf: chosenFiles[0])
let imageProvider = CGDataProvider(data: imageAsData as CFData)
var myImage = CGImage(jpegDataProviderSource: imageProvider!, decode: nil, shouldInterpolate: true, intent: .defaultIntent)
let imageWidth = myImage!.width
let imageHeight = myImage!.height
// with NSImage, now
//
let imageAsNSImage=NSImage(contentsOf: chosenFiles[0])
let imageSize=imageAsNSImage?.size // ---> 0.36 * pixels
// creating a CG context and drawing
//
let colorSpace:CGColorSpace = CGColorSpaceCreateDeviceRGB()
let gc = CGContext(data: nil, width: imageWidth, height: imageHeight, bitsPerComponent: 8, bytesPerRow: 0,space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)
let NSGContext = NSGraphicsContext(cgContext: gc, flipped: true)
let currentContext = NSGraphicsContext.current() // Cocoa GC object appropriate for the current drawing environment
NSGraphicsContext.saveGraphicsState()
NSGraphicsContext.current = NSGContext
NSGContext?.beginPath()
NSGContext?.setStrokeColor(redColor)
NSGContext?.setLineWidth(50.0)
NSGContext?.move(to: targetStart)
NSGContext?.addLine(to: targetEnd)
NSGContext?.setStrokeColor(grayColor)
NSGContext?.setFillColor(grayColor)
NSGContext?.addRect(ROIRect)
NSGContext?.closePath()
NSGContext.restoreGraphicsState()
imageAsNSImage?.draw(at: NSZeroPoint, from: NSZeroRect, operation: NSCompositeSourceOver, fraction: 1.0)
imageAsNSImage?.unlockFocus()
NSGraphicsContext.setcurrent(currentContext)
myImageView.image = imageAsNSImage // image & drawings should show in View

Drawing a simple line on a JPEG image
// load JPEG from main bundle
guard let path = Bundle.main.pathForImageResource(NSImage.Name("picture.jpg")),
let image = NSImage(contentsOfFile: path)
else { fatalError() }
let size = image.size
image.lockFocus() // prepare image for drawing
NSColor.red.setStroke()
NSBezierPath.strokeLine(from: .zero, to: NSPoint(x: size.width, y: size.height))
image.unlockFocus() // drawing commands done
The code above strokes a red line from lower left corner to top right.
If you have an NSImageView at hand you can use the image directly:
#IBOutlet weak var imageView: NSImageView!
...
imageView.image = image

Thanks to djromero, here is the solution I just reached:
// Load the JPEG image from disk into a CGImage
//
let imageAsData = try Data(contentsOf: chosenFiles[0])
let imageProvider = CGDataProvider(data: imageAsData as CFData)
var myImage = CGImage(jpegDataProviderSource: imageProvider!, decode: nil, shouldInterpolate: true, intent: .defaultIntent)
// Create a NSImage from the CGImage (with the same width and height in pixels)
//
let imageAsNSImage=NSImage(cgImage: myImage!, size: NSZeroSize)
// Drawing a simple line
//
imageAsNSImage.lockFocusFlipped(true) // Otherwise, the origin is at the lower left corner
NSColor.red.setStroke()
NSBezierPath.strokeLine(from: targetStart, to: targetEnd)
imageAsNSImage.unlockFocus()
// Show the NSImage in the NSImageView
//
myImageView.image = imageAsNSImage

Related

Adding gradient behind image returns empty image

I have a round avatar image with a transparent background. I want to create a new round image of the same size out of the initial image, with a gradient background behind it. So it looks like standing in sky instead of having a transparent background.
Since I will use this image as tabbaritem’s image, I couldn’t use uiview and edit it’s background layer.
And to make it reusable I wanted to create a UIImage extension.
Below is what I do:
extension UIImage {
func gradientImage() -> UIImage? {
let width = self.size.width
let height = self.size.height
UIGraphicsBeginImageContextWithOptions(size, false, 0)
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue)
let colorSpace = CGColorSpaceCreateDeviceRGB()
guard let bitmapContext = CGContext(data: nil,
width: Int(width),
height: Int(height),
bitsPerComponent: 8,
bytesPerRow: 0,
space: colorSpace,
bitmapInfo: bitmapInfo.rawValue) else { return nil }
let locations: [CGFloat] = [0.0, 1.0]
let top = R.color.duckDimDarkGrey()?.cgColor
let bottom = R.color.duckPencilDark()?.cgColor
let colors = [top, bottom] as CFArray
guard let gradient = CGGradient(colorsSpace: colorSpace, colors: colors, locations: locations) else {
return nil
}
bitmapContext.drawLinearGradient(gradient, start: CGPoint.zero, end: CGPoint(x: 0, y: size.height), options: CGGradientDrawingOptions())
guard let cgImage = UIGraphicsGetImageFromCurrentImageContext()?.cgImage else { return nil }
UIGraphicsEndImageContext()
let img = UIImage(cgImage: cgImage)
return img
}
}
Here is how I use it:
Let image1 = UIImage(named: “test.png”)
self.tabBar.items[3].image = image1.gradientImage()
However I am getting an empty image somehow.

Convert a CGImage to MTLTexture without premultiplication

I have a UIImage which I've previously created from a png file:
let strokeUIImage = UIImage(data: pngData)
I want to convert strokeImage (which has opacity) to an MTLTexture for display in an MTKView, but doing the conversion seems to perform an unwanted premultiplication, which darkens all the semitransparent edges.
My blending settings are as follows:
pipelineDescriptor.colorAttachments[0].isBlendingEnabled = true
pipelineDescriptor.colorAttachments[0].rgbBlendOperation = .add
pipelineDescriptor.colorAttachments[0].alphaBlendOperation = .add
pipelineDescriptor.colorAttachments[0].sourceRGBBlendFactor = .one
pipelineDescriptor.colorAttachments[0].sourceAlphaBlendFactor = .one
pipelineDescriptor.colorAttachments[0].destinationRGBBlendFactor = .oneMinusSourceAlpha
pipelineDescriptor.colorAttachments[0].destinationAlphaBlendFactor = .oneMinusSourceAlpha
I've tried two methods of conversion:
let stampTexture = try! MTKTextureLoader(device: self.device!).newTexture(cgImage: strokeUIImage.cgImage!, options: nil)
and the more elaborate dataProvider-driven method:
let image = strokeUIImage.cgImage!
let imageWidth = image.width
let imageHeight = image.height
let bytesPerPixel:Int! = 4
let rowBytes = imageWidth * bytesPerPixel
let texDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: .rgba8Unorm_srgb,
width: imageWidth,
height: imageHeight,
mipmapped: false)
guard let stampTexture = device!.makeTexture(descriptor: texDescriptor) else { return }
let srcData: CFData! = image.dataProvider?.data
let pixelData = CFDataGetBytePtr(srcData)
let region = MTLRegionMake2D(0, 0, imageWidth, imageHeight)
stampTexture.replace(region: region, mipmapLevel: 0, withBytes: pixelData!, bytesPerRow: Int(rowBytes))
both of which yield the same unwanted premultiplied result.
The latter I tried, as there were some posts suggesting that the old swift3 method CGDataProviderCopyData() extracts raw pixel data from the image which is not premultiplied. Sadly, the equivalent:
let srcData: CFData! = image.dataProvider?.data
does not seem to do the trick. Am I missing something?
Any pointers would be appreciated.
After much experimenting, I've come to a solution which addresses the pre-multiplication issue inherent in CoreGraphics images. Thanks to Warren's tip regarding using an Accelerate function (vImageUnpremultiplyData_ARGB8888 in particular), I thought, why not build a CGImage using vImage_CGImageFormat which will allow me to play with the bitmapInfo setting that specifies how to interpret alpha...The result is not perfect, as demonstrated by the image attachment below:
Somehow, in the translation the alpha values are getting punched up slightly, (possibly the rgb as well, but not significantly). By the way, I should point out that the png pixel format is sRGB, and the MTKView I'm using is set to MTLPixelFormat.rgba16Float (app requirement)
Below is the full metalDrawStrokeUIImage routine I implemented. Of particular note is the line:
bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue)
which essentially unassociates the alpha (I think) without calling vImageUnpremultiplyData_ARGB8888. Looking at the resulting image certainly looks like an un-premultiplied image...
Lastly, to get back a premultiplied texture on the MTKView side, I let the fragment shader handle the pre-multiplication:
fragment float4 premult_fragment(VertexOut interpolated [[stage_in]],
texture2d<float> texture [[texture(0)]],
sampler sampler2D [[sampler(0)]]) {
float4 sampled = texture.sample(sampler2D, interpolated.texCoord);
// this fragment shader premultiplies incoming rgb with texture's alpha
return float4(sampled.r * sampled.a,
sampled.g * sampled.a,
sampled.b * sampled.a,
sampled.a );
} // end of premult_fragment
The result is pretty close to the input source, but the image is maybe 5% more opaque than the incoming png. Again, png pixel format is sRGB, and the MTKView I'm using to display is set to MTLPixelFormat.rgba16Float . So, I'm sure something is getting mushed somewhere. If anyone has any pointers, I'd sure appreciate it.
Below is the rest of the relevant code:
func metalDrawStrokeUIImage (strokeUIImage: UIImage, strokeBbox: CGRect) {
self.metalSetupRenderPipeline(compStyle: compMode.strokeCopy) // needed so stampTexture is not modified by fragmentFunction
let bytesPerPixel = 4
let bitsPerComponent = 8
let width = Int(strokeUIImage.size.width)
let height = Int(strokeUIImage.size.height)
let rowBytes = width * bytesPerPixel
//
let texDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: .rgba8Unorm_srgb,
width: width,
height: height,
mipmapped: false)
guard let stampTexture = device!.makeTexture(descriptor: texDescriptor) else { return }
//let cgImage: CGImage = strokeUIImage.cgImage!
//let sourceColorSpace = cgImage.colorSpace else {
guard
let cgImage = strokeUIImage.cgImage,
let sourceColorSpace = cgImage.colorSpace else {
print("Unable to initialize cgImage or colorSpace.")
return
}
var format = vImage_CGImageFormat(
bitsPerComponent: UInt32(cgImage.bitsPerComponent),
bitsPerPixel: UInt32(cgImage.bitsPerPixel),
colorSpace: Unmanaged.passRetained(sourceColorSpace),
bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue),
version: 0, decode: nil,
renderingIntent: CGColorRenderingIntent.defaultIntent)
var sourceBuffer = vImage_Buffer()
defer {
free(sourceBuffer.data)
}
var error = vImageBuffer_InitWithCGImage(&sourceBuffer, &format, nil, cgImage, numericCast(kvImageNoFlags))
guard error == kvImageNoError else {
print ("[MetalBrushStrokeView]: can't vImageBuffer_InitWithCGImage")
return
}
//vImagePremultiplyData_RGBA8888(&sourceBuffer, &sourceBuffer, numericCast(kvImageNoFlags))
// create a CGImage from vImage_Buffer
var destCGImage = vImageCreateCGImageFromBuffer(&sourceBuffer, &format, nil, nil, numericCast(kvImageNoFlags), &error)?.takeRetainedValue()
guard error == kvImageNoError else {
print ("[MetalBrushStrokeView]: can't vImageCreateCGImageFromBuffer")
return
}
let dstData: CFData = (destCGImage!.dataProvider!.data)!
let pixelData = CFDataGetBytePtr(dstData)
destCGImage = nil
let region = MTLRegionMake2D(0, 0, Int(width), Int(height))
stampTexture.replace(region: region, mipmapLevel: 0, withBytes: pixelData!, bytesPerRow: Int(rowBytes))
let stampColor = UIColor.white
let stampCorners = self.stampSetVerticesFromBbox(bbox: strokeBbox)
self.stampAppendToVertexBuffer(stampLayer: stampLayerMode.stampLayerFG, stampCorners: stampCorners, stampColor: stampColor)
self.metalRenderStampSingle(stampTexture: stampTexture)
self.initializeStampArray() // clears out the stamp array so we always draw 1 stamp at a time
} // end of func metalDrawStrokeUIImage (strokeUIImage: UIImage, strokeBbox: CGRect)

Merging two Images & text on a final image

What I'm trying to do in Swift is merge two images together, the images I'm merging together is me dragging one image on top of another image with touch gestures. I get the images to composite, BUT the image I dragged on top of the main image does not keep it's position. How do I get it to stay where I placed it for the composite?
Here's my code:
extension CIHHatSelectionViewController{
// Return composite image of image2 overlayed on image1
//
func compositeImage(_ image1: UIImage, image2: UIImage, drawText: String, imageView1: CGPoint, imageView2:CGRect) -> UIImage {
let catPos = (image1.size.height - imageView2.origin.y) - imageView2.height
let bounds1 = CGRect(x: 0, y: 0, width: image1.size.width, height: image1.size.height)
let bounds2 = CGRect(x:imageView2.origin.x, y:catPos, width:imageView2.size.width, height:imageView2.size.height)
_ = CGColorSpaceCreateDeviceRGB()
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)
let ctx = CGContext(data: nil,
width: image1.cgImage!.width,
height: image1.cgImage!.height,
bitsPerComponent: image1.cgImage!.bitsPerComponent,
bytesPerRow: image1.cgImage!.bytesPerRow,
space: image1.cgImage!.colorSpace!,
bitmapInfo: bitmapInfo.rawValue)!
ctx.draw(image1.cgImage!, in: bounds1, byTiling: false)
ctx.setBlendMode(.normal) // one image over the other
ctx.draw(image2.cgImage!, in: bounds2, byTiling: false)
let finalImage = textToImage(drawText, inImage: UIImage(cgImage: ctx.makeImage()!), atPoint: imageView1)
return finalImage
}

change resolution and size of image with cocoa/osx/swift (no mobile apps)

I try to change the size and the resolution of an image programmatically, afterwards I save this image.
The imagesize in the imageView is changing, but when I look at my file "file3.png" it always has the original resolution of 640x1142.
I googled around but can't find a solution. I try to redraw the image. But maybe it's the wrong strategy.
thanks
#IBAction func pickOneImageBtn(sender: AnyObject) {
//load image from path
pickedImage.image = loadImageFromPath(fileInDocumentsDirectory("Angebote.png"))
let newSize = NSSize(width: 10, height: 10)
if let image = pickedImage.image {
print("found image")
//cast to CGImage
var imageRect:CGRect = CGRectMake(0, 0, image.size.width, image.size.height)
let imageRef = image.CGImageForProposedRect(&imageRect, context: nil, hints: nil)
if let imageRefExists = imageRef {
print("Cast to CGImage worked \(imageRefExists)")
}
//redraw to NSImage with new size
let imageWithNewSize = NSImage(CGImage: imageRef!, size: newSize)
//save on disk
let imgData: NSData! = imageWithNewSize.TIFFRepresentation!
let bitmap: NSBitmapImageRep! = NSBitmapImageRep(data: imgData!)
if let pngCoverImage = bitmap!.representationUsingType(NSBitmapImageFileType.NSPNGFileType, properties: [:]) {
pngCoverImage.writeToFile("/...correctpath.../imageSourceForResize/file3.png", atomically: false)
print("saved new image")
}
//the size is smaller
pickedImage.image = imageWithNewSize
}
}
Change
let imgData: NSData! = pickedImage.image!.TIFFRepresentation!
to
let imgData: NSData! = imageWithNewSize.TIFFRepresentation!
I tried to change the size of a NSImage for Mac application and here is the working function to resize an image written in swift.
func resize(image: NSImage, w: Int, h: Int) -> NSImage
{
let destSize = NSMakeSize(CGFloat(w), CGFloat(h))
let newImage = NSImage(size: destSize)
newImage.lockFocus()
image.drawInRect(NSMakeRect(0, 0, destSize.width, destSize.height), fromRect: NSZeroRect, operation: NSCompositingOperation.CompositeCopy, fraction: 1.0)
newImage.unlockFocus()
newImage.size = destSize
return NSImage(data: newImage.TIFFRepresentation!)!
}
You need to pass 3 parameters to call this function i.e NSImage, width, height and this function will return resized image.
targetimage = resize(source, w: Int(targetwidth), h: Int(targetheight))

Asynchronous function causing crashing

I'm generating a QR Code to put into a UIImage. I'm running the generation function asynchronously but for some reason the app crashes when I run it on my phone, but doesn't crash in the simulator. I'm not really sure what's going on... Any ideas?
Setup Image
let QR = UIImageView()
dispatch_async(dispatch_get_global_queue(Int(QOS_CLASS_USER_INITIATED.value), 0)) { // 1
var img = self.generateQRImage(self.arr[sender.tag],withSizeRate: self.screenWidth-40)
dispatch_async(dispatch_get_main_queue()) { // 2
QR.image = img
}
}
QR.frame = CGRectMake(0,0,screenWidth-40,screenWidth-40)
QR.center = CGPoint(x:screenWidth/2,y:screenHeight/2)
sView.addSubview(QR)
Generate QR
func generateQRImage(stringQR:NSString, withSizeRate rate:CGFloat) -> UIImage
{
var filter:CIFilter = CIFilter(name:"CIQRCodeGenerator")
filter.setDefaults()
var data:NSData = stringQR.dataUsingEncoding(NSUTF8StringEncoding)!
filter.setValue(data, forKey: "inputMessage")
var outputImg:CIImage = filter.outputImage
var context:CIContext = CIContext(options: nil)
var cgimg:CGImageRef = context.createCGImage(outputImg, fromRect: outputImg.extent())
var img:UIImage = UIImage(CGImage: cgimg, scale: 1.0, orientation: UIImageOrientation.Up)!
var width = img.size.width * rate
var height = img.size.height * rate
UIGraphicsBeginImageContext(CGSizeMake(width, height))
var cgContxt:CGContextRef = UIGraphicsGetCurrentContext()
CGContextSetInterpolationQuality(cgContxt, kCGInterpolationNone)
img.drawInRect(CGRectMake(0, 0, width, height))
img = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return img
}
The intent of withSizeRate is clearly to be a scaling factor to apply to the QR image (which is 27x27). But you are using the screen width as the multiplier. That results in an exceedingly large image (once it's uncompressed, used in image view; don't go by the size of the resulting JPEG/PNG file). The theoretical internal, uncompressed representation of this image is extremely large (300 mb on iPhone 6 and nearly 400 mb on iPhone 6+). When I ran it through the iPhone 6 simulator, memory usage actually spiked to 2.4 gb:
I would suggest using a smaller scaling factor. Or just create an image that is precisely the size of the imageview (though use zero for the scale with UIGraphicsBeginImageContextWithOptions).
For example, you could simply pass the CGSize of the image view to generateQRImage, and adjust the method like so:
func generateQRImage(stringQR: String, size: CGSize) -> UIImage {
let filter = CIFilter(name:"CIQRCodeGenerator")
filter.setDefaults()
let data = stringQR.dataUsingEncoding(NSUTF8StringEncoding)!
filter.setValue(data, forKey: "inputMessage")
let outputImage = filter.outputImage
let context = CIContext(options: nil)
let cgImage = context.createCGImage(outputImage, fromRect: outputImage.extent())
var image = UIImage(CGImage: cgImage, scale: 1.0, orientation: UIImageOrientation.Up)!
let width = size.width
let height = size.height
UIGraphicsBeginImageContextWithOptions(CGSizeMake(width, height), true, 0)
let cgContext = UIGraphicsGetCurrentContext()
CGContextSetInterpolationQuality(cgContext, kCGInterpolationNone)
image.drawInRect(CGRectMake(0, 0, width, height))
image = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return image
}