I'm receiving raw pixels of an image and successfully transforming them into a CGImage with the following function:
func imageFromTexturePixels(raw: UnsafePointer<UInt8>, w: Int, h: Int) -> CGImage? {
// 4 bytes(rgba channels) for each pixel
let bytesPerPixel: Int = 4
// (8 bits per each channel)
let bitsPerComponent: Int = 8
let bitsPerPixel = bytesPerPixel * bitsPerComponent
// channels in each row (width)
let bytesPerRow: Int = w * bytesPerPixel
let cfData = CFDataCreate(nil, raw, w * h * bytesPerPixel)
let cgDataProvider = CGDataProvider(data: cfData!)!
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue)
let image: CGImage! = CGImage(width: w,
height: h,
bitsPerComponent: bitsPerComponent,
bitsPerPixel: bitsPerPixel,
bytesPerRow: bytesPerRow,
space: deviceColorSpace,
bitmapInfo: bitmapInfo,
provider: cgDataProvider,
decode: nil,
shouldInterpolate: false,
intent: CGColorRenderingIntent.defaultIntent)
return image
}
What I really want in the end is a PixelBuffer, so I'm transforming that image into a pixelbuffer using this extension.
Although all this works, it looks a bit inefficient and I'd like to know how to transform the raw pixels directly into a CVPixelBuffer, without converting them into a CGImage before.
Related
I'm trying to create a CGImage from a byte array, with something similar to this example function that generates a red square on a black field:
var bgrArray: [UInt8] = Array(repeating: 0, count: 480*480*4)
for i in 50..<250 {
for j in 50..<250 {
bgrArray[(i*480+j)*4] = 255
bgrArray[(i*480+j)*4+1] = 0
bgrArray[(i*480+j)*4+2] = 0
bgrArray[(i*480+j)*4+3] = 0
}
}
let releaseMaskImagePixelData: CGDataProviderReleaseDataCallback = { (info: UnsafeMutableRawPointer?, data: UnsafeRawPointer, size: Int) -> () in
return
}
let provider = CGDataProvider(dataInfo: nil, data: bgrArray, size: bgrArray.count, releaseData: releaseMaskImagePixelData)!
let colorspace = CGColorSpaceCreateDeviceRGB()
let bitsPerComponent = 8;
let bitsPerPixel = 32;
let bytesPerRow = 4 * 480;
let bitmapInfo: CGBitmapInfo = [.byteOrder32Big, CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipLast.rawValue)]
let img = CGImage(width: 480, height: 480, bitsPerComponent: bitsPerComponent, bitsPerPixel: bitsPerPixel, bytesPerRow: bytesPerRow, space: colorspace, bitmapInfo: bitmapInfo, provider: provider, decode: nil, shouldInterpolate: false, intent: .defaultIntent)
let uiimage = UIImage(cgImage: img!)
return CIImage(image: uiimage)!
I have this exact bit of code copied into two different projects, in one project it always succeeds and in the other the line let img = CGImage(width: 480, height: 480, bitsPerComponent: bitsPerComponent, bitsPerPixel: bitsPerPixel, bytesPerRow: bytesPerRow, space: colorspace, bitmapInfo: bitmapInfo, provider: provider, decode: nil, shouldInterpolate: false, intent: .defaultIntent) always fails with a EXC_BAD_ACCESS. The fact that it succeeds in one project and fails in another confuses me, is this bit of code able to be impacted by other things going on in an app?
Turns out this is due to optimization, seems like the exception occurs when optimization is turned on.
I'm trying to extract the array of numbers from a UIImage in swift but at the end I got only a bunch of zeros no useful information at all.
that's the code I wrote to try accomplishing this.
var photo = UIImage(named: "myphoto.jpg")!
var withAlpha = true
var bytesPerPixels: Int = withAlpha ? 4 : 3
var width: Int = Int(photo.size.width)
var height: Int = Int(photo.size.height)
var bitsPerComponent: Int = 8
var bytesPerRow = bytesPerPixels * width
var totalPixels = (bytesPerPixels * width) * height
var alignment = MemoryLayout<UInt32>.alignment
var data = UnsafeMutableRawPointer.allocate(byteCount: totalPixels, alignment: alignment )
var bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue).rawValue
var colorSpace = CGColorSpaceCreateDeviceRGB()
let ctx = CGContext(data: data, width: width, height: height, bitsPerComponent: bitsPerComponent, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo)
let bindedPointer: UnsafeMutablePointer<UInt32> = data.bindMemory(to: UInt32.self, capacity: totalPixels)
var pixels = UnsafeMutableBufferPointer.init(start: bindedPointer, count: totalPixels)
for p in pixels{
print(p, Date())
}
At the end I tried to bind the unsafeMutableRawPointer to extract the values but got no success,
what could I be missing here?
Thank you all in advance.
A few observations:
You need to draw the image to the context.
I’d also suggest that rather than creating a buffer that you have to manage manually, that you pass nil and let the OS create (and manage) that buffer for you.
Note that totalPixels should be just width * height.
Your code assumes the scale of the image is 1. That’s not always a valid assumption. I’d grab the cgImage and use its width and height.
Even if you have only three components, you still need to use 4 bytes per pixel.
Thus:
guard
let photo = UIImage(named: "myphoto.jpg”),
let cgImage = photo.cgImage
else { return }
let bytesPerPixels = 4
let width = cgImage.width
let height = cgImage.height
let bitsPerComponent: Int = 8
let bytesPerRow = bytesPerPixels * width
let totalPixels = width * height
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue).rawValue
let colorSpace = CGColorSpaceCreateDeviceRGB()
guard
let ctx = CGContext(data: nil, width: width, height: height, bitsPerComponent: bitsPerComponent, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo),
let data = ctx.data
else { return }
ctx.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height))
let pointer = data.bindMemory(to: UInt32.self, capacity: totalPixels)
let pixels = UnsafeMutableBufferPointer(start: pointer, count: totalPixels)
for p in pixels {
print(String(p, radix: 16), Date())
}
You need to draw the image into the context.
ctx?.draw(photo.cgImage!, in: CGRect(origin: .zero, size: photo.size))
Add that just after creating the CGContext.
I'm trying to create and save a .png from raw pixel data. I create an array of UInt8 where each number is a rgba value, kinda like this: [r, g, b, a, r, g, b, a...]. I can use this array to create a CGImage just fine. I can even use the CGImage to create an NSImage. The NSImage displays how I expect it to display when its loaded into an NSImageView.
What I want to do is to save an NSImage to disk. I've tried calling TIFFRepresentation on the NSImage and saving the NSData to "~/Desktop", but no file is saved. Any thoughts?
var pixels = [UInt8]()
for wPixel in 0...width {
for hPixel in 0...height {
pixels.append(0xff)
pixels.append(0xaa)
pixels.append(UInt8(wPixel % 200))
pixels.append(0x00)
}
}
let image = createImage(100, height:100, pixels: pixels)
let nsImage = NSImage(CGImage: image, size: CGSize(width: 100, height: 100))
NSBitmapImageRep(data: nsImage.TIFFRepresentation!)!.representationUsingType(.NSPNGFileType, properties: [:])!.writeToFile("~/Desktop/image.png", atomically: true)
func createImage(width: Int, height: Int, pixels:Array<UInt8>) -> CGImage{
let componentsPerPixel: Int = 4; // rgba
let provider: CGDataProviderRef = CGDataProviderCreateWithData(nil,
pixels,
width * height * componentsPerPixel,
nil)!;
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo:CGBitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.None.rawValue)
let bitsPerComponent = 8
let bitsPerPixel = 32
let bytesPerRow = (bitsPerComponent * width) ;
let cgImage = CGImageCreate(
width,
height,
bitsPerComponent,
bitsPerPixel,
bytesPerRow,
rgbColorSpace,
bitmapInfo,
provider,
nil,
true,
.RenderingIntentDefault
)
print(cgImage.debugDescription)
return cgImage!;
}
I've successfully converted many errors to Swift 3 except for the last line. It works in Xcode 7 but not Xcode 8.
It's also worth noting that Xcode 7 has documentation on CGColorRenderingIntent but Xcode 8 doesn't.
Type 'CGColorRenderingIntent' has no member 'RenderingIntentDefault'
Code I'm working with:
import CoreImage
// omitted code
public func imageFromPixels(pixels: ([Pixel], width: Int, height: Int)) -> CIImage {
let bitsPerComponent = 8
let bitsPerPixel = 32
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue) // alpha is last
let providerRef = CGDataProvider(data: NSData(bytes: pixels.0, length: pixels.0.count * sizeof(Pixel)))
let image = CGImageCreate(pixels.1, pixels.2, bitsPerComponent, bitsPerPixel, pixels.1 * sizeof(Pixel), rgbColorSpace, bitmapInfo, providerRef!, nil, true, CGColorRenderingIntent.RenderingIntentDefault)
return CIImage(CGImage: image!)
}
Apple documentation:
enum CGColorRenderingIntent : Int32 {
case RenderingIntentDefault
case RenderingIntentAbsoluteColorimetric
case RenderingIntentRelativeColorimetric
case RenderingIntentPerceptual
case RenderingIntentSaturation
}
Updated Code:
let image = CGImage(width: pixels.1,
height: pixels.2,
bitsPerComponent: bitsPerComponent,
bitsPerPixel: bitsPerPixel,
bytesPerRow: pixels.1 * sizeof(Pixel),
space: rgbColorSpace,
bitmapInfo: bitmapInfo,
provider: providerRef!,
decode: nil,
shouldInterpolate: true,
intent: .defaultIntent)
return CGImage(CGImage: image!) // Incorrect argument label in call (have 'CGImage:', expected 'copy:')
⌘-click on the symbol CGColorRenderingIntent and you will see
public enum CGColorRenderingIntent : Int32 {
case defaultIntent
case absoluteColorimetric
case relativeColorimetric
case perceptual
case saturation
}
So it's
let image = CGImage(width: pixels.1,
height: pixels.2,
bitsPerComponent: bitsPerComponent,
bitsPerPixel: bitsPerPixel,
bytesPerRow: pixels.1 * sizeof(Pixel),
space: rgbColorSpace,
bitmapInfo: bitmapInfo,
provider: providerRef!,
decode: nil,
shouldInterpolate: true,
intent: .defaultIntent)
return CIImage(cgImage: image!)
even the initializers of CGImage and CIImage have been changed.
It is commonly required to get the pixel data from an image or reconstruct that image from pixel data. How can I take an image, convert it to an array of pixel values and then reconstruct it using the pixel array in Swift using CoreGraphics?
The quality of the answers to this question have been all over the place so I'd like a canonical answer.
Get pixel values as an array
This function can easily be extended to a color image. For simplicity I'm using grayscale, but I have commented the changes to get RGB.
func pixelValuesFromImage(imageRef: CGImage?) -> (pixelValues: [UInt8]?, width: Int, height: Int)
{
var width = 0
var height = 0
var pixelValues: [UInt8]?
if let imageRef = imageRef {
let totalBytes = imageRef.width * imageRef.height
let colorSpace = CGColorSpaceCreateDeviceGray()
pixelValues = [UInt8](repeating: 0, count: totalBytes)
pixelValues?.withUnsafeMutableBytes({
width = imageRef.width
height = imageRef.height
let contextRef = CGContext(data: $0.baseAddress, width: width, height: height, bitsPerComponent: 8, bytesPerRow: width, space: colorSpace, bitmapInfo: 0)
let drawRect = CGRect(x: 0.0, y:0.0, width: CGFloat(width), height: CGFloat(height))
contextRef?.draw(imageRef, in: drawRect)
})
}
return (pixelValues, width, height)
}
Get image from pixel values
I reconstruct an image, in this case grayscale 8-bits per pixel, back into a CGImage.
func imageFromPixelValues(pixelValues: [UInt8]?, width: Int, height: Int) -> CGImage?
{
var imageRef: CGImage?
if let pixelValues = pixelValues {
let bitsPerComponent = 8
let bytesPerPixel = 1
let bitsPerPixel = bytesPerPixel * bitsPerComponent
let bytesPerRow = bytesPerPixel * width
let totalBytes = width * height
let unusedCallback: CGDataProviderReleaseDataCallback = { optionalPointer, pointer, valueInt in }
let providerRef = CGDataProvider(dataInfo: nil, data: pixelValues, size: totalBytes, releaseData: unusedCallback)
let bitmapInfo: CGBitmapInfo = [CGBitmapInfo(rawValue: CGImageAlphaInfo.none.rawValue), CGBitmapInfo(rawValue: CGImageByteOrderInfo.orderDefault.rawValue)]
imageRef = CGImage(width: width,
height: height,
bitsPerComponent: bitsPerComponent,
bitsPerPixel: bitsPerPixel,
bytesPerRow: bytesPerRow,
space: CGColorSpaceCreateDeviceGray(),
bitmapInfo: bitmapInfo,
provider: providerRef!,
decode: nil,
shouldInterpolate: false,
intent: .defaultIntent)
}
return imageRef
}
Demoing the code in a Playground
You'll need an image copied into the Playground's Resources folder and then change the filename and extension below to match. The result on the last line is a UIImage constructed from the CGImage.
import Foundation
import CoreGraphics
import UIKit
import PlaygroundSupport
let URL = playgroundSharedDataDirectory.appendingPathComponent("zebra.jpg")
print("URL \(URL)")
var image: UIImage? = nil
if FileManager().fileExists(atPath: URL.path) {
do {
try NSData(contentsOf: URL, options: .mappedIfSafe)
} catch let error as NSError {
print ("Error: \(error.localizedDescription)")
}
image = UIImage(contentsOfFile: URL.path)
} else {
print("File not found")
}
let (intensityValues, width, height) = pixelValuesFromImage(imageRef: image?.cgImage)
let roundTrippedImage = imageFromPixelValues(pixelValues: intensityValues, width: width, height: height)
let zebra = UIImage(cgImage: roundTrippedImage!)
I was having trouble getting Cameron's code above to work, so I wanted to test another method. I found Vacawama's code, which relies on ARGB pixels. You can use that solution and convert each grayscale value to an ARGB value by simply mapping on each value:
/// Assuming grayscale pixels contains floats in the range 0...1
let grayscalePixels: [Float] = ...
let pixels = grayscalePixels.map {
let intensity = UInt8(round($0 / Float(UInt8.max)))
return PixelData(a: UInt8.max, r: intensity, g: intensity, b: intensity)
}
let image = UIImage(pixels: pixels, width: width, height: height)