iOS 11 doesn't grab screen from MKMapView - swift

I have an app that displays the locations the user has walked on an MKMapView. When the user leaves the map view the app grabs the screen and saves the image on disk. Up til iOSS 10.3 this method was always successful. With iOS 11.0 the screen grab is a blank image. I get no notification from xcode that there were some changes and that I need to adjust the code.
Interestingly, screen grabs from text pages are still grabbed and saved successfully.
Did anyone encounter the same problem and got the solution?
The code that has always been successful up til now, is:
override func viewWillDisappear(_ animated: Bool) {
//Set the full file name under which the track will be saved.
let fileBaseName = self.imageName.appending(String(describing: (self.display?.trackDate)!))
let fileFullName = fileBaseName.appending(".png")
//Check if the image already has been saved
if !AuxiliaryObjects.shared.getImageFileName(with: fileFullName ) {
//Create the sizes of the capture
let screenRect = self.trackMapView.frame
let screenSize = screenRect.size
let screenScale = UIScreen.main.scale
var grabRect = self.trackMapView.convertRegion(self.mapRegion, toRectTo: self.view)
var heightAdjustment : CGFloat = 0.0
//Grab the image from the screen
UIGraphicsBeginImageContextWithOptions(screenSize, false, screenScale)
self.trackMapView.drawHierarchy(in: screenRect, afterScreenUpdates: true)
let myImage = UIGraphicsGetImageFromCurrentImageContext()
grabRect.origin.x *= (myImage?.scale)!
grabRect.origin.y *= (myImage?.scale)!
grabRect.size.width *= (myImage?.scale)!
grabRect.size.height *= (myImage?.scale)!
let grabImage = (myImage?.cgImage)!.cropping(to: grabRect)
let mapImage = UIImage(cgImage: grabImage!)
UIGraphicsEndImageContext()
AuxiliaryObjects.shared.save(image: mapImage, with: fileFullName, and: self.imageName)
self.display?.displayImage = AuxiliaryObjects.shared.getImage(with: fileFullName, with: self.tableImageRect)!
} else {
self.display?.displayImage = AuxiliaryObjects.shared.getImage(with: fileFullName, with: self.tableImageRect)!
}
}

I submitted a code level support request at Apple to get the answer to the question. Apple does not support the use of drawHierarhy in grabbing a MapKit screen. The way to go is using the MKMapSnapshotter utility to create an MKMapSnapshot and then draw in the lines and annotations by converting all the map coordinates to view coordinates.
Since this gave me some problems with getting the a mirrored image and translating the coordinates properly, I decided to use the layer method render(in: CGContext) this provided me a well functioning very efficient screen grab.
func creatSnapshot(with fileName: String) {
UIGraphicsBeginImageContextWithOptions(self.trackMapView.frame.size, false, UIScreen.main.scale)
let currentContext = UIGraphicsGetCurrentContext()
self.trackMapView.layer.render(in: currentContext!)
let contextImage = (UIGraphicsGetImageFromCurrentImageContext())!
UIGraphicsEndImageContext()
let region = self.trackMapView.region
var cropRect = self.trackMapView.convertRegion(region, toRectTo: self.trackMapView.superview)
cropRect.origin.x *= contextImage.scale
cropRect.origin.y *= contextImage.scale
cropRect.size.height *= contextImage.scale
cropRect.size.width *= contextImage.scale
let cgMapImage = contextImage.cgImage?.cropping(to: cropRect)
let mapImage = UIImage(cgImage: cgMapImage!)
AuxiliaryObjects.shared.save(image: mapImage, with: fileName, and: self.imageName)
self.displayTrack?.displayImage = AuxiliaryObjects.shared.getImage(with: fileName, with: self.tableImageRect)!
NotificationCenter.default.post(name: self.imageSavedNotification, object: self)
}

Related

IMessage MSSticker view created from UIView incorrect sizing

Hey I have been struggling with this for a couple of days now and can't seem to find any documentation out side of the standard grid views for MSStickerView sizes
I am working on an app that creates MSStickerViews dynamically - it does this via converting a UIView into an UIImage saving this to disk then passing the URL to MSSticker before creating the MSStickerView the frame of this is then set to the size of the original view.
The problem I have is that when I drag the MSStickerView into the messages window, the MSStickerView shrinks while being dragged - then when dropped in the messages window, changes to a larger size. I have no idea how to control the size when dragged or the final image size
Heres my code to create an image from a view
extension UIView {
func imageFromView() -> UIImage? {
UIGraphicsBeginImageContextWithOptions(self.bounds.size, self.isOpaque, 0.0)
defer { UIGraphicsEndImageContext() }
if let context = UIGraphicsGetCurrentContext() {
self.layer.render(in: context)
let image = UIGraphicsGetImageFromCurrentImageContext()
return image
}
return nil
}
}
And here's the code to save this to disk
extension UIImage {
func savedPath(name: String) -> URL{
let paths = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)
let filePath = "\(paths[0])/name.png"
let url = URL(fileURLWithPath: filePath)
// Save image.
if let data = self.pngData() {
do {
try data.write(to: url)
} catch let error as NSError {
}
}
return url
}
}
finally here is the code that converts the data path to a Sticker
if let stickerImage = backgroundBox.imageFromView() {
let url = stickerImage.savedPath(name: textBox.text ?? "StickerMCSticker")
if let msSticker = try? MSSticker(contentsOfFileURL: url, localizedDescription: "") {
var newFrame = self.backgroundBox.frame
newFrame.size.width = newFrame.size.width
newFrame.size.height = newFrame.size.height
let stickerView = MSStickerView(frame: newFrame, sticker: msSticker)
self.view.addSubview(stickerView)
print("** sticker frame \(stickerView.frame)")
self.sticker = stickerView
}
}
I wondered first off if there was something I need to do regarding retina sizes, but adding #2x in the file just breaks the image - so am stuck on this - the WWDC sessions seem to show stickers being created from file paths and not altering in size in the transition between drag and drop - any help would be appreciated!
I fixed this issue eventually by getting the frame from the view I was copying's frame then calling sizeToFit()-
init(sticker: MSSticker, size: CGSize) {
let stickerFrame = CGRect(x: 0, y: 0, width: size.width, height: size.height)
self.sticker = MSStickerView(frame: stickerFrame, sticker: sticker)
self.sticker.sizeToFit()
super.init(nibName: nil, bund
as the StickerView was not setting the correct size. Essentially the experience I was seeing was that the sticker size on my view was not accurate with the size of the MSSticker - so the moment the drag was initialized, the real sticker size was implemented (which was different to the frame size / autoLayout I was applying in my view)

Google Maps iOS is not allowing custom map marker image

I am using the Google Maps SDK for iOS - https://developers.google.com/maps/documentation/ios-sdk/marker#use_the_markers_icon_property
Combined with the Maps SDK for iOS Utility Library https://developers.google.com/maps/documentation/ios-sdk/utility/kml-geojson#render-kml-data
I am trying to use the utility library to render a kml file on a map. It mostly works, however the custom icons for the markers are not loading. The markers with their titles, snippets, and locations all load correctly. The only thing that does not work is the custom icon for the marker.
Originally, I thought it was an issue with the utility library, so I spent some time trying to write my own code to go through the kml file and add the custom markers myself. However, before I got too far I noticed that even when I try to add a basic marker with a custom icon, I cannot. This led me to believe it was an issue not with the utility library but with the Maps SDK for iOS. I've tried moving the folder that the image is in, and making sure that the code can see the path to the images, but I cannot get it to work.
This is the code that I have in my project
let path = Bundle.main.path(forResource: testFile, ofType: "kml")
let url = URL(fileURLWithPath: path!)
let kmlParser = GMUKMLParser(url: url)
kmlParser.parse()
let camera = GMSCameraPosition.camera(withLatitude: lat, longitude: long, zoom: zoom)
let mapView = GMSMapView.map(withFrame: CGRect.zero, camera: camera)
mapView.mapType = GMSMapViewType.terrain
mapView.isMyLocationEnabled = true
mapView.settings.zoomGestures = true
mapView.settings.myLocationButton = true
let renderer = GMUGeometryRenderer(map: mapView, geometries: kmlParser.placemarks, styles: kmlParser.styles, styleMaps: kmlParser.styleMaps)
renderer.render()
This also does not work
let position = CLLocationCoordinate2D(latitude: lat, longitude: long)
let marker = GMSMarker(position: position)
marker.title = "Test"
marker.icon = UIImage(named: "icon-1")
marker.map = mapView
Thanks in advance for any help
I haven't figured out why the utils library wasn't working, but I did come up with my own fix. It's horrible, but I can come back and make it better later after we've finished adding all the other necessary features to the app and can focus on cleaning up the code .
First, I made a new array of placemarks that had everything except the map markers. I then used this array of placemarks instead of kmlParser.placemarks, so that everything else could be added by the utility library.
//Removing markers without icons
var myIndex = 0
var removed = [GMUGeometryContainer]()
for mark in kmlParser.placemarks{
if(mark.geometry.type != "Point"){
removed.append(kmlParser.placemarks[myIndex])
}
myIndex += 1
}
let renderer = GMUGeometryRenderer(map: mapView, geometries: removed, styles: kmlParser.styles, styleMaps: kmlParser.styleMaps)
renderer.render()
After that, I made my own horrible horrible method that reads the kml file again, and only picks out the placemarks and styles for them and returns an array of Markers.
func addMarkers(fileName:String) -> [GMSMarker]{
var markers = [GMSMarker]()
if let path = Bundle.main.path(forResource: fileName, ofType: "kml"){
do{
let data = try String(contentsOfFile: path, encoding: .utf8)
let myStrings = data.components(separatedBy: .newlines)
var styleToIcon = [String: String]()
var lineNum = 0
for line in myStrings{
//Detecting new style that will be used in placemarks
if line.contains("Style id") && line.contains("normal") && !line.contains("line-"){
let newKey = String(line.split(separator: "\"")[1])
let newValue = String(myStrings[lineNum+4].split(separator: ">")[1].split(separator: "/")[1].split(separator: "<")[0])
styleToIcon[newKey] = newValue
}
//Detecting new placemark on map
else if(line.contains("<Placemark>") && !myStrings[lineNum+2].contains("#line")){
//Get name
var name = myStrings[lineNum+1].split(separator: ">")[1].split(separator: "<")[0]
//Sometimes name has weird CDATA field in it that needs to be removed
if(name.contains("![CDATA")){
name = name.split(separator: "[")[2].split(separator: "]")[0]
}
//Get snippet (description)
var snippet = myStrings[lineNum+2].split(separator: ">")[1].split(separator: "<")[0]
//Sometimes snippet has weird CDATA field in it that needs to be removed
if(snippet.contains("![CDATA")){
snippet = snippet.split(separator: "[")[2].split(separator: "]")[0]
}
//Get style
let style = String(myStrings[lineNum+3].split(separator: ">")[1].split(separator: "#")[0].split(separator: "<")[0] + "-normal")
//Get Coordinates
let coordStringSplit = myStrings[lineNum+6].split(separator: ",")
var lat = 0.0
var long = 0.0
if(coordStringSplit[0].contains("-")){
long = Double(coordStringSplit[0].split(separator: "-")[1])! * -1.0
}else{
long = Double(coordStringSplit[0])!
}
if(coordStringSplit[1].contains("-")){
lat = Double(coordStringSplit[1].split(separator: "-")[1])! * -1.0
}else{
lat = Double(coordStringSplit[1])!
}
//Create marker and add to list of markers
let position = CLLocationCoordinate2D(latitude: lat, longitude: long)
let marker = GMSMarker(position: position)
marker.title = String(name)
marker.snippet = String(snippet)
marker.icon = UIImage(named: styleToIcon[style]!)
markers.append(marker)
}
lineNum += 1
}
}catch{
print(error)
}
}
return markers
}
This is so heavily related to how my kml files look that I doubt it will help anyone else, but I thought I should post it just in case.
Now that we have that method, all we need to do is go back to where we were rendering all of the kml data and render those markers on the map
//Adding markers with icons
let newMarkers = addMarkers(fileName: courseName)
for mark in newMarkers{
mark.map = mapView
}
I also had to go through my kml files manually and fix some of the image names, but that wasn't a big deal. Even if the utility library worked I would need to do that because the utility library only does kml files and not kmz, so each kml file references the same folder for images and uses the same names for images. It's fine, only takes a few minutes per file. Would be nice if there was a kmz library but oh well.
Hopefully this helps someone else, and hopefully I can find the real solution soon (unless its a problem with the utility library in which case hopefully it's fixed soon).
//call method by passing ;
if userLocation.coordinate.latitude != 0.0 && userLocation.coordinate.longitude != 0.0
{
self.updateCurrentPositionMarker(currentLocation: CLLocation(latitude: userLocation.coordinate.latitude, longitude:userLocation.coordinate.longitude))
}
//methods
func updateCurrentPositionMarker(currentLocation: CLLocation) {
self.currentPositionMarker.map = nil
self.currentPositionMarker = GMSMarker(position: currentLocation.coordinate)
if self.imageDataUrl != ""
{
let camera: GMSCameraPosition = GMSCameraPosition.camera(withLatitude: currentLocation.coordinate.latitude, longitude: currentLocation.coordinate.longitude, zoom: 18.0)
self.mapView.camera = camera
//self.imageDataUrl == image to show
self.currentPositionMarker.iconView = self.drawImageWithProfilePic(urlString:self.imageDataUrl,image: UIImage.init(named: “backgroungImage”)!)
self.currentPositionMarker.zIndex = 1
}
self.currentPositionMarker.map = self.mapView
self.mapView.reloadInputViews()
}
func drawImageWithProfilePic(urlString:String, image: UIImage) -> UIImageView {
let imgView = UIImageView(image: image)
imgView.frame = CGRect(x: 0, y: 0, width: 90, height: 90)
let picImgView = UIImageView()
picImgView.sd_setImage(with:URL(string: urlString))
picImgView.frame = CGRect(x: 0, y: 0, width: 40, height: 40)
imgView.addSubview(picImgView)
picImgView.center.x = imgView.center.x
picImgView.center.y = imgView.center.y-10
picImgView.layer.cornerRadius = picImgView.frame.width/2
picImgView.clipsToBounds = true
imgView.setNeedsLayout()
picImgView.setNeedsLayout()
// let newImage = imageWithView(view: imgView)
// return newImage
return imgView
}

Taking a snapshot of UIImageView instead of main View

I have in my app in my View a Button for taking snapshot with below code
open func takeScreenshot(_ shouldSave: Bool = true) -> UIImage? {
var screenshotImage :UIImage?
let layer = UIApplication.shared.keyWindow!.layer
let scale = UIScreen.main.scale
UIGraphicsBeginImageContextWithOptions(layer.frame.size, false, scale);
guard let context = UIGraphicsGetCurrentContext() else {return nil}
layer.render(in:context)
screenshotImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
if let image = screenshotImage, shouldSave {
UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil)
}
return screenshotImage
}
But in the same View is UIImageView and I want to take a snapshot of this UIImageView. How should I change the code? When I take a snapshot with this code, it will save only black screen with button for play/pause video and with button for taking snapshot.
Instead you using
let layer = UIApplication.shared.keyWindow!.layer
Just use your imageview Layer
let layer = YourImageView.layer

How to Take a Snapshot of a Sceneview with the Sceneviews Autolighting

For anybody who has worked with snapshotting sceneview screens, you would know what I mean when I say the photo output appears much darker then the screen you are capturing. How can I capture photo output of the sceneview that shows the sceneviews brightness. Im not sure how to ask this question better but essentially this is how I am capturing the sceneview.
#IBAction func ARSnapTapped(_ sender: Any) {
if !draw {
let newImg: UIImage = self.sceneView.snapshot()
DispatchQueue.main.async {
self.imageTaken.image = newImg
self.imageTakenView.isHidden = false
}
self.image = newImg
}
}
This is the solution for anybody looking to enhance the snapshot output lighting when taking snapshots of sceneview in swift.
if let camera = sceneView.pointOfView?.camera {
camera.wantsHDR = true
camera.wantsExposureAdaptation = true
camera.whitePoint = 1.0
camera.exposureOffset = 1
camera.minimumExposure = 1
camera.maximumExposure = 1
}
Adjusting the values for exposureOffset and min/max will brighten/darken the output from the screenshot.

Binarize Picture with Core Image on iOS

I was wondering if it is possible to binarize an image (convert to black and white only) with Core Image?
I made it with OpenCV and GPUImage, but would prefer it to use Apple Core Image, if that's possible
You can use MetalPerformanceShaders for that. And the CIImageProcessingKernel.
https://developer.apple.com/documentation/coreimage/ciimageprocessorkernel
Here is the code of the class needed.
class ThresholdImageProcessorKernel: CIImageProcessorKernel {
static let device = MTLCreateSystemDefaultDevice()
override class func process(with inputs: [CIImageProcessorInput]?, arguments: [String : Any]?, output: CIImageProcessorOutput) throws {
guard
let device = device,
let commandBuffer = output.metalCommandBuffer,
let input = inputs?.first,
let sourceTexture = input.metalTexture,
let destinationTexture = output.metalTexture,
let thresholdValue = arguments?["thresholdValue"] as? Float else {
return
}
let threshold = MPSImageThresholdBinary(
device: device,
thresholdValue: thresholdValue,
maximumValue: 1.0,
linearGrayColorTransform: nil)
threshold.encode(
commandBuffer: commandBuffer,
sourceTexture: sourceTexture,
destinationTexture: destinationTexture)
}
}
And this is how you can use it:
let context = CIContext(options: nil)
if let binaryCIImage = try? ThresholdImageProcessorKernel.apply(
withExtent: croppedCIImage.extent,
inputs: [croppedCIImage],
arguments: ["thresholdValue": Float(0.2)]) {
if let cgImage = context.createCGImage(binaryCIImage, from: binary.extent) {
DispatchQueue.main.async {
let resultingImage = UIImage(cgImage: cgImage)
if resultingImage.size.width > 100 {
print("Received an image \(resultingImage.size)")
}
}
}
}
Yes. You have at least two options, CIPhotoEffectMono or a small custom CIColorKernel.
CIPhotoEffectMono:
func createMonoImage(image:UIImage) -> UIImage {
let filter = CIFilter(name: "CIPhotoEffectMono")
filter!.setValue(CIImage(image: image), forKey: "inputImage")
let outputImage = filter!.outputImage
let cgimg = ciCtx.createCGImage(outputImage!, from: (outputImage?.extent)!)
return UIImage(cgImage: cgimg!)
}
Note, I'm writing this quickly, you may need to tighten up things for nil returns.
CIColorKernel:
The FadeToBW GLSL (0.0 factor full color, 1.0 factor is no color):
kernel vec4 fadeToBW(__sample s, float factor) {
vec3 lum = vec3(0.299,0.587,0.114);
vec3 bw = vec3(dot(s.rgb,lum));
vec3 pixel = s.rgb + (bw - s.rgb) * factor;
return vec4(pixel,s.a);
}
The code below opens this as a file called FadeToBW.cikernel. You can also post this as a String directly into the openKernelFile call.
The Swift code:
func createMonoImage(image:UIImage, inputColorFade:NSNumber) -> UIImage {
let ciKernel = CIColorKernel(string: openKernelFile("FadeToBW"))
let extent = image.extent
let arguments = [image, inputColorFade]
let outputImage = ciKernel.applyWithExtent(extent, arguments: arguments)
let cgimg = ciCtx.createCGImage(outputImage!, from: (outputImage?.extent)!)
return UIImage(cgImage: cgimg!)
}
Again, add some guards, etc.
I have had success by converting it to greyscale using CIPhotoEffectMono or equivalent, and then using CIColorControls with a ridiculously high inputContrast number (I used 10000). This effectively makes it black and white and thus binarized. Useful for those who don't want to mess with custom kernels.
Also, you can use an example like Apple's "Chroma Key" filter which uses Hue to filter, but instead of looking at Hue you just give the rules for binarizing the data (ie: when to set RGB all to 1.0 and when to set to 0.0).
https://developer.apple.com/documentation/coreimage/applying_a_chroma_key_effect
Found this thread from a Google search, and thought I'd mention that as of iOS 14 and OSX 11.0, CoreImage includes CIColorThreshold and CIColorThresholdOtsu filters (the latter using Otsu's method to calculate the threshold value from the image histogram)
See:
https://cifilter.io/CIColorThreshold/
https://cifilter.io/CIColorThresholdOtsu/
let outputImage = inputImage.applyingFilter("CIColorMonochrome",
parameters: [kCIInputColorKey: CIColor.white])
In you want to play with every out of 250 CIFilters please check this app out: https://apps.apple.com/us/app/filter-magic/id1594986951