Not sure if it's a bug or what. I made an app that add Vim bindings all over macOS, but it fails in Catalyst apps. The text is readable mostly—although line numbers are wrong and line ranges too—but I can't set the selected text. It does nothing expect selecting the text, but not replacing it with the new one. It works well in all native apps that support the Accessibility.
Am I doing things wrong or is this a confirmed bug? (I've been reporting bugs in the Accessibility API for two years, with no change from Apple's part.)
The code is fairly simple. As an example:
func ....() {
var selectedTextRange = CFRange()
selectedTextRange.location = 2
selectedTextRange.length = 3
let newValue = AXValueCreate(.cfRange, &selectedTextRange)
guard AXUIElementSetAttributeValue(axFocusedElement, kAXSelectedTextRangeAttribute as CFString, newValue!) == .success else { return false }
guard AXUIElementSetAttributeValue(axFocusedElement, kAXSelectedTextAttribute as CFString, "hehe" as CFTypeRef) == .success else { return false }
return true
}
The func returns true. But in Catalyst apps there's a selection from location 2 to 5, but the text is not replaced by "hehe".
I've got an Xcode project also that shows the issue: https://github.com/godbout/AXBugsWithCatalystApps
Thanks.
Related
In my function, I run the following code, when a specific event shows up and Safari is in foreground:
if win.safariIsForeground() {
let el = AXUIElementCreateApplication(win.getSafariPid())
var ptr: CFArray?
_ = AXUIElementCopyAttributeNames(el, &ptr)
}
The pointer returns an array that looks like this:
["AXFunctionRowTopLevelElements", "AXFrame", "AXChildren",
"AXFocusedUIElement", "AXFrontmost", "AXRole", "AXExtrasMenuBar",
"AXMainWindow", "AXFocusedWindow", "AXTitle",
"AXChildrenInNavigationOrder", "AXEnhancedUserInterface",
"AXRoleDescription", "AXHidden", "AXMenuBar", "AXWindows", "AXSize",
"AXPosition"]
I'd like to make Safari go one site back in the history. I think I will need AXUIElementCopyAttributeValue and AXUIElementPerformAction to do that but how do I find out the button's attribute and how do I call check AXUIElementCopyAttributeValue for that?
The easiest way to do that is by accessing the menu item. Using AXUIElementCopyAttributeValue works best with the provided constants:
// get menu bar
var menuBarPtr: CFTypeRef?
_ = AXUIElementCopyAttributeValue(safariElement, kAXMenuBarRole as CFString, &menuBarPtr)
guard let menuBarElement = menuBarPtr as! AXUIElement? else {
fatalError()
}
Accessibility Inspector shows me, what items are child of the menu bar:
so lets get the children using kAXChildrenAttribute:
// get menu bar items
var menuBarItemsPtr: CFTypeRef?
_ = AXUIElementCopyAttributeValue(menuBarElement, kAXChildrenAttribute as CFString, &menuBarItemsPtr)
guard let menuBarItemsElement = menuBarItemsPtr as AnyObject as! [AXUIElement]? else {
fatalError()
}
And so on all the way down to the menu item. Items also have an unique identifier that can look like _NS:1008. I'm not sure how to access them directly but by using AXUIElementPerformAction I can simply simulate pressing the menu item (action will be kAXPressAction).
I can use kAXRoleAttribute to identify the type of the item and where it occurs in the Accessibility hierarchy (see "Role:")
As I'm still a beginner at Swift, that was a quite challenging task as this is also not documented very well. Thanks a lot to Dexter who also helped me to understand this topic: kAXErrorAttributeUnsupported when querying Books.app
In your case you don't need necessarily need to use AXUI accessibility API. It's arguably simpler to send key strokes. You can go back in Safari history to previous page with CMD[. As an added bonus Safari does not need to be in foreground anymore.
let pid: pid_t = win.getSafariPid();
let leftBracket: UInt16 = 0x21
let src = CGEventSource(stateID: CGEventSourceStateID.hidSystemState)
let keyDownEvent = CGEvent(keyboardEventSource: src, virtualKey: leftBracket, keyDown: true)
keyDownEvent?.flags = CGEventFlags.maskCommand
let keyUpEvent = CGEvent(keyboardEventSource: src, virtualKey: leftBracket, keyDown: false)
keyDownEvent?.postToPid(pid)
keyUpEvent?.postToPid(pid)
Your app running on Mojave and newer MacOS versions will need the System Preferences -> Security & Privacy -> Accessibility permisions granted to be eligible for sending keystrokes to other apps.
The keyboard codes can be looked up:
here & here visually
I'm offering a custom keyboard app.
I am checking'full access' with the code below to activate some functions. It's a valuable code I discovered through this site a few years ago. But I found out that the code below doesn't run on iOS 14.
UIPasteboard.general.hasStrings always returns false. hasImages/hasColors/hasURLs all return false. But it doesn't seem to be real. If you paste, there is a previously copied content or only ‘TEST’ is pasted.
'TEST' is for checking purposes and should not be printed.
let pasty = UIPasteboard.general
if pasty.hasURLs || pasty.hasColors || pasty.hasStrings || pasty.hasImages {
hasFullAccess = true
} else {
pasty.string = "TEST"
if pasty.hasStrings {
hasFullAccess = true
pasty.string = ""
}
}
I am deeply reflecting on what I only found out about this. Also, I'm really sorry for the users who use my app. So I'm asking here because I want to solve it somehow. So, if anyone knows how to fix it, I'd like to let you know.
What I have done so far
Update ‘firebase’ and ‘realm’ to the latest version with ‘cocoapod’
Fix a problem after pod update
Try modifying the code continuously by turning on/off Allow Full Access in iOS settings
Google search…
Let me know if someone knows a better solution. Or, I'd like to give you a clue as to the cause of the problem. I hope that people who have the same problem will consider it together.
Thanks for reading this far. And I'll wait for someone's help. please.
We found the same issue. Our current solution is to use viewWillAppear instead of viewDidLoad when checking for full access.
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
// We need to check fullAccess after view has appeared in order for UIPasteboard to be available.
if self.hasFullyAccess() == false {
// Do whatever.
}
}
public func hasFullyAccess() -> Bool {
var hasFullAccess = false
if #available(iOS 10.0, *) {
let pasty = UIPasteboard.general
if pasty.hasURLs || pasty.hasColors || pasty.hasStrings || pasty.hasImages {
hasFullAccess = true
} else {
// We test if we can put something in UIPasteboard, if anything we have access.
pasty.string = "TEST"
if pasty.hasStrings {
hasFullAccess = true
pasty.string = "" // Removes string from pasty again.
}
}
}
return hasFullAccess
}
We have an app that downloads assets and media selections using .aggregateAssetDownloadTask. In iOS 12.4 and below we have been able to access the captions tracks when offline using the asset's assetCache. On an iOS 13 device, the assetCache for the captions is now empty despite being downloaded. We can get the captions if we directly access the assets tracks using asset.mediaSelectionGroup(forMediaCharacteristic: characteristic).
We access the tracks like this:
guard let asset = currentItem.asset as? AVURLAsset else { return [] }
let cache = asset.assetCache
if let group = asset.mediaSelectionGroup(forMediaCharacteristic: characteristic), let languages = cache?.mediaSelectionOptions(in: group) {
if medium == .captions {
return languages.filter({ $0.hasMediaCharacteristic(AVMediaCharacteristic.containsOnlyForcedSubtitles) == false }).compactMap({ $0 as Language})
} else {
return languages
}
}
We've filed a radar with apple but we're wondering if anyone else has ran into this issue? If so what workaround you're using? We're hesitant to access the mediaSelectionGroup on the asset.
THIS IS NOT A DUPLICATE QUESTION
I have searched for everything that I can, but I didn't find something meaningful.
Here is the question:
I want to download a font using CoreText API but it works in iOS 12, NOT available on iOS 13.
There is a demo project from Apple in 2013. But The API it used seems to be private since iOS 13.
/*
#param progressBlock
Callback block to indicate the progress.
Return true to continue, and return false to cancel the process.
This block is called on a private serial queue on OS X 10.15, iOS 13, and later.
*/
#available(iOS 6.0, *)
public func CTFontDescriptorMatchFontDescriptorsWithProgressHandler(_ descriptors: CFArray, _ mandatoryAttributes: CFSet?, _ progressBlock: #escaping CTFontDescriptorProgressHandler) -> Bool
As the documentation declared:
This block is called on a private serial queue on OS X 10.15, iOS 13, and later.
Here is my code for font download:
let fontName = "STXingkai-SC-Light"
let attributes = [kCTFontNameAttribute : fontName] as CFDictionary
let fontDescription = CTFontDescriptorCreateWithAttributes(attributes)
// create font font descriptor and add it in an array
let descs = [fontDescription] as CFArray
CTFontDescriptorMatchFontDescriptorsWithProgressHandler(descs, nil) { (state, progressParamater) -> Bool in
let progressValue = (progressParamater as Dictionary)[kCTFontDescriptorMatchingPercentage]?.doubleValue
switch state {
case .didBegin: print("didBegin")
case .didFinish: print("didFinish")
case .willBeginDownloading: print("willBeginDownloading")
case .didFinishDownloading:
print("--> download finish")
DispatchQueue.main.async {
self.fontLabel.font = UIFont(name: self.fontName, size: 20)
}
case .downloading:
print("downloading#####\(progressValue ?? 0.0)")
DispatchQueue.main.async {
self.progressView.progress = Float(progressValue ?? 0.0)
}
case .didFailWithError:
if let error = (progressParamater as Dictionary)[kCTFontDescriptorMatchingError] as? NSError {
print(error.description)
} else {
print("ERROR MESSAGE IS NOT AVAILABLE")
}
default: print(String(reflecting: state))
}
return true
}
I tried this API on iOS 12 and Xcode 10, everything works fine and I can download a font with provided fontName.
But, when I use this one on Xcode 11 Version 11.0 beta 6 (11M392q) and macOS Catalina 10.15 Beta (19A526h), something went wrong and that method is no longer effective. And I got some output:
didBegin
__C.CTFontDescriptorMatchingState
didFinish
done
"Error Domain=com.apple.CoreText.CTFontManagerErrorDomain Code=303 \"0
font descriptors do not have information to specify a font file.\"
UserInfo={NSLocalizedDescription=0
font descriptors do not have information to specify a font file.
0 font descriptors do not have information to specify a font file
The state will only have didBegin and didFinish, not calling downloading.
WWDC19 seeesion 227 has announced that CTFontManagerRegisterFontDescriptors(fontDescriptors:scope:enabled:registrationHandler:) is available and something about font management. But the source code from the session pdf is not clearly and I've tried, got an other error:
done
[Error Domain=com.apple.CoreText.CTFontManagerErrorDomain Code=306 "The file is not in an allowed location. It must be either in the application’s bundle or an on-demand resource." UserInfo={NSLocalizedDescription=The file is not in an allowed location. It must be either in the application’s bundle or an on-demand resource., CTFontManagerErrorFontURLs=(
"http://iweslie.com/fonts/HanziPenSC-W3.ttf"
), NSLocalizedFailureReason=Font registration was unsuccessful.}]
done
Here is my code referencing from WWDC19 session 227:
let urlString = "http://iweslie.com/fonts/HanziPenSC-W3.ttf"
let url = URL(string: urlString)! as CFURL
let fontURLArray = [url] as CFArray
CTFontManagerRegisterFontURLs(fontURLArray, .persistent, true) { (errors, done) -> Bool in
if done {
print("done")
}
print(errors as Array)
return true
}
And got the error:
Error Domain=com.apple.CoreText.CTFontManagerErrorDomain Code=306
"The file is not in an allowed location. It must be either in the application’s bundle or an on-demand resource."
For other people looking for a solution for iOS 13
The easiest way to do it is probably using Apple Bundle and the 'new' CTFontManagerRegisterFontURLs api.
Just gather all your urls in a list by utilizing Bundle.main.url:
let fontUrl = Bundle.main.url(forResource: fileName, withExtension: "ttf")
Then just register your fonts:
CTFontManagerRegisterFontURLs([fontUrl] as CFArray, .persistent, true) { (errors, done) -> Bool in
if(done) {
print("Done")
}
print(errors as Array)
return true
}
If you're using another solution and having problems with the font name it might be xcode which is at fault. A tip I saw in another thread for figuring out what xcode is naming your fonts are:
for family in UIFont.familyNames.sorted() {
let names = UIFont.fontNames(forFamilyName: family)
print("Family: \(family) Font names: \(names)")
}
Also remember to enable Install Fonts capability under the Signing & Capabilities in your project settings.
I have facing the same problem. and I finaly found this.
https://juejin.im/post/5aebd428f265da0ba266d897
the problem is you are using the wrong font postscriptname. the correct font name is "STXingkaiSC-Light".
I have test this on xcode 12, iOS 13 simulator.
you can find the correct font postscript name using "Font BooK.app".
I am donating an INInteraction and everything is working ok, however I can't understand why the donation is showing up duplicated in Spotlight.
Is there a property that needs to be set in order to prevent this??
let viewUsageIntent = UsageIntent()
var susbcribers = [INObject]()
for sub in account.subscribers {
let inObject = INObject(identifier: sub.phoneNumber, display: sub.id)
susbcribers.append(inObject)
}
viewUsageIntent.suggestedInvocationPhrase = phrase
viewUsageIntent.ban = account.ban
viewUsageIntent.subs = susbcribers
let interaction = INInteraction(intent: viewUsageIntent, response: nil)
interaction.donate(completion: {
error in
if let err = error {
MyAppServices.Logger.error(tag: "UsageIntentDonation", message: "Donation for ban \(account.ban) could not be completed: \(err.localizedDescription)")
}
})
Anybody dealing with this issue?. Thanks.
This is not a bug. This is the default behaviour when you set your simulator or iPhone to display recent shortcuts in the developer section. I was just confused by it.
Is not a duplication, it is just displaying the newest one over older ones for development sake