In my function, I run the following code, when a specific event shows up and Safari is in foreground:
if win.safariIsForeground() {
let el = AXUIElementCreateApplication(win.getSafariPid())
var ptr: CFArray?
_ = AXUIElementCopyAttributeNames(el, &ptr)
}
The pointer returns an array that looks like this:
["AXFunctionRowTopLevelElements", "AXFrame", "AXChildren",
"AXFocusedUIElement", "AXFrontmost", "AXRole", "AXExtrasMenuBar",
"AXMainWindow", "AXFocusedWindow", "AXTitle",
"AXChildrenInNavigationOrder", "AXEnhancedUserInterface",
"AXRoleDescription", "AXHidden", "AXMenuBar", "AXWindows", "AXSize",
"AXPosition"]
I'd like to make Safari go one site back in the history. I think I will need AXUIElementCopyAttributeValue and AXUIElementPerformAction to do that but how do I find out the button's attribute and how do I call check AXUIElementCopyAttributeValue for that?
The easiest way to do that is by accessing the menu item. Using AXUIElementCopyAttributeValue works best with the provided constants:
// get menu bar
var menuBarPtr: CFTypeRef?
_ = AXUIElementCopyAttributeValue(safariElement, kAXMenuBarRole as CFString, &menuBarPtr)
guard let menuBarElement = menuBarPtr as! AXUIElement? else {
fatalError()
}
Accessibility Inspector shows me, what items are child of the menu bar:
so lets get the children using kAXChildrenAttribute:
// get menu bar items
var menuBarItemsPtr: CFTypeRef?
_ = AXUIElementCopyAttributeValue(menuBarElement, kAXChildrenAttribute as CFString, &menuBarItemsPtr)
guard let menuBarItemsElement = menuBarItemsPtr as AnyObject as! [AXUIElement]? else {
fatalError()
}
And so on all the way down to the menu item. Items also have an unique identifier that can look like _NS:1008. I'm not sure how to access them directly but by using AXUIElementPerformAction I can simply simulate pressing the menu item (action will be kAXPressAction).
I can use kAXRoleAttribute to identify the type of the item and where it occurs in the Accessibility hierarchy (see "Role:")
As I'm still a beginner at Swift, that was a quite challenging task as this is also not documented very well. Thanks a lot to Dexter who also helped me to understand this topic: kAXErrorAttributeUnsupported when querying Books.app
In your case you don't need necessarily need to use AXUI accessibility API. It's arguably simpler to send key strokes. You can go back in Safari history to previous page with CMD[. As an added bonus Safari does not need to be in foreground anymore.
let pid: pid_t = win.getSafariPid();
let leftBracket: UInt16 = 0x21
let src = CGEventSource(stateID: CGEventSourceStateID.hidSystemState)
let keyDownEvent = CGEvent(keyboardEventSource: src, virtualKey: leftBracket, keyDown: true)
keyDownEvent?.flags = CGEventFlags.maskCommand
let keyUpEvent = CGEvent(keyboardEventSource: src, virtualKey: leftBracket, keyDown: false)
keyDownEvent?.postToPid(pid)
keyUpEvent?.postToPid(pid)
Your app running on Mojave and newer MacOS versions will need the System Preferences -> Security & Privacy -> Accessibility permisions granted to be eligible for sending keystrokes to other apps.
The keyboard codes can be looked up:
here & here visually
Related
its not hard to get a specified application by name
NSWorkspace.shared.runningApplications.filter{$0.localizedName == "Safari"}.first
but how to get the first window of this application, and perform miniaturize with this window?
something similar with this
let app = NSWorkspace.shared.runningApplications.filter{$0.localizedName == "Safari"}.first
app.frontmostWindow.miniaturize()
You can do this using the ScriptingBridge
import ScriptingBridge
let safari = SBApplication(bundleIdentifier: "com.apple.Safari")
let windowClass = appleEvent(keyword: "cwin")
let miniaturized = appleEvent(keyword: "pmnd")
let windows = safari?.elementArray(withCode: windowClass)
let frontMostWindow = (windows?.firstObject as? SBObject)?.get() as? SBObject
frontMostWindow?.property(withCode: miniaturized).setTo(true)
func appleEvent(keyword: StaticString) -> AEKeyword {
keyword
.utf8Start
.withMemoryRebound(to: DescType.self, capacity: 1, \.pointee)
.bigEndian
}
To be able to run this code you will need a code signed app with the com.apple.security.automation.apple-events entitlement set to true (which allows posting of AppleEvents to other applications)
I have a simple To do style list app, where an added item can have an intent donated so that user can find and mark the item as "completed" without opening the app.
In the Note class I have this function to donate the intent, which works as expected
public func donateMarkNoteAsCompleteIntent() {
let intent = MarkNoteAsCompleteIntent()
intent.content = self.content
let dateFormatter = DateFormatter()
dateFormatter.dateFormat = "dd/MM/YYYY"
intent.addedDate = dateFormatter.string(from: self.addedDate)
intent.id = self.id
let interaction = INInteraction(intent: intent, response: nil)
interaction.groupIdentifier = self.id
interaction.donate(completion: nil)
}
My only issue is, when the user uses the shortcut and triggers the app to update the Note item, I want to remove the shortcut so that the user can't trigger it again.
In my intent handle function, I end up calling this function
public func removeMarkNoteAsCompleteIntent() {
INInteraction.deleteAll(completion: nil)
let index = CSSearchableIndex()
index.deleteAllSearchableItems(completionHandler: nil)
}
No matter what combination of things I do here I can't seem to remove the donated shortcut. As soon as a user accepts marking the task as complete, I want the shortcut to no longer be visible in searches from Spotlight, etc. Everything else in the intent handling code is working, its updating the Note item in my database perfectly.
Help would be greatly appreciated.
I am working on NEHotspotHelper and trying to register but not receiving call back. Firstly,
I enabled Capability : Network Extensions
Then added this following code,
let options: [String: NSObject] = [kNEHotspotHelperOptionDisplayName : "ABC" as NSObject]
let queue: DispatchQueue = DispatchQueue(label: "com.ABC", attributes: DispatchQueue.Attributes.concurrent)
NSLog("Started wifi scanning.")
NEHotspotHelper.register(options: options, queue: queue) { (cmd: NEHotspotHelperCommand) in
NSLog("Received command: \(cmd.commandType.rawValue)")
if cmd.commandType == NEHotspotHelperCommandType.filterScanList {
//Get all available hotspots
let list: [NEHotspotNetwork] = cmd.networkList!
//Figure out the hotspot you wish to connect to
print(list)
} else if cmd.commandType == NEHotspotHelperCommandType.evaluate {
if let network = cmd.network {
//Set high confidence for the network
network.setConfidence(NEHotspotHelperConfidence.high)
let response = cmd.createResponse(NEHotspotHelperResult.success)
response.setNetwork(network)
response.deliver() //Respond back
}
} else if cmd.commandType == NEHotspotHelperCommandType.authenticate {
//Perform custom authentication and respond back with success
// if all is OK
let response = cmd.createResponse(NEHotspotHelperResult.success)
response.deliver() //Respond back
}
}
Kindly let me know if I am missing any step.
You should check the result of the register() function. If it's returning false, something is probably not configured correctly. See the full list of configuration instructions below.
Also in the screenshot you provided, you have the entitlements enabled for Hotspot Configuration, but the API you're calling is for Hotspot Helper. The two features require very different entitlements. You'll need to make sure everything is configured for Hotspot Helper to call that API. Again, see below for full details. See Hotspot Helper vs. Hotspot Configuration for more details about the differences of these similarly named APIs.
To use NEHotspotHelper:
Apply for the Network Extension entitlement.
This needs to be done at Apple's website here.
Modify your Provisioning Profile.
Go to http://developer.apple.com. Hit Edit near your profile. On the bottom where it says Entitlements, choose the one that contains the Network Extension entitlement.
Update your app's entitlements file.
The application must set com.apple.developer.networking.HotspotHelper as one of its entitlements. The value of the entitlement is a boolean set to true.
Add Background Mode
The application's Info.plist must include a UIBackgroundModes array containing network-authentication.
Note that unlike all the other background modes that are converted to human readable strings, this one will stay as network-authentication.
Call the NEHotspotHelper.register() function.
This method should be called once when the application starts up. Invoking it again will have no effect and result in false being returned.
You should make sure the function returns true. Otherwise something one of the above steps is probably not configured properly.
Understand when this callback will be called.
From the documentation, it's not entirely clear when exactly this callback will be called. For example, one might assume that NEHotspotHelper could be used to monitor for network connections. However, the callback will (only?) be called when the user navigates to the Settings app and goes to the Wi-Fi page.
Since your callback will be called only while the user in the Settings app, you should attach to the debugger and use print().
Swift Example
let targetSsid = "SFO WiFi"
let targetPassword = "12345678"
let targetAnnotation: String = "Acme Wireless"
let options: [String: NSObject] = [
kNEHotspotHelperOptionDisplayName: targetAnnotation as NSString
]
let queue = DispatchQueue(label: "com.example.test")
let isAvailable = NEHotspotHelper.register(options: options, queue: queue) { (command) in
switch command.commandType {
case .evaluate,
.filterScanList:
let originalNetworklist = command.networkList ?? []
let networkList = originalNetworklist.compactMap { network -> NEHotspotNetwork? in
print("networkName: \(network.ssid); strength: \(network.signalStrength)")
if network.ssid == targetSsid {
network.setConfidence(.high)
network.setPassword(targetPassword)
return network
}
return nil
}
let response = command.createResponse(.success)
response.setNetworkList(networkList)
response.deliver()
default:
break
}
}
assert(isAvailable)
Sources:
https://developer.apple.com/documentation/networkextension/nehotspothelper/1618965-register
https://medium.com/#prvaghela/nehotspothelper-register-an-app-as-a-hotspot-helper-cf92a6ed7b72
https://stackoverflow.com/a/39189063/35690
I'm trying to retrieve a list of all the available Finder tags.
I found NSWorkspace().fileLabels, which does return an array, but only an array of the tag colours, not the tags themselves:
print(NSWorkspace.shared().fileLabels) // prints ["None", "Gray", "Green", "Purple", "Blue", "Yellow", "Red", "Orange"]
Which as you can see is not even all the default tags, it's missing Home, Work and Important, and obviously doesn't have any of the custom ones that I created. It looks like it's just the nice names that go with fileLabelColors.
I found NSMetadataQuery for actually searching for things, but how do I get a list of all the tags I have created in the Finder?
After some digging with "Hopper Disassembler" and my own "Find Any File" (for text search in files), I figured out where the Tags are now stored since Monterey:
The file ~/Library/SyncedPreferences/com.apple.kvs/com.apple.KeyValueService-Production.sqlite contains the same plist data that was previously stored in ~/Library/SyncedPreferences/com.apple.finder.plist.
But it's now hidden inside a database record:
If you look into the ZSYSDMANAGEDKEYVALUE table, you'll find a single entry with ZKEY="FinderTagDict". The ZPLISTDATAVLUE contains a bplist record, which contains the pblist (binary plist) structure, from which you can then extract the tags.
NSWorkspace.shared().fileLabels only returns the system tags that were available when the user account was created (the default system tags).
There's unfortunately no API in macOS to retrieve the tags that you have created yourself in the Finder.
The solution is to parse the ~/Library/SyncedPreferences/com.apple.finder.plist:
func allTagLabels() -> [String] {
// this doesn't work if the app is Sandboxed:
// the users would have to point to the file themselves with NSOpenPanel
let url = URL(fileURLWithPath: "\(NSHomeDirectory())/Library/SyncedPreferences/com.apple.finder.plist")
let keyPath = "values.FinderTagDict.value.FinderTags"
if let d = try? Data(contentsOf: url) {
if let plist = try? PropertyListSerialization.propertyList(from: d, options: [], format: nil),
let pdict = plist as? NSDictionary,
let ftags = pdict.value(forKeyPath: keyPath) as? [[AnyHashable: Any]]
{
return ftags.flatMap { $0["n"] as? String }
}
}
return []
}
let all = allTagLabels()
print(all)
This gets all Finder tags labels.
You can also select only the custom tags (ignore the system ones):
func customTagLabels() -> [String] {
let url = URL(fileURLWithPath: "\(NSHomeDirectory())/Library/SyncedPreferences/com.apple.finder.plist")
let keyPath = "values.FinderTagDict.value.FinderTags"
if let d = try? Data(contentsOf: url) {
if let plist = try? PropertyListSerialization.propertyList(from: d, options: [], format: nil),
let pdict = plist as? NSDictionary,
let ftags = pdict.value(forKeyPath: keyPath) as? [[AnyHashable: Any]]
{
return ftags.flatMap { tag in
if let n = tag["n"] as? String,
tag.values.count != 2
{
return n
}
return nil
}
}
}
return []
}
This is an attempt to solve the question for Monterey (and it also works in earlier systems such as High Sierra).
It might also work in sandboxed apps, as long as you're allowed to use System Events in scripts (I know from own experience that it's generally not allowed for apps in the Mac App Store, though). At worst, you'd have to support user-installed scripts and then instruct your users to install the script below manually (and I also suggest to avoid bothering the user with this operation within the first hour since the first launch of the app, or the reviewer may get triggered and reject your app regardless).
The best I could come up with is to use AppleScript to read the tag names from the Finder's Preferences window, see below.
But it's lame because it has to actually open the Finders's Prefs window briefly for this. Still, better than nothing.
use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
set prefsWinWasOpen to false
tell application "Finder"
set prefs to Finder preferences
set prefsWin to window of prefs
tell prefsWin
try
get id
set prefsWinWasOpen to true
on error
set prefsWinWasOpen to false
open
end try
set current panel to Label Preferences panel
end tell
end tell
set rowNames to {}
tell application "System Events"
tell front window of (first application process whose name is "Finder")
set uiElems to entire contents
repeat with uiElem in uiElems
if class of uiElem is table then
set theTable to uiElem
exit repeat
end if
end repeat
set itsRows to rows of theTable
repeat with oneRow in itsRows
set end of rowNames to name of first UI element of oneRow
end repeat
end tell
end tell
if not prefsWinWasOpen then
tell application "Finder"
close prefsWin
end tell
end if
return rowNames
but I'm not new to programming. I'm trying to have an always running menubar app listen for a specific window in an application to be opened and then close that window, but not the application. Here is my current approach, it is mostly a mix of other SO answers I found.
func applicationDidFinishLaunching(aNotification: NSNotification) {
// listener for when user switches applications
NSWorkspace.sharedWorkspace().notificationCenter.addObserver(self,
selector: #selector(activated),
name: NSWorkspaceDidActivateApplicationNotification,
object: nil)
}
func activated(notification: NSNotification) {
// i feel like i should be using that NSNotification but I'm not
let options = CGWindowListOption(arrayLiteral: CGWindowListOption.ExcludeDesktopElements, CGWindowListOption.OptionOnScreenOnly)
let windowListInfo = CGWindowListCopyWindowInfo(options, CGWindowID(0))
let infoList = windowListInfo as NSArray? as? [[String: AnyObject]]
for item in infoList! {
// if i find the window i want to close
if item[kCGWindowName as String] == "Warnings" {
// close window item[kCGWindowName]
}
}
My question is, if I am even doing any of this right, how do I close the window based off of having its windowListInfo?
Also, the item[kCGWindowName as String] is saying Cannot convert value of type 'AnyObject?' to expected argument type String. How would I go about checking windows for the name I want without casting to a String? I have called print() on all objects in infoList and have seen a kCGWindowName value with the name of the window I am looking for, so I know its there.
Thanks in advance.
You can use the accessibility API:
You need to enable accessibility for your application or Xcode (if
you test your application from Xcode).
Open the "Security & Privacy" pane on the "System Preferences"
window. Click the "Privacy" tab.
Select "Accessibility" from the list in the left pane.
Click the lock icon to make changes.
Drag/drop your application or Xcode in the right pane.
Here's an example of the swift code (Swift version 2.2) to simulate a click on the red button of the window whose the title start with "Warnings":
func activated(notification: NSNotification) {
if (notification.userInfo![NSWorkspaceApplicationKey]!.localizedName) == "Pages" { // when "Pages" application is active only
let myApp = AXUIElementCreateApplication(notification.userInfo![NSWorkspaceApplicationKey]!.processIdentifier).takeUnretainedValue()
let val1 = UnsafeMutablePointer<AnyObject?>.alloc(1)
if AXUIElementCopyAttributeValue(myApp, kAXWindowsAttribute, val1).rawValue == 0 { // get windows of the "Pages" application
let windowList = val1.memory as? [AXUIElement] ?? []
for w in windowList { // loop each window, get the title of the window, and check if the title starts with "Warning"
if AXUIElementCopyAttributeValue(w, kAXTitleAttribute, val1).rawValue == 0 && (val1.memory as! String).hasPrefix("Warnings") {
if AXUIElementCopyAttributeValue(w, kAXCloseButtonAttribute, val1).rawValue == 0 {// get the red button of the window
AXUIElementPerformAction(val1.memory as! AXUIElement, kAXPressAction); // close the window
}
}
}
}
val1.dealloc(1)
}
}
To check the title equal "Warnings": use val1.memory as! String == "Warnings"
Informations on the infoList from your code:
How would I go about checking windows for the name I want without
casting to a String?
Use a string instead of the constant, like this:
if item["kCGWindowName"]! as! String == "Warnings" {