How to detect keystrokes globally in Swift on macOS? - swift

Here is what I tried:
NSEvent.addGlobalMonitorForEvents(matching: [.keyDown]) { (event) in
print(event.keyCode)
}
Unfortunately, it does not print anything.
And no, it's not a duplicate of this, that question is about modifier keys, my question is about keystrokes.


Looks like the "duplicate" mark got removed, but so has the answer that I kludged into the comments section. So, for posterity:
The reason this doesn't work is because global monitors for .keyDown events require more permissions than some of the other event handlers, including the one that somebody thought this was a duplicate of. This is mainly because global .keyDown monitors can be used for nefarious purposes, such as keyloggers. So there are additional security measures in place to make sure we're legit:
1) Your app needs to be code-signed.
2) Your app needs to not have the App Sandbox enabled, and:
3) Your app needs to be registered in the Security and Privacy preference pane, under Accessibility.
The third one of these things has to be enabled by the user, but you can nudge them in that direction with this code:
let options: NSDictionary = [kAXTrustedCheckOptionPrompt.takeUnretainedValue() as String : true]
let accessEnabled = AXIsProcessTrustedWithOptions(options)
if !accessEnabled {
print("Access Not Enabled")
}
This will prompt the user, giving him/her the option to automatically open the appropriate preference pane where the user can allow your app to control the computer via the Accessibility API, which, assuming your app is signed and not sandboxed, will allow your global .keyDown monitor to work.


if you only want global hotkey support all this is unnecessary (and not all random key or mouse events) you can do that easily with the hotkey API. look at e.g. PTHotkey :)
or a newer api .. seee also: How to implement shortcut key input in Mac Cocoa App?

Related

How to use System KeyChain along with AccessGroup?

I want to have my keys stored in System Keychain and a couple of apps(keySharing and AppGroup enabled) to access them
My current approach
I am using
 let path:String = "/Library/Keychains/System.keychain"
 var keychain: SecKeychain?
 SecKeychainOpen(path, &keychain)
For options in SecItemAdd() ,I added
[kSecUseKeychain as String: keychain! ]
with this approach, I am able to write to System keyChain but only the current app is able to access this KeyChainItem,
kSecAttrAccessGroup
is completely ignored for this keychain item.
After some digging I found
kSecUseDataProtectionKeychain = true
is mandatory to use AccessGroup
When I included this in the options of SecAddItem(), Yes I am able to utilize access groups after this, but it is ignoring kSecUseKeychain now. Though my app is running in root and kSecUseKeychain is set, SecAddtem() is writing to login KeyChain (and apps inside my access are able to access them) instead of System Keychain
Is there a way to write to System KeyChain and allow a set of apps to modify them?
In short
[kSecUseKeychain as String: keychain! ]
is ignored when
kSecUseDataProtectionKeychain = true
Thanks
Any alternatives related to having keys in System Context is much appreciated
Xcode -12.2
macOS :10.15

How to use system functionality programmatically via swift?

If we look on hotkeys/shortcuts of the macOS, we can find there a lot of functionality. Some functionality is assigned to hotkeys, some is not.
As example:
make screenshot(cmd+shif+F5)
or to show desktop (hide all apps) (F11)
and lot of other built-in into the OS functions:
Question is the following: how to use this functionality from my code without using of hotkeys pressing imitation?
The Goal is do not write custom code in case of such functionality already excist in OS.
I have found only how to open some sort of system preferences, maybe this is direction that I need to go, but I didn't found any details via google.
import Cocoa
// Open Siri prefs
NSWorkspace.shared.open(URL(fileURLWithPath: "/System/Library/PreferencePanes/Speech.prefPane"))
// Open Siri prefs with another way
NSWorkspace.shared.open(URL(fileURLWithPath: "x-apple.systempreferences:com.apple.preference.speech"))
// Open System Preferences window
NSWorkspace.shared.open(URL(fileURLWithPath: "x-apple.systempreferences:com.apple.preference"))

The area you are looking at is System Services, and actually there is no one answer on your question, because some of those items available via own API, some of them are private macOS features, some of them available via actually services.
You can start your findings with:
BOOL NSPerformService(NSString *serviceItem, NSPasteboard *pboard)
and with Services Implementation Guide, which depicts details of that.
The other mechanism to use those things was AppleScript... very wide area.

Swift 5.2 UIAccessibility.isAssistiveTouchRunning never returns true nor does UIAccessibility.assistiveTouchStatusDidChangeNotification get triggered

I would like to know if AssistiveTouch is enabled or not, because my App wants to give the user instructions how to set it up properly for a certain use case.
I tried getting UIAccessibility.isAssistiveTouchRunning but it never returns true. I even put it in a timer and switched AssistiveTouch on and off but it always returns false. I then added UIAccessibility.assistiveTouchStatusDidChangeNotification like so:
NotificationCenter.default.addObserver(forName: UIAccessibility.assistiveTouchStatusDidChangeNotification, object: self, queue: OperationQueue.current, using: { notification in
print(notification)
})
but this never gets triggered.
What am I missing? Is this feature not available?
Greetings krjw
EDIT 1
I tried isVoiceOverRunning and it works. I can't seem to get further with this.

I've encountered same behaviour. I've opened a TSI with Apple and as for their response - It only works if Guided Access is also turned on and active. I'm trying to figure out if there's any way we can determine if only assistive touch is running. Will update ...
UPDATE
Apple has responded that in order for it to work, the settings option "GUIDED ACCESS" has to be also enabled. Which makes no sense as there's no relation between the two features.....
I'll send a feedback via Feedback Assistant and request an option to know if ONLY assistive touch is running.

Error 1002 in applescript that uses "key code" when called from macOS app

I'm trying to run an AppleScript from my macOS app (Mojave, Xcode 10.3). However, when the AppleScript is run from the app, I get this error:
/Users/my_username/Desktop/test_apple_script.scpt: execution error: System Events got an error: osascript is not allowed to send keystrokes. (1002)
Based on other posts, I have made sure that my app can control my computer (System Preferences > Security & Privacy > Privacy > Accessibility), and it also has control over System Events (System Preferences > Security & Privacy > Privacy > Automation).
Here is my AppleScript:
tell application "System Events"
key code 48 using {command down}
end tell
The script works when run in Script Editor (I only had to give Script Editor the access in System Preferences > Security & Privacy > Privacy > Accessibility).
This is the code I am using to run the Apple Script from my app (via osascript):
let proc = Process()
proc.launchPath = "/usr/bin/env"
proc.arguments = ["/usr/bin/osascript","/Users/my_username/Desktop/test_apple_script.scpt"]
proc.launch()
This code works perfectly when I run a simple AppleScript that just displays a notification:
tell application "System Events"
display dialog "Test"
end tell
How can I make it so that my AppleScript runs from my macOS app just as it does when I run it in Script Editor?
Edit: For some more context, the command tab case here is just an example. I am trying to find a generalizable approach to simulating keyboard shortcuts by clicking buttons in my app (e.g., the user clicks on the "cut" button and the app acts as if the user has just pressed command + x). There may be a better approach to this than using an AppleScript, but I know that it is possible to do what I want using AppleScripts.
Edit: In case it wasn't clear from my original question, the simple AppleScript that displays a notification was already working from within my app. The problem arises when I try to run an AppleScript that simulates user input from the keyboard.
Edit: This code here worked for me temporarily, but it no longer works. I did not change the code at all. I simply ran my app again.
let PlayPauseScript = "tell application \"Spotify\"\n playpause\n end tell"\"test\"\n end tell"
if let scriptObject = NSAppleScript(source: PlayPauseScript) {
scriptObject.executeAndReturnError(nil)
}
Edit:
Here is where the problem currently stands. Based on suggestions, I added this to my info.plist, and now the Spotify AppleScript consistently works (when called via NSAppleScript):
<key>NSAppleEventsUsageDescription</key>
<string>...this is why I need permission...</string>
It seems my problem (now) is just with AppleScripts that use System Events. Despite making sure that both my app and osascript are allowed to control my computer (via security & privacy > privacy > accessibility), I now get the same error (1002) when using NEAppleScript and osascript. My app is also allowed to control System Events (security & privacy > privacy > automation). The osascript error prints to the console in Xcode, and I uncovered the error code for NEAppleScript by using this code (in the #IBAction of the button):
let script =
"""
try
tell application "System Events"
keystroke "c" using {command down}
end tell
on error errMsg number errorNumber
display dialog "An unknown error occurred: " & errorNumber as text
end try
"""
if let scriptObject = NSAppleScript(source: script) {
scriptObject.executeAndReturnError(nil)
}
The above code caused this to appear in the dialog box:
An unknown error occurred: 1002
When my app is run, it does prompt me to give it control over System Events, which I always do. My app is not sandboxed. I believe that both NSAppleScript and oascript did work for me once, but stopped working when I ran the app again (despite my not having made any changes to my code, I'm almost positive).
I don't know if this is informative, but the only other clue I have as to what might be going on is that I keep having to check and uncheck Script Editor's permission in security & privacy > privacy > accessibility to get it to run these AppleScripts because it tells me that Script Editor is not allowed to send keystrokes.

Security & Privacy -> Privacy tab -> Accessibility
Add /usr/bin/osascript
NOTE: You may also need to add your terminal (iTerm in my case)

To fix osascript 1002 permission issue.
Got to Security & Privacy -> Privacy tab -> Accessibility -> Add osascript (/usr/bin/osascript)
Or Use NSAppleScript

This is a couple of things for you to consider, more than a real answer to your question. If you'd like more details, I'll happily edit them in on request. (also, I'll put code examples in objective-C, because I'm still learning swift)
First, if you're going to be doing a lot of scripting in your app, it may be worth your while to switch over to Scripting Bridge. Scripting Bridge is a cocoa interface for applescripting other apps, so (to steal an example I found elsewhere) if you want to get the name of the current track in iTunes, you'd do something like:
iTuneApplication *iTunes = [SBApplication applicationWithBundleIdentifier:#"com.apple.iTunes"];
currentTrack = iTunes.currentTrack.name;
If you want specific code for Spotify I'll have to install that app; let me know.
Second, if you want to do keystrokes, you should look into Quartz Event Services. QES allows you to create, send, and capture input events. So if you want to send a ⌘C, you might do something like:
// '8' is the key code for 'c'
comCDownEvent = CGEventCreateKeyboardEvent(nil, 8, true);
CGEventSetFlags(comCDownEvent, kCGEventFlagMaskCommand);
comCUpEvent = CGEventCreateKeyboardEvent(nil, 8, false);
CGEventSetFlags(comCUpEvent, kCGEventFlagMaskCommand);
CGEventPostToPSN(targetAppProcessSerialNumber, comCDownEvent);
CGEventPostToPSN(targetAppProcessSerialNumber, comCUpEvent);
You can't run this code from sandboxed apps, but I think you'll have that problem any way you go.
The main advantage of these approaches is that they avoid switching contexts: it's all done from cocoa, so you don't need to invoke a shell or create an AppleScript environment, and avoid all the translation problems that involves.

Swift 3 osx - Popup every time I use SecIdentitySetPreferred

I am editing a lot of identity preferences in a swift 3 app. For every single domain, there is an annoying popup: "[app name] wants to access key "[domain]" in your keychain. Do you want to allow access to this item?".
How do I get rid of this popup / automatically give my app the right access any keys in my keychain?
Edit: my question is not the same as this question because I need a programmatic solution. How do I change that setting programmatically with either Swift or AppleScript? Thanks.