I am writing a voice command in advanced scripting in Dragon NaturallySpeaking. I would like to access the last element of the recognition history (e.g., to place it in the clipboard, or repeat). How cannot achieve that?
For example, given this recognition history:
I would like to have a voice command so that when I say repeat Dragon NaturallySpeaking sends the keys show recognition history.
Well, I don't know if this is what you intend, but whatever the command name is will be the last recognition. Do you mean the recognition before that one? That is, the one you said before you say the command? In that case, try the following, but be advised, it's not like saying the phrase again. It's literally copying out what Dragon heard you say (recognized). So if you say at the start of a sentence "this is a test period" to have Dragon type out "This is a test.", your new command will type out "this is a test period".
Sub Main
Dim engine As New DgnEngineControl
engine.DlgShow(dgndlgRecognitionHistory,0,,0) ' Call up Recognition History
Wait .5 ' Need a short delay
SendKeys "{Up}", True ' move up to prior utterance
SendKeys "{Tab}", True ' move to utterance selection
Wait .3
SendKeys "^c", True ' Copy to the clipboard
Wait .5 ' Need a short delay for clipboard
SendKeys "{Esc}", True ' Close recognition history
Wait .3
SendKeys "^v", True ' Paste from clipboard
End Sub
To convert the recognition to a newly recognized utterance requires some additional work.
Hth
Related
I am having some serious struggles fully grasping the control on activating windows and forcing their focus and foremost position.
In order to debug a larger script I made a separate script to test the use of WinActivate and again I am observing frustrating behaviour as it either all together ignores the title I have defined or is failing in some other way. In the smaller test script I am simply requesting that the window in which the hotkey was triggered be set as active after another action, specifically an input box
Below is the simple code for testing:
F10::
SetTitleMatchMode, 1
DetectHiddenWindows, Off
WinGetTitle, startTitle, A
msgbox % "Start Title = <" . startTitle . ">"
;WinActivate, startTitle
inputbox, mode, Test box, Testing,,260,160
sleep 500
WinActivate, startTitle
Return
This code does not properly activate the starting window. For example I execute the hotkey in an empty notepad window and upon submitting blank into the input box the focus becomes notepad++ on my second monitor. The second time I press the hotkey from within notepad (or another application) notepad does not lose focus. In a third execution I begin from notepad again and after the input box appears I switch the focus to another window. I again submit blank to the inputbox but that new window remains the focus and notepad is not activated or brought to the foremost position.
Can someone please explain to me what is going on with WinActivate?
I was having similar frustration with unexpected results making a windows script host file and I think I must be missing some fundamental detail in windows.
You are trying to activate a window that start with the literal text "startTitle".
You forgot(?) to either enter expression syntax with % or use the legacy way of referring to a variable %startTitle% (please don't use legacy).
Extra stuff:
You shouldn't specify SetTitleMatchMode and DetectHiddenWindows inside your hotkey statement. There is no need (unless there actually is) to set those every time you hit the hotkey. Just specify them at the top of your script once.
Both of them are useless for you though, below I'll show why. Also DetectHiddenWindows is already off by default.
WinGetTitle is not good to use for this. What you actually want to do is get the hwnd of the window you wish by using e.g. WinExist().
And then refer to the window by its hwnd. Much better than working with window titles, and impossible to match the wrong window as well. To refer to a window by its hwnd, you specify ahk_id followed by the hwnd on a WinTitle parameter.
And lastly, the concatenation operator . is redundant. Of course you may prefer to use it, but in case you didn't know, it can just be left out.
Here's your revised code:
F10::
_HWND := WinExist("A")
MsgBox, % "Start hwnd = <" _HWND ">"
InputBox, mode, Test box, Testing,,260,160
Sleep, 500
WinActivate, % "ahk_id " _HWND
Return
So, I want to detect if a user writes a specific sentence on the keyboard, and if so afterwards perform a function.
For example, if the user writes:
if user types /hello
script activates and for example puts out
Send, Hello. How are you today?
According to the documentation for Hotstrings, your example would look like this:
::if user types /hello::Send, Hello. How are you today?
Although, this requires typing an additional "ending character" such as a space or tab to trigger the transformation.
To immediately replace the text after entering the last character you add the * option:
:*:if user types /hello::Send, Hello. How are you today?
In Matlab I can start external .exe files that sometime have a pop up that requires an enter key pressed. For example:
system('C:\Program Files (x86)\WinZip\WINZIP32.EXE')
will start Winzip, and then in order to use it you need to pass the "buy now" pop up window by pressing enter.
Now my problem is not with winzip, I only gave it as an example (i use winrar anyway :).
How can I programmatically press an enter key in Matlab in such cases ? (I use win 7)
Can an event listener be used to solve that?
EDIT: The java.awt.Robot class indeed works on explorer, but not on any software that has a pop up window with an OK button that needs to be pressed. I don't know why it doesn't work for that. I gave the winzip example because I assume everybody has winzip/winrar installed in their machine. The actual software I have is different and irrelevant for the question.
There is a way using Java from Matlab, specifically the java.awt.Robot class. See here.
Apparently there are two types of programs, regarding the way they work when called from Matlab with system('...'):
For some programs, Matlab waits until the program has finished before running the next statement. This happens for example with WinRAR (at least in my Windows 7 machine).
For other programs this doesn't happen, and Matlab proceeds with the next statement right after the external program has been started. An example of this type is explorer (the standard Windows file explorer).
Now, it is possible to return execution to Matlab immediately even for type 1 programs: just add & at the end of the string passed to system. This is standard in Linux Bash shell, and it also works in Windows, as discussed here.
So, you would proceed as follows:
robot = java.awt.Robot;
command = '"C:\Program Files (x86)\WinRAR\WinRAR"'; %// external program; full path
system([command ' &']); %// note: ' &' at the end
pause(5) %// allow some time for the external program to start
robot.keyPress (java.awt.event.KeyEvent.VK_ENTER); %// press "enter" key
robot.keyRelease (java.awt.event.KeyEvent.VK_ENTER); %// release "enter" key
If your applications are only on Windows platform, you can try using .net objects.
The SendWait method of the SendKeys objects allows to send virtually any key, or key combination, to the application which has the focus, including the "modifier" keys like Alt, Shift, Ctrl etc ...
The first thing to do is to import the .net library, then the full syntax to send the ENTER key would be:
NET.addAssembly('System.Windows.Forms');
System.Windows.Forms.SendKeys.SendWait('{ENTER}'); %// send the key "ENTER"
If you only do it once the full syntax is OK. If you plan to make extensive use of the command, you can help yourself with an anonymous helper function.
A little example with notepad
%% // import the .NET assembly and define helper function
NET.addAssembly('System.Windows.Forms');
sendkey = #(strkey) System.Windows.Forms.SendKeys.SendWait(strkey) ;
%% // prepare a few things to send to the notepad
str1 = 'Hello World' ;
str2 = 'OMG ... my notepad is alive' ;
file2save = [pwd '\SelfSaveTest.txt'] ;
if exist(file2save,'file')==2 ; delete(file2save) ; end %// this is just in case you run the test multiple times.
%% // go for it
%// write a few things, save the file then close it.
system('notepad &') ; %// Start notepad, without matlab waiting for the return value
sendkey(str1) %// send a full string to the notepad
sendkey('{ENTER}'); %// send the {ENTER} key
sendkey(str2) %// send another full string to the notepad
sendkey('{! 3}'); %// note how you can REPEAT a key send instruction
sendkey('%(FA)'); %// Send key combination to open the "save as..." dialog
pause(1) %// little pause to make sure your hard drive is ready before continuing
sendkey(file2save); %// Send the name (full path) of the file to save to the dialog
sendkey('{ENTER}'); %// validate
pause(3) %// just wait a bit so you can see you file is now saved (check the titlebar of the notepad)
sendkey('%(FX)'); %// Bye bye ... close the Notepad
As explained in the Microsoft documentation the SendKeys class may have some timing issues sometimes so if you want to do complex manipulations (like Tab multiple times to change the button you actually want to press), you may have to introduce a pause in your Matlab calls to SendKeys.
Try without first, but don't forget you are managing a process from another without any synchronization between them, so timing all that can require a bit of trial and error before you get it right, at least for complex sequences (simple one should be straightforward).
In my case above for example I am running all my data from an external hard drive with an ECO function which puts it into standby, so when I called the "save as..." dialog, it takes time for it to display because the HDD has to wake up. If I didn't introduce the pause(1), sometimes the file path would be imcomplete (the first part of the path was send before the dialog had the focus).
Also, do not forget the & character when you execute the external program. All credit to Luis Mendo for highlighting it. (I tend to forget how important it is because I use it by default. I only omit it if I have to specifically wait for a return value from the program, otherwise I let it run on its own)
The special characters have a special code. Here are a few:
Shift +
Control (Ctrl) ^
Alt %
Tab {TAB}
Backspace {BACKSPACE}, {BS}, or {BKSP}
Validation {ENTER} or ~ (a tilde)
Ins Or Insert {INSERT} or {INS}
Delete {DELETE} or {DEL}
Text Navigation {HOME} {END} {PGDN} {PGUP}
Arrow Keys {UP} {RIGHT} {DOWN} {LEFT}
Escape {ESC}
Function Keys {F1} ... {F16}
Print Screen {PRTSC}
Break {BREAK}
The full list from Microsoft can be found here
There is a small javascript utility that simulates keystrokes like this on the Windows javascript interpreter.
Just create a js file with following code:
var WshShell = WScript.CreateObject("WScript.Shell");
WshShell.SendKeys(WScript.Arguments(0));
then call it from Matlab after the necessary timeout like this:
system('c:\my\js\file\script.js {Enter}');
Can't test here now, but I think this should work...
If you need to run a console-only program in a context that permits full DOS redirection, you can create a file called, say, CR.txt containing a carriage return and use the '<' notation to pipe the value into the program.
This only works if you can provide all the keyboard input can be recorded in the file. It fails dismally if the input has to vary based on responses.
An alternative is to duplicate the input (and possibly output) stream(s) for the program and then pipe data into and out of the program. This is more robust and can permit dynamic responses to the data, but will also likely require substantial effort to implement a robot user to the application.
Rog-O-Matic is an example of a large application completely controlled by a program that monitors screen output and simulates keyboard input to play an early (1980s) ASCII graphic adventure game.
The other responses will be required for GUI-based applications.
Python package pywinauto can wait any dialog and click buttons automatically. But it's capable for native and some .NET applications only. You may have problems with pressing WPF button (maybe QT button is clickable - not checked), but in such case code like app.DialogTitle.wait('ready').set_focus(); app.DialogTitle.type_keys('{ENTER}') may help. Your case is quite simple and probably some tricks with pywinauto are enough. Is your "app with popup" 64-bit or 32-bit?
wait and wait_not functions have timeout parameter. But if you need precisely listener with potentially infinite loop awaiting popups, good direction is global Windows hooks (pyHook can listen mouse and keybd events, but cannot listen dialog opening). I'll try to find my prototype that can detect new windows. It uses UI Automation API event handlers... and... ops... it requires IronPython. I still don't know how to set UI Automation handler with COM interface from standard CPython.
EDIT (2019, January): new module win32hooks was implemented in pywinauto a while ago. Example of usage is here: examples/hook_and_listen.py.
Okay, I need to add some emails to my Daylite tasks automatically based on certain criteria, so I set up a Mail.app rule and an AppleScript to accomplish this. There's only one problem: when the AppleScript gets the content of the message, it adds an 'a' at the end (and makes a beep as well, letting me know it tried some keystroke that apparently wasn't allowed).
Here's what the AppleScript looks like:
using terms from application "Mail"
on perform mail action with messages theMessages for rule theRule
tell application "Mail"
repeat with eachMessage in theMessages
try
set this_subject to subject of eachMessage
set this_body to content of eachMessage
if this_subject is "" then error
on error
set this_subject to "No subject."
end try
activate application "Daylite"
tell application "System Events"
keystroke "t" using {command down, shift down}
keystroke this_subject
keystroke tab
keystroke this_body
keystroke "s" using command down
end tell
end repeat
end tell
end perform mail action with messages
end using terms from
This results in me getting a new task in Daylite, where the task title is the email Subject, and is displayed correctly, but the attached note, which is the email's body, is not displayed correctly.
So, if I receive an email that matches my rule, and the email is:
From: Hypothetical Wife
Subject: Don't forget to take out the trash!
Body: If you forget one more time, I'm filing for a divorce.
That ends up with a task that looks like this:
☐ Don't forget to take out the trash!
If you forget one more time, I'm filing for a divorce.a
… making my hypothetical wife in this scenario sound like a barely literate Canadian. (Plus there's an system alert sound, letting me know it apparently tried to type something that's not allowed.)
If the body has multiple lines of text, it also annoyingly removes the line-breaks.
I know I could set it to:
set the clipboard to this_body
and then replace
keystroke this_body
with
keystroke "v" using command down
…but that's a really inelegant solution and I don't want to replace whatever's in my clipboard every time the rule runs.
What am I doing wrong here? I've tested this in TextEdit as well, and the same problem occurs there too, so it's not a Daylite problem. I've also tried appending "as Unicode text" or "as string" or "as «class UTF8»" to the end of the "set this_body…" line, but none of those fixed the problem either.
Please, explain it to me like I'm a complete idiot because, as a complete idiot, that's the only way I'll understand you. Thanks.
One of the basic mistakes many people make is telling an application to do things that the application does not know how to do. You should only tell an application to do things that you can find in its applescript dictionary because that's really all the application knows how to do... besides some basic general purpose applescript stuff like performing a repeat loop.
You have fallen into this trap. You are telling Mail to tell Daylight and System Events to do things because all of that code is inside the "tell application Mail" block of code. This type of error often leads to problems that are very difficult to track down.
As such, my first suggestion would be to stop telling Mail to do unnecessary things. Here's how I would write your code to separate the different application commands from each other.
using terms from application "Mail"
on perform mail action with messages theMessages for rule theRule
repeat with eachMessage in theMessages
tell application "Mail"
set this_subject to subject of eachMessage
set this_body to content of eachMessage
end tell
if this_subject is "" then set this_subject to "No subject."
activate application "Daylite"
delay 0.2
tell application "System Events"
tell process "Daylite"
keystroke "t" using {command down, shift down}
delay 0.2
keystroke this_subject
delay 0.2
keystroke tab
delay 0.2
keystroke this_body
delay 0.2
keystroke "s" using command down
end tell
end tell
end repeat
end perform mail action with messages
end using terms from
You'll also notice I added a small delay between some of your code. Sometimes the code runs more quickly than the computer interface can handle, so that too may be a source of your error. You can also try increasing the length of the delay if needed (any delay over 1 second is overkill and shouldn't be necessary).
If these changes and a longer delay don't work then a simple solution would be to check if the text ends with "a" and just remove it. Something like this between the keystroke tab and keystroke this_body lines...
if this_body ends with "a" then set this_body to text 1 thru -2 of this_body
Good luck!
EDIT: the "keystroke" process doesn't seem to like the return characters from the email. I believe that's what causing the noise... when it hits a return character it beeps. Anyway, you can try something like this. Just open a blank TextEdit document and select an email in Mail and run the code in AppleScript Editor. You can incorporate this idea in your Daylight script.
tell application "Mail"
set eachMessage to item 1 of (get selection)
set this_subject to subject of eachMessage
set this_body to content of eachMessage
end tell
set bodyList to paragraphs of this_body
activate application "TextEdit"
tell application "System Events"
tell process "TextEdit"
repeat with i from 1 to count of bodyList
keystroke (item i of bodyList)
keystroke return
end repeat
end tell
end tell
I am writing an Advanced Scripting voice command in Dragon NaturallySpeaking Professional 11.5 to write some MATLAB usual figure commands by voice, namely:
title('')
xlabel('')
ylabel('')
As I would like to write these three lines in one voice command, and the fastest way two output text using Advanced Scripting is to put the text within the keyboard and press Ctr+v, I'm looking for a way to represent new line characters in the Clipboard.
My current script is the following:
Sub Main
Clipboard("title(''); xlabel('') ; ylabel('');")
SendKeys "^v"
SendKeys "{Enter}"
End Sub
and output:
title(''); xlabel('') ; ylabel('');
vbCrLf will add a new line:
Sub Main
Clipboard("title('')" & vbCrLf & "xlabel('')" & vbCrLf & "ylabel('')")
SendKeys "^v"
SendKeys "{Enter}"
End Sub
will output:
title('')
xlabel('')
ylabel('')