Greetings and salutations!
I'm working on a UI automation project for a windows desktop app (FrameworkId: Win32)
Stack: Python (3.7) + Appium (1.15.1) + WinAppDriver (v1.1).
I have identified an element using Inspect.exe, but when I try to code, whatever I do I receive this error:
selenium.common.exceptions.NoSuchElementException: Message: An element could not be located on the page using the given search parameters.
The locator strategy I'm using is xpath:
self.driver.find_element_by_xpath("//*[#LocalizedControlType='text' and #IsControlElement='false']")
As you can see, Inspect.exe has shown that it has the property "IsControlElement='false'", but I cannot for the life of me "access" it via code.
I would also like to point out that any elements that had the IsControlElement='true' are properly found and I can "interact" with them.
Thank you very much for your help!
Source of issue
This is probably an issue within Microsoft's UI Automation implementation in .NET.
The property IsControlElement should have returned true while it didn't.
From my tests, it seems to be an issue somewhere within UIAutomationCore.dll.
I speculate that the root cause is that the automation implementation was targeted for accessibility in mind, and they have mistakenly ignored some controls which are NOT readable (Image, Geometry, etc.).
Workaround for some cases
Try to use UI control from a type that has a text.
if it's already a textual control, try to use a different textual control type. for example - in WPF project - use Label instead of TextBlock
if it's NOT a textual element, if possible, wrap the control in a textual element. in WPF projects you can use a <Label Padding="0"> as a wrapper.
Other things to consider
Try to use UIAComWrapper
Related issues
https://stackoverflow.com/a/46452431/426315
UIAutomation won't retrieve children of an element
UI Automation - #32770 (Dialog) shows in Insepct.exe but not in VisualUIAVerifyNative.exe
Side Note
Since you haven't specified which Python GUI library are you using, I was not able to provide examples for your library. Sorry.
Related
I am using the UI Automation add-in to automate and test an application that contains HTML objects in a Java window. I have the UIATable identified & saved in my object repository and the following methods work fine:
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").RowCount 'Prints 3
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").ColumnCount 'Prints 5
However, when I try to get cell value using any of the below methods:
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").GetCellValue(1,1) 'Error
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").GetCellData(1,1) 'Error
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").GetCellName(1,1) 'Error
I get an error pop up with the following message:
The test run cannot continue due to an unrecoverable error.
<0x80070057> Failed due to a lacking or broken API call inherited from
UI Automation.
I am using UFT 14.02. What might be the possible reason for this error and is there something I can do to resolve this?
Have a look at the UFT 14 Product Availability Matrix. You want the section "UFT GUI Testing UI Automation Add-in".
JavaFX is supported but HTML is not supported by the UI Automation Framework in UFT.
That might be why some methods work and others do not. i.e. You can read the java table, but cannot validate the html content.
(I assume you are testing against a javaFX application? - you just say java)
It's worth saying that "Not Supported" doesn't mean it will not work, just that it's not been fully tested and certified by Microfocus.
Additionally, if you check the support pages it has a big note:
Note: The test objects and methods available are completely dependent on the properties and patterns implemented in your application. We recommend that you familiarize yourself with the properties of your application's objects - specifically the Control Type IDs and supported patterns to understand what test objects and methods you can use.
So the error might not be you, and might not be UFT. It might be a result of the delivery of the application under test.
Things you can try...
Try the actual java add-in - it is possible to use multiple add-ins concurrently - even if it's a work around for just one object.
Try the standard windows object identifiers.
Confirm the application is built to support Microsoft's UI Automation
Update to the latest UFT (UFT 15.01 at time of writing, now also called UFT One) to make sure your libraries are as up to date as possible
If all that fails let me know. UFT is very flexible around the GUI and depending on how you need to interact with the table there are some other solutions we can try.
I am trying to build a video player using flutter for Desktop. There is a video_player plugin available for iOS and Android, but not for Desktop. So, for the time being thought of trying to use gstreamer for decoding and hardware rendering in C++ code as back-end to flutter. The idea is to pass the Window Id of the flutter window to gstreamer's glimagesink plugin for rendering the video.
I am using the latest code from https://github.com/google/flutter-desktop-embedding as the base for my experiments. Below mentioned points are with reference to this repo.
In file flutter-desktop-embedding/example/linux/main.cc, FlutterWindowController object is created as shown below.
flutter::FlutterWindowController flutter_controller(icu_data_path);
This internally calls
FlutterDesktopInit();
While hovering the mouse pointer on the above method, VS code shows the following
bool FlutterDesktopInit()
Sets up the library's graphic context. Must be called before any other
methods.
Note: Internally, this library uses GLFW, which does not support multiple
copies within the same process. Internally this calls glfwInit, which will
fail if you have called glfwInit elsewhere in the process.
It is clear that FlutterDesktopInit() uses GLFW to create window. Checked whether I can get the window handle. But, no luck. I could only get the FlutterWindow object as shown below.
flutter::FlutterWindow *win = flutter_controller.window();
Appreciate if somebody can give some hint on how to get the GLFW window handle, which can be used with glimagesink.
You can't get references to any GLFW objects through that API. This is by design because, as the comment you quoted says, you can't have multiple copies of GLFW within the same process. GLFW is statically linked into the Linux Flutter embedding, so you can't use GLFW in the runner or a plugin.
Implementing a video player should be done using the texture APIs, which will be added for GLFW in this PR.
I'm using VS 2013 with CodedUI to automate UI tests on an application that is not built by my client (it's an implementation project). When inspecting the UI Control using inspect or coded UI, I see that the Automation ID keeps changing and I have no real way (beside position based) to capture my controls (the application is developed in Delphi).
So I'm wondering if there exist some library or add-ons (or something not even related to Coded UI and VS) that can help with this? For example some tools that can capture a screen shot of the control and then map it (the screenshot) to an Control Id that I will define and use that to automate?
Wow....I was able to find a way to do what I need using sikuli (http://www.sikuli.org/) checkout this post. Ill actually try it out tomorrow. But I found on the web (link below) that it`s possible.
From Coded UI we can call the sikuli script like that:
Process process = new Process();
ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
startInfo.FileName = #"D:\Sikuli\ds.bat";
process.StartInfo = startInfo;
process.Start();
process.WaitForExit();
(code from) https://answers.launchpad.net/sikuli/+question/232233 , read this post guys!
the problem is: my web application uses ZK, which automatically generates random UUID for each web element.
When I try to record some basic test-case with Selenium IDE, it automatically tries to use these randomly-generated ID's, without even giving me a good alternative.
Is there a way to forbid Selenium IDE to use IDs while locating elements?
Possible workaraounds:
Implement ID generator in ZK: I've thrown away this possibility, because the application GUI is too complex for this task, and ID should be unique for whole sesion, which make this workaraound really hard to implement, when you have same elements on different page.
Find another recording tool: I've only found XLT script developer, which does the work by writing DOM-path using classes (which zk gives plenty) - but sometimes the location strategy gives false path, which is then not reproducible. Any good alternatives here?
You can change locater builder by changing the order of the locater in options>locater builder.
For example if you want to give first preference to css: name drag it on the top so when you start recording it will first give the preference to css name
Hope this will help you
I was googling a lot in order to find a solution for my problems with UI Automation. I found a post that nice summarizes the issues:
There's no way to run tests from the command line.(...)
There's no way to set up or reset state. (...)
Part of the previous problem is that UI Automation has no concept of discrete tests. (...)
There's no way to programmatically retrieve the results of the test run. (...)
source: https://content.pivotal.io/blog/iphone-ui-automation-tests-a-decent-start
Problem no. 3 can be solved with jasmine (https://github.com/pivotal/jasmine-iphone)
How about other problems? Have there been any improvements introduced since that post (July 20, 2010)?
And one more problem: is it true that the only existing method for selecting a particular UI element is adding an accessibility label in the application source code?
While UI Automation has improved since that post was made, the improvements that I've seen have all been related to reliability rather than new functionality.
He brings up good points about some of the issues with using UI Automation for more serious testing. If you read the comments later on, there's a significant amount of discussion about ways to address these issues.
The topic of running tests from the command line is discussed in this question, where a potential solution is hinted at in the Apple Developer Forums. I've not tried this myself.
You can export the results of a test after it is run, which you could parse offline.
Finally, in regards to your last question, you can address UI elements without assigning them an accessibility label. Many common UIKit controls are accessible by default, so you can already target them by name. Otherwise, you can pick out views from their location in the display hierarchy, like in the following example:
var tableView = mainWindow.tableViews()[0];
As always, if there's something missing from the UI Automation tool that is important to you, file an enhancement request so that it might find its way into the next version of the SDK.
Have you tried IMAT? https://code.intuit.com/sf/sfmain/do/viewProject/projects.ginsu . It uses the native javascript sdk that Apple provides and can be triggered via command line or Instruments.
In response to each of your questions:
There's no way to run tests from the command line.(...)
Apple now provides this. With IMAT, you can kick off tests via command line or via Instruments. Before Apple provided the command line interface, we were using AppleScript to bring up Instruments and then kick off the tests - nasty.
There's no way to set up or reset state. (...)
Check out this state diagram: https://code.intuit.com/sf/wiki/do/viewPage/projects.ginsu/wiki/RecoveringFromTestFailures
Part of the previous problem is that UI Automation has no concept of discrete tests. (...)
Agreed. Both IMAT and tuneup.js (https://github.com/alexvollmer/tuneup_js#readme) allow for this.
There's no way to programmatically retrieve the results of the test run. (...)
Reading the trailing plist file is not trivial. IMAT provides a jUnit like report after a test run by reading the plist file and this is picked up by my CI Tool (Teamcity, Jenkins, CruiseControl)
Check out http://lemonjar.com/blog/?p=69
It talks about how to run UIA from the command line
Try to check the element hierarchy, the table can be placed over a UIScrollView.
var tableV = mainWindowTarget.scrollViews()[0].tableViews()[0].scrollToElementWithName("Name of element inside the cell");
the above script will work even the element is in 12th cell(but the name should be exactly the same as mentioned inside the cell)