I was googling a lot in order to find a solution for my problems with UI Automation. I found a post that nice summarizes the issues:
There's no way to run tests from the command line.(...)
There's no way to set up or reset state. (...)
Part of the previous problem is that UI Automation has no concept of discrete tests. (...)
There's no way to programmatically retrieve the results of the test run. (...)
source: https://content.pivotal.io/blog/iphone-ui-automation-tests-a-decent-start
Problem no. 3 can be solved with jasmine (https://github.com/pivotal/jasmine-iphone)
How about other problems? Have there been any improvements introduced since that post (July 20, 2010)?
And one more problem: is it true that the only existing method for selecting a particular UI element is adding an accessibility label in the application source code?
While UI Automation has improved since that post was made, the improvements that I've seen have all been related to reliability rather than new functionality.
He brings up good points about some of the issues with using UI Automation for more serious testing. If you read the comments later on, there's a significant amount of discussion about ways to address these issues.
The topic of running tests from the command line is discussed in this question, where a potential solution is hinted at in the Apple Developer Forums. I've not tried this myself.
You can export the results of a test after it is run, which you could parse offline.
Finally, in regards to your last question, you can address UI elements without assigning them an accessibility label. Many common UIKit controls are accessible by default, so you can already target them by name. Otherwise, you can pick out views from their location in the display hierarchy, like in the following example:
var tableView = mainWindow.tableViews()[0];
As always, if there's something missing from the UI Automation tool that is important to you, file an enhancement request so that it might find its way into the next version of the SDK.
Have you tried IMAT? https://code.intuit.com/sf/sfmain/do/viewProject/projects.ginsu . It uses the native javascript sdk that Apple provides and can be triggered via command line or Instruments.
In response to each of your questions:
There's no way to run tests from the command line.(...)
Apple now provides this. With IMAT, you can kick off tests via command line or via Instruments. Before Apple provided the command line interface, we were using AppleScript to bring up Instruments and then kick off the tests - nasty.
There's no way to set up or reset state. (...)
Check out this state diagram: https://code.intuit.com/sf/wiki/do/viewPage/projects.ginsu/wiki/RecoveringFromTestFailures
Part of the previous problem is that UI Automation has no concept of discrete tests. (...)
Agreed. Both IMAT and tuneup.js (https://github.com/alexvollmer/tuneup_js#readme) allow for this.
There's no way to programmatically retrieve the results of the test run. (...)
Reading the trailing plist file is not trivial. IMAT provides a jUnit like report after a test run by reading the plist file and this is picked up by my CI Tool (Teamcity, Jenkins, CruiseControl)
Check out http://lemonjar.com/blog/?p=69
It talks about how to run UIA from the command line
Try to check the element hierarchy, the table can be placed over a UIScrollView.
var tableV = mainWindowTarget.scrollViews()[0].tableViews()[0].scrollToElementWithName("Name of element inside the cell");
the above script will work even the element is in 12th cell(but the name should be exactly the same as mentioned inside the cell)
Related
Is there a replacement for the RadioInfo that was removed as of OS11? (com.android.settings/.RadioInfo)
If not, where else can this information be found in UI? Or adb? (preferably without rooting the device)
Our team relies heavily on the use of the data in RadioInfo for QA testing, especially since it works on nearly all Android devices (rather than being OEM-dependent like engineering short codes).
It also offered ideal granularity in network selection, moreso than the basic Settings UI.
Also, why was it removed? I looked back about 11 months in logs and didn't see a single mention of it (though maybe I'm looking in the wrong place - if there's a comment on it somewhere, please do share the link).
Looks like it just got moved.
10 and earlier:
com.android.settings/.RadioInfo
as of 11:
com.android.phone/.settings.RadioInfo
I am using the UI Automation add-in to automate and test an application that contains HTML objects in a Java window. I have the UIATable identified & saved in my object repository and the following methods work fine:
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").RowCount 'Prints 3
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").ColumnCount 'Prints 5
However, when I try to get cell value using any of the below methods:
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").GetCellValue(1,1) 'Error
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").GetCellData(1,1) 'Error
MsgBox UIAWindow("**").UIAObject("**").UIATable("**").GetCellName(1,1) 'Error
I get an error pop up with the following message:
The test run cannot continue due to an unrecoverable error.
<0x80070057> Failed due to a lacking or broken API call inherited from
UI Automation.
I am using UFT 14.02. What might be the possible reason for this error and is there something I can do to resolve this?
Have a look at the UFT 14 Product Availability Matrix. You want the section "UFT GUI Testing UI Automation Add-in".
JavaFX is supported but HTML is not supported by the UI Automation Framework in UFT.
That might be why some methods work and others do not. i.e. You can read the java table, but cannot validate the html content.
(I assume you are testing against a javaFX application? - you just say java)
It's worth saying that "Not Supported" doesn't mean it will not work, just that it's not been fully tested and certified by Microfocus.
Additionally, if you check the support pages it has a big note:
Note: The test objects and methods available are completely dependent on the properties and patterns implemented in your application. We recommend that you familiarize yourself with the properties of your application's objects - specifically the Control Type IDs and supported patterns to understand what test objects and methods you can use.
So the error might not be you, and might not be UFT. It might be a result of the delivery of the application under test.
Things you can try...
Try the actual java add-in - it is possible to use multiple add-ins concurrently - even if it's a work around for just one object.
Try the standard windows object identifiers.
Confirm the application is built to support Microsoft's UI Automation
Update to the latest UFT (UFT 15.01 at time of writing, now also called UFT One) to make sure your libraries are as up to date as possible
If all that fails let me know. UFT is very flexible around the GUI and depending on how you need to interact with the table there are some other solutions we can try.
I have added a 'number_of_members' value to the Customer DocType via customization.
In my application I have tried several ways to update the value. However the value never updates in the webpage. I feel like I'm missing some sort of save or update or commit step.
For example I have tried:
frappe.client.set_value('Customer', '00042', 'number_of_members', 8887)
frappe.set_value('Customer', '00042', 'number_of_members', 8887)
frappe.db.set_value('Customer', '00042', 'number_of_members', 8887)
and also
customer = frappe.get_doc('Customer', '00042')
customer.number_of_members = 8887
customer.save()
In each case I can do something like frappe.get_value, or frappe.get_doc and it shows the value is set to 8887. However it never updates in the web side. This is what makes me think I'm updating some sort of cache or database transaction and I need some way to save it, but have not had any luck.
I am mostly testing this via bench console if that has any bearing on it, but I've tried a couple of the methods in my application code as well.
Relevant documentation:
Frappe Developer API - Document
Frappe Developer API - Database
Turns out the answer is to call frappe.db.commit() after making changes. If someone can point this out in the documentation so I can better understand how I'm missing stuff, I would appreciate it.
I also noticed if you try to Save something in the UI before you send frappe.db.commit() the UI will hang.
I have a single-page web app that presents a multi-step photo management "wizard", split up across several discrete steps (photo upload, styling, annotation, publishing) via a tab strip. On switching steps I set the URL hash to #publishing-step (or whichever step was activated).
How do I set up Optimizely tests to run on the various discrete steps of the wizard?
The browser never leaves the page, so it only gets a single window.load event. Its DOM isn't getting scrapped or regenerated, but just switches what page elements are visible at any one time via display: none or block, so the part I am trying to figure out is really mostly about in what way I go about the Optimizely test setup itself - it's fine (and likely necessary) if all edits get applied at once.
This thing unfortunately has to work in IE9, so I can't use history.pushState to get pretty discrete urls for each step.
There's actually several ways you could go about doing this, and which option you choose will largely depend on what's easiest for you AND how you plan to analyze the data.
If you want to use Optimizely's analytics dashboard:
I would recommend creating one experiment which will activate a bunch of other experiments at different times. The activation experiment will be targeted to everyone and run immediately when they get to your wizard. The other experiments will be set up with manual activation and triggered by this experiment.
The activation experiment would have code like:
window.optimizely = window.optimizely || [];
function hashChanged() {
if(location.hash === 'publishing-step') {
window.optimizely.push(['activate', 0000000000]);
}
if(location.hash === 'checkout-step') {
window.optimizely.push(['activate', 1111111111]);
}
}
window.addEventListener('hashchange', hashChanged, false);
Or you could call window.optimizely.push(['activate', xxxxxxxxx]); directly from your site's code instead of creating an activation experiment and listening for hashchange.
If you want to use a 3rd party analytics tool like Google Analytics:
You could do this all in one experiment with code similar to above, but in each "if" section instead of activating an experiment, you could run your variation code that makes changes to the wizard and sends special tracking information to your analytics sweet for later reporting. You'll have to do your own statistical significance calculation for this method (as Optimizely's data won't be "clean"), but this method actually works out better usually if properly configured.
Alternatively you could use the method outlined above but still try to use the Optimizely analytics dashboard by creating custom events on your experiment and sending data to them using calls like window.optimizely.push(["trackEvent", "eventName"]);
This article may also be helpful to you.
You'll probably need to do this yourself, using Optimizely's JS API to trigger actions on their end and tell it what your users did: https://www.optimizely.com/docs/api
I eventually couldn't get any further with my program due to the various shortcomings of VB.NET (bad audio support, no reading events in the middle of execution, very weak keyboard input, etc). So I tried SDL.NET 6.1.
Despite its terrible documentation, I was able to fix my code to use it and I love it!
But there's a problem. I don't know how to set up my application settings for it. The Startup Object definitely should be a class (the examples always are in classes, never modules), but a startup class specifically has to be a form! This is bad because SDL makes its own window via SetVideoMode; you don't need a form. So when the form constructor New() finishes, a useless form is created and you have two windows.
I tried placing a call to the game engine loop within New() so that the game starts up without New() ever finishing. The game runs normally, and this solves the "second window" problem... but it can't be closed! X button does nothing, calls to Events.QuitApplication or Me.Close are blatantly ignored, etc.
I'm stumped. It seems I need to set a non-form class as the startup object, but it won't let me.
Oh, by the way, it seems that there are two things called "SDL NET". To clarify, I'm using this one, which exists in the SdlDotNet namespace.
Oh, I forgot to mention, I also noticed that a lot of the examples have a line that says "[STAThread]". Is this is important?
EDIT:
I've already received and accepted an answer for my question, but I want to tell other people what the problem is with exiting/closing the app, even though that wasn't my question:
While SDL.NET allows you to receive input and run other events without having to stop running logic, the application still cannot quit while logic is being run. So I find the best way to tell your SDL.NET application to Quit in the middle of running logic is to use the following TWO lines:
SdlDotNet.Core.Events.QuitApplication
End
Place these in the handler for the SdlDotNet.Core.Events.Quit event, as well as anywhere else you want your program to quit.
The Startup Object definitely should be a class (the examples always are in classes, never modules)
Here's your mistake. There's no real difference between a class and a VB module from CLR perspective. So just make it a module with Main and go on. There is no need for a class. I suspect you're looking at C# examples, which use classes - but that's because there is no such thing as a module in C#.
[STAThread] probably won't make any difference for SDL. It is important for UI applications (both WinForms and WPF require it), but I don't think that SDL does any COM calls, so it shouldn't care whether your thread is STA or not. It's just something that Visual Studio puts on Main in new projects by default.