Obviously, the answer to the question depends on a number of environmental factors.
In general, I'm wondering what people's experiences are with HtmlUnitDriver as a reliable tool that can be "trusted" to navigate a website basically the same way other browsers do.
Of course, I realize "the way other browsers do" is pretty nebulous; naturally every browser will have its quirks. But I am on a project where we have hundreds of acceptance test scenarios (written in JBehave) and using FirefoxDriver and InternetExplorerDriver, running all of them takes over two hours, which is kind of rough from a continuous integration standpoint. So I'm wondering if it's at least feasible that we could switch our acceptance tests over to use HtmlUnitDriver and expect much faster times with mostly the same behavior (and perhaps we could expect a handful of tests to fail using HtmlUnitDriver and specifically run those tests with a browser-based driver).
Our UI uses GWT, which may or may not complicate things (I don't know).
Basically, in others' experience, does HtmlUnitDriver operate about as well as another browser, or is it really only appropriate for very simple HTML websites with minimal JavaScript and should not be used for an enterprise web application?
From my experiences with using HtmlUnitDriver I would say that if you don't use it as your baseline browser when writing your tests then converting them to use it becomes a bit of a nightmare. This is especially true when it comes to javascript heavy sites.
The main reason for this is the obvious underlying use of htmlunit which, by default, uses the Rhino javascript engine. In the past I've always had to specify that HtmlUnitDriver start htmlunit using Firefox's javascript engine. This, for the most part, solved the javascript issues I was finding while running tests using HtmlUnitDriver.
One of the biggest issues I faced when it came to using the same test code for each browser was if, on the site under test, the UI developers had assigned javascript events such as onClick() to html elements such as a <span>.
The reason for this is that if you were to use WebDriver's .click() method on a WebElement representing the <span>, then htmlunit would not do anything (it expects an onClick() to be called on elements such as an <input>).
To get around this I had to manually call a click() event in javascript. You can do this either by using WebDriver's JavascriptExecutor or by using a WebDriverBackedSelenium and Selenium's .fireEvent() method.
So if your site uses such events then I'd say switching to use HtmlUnitDriver could be a big task.
Despite this, I actually use HtmlUnitDriver for all my tests. However, I went through the pains of discovering all of the above a while back, so now use HtmlUnitDriver as my baseline browser when writing tests.
Related
I was talking to a friend of mine who knows a lot about js and wasm.He told me the technology goes far beyond web, since it is basicly a way to run near native applications on devices without actually giving them access to the computer.
Which means that thrid party or untrusted code on a smartphone for instance cannot accidentally or intentional change other apps or parts of the system.
This seamed to me like the perfect conditions to build a plugin system for an application I am working on.
I asked him about it but he was unable to give me a clear answer.
So the question is, can I use webassembly outside of a webbrowser, with custom bindings to safely allow users to extend the functionality of my application (a special image viewer) without sacrificing too much speed? It seams it should work using libnode or something, but is there a problem I might run into?
I don't know how much you know about web assembly but it depends on what your plugins actually should do. If it basically handle Arrays and numeric data with not that match interacting with host applications then it might fit. But when you have heavy object handling then it will not fit at the moment. So for image processing it might be perfect match like it is used in some web examples. Also be aware that some web assembly targeting system or not suitable for none web targets as they generate also some javascript code to be used in browsers beside the generate wasm. Some wasm modules for example require that you call malloc and free for string handling other have functions like new and gc for the nearly the same.
Besides being able to leverage Angular locator methods, why would one use the Protractor testing framework instead of the Intern testing framework for Angular end to end testing?
Aside from AngularJS specific locators like by.model, by.repeater, protractor knows when the page is completely loaded, when Angular is settled down and ready - it makes the tests running naturally, there is usually no need in using explicit waits or introducing artificial delays in the testing code. In other words, it always works in sync with Angular:
You no longer need to add waits and sleeps to your test. Protractor
can automatically execute the next step in your test the moment the
webpage finishes pending tasks, so you don’t have to worry about
waiting for your test and webpage to sync.
Besides, protractor has a very convenient and rich API. It's not only wrapping WebdriverJS, but also extending it introducing new features on top. For instance, there are multiple functional programming functions available on an array of web elements, like map() or reduce(). I also like the way it allows to work with "repeaters" through rows and columns. Additionally, there is a nice Plugin API and a set of built-in plugins, like accessibility or timeline.
As a side bonus, there is a protractor-perf package that uses protractor and browser-perf for performance regression testing. You might even use your existing e2e tests as a base for performance tests wrapping the desired testing code blocks into perfRunner.start() and perfRunner.stop().
The big pro for Protractor is that it solve the asynchronicity problem by binding to AngularJS elements to check when the elements have finished loading. It also got an easyer to read syntax (if you come from a ruby background) and a lot more practical tutorial.
There is a more detailed comparrison between Intern and Protractor in this blog post here.
I am trying to find a high-level Clojure library for making HTTP and HTTPS requests, parsing out forms and links from responses and then POST-ing updated forms or following links. Ideally something that would automatically handle redirects and cookies (i.e. sessions). That is, I'd like to find something whereby my code can as closely as possible mimic a user driving a webapp from a browser, without the browser.
A number of years ago we used Hpricot and Ruby for a similar task but I'm prefer to do this in Clojure if at all possible. From memory - and I haven't used Hpricot for years - we were able to do all this with minimal effort: we were able to concentrate on the 'what' of driving the application, not the 'how'.
I found clj-http https://github.com/dakrone/clj-http but this seems to be one step lower-level than I'm looking for (no form parsing) - although it is based on Apache HttpComponents http://hc.apache.org/httpcomponents-client-ga/ which does seem to expose a nice, fluent, API for forms http://hc.apache.org/httpcomponents-client-ga/tutorial/html/fluent.html.
Screen scraping in clojure asks about screen-scraping in Clojure, and there are several good suggestions for that, but nothing that really addresses the above.
HTTP Kit http://www.http-kit.org/client.html looks like it would be a great foundation for the above but doesn't do form parsing or session management (as far as I can see).
Currently I've veering toward using the Apache HttpComponents Java library directly from Clojure. Can anyone suggest any better - perhaps more Clojure idiomatic - alternative? Or anything that they found worked well in similar circumstances? My goal is to write the minimal amount of code quickly to investigate a problem with a web service. This is not production code. Saving time, rather than getting an 'ideal' solution is my main concern.
[The background is that I am trying to mimic certain forms of user behaviour in order to first reproduce and then try and track down an intermitent bug in a large body of legacy Java/EJB code. However the problem only seems to occur one time per several thousand POSTs. (The suspicious is of some form of caching issue.) The existence of the problem, after the fact, is easy to detect however.]
Have you looked at the Enlive library yet? Here is a good tutorial on it.
You seem to really have 2 parts here. The first part is (1) a Selenium-like client, which drives (2) a webserver.
For part (1), either Selenium, Enlive, or something similar will allow you to simulate a browser to submit data, read the responses, and respond from there. For part (2), it seems you just need a regular Clojure web framework such as Ring/Compojure (older & simpler) or Pedestal (newer & more powerful).
I usually made php forms and "try" to use "good practices" in them.
I'm concerned about the real safety and error-free of that forms and I want to do some tests simulating the customer behavior, and I do it manually, but I find that is a hard work specially when the form is large and I know that there are a lot of combinations that I can't test, so usually I find bugs in the production phase.
Is there a tool that do this? I listened about Selenium, did somebody use it in the way I need? Or how can I create my own test tools that simulate user inputs at random?
User inputs implies: not filling/checking all the fields, putting in invalid data, using differents setups (no javascript, browser versions, ...), SQL injections, and I don't know so more...
You'll need to consider a combination of approaches here: good test case design, data driving those tests with various input combinations, and an automation tool such as Selenium, WebDriver, Telerik's Test Studio (commercial tool I help promote), or some other automation tool.
Design your test cases such that you're focusing on groups of behavior (a successful path case, a case validating invalid input, a case validating protection against SQL injection, etc.). Then you can look to (perhaps) data drive those test cases with sets of inputs and expected results. You can randomize that as needed through your test case code.
Most good functional automation tools support multiple browsers running the same test script, so that's a good help for hitting multi-browser testing.
Above all, start your automation efforts with small steps and focus first on high-value tests. Don't spend time trying to automate everything because that costs you a lot of time.
Selenium is used to automate browsers in exactly the way you described.
Its used for what is called Functional Testing. Where you test the external aspects of an application to ensure that they meet the specifications.
Its is most often combined with unit tests that test the internal aspects. For example to test that your application is safe against different forms of SQL injection.
Each programming language usually has several different frameworks for writing unit tests.
This are often used together this with an approach called test driven development (TDD) where you write the tests before the application code.
Our company is currently writing a GUI automation testing tool for compact framework applications. We have initially searched many tools but none of them was right for us.
By using the tool you can record test-cases and group them together to test-suites. For every test-suite there is generated an application, which launches the application-under-test and simulates user-input.
In general the tool works fine, but as we are using window handles for simulation user input, you can't do very many things. For example it is impossible for us to get the name of a control (we just get the caption).
Another problem using window handles is checking for a change. At the moment we simulate a click on a control and depending on the result we know if the application has gone to the next step.
Is there any other (simpler) way for doing such things (for example the message queue or anything else)?
Interesting problem! I've not done any low-level (think Win32) Windows programming in a while, but here's what I would do.
Use a named pipe and have your application listen to it. Using this named pipe as a communication medium, implement a real simple protocol whereby you can query the application for the name of a control given its HWND, or other things you find useful. Make sure the protocol is rich enough so that there is sufficient information exchanged between your application and the test framework. Make sure that the test framework does not yield too much "special behavior" from the app, because then you wouldn't really be testing the features, but rather your test framework.
There's probably way more elegant and cooler ways to implement this, but this is what I remember from the top of my head, using only simple Win32 API calls.
Another approach, which we have implemented for our product at work, is to record user events, such as mouse clicks and key events in an event script. This should be rich enough so that you can have the application play it back, artificially injecting those events into the message queue, and have it behave the same way it did when you first recorded the script. You basically simulate the user when you play back the script.
In addition to that, you can record any important state (user's document, preferences, GUI controls hierarchy, etc.), once when you record the script, and once when you play it back. This gives you two sets of data you can compare, to make sure for instance that everything stays the same. This solution gives you tests that not easy to modify (you have to re-record if your GUI changes), but that provide awesome regression testing.
(EDIT: This is also a terrific QA tool during beta testing, for instance: just have your users record their actions, and if there's a crash, you have a good chance of easily reproducing the problem by just playing back the script)
Good luck!
Carl
If the Automated GUI testing tool has knowledge about the framework the application is written in it could use that information to make better or more advanced scripts. TestComplete for example knows about Borland's VCL and WinForms. If you test applications build using Windows Presentation Foundation has advanced support for this build in.
use NUnitForms. I've used them with great success for single and multi threading apps and you don't have to worry about handles and stuff like that
Here are some posts about NUnitForms worth reading
NUnitForms and failed DragDrop registration - problem of MTA vs STA
Compiled application exe GUI testing with NUnitForms
I finally found a solution to communicate between the testing-application and the application-under-test: Managed Spy. It's basically a .NET application build on top of ManagedSpyLib.
ManagedSpyLib allows programmatic access to the Windows Forms controls of another process. For this it uses Window Hooks and memory-mapping files.
Thanks for all who helped me to get to this solution!
Managed Spy does not provide a solution for compact framework applications.
The company Jamo Solutions (www.jamosolutions.com) meets the requirements for automation testing on mobile devices, including .net compact framework applications.