This question already has an answer here:
KARATE integration with SauceLabs
(1 answer)
Closed 1 year ago.
I'm trying to run a simple test scenario on sauce-labs, but the issue is that sauce-labs doesn't identify the test-status as passed or failed on its own. I've gone through multiple solutions but couldn't make any of them work.
I would love to know if we can do something in the feature file rather than doing something externally if possible. Following are my feature and java files.
Feature File
Java File
Any help will be appreciated. I'm a JAVA newbie & just starting on the Karate Framework and do not have in-depth knowledge of how things are working.
The Remote Driver is just sending along requests for actions and requests for information. What that information is "supposed" to be is decided by the assertions in your test. Sauce Labs has no way of knowing what that is without your input.
There are 2 ways to tell Sauce Labs the status of the test.
The recommended way is to use the API:
https://wiki.saucelabs.com/display/DOCS/Job+Methods#JobMethods-UpdateJob
You just need your sauce username, access key and the job ID (job ID is the same as the driver's session ID).
The less reliable, but easier to implement way is with a custom JavaScript command:
https://wiki.saucelabs.com/display/DOCS/Annotating+Tests+with+Selenium%27s+JavaScript+Executor
"sauce:job-result=passed"
Either of these can be put into an AfterClass or TestWatcher methods.
Related
Is there's a way to update the nextBuildNumber directly via Rest? I found the parameter here:
/job/MyJob/api/xml?tree=nextBuildNumber
and in the job directory, there is a nextBuildNumber file
We already use the Rest Api for creating/updating jobs and views, so it would be nice to stick to this, instead of using cli or the nextbuildnum plugin.
Edit: new approach brings another question
Java send integer value with HTTP POST
The REST API does not currently (at time of writing) support changing the value of the nextBuildNumber. As you have found you can (only) read it.
The easiest way I know to manipulate this value is through the Next Build Number plugin.
I am looking for the best way to test this scenario using automation. Before you mention or ask, yes, i have throughly searched to make sure or even get info or knowledge on this topic.
I have a web service that provides various pricing for health insurance. The pricing is based on different class category of race, age, location etc. All that is inputed into the web service to get the fee for the client.
Right now everything is done locally with SoupUI. As you know SoupUI does not have a way for you to randomly change the data in the body, but i need a way to automate this so random age, race, location, etc can be inputed in and verify the result against a CSV file with the information.
Is this possible or is there a better way to test this web service?
-
Per #lloyd request, because he is King here and other known methods i know:
UI Option: We build a UI to interface with the API and use Selenium to run the countless scenarios we have, but they do not want to do this. They are more incline on testing the API itself.
Ruby: Use Ruby to connect to the API, using Rspec and Airborne (https://github.com/brooklynDev/airborne), the downsize of this is also the same as using SoupUI. Everything is entered in manually, and the code just runs through the various variations we have to get the return and match it against what it should be.
I also found another tool, https://assertible.com/ but the downsize of that is we cannot limit the random generator between 2 selected numbers, plus no way to do the comparison.
Then there is Postman, which is also similar to SoupUI so not a real automation since we cannot change the values in the body. http://blog.getpostman.com/2015/09/03/how-to-write-powerful-automated-api-tests-with-postman-newman-and-jenkins/
We are using HP ALM for QC in our projects and we are planning to automate the following things uisng HP ALM's REST api.
1.Dynamically upload the test cases in Test Plan and Test lab.-From the api code sample i can understand that TC's are upload to Test lab by creating test set folders.But how to upload the TC's in Test plan ?
Map the test cases in Test plan and Test lab -No where i have seen any examples which explains how to map the TC's in Test Plan and Lab.
3.Dynamically update the status of the test cases with Success or failure.
4.How to map the TC's with work items ?
Can you please advise us that the above items are achievable ?
of course what you mentioned is possible. First of all it may be different on the HP QC REST ALM version you are using. For instance not all the features are enabled on the first 11.x version, but most of bugs got fixed in the 12.53 for instance.
Anyway coming to your point:
1) Always refer to the HP REST ALM library to check the REST msg you should send:
http://alm-help.saas.hpe.com/en/12.53/api_refs/REST/#Overview.htm
2) You have to choose a way to send and receive those msgs. Nowadays it is quite common the usage of python (I am using that). Please have a look at the "REQUESTS" module that will fit for this task!
First action is to login/authenticate into the QC ALM, then you can start sending next operation according to the library above and keeping the correct information in the header (for instance the LSSOCookie).
3) Plenty of questions are already answered on stackOverflow. Always try to check a specific task.
4) Coming to your questions: if test-set-folders is used for folders in test lab, then test-folders is used as entity for the folder under Test Plan.
Entity= "test-instances" is used then to link a Test Case to Test Set :-)
To update a test-set or test-case you have to send a proper update for that. (some bugs are visible in that area for the HP 11.52 --> for instance before create a test as "No-run" and then update to "pass or fail". This you will experience after you have implemented first points.
you can then create run, run-steps, design-steps, attached files and whatever is mentioned on the HP ALM library.
Please vote the question if this solved your query ;) that you can close the same! I wish you luck in your project! ciao ciao!
I am trying to determine whether I am able to inject test case information at run time and leverage the SOAPUI tool. I understand that I can create test cases on the GUI but is this my only option?
Background info if interested: Currently I am working on creating an automation framework at my company. We currently have web page testing and soon to be added SOAP testing. As many of these tests (at one point in the future as I am told my the architect) could be run from both a web page and soap I think it's best to store the test cases in some format (Json, YAML, etc.) to document all the test cases and then inject them into test steps at run time.
However my company enjoys working with SOAPUI. I've used the tool and created test cases, assertions, et al on the GUI (of course) but I cannot find any documentation which suggests that instead of defining the test cases in this way I could inject the test information at run time (similar to what you can do with the wsdl2java apache tool). Can this be done with testrunner? This way I can reuse the test cases. Is this possible? Does this even make sense? I just want to attempt to incorporate a tool I've been asked to use.
Any thoughts are greatly appreciated!
Here is an example of what data may look like:
Partner : [
Organization : [
Company Name:
Company URL:
]
Contact Information : [
Name:
Address:
]
] (sorry i can't get the indents to work properly...)
As I stated below in a comment, I know on the SoapUI GUI I can create a test suite, test case and add test steps. But I want to store the test step information in a different place so I can use the test steps for different kinds of tests.
Your question is way too broad for me to even attempt a complete answer.
SoapUI GUI you use to create the tests. Your data can be stored, and read by SoapUI, in Excel, database, flat file, generated dynamically, whatever you want. You can run everything using the testrunner from command line, or using the Maven plugin from Jenkins.
Seriously, spend some time with the documentation.
I am trying to use celery to create a task management system, where the tasks are written in phantomjs. Basically the task is to crawl a URL and return a JSON object (that I'll want to use once the task is complete). There will be a list of URLs which will each have it's own crawl task.
I looked at this http://docs.celeryproject.org/en/latest/userguide/remote-tasks.html#calling-webhook-tasks to implement tasks in another language. However celery does not provide very good documentation at all on how the external webhook script should be written, what dependencies are required, and how it is run.
Should the task simply be stored on a server, which I will call with the HTTPCallback? How is it run in the PhantomJS framework. How are results from here stored or returned and how do I keep track of it? I have read a lot of docs and I still cannot find the exact interface or API for the other side of the webhook API.
I realize my question may not be too clear, so let me know, otherwise some good examples of this HTTPCallback API workflow would be helpful for a basic understanding, and how the webhook script in a different language is structured.
Any insight would be greatly appreciated. Thanks a lot!