I execute a script from a Groovy script in SoapUI project.
def tCase = testRunner.testCase.testSuite.testCases["TC1"]
def tStep = tCase.testSteps["attachIt"]
tStep.run(testRunner, context)
There is another loop in this script, which is to execute a SOAP request. After that two more Groovy scripts should be executed to process output. These are in same testcase after SOAP request step, but aren't executed automatically.
lets Assume you have a testcase which has 5 steps
As per my understanding you are trying to execute few steps(say Step1,step2) of the testcase via a groovy(step 3) and you are expecting that step4 and step5 should execute automatically.
That will not happen
So you can disable step 1 & step 2
So now what will happen when you run the test case(not step) system will execute step 1 & step2 via step3 and once step 3 is done step4 & step5 will run automatically.(since you ran the test case_
In case you like that step4 & step5 should execute automatically you can use the same code as you have shared and use groovy script teststep names in it and disable those steps
Related
I have 3 normal and 1 explicit test but when I run my test using the Test Explorer window I get this output under "Tests" in the Output window
========== Starting test run ==========
NUnit Adapter 4.2.0.0: Test execution started
Running all tests in xyz.dll
NUnit3TestExecutor discovered 1 of 1 NUnit test cases using Current Discovery mode, Explicit run
ExplicitMethod(03/10/2022 08:00:00,03/10/2022 16:00:00): OneTimeSetUp:
NUnit Adapter 4.2.0.0: Test execution complete
========== Test run finished: 1 Tests (0 Passed, 0 Failed, 0 Skipped) run in 462 ms ==========
*Note I'm using NUnit Adapter 4.2.1 instead of 4.2.0 so that's already weird
And this is how the Test Explorer window looks
Test Explorer window
*Note the full blue test is the Explicit test that get's skipped like it should
This is a problem because it does seem to only discover tests which are explicit.
The tests I want to run are of course not explicit, here is an example
[Test]
[TestCaseSource(nameof(TestNameData))]
public async Task<float> TestName(DateTime start, DateTime end, List<CalculateHoursObj>? list = default)
{
if (list == null) list = new List<CalculateHoursObj>();
return await EmployeeService.CalculateOverTimeHours(start, end, list);
}
public static IEnumerable TestNameData
{
get
{
yield return new TestCaseData(TenthMarch8_2022, TenthMarch16_2022).Returns(8.0f);
}
}
It fails to discover and/or run this test.
But curiously if I break all my non explicit tests by making the data non-static like this
public IEnumerable TestNameData
{
get
{
yield return new TestCaseData(TenthMarch8_2022, TenthMarch16_2022).Returns(8.0f);
}
}
It of course breaks the test and when I run all tests it does actually discover all tests
========== Starting test run ==========
NUnit Adapter 4.2.0.0: Test execution started
Running selected tests in xyz.dll
NUnit3TestExecutor discovered 2 of 4 NUnit test cases using Current Discovery mode, Non-Explicit run
NUnit Adapter 4.2.0.0: Test execution complete
========== Test run finished: 2 Tests (0 Passed, 2 Failed, 0 Skipped) run in 394 ms ==========
But even now it only runs 2 of the 3 broken non explicit tests and of course they all fail
I have looked up everything online for 1.5 hours and really can't find a solution.
Don't bother responding with "have you updated visual studio or the nugget packages"
This question was part of a bug the solution is to not use new DateTime() withing nunit tests or to have nunit test be discover with id's using the <UseParentFQNForParametrizedTests>true</UseParentFQNForParametrizedTests> <UseNUnitIdforTestCaseId>true</UseNUnitIdforTestCaseId> flags
This was a bug because it was not reported correctly in the output window and/or Error List window.
I've created an Issue on GitHub about this if anyone wants to read.
Now stop editing this post because it truly is answered and the Issue link which I included shows the valid, correct and readable answer by "OsirisTerje" whom explains it way better than I could.
For my project I want to run the exact same test cases twice, once locally and on a different VM in parallel in the cloud (Azure in my case).
I duplicated the TestCase and tagged one Category("Local") and the other Category("Cloud").
Running nunit3 from the console with --where="cat == Cloud" will thus run all TestCases of every test that has one or more TestCases tagged with Category("Cloud").
Is there a different way of only running selected TestCases by a commandline switch?
Simplified example:
[TestCase(TestName = "Canary, Run in cloud."), Category("Cloud")]
[TestCase(TestName = "Canary, Run locally."), Category("Local")]
public void Canary()
{
Assert.True(true);
}
Found a work-around.
Using --params:Cloud=true as command line argument and in the code
private bool ShallRunInCloud => TestContext.Parameters["Cloud"]?.ToLowerInvariant() == "true";
Is it possible to send the Katalon's test result to slack when we are using command line?
I have created the integration for slack inside the Katalon-Studio and it is working fine, but when I tried to execute that test using command-line, the test is running, but the result is not sending to slack.
katalon -noSplash -runMode=console -consoleLog -noExit -projectPath="C:\Users\test\Katalon Studio\Web.Portal\test.prj" -retry=0 -testSuitePath="Test Suites/New Test Suite" -executionProfile="default" -browserType="Chrome"
I expected to receive the same message that I've received when I executed the tests through Katalon-Studio.
eg.
Summary execution result of test suite: Test Suites/New Test Suite
Total test cases: 1
Total passes: 1
Total failures: 0
Total errors: 0
Total skipped: 0
All you need is Katalon Plugins
To test suite in console mode and you want Katalon to send a summary report to Slack,
You need to add one more parameter -apiKey into your command as well.
To get an API Key, you can access the store (https://store.katalon.com), log in with your account then navigate to the Setting page to add a new apiKey.
Example in CMD mode
./Katalon\ Studio.app/Contents/Ms/katalon --args -runMode=console -projectPath="/Users/Shared/Jenkins/Home/workspace/QA-katalon/kyy.prj" -reportFolder=“Reports” -reportFileName=“report” -retry=0 -testSuitePath=“Test Suites/Employees” -browserType=“Chrome” -apiKey=“de544cef-a56b-4eb5-a051-391bc0537b59(your key)”
Find more details here > https://docs.katalon.com/katalon-studio/docs/slack-plugin-integration.html#obtain-slack-legacy-token
I'm running into a problem like this. I'm currently using pytest to run test cases, and reducing execution time using xdist to run tests in parallel and publishing tests results to TestRail. The issue is when using xdist, pytest-testrail plugin creates Test-Run for each xdist workers and then publishes test cases like Untested.
I tried this hook pytest_terminal_summary to prevent pytest_sessionfinish plugin hook from being call multiple times.
I expect only one test run is created, but still multiple test runs are created.
I ran in to the same problem, but found a kind of workaround with duct tape.
I found that all results are collecting properly in test run, if we run the tests with --tr-run-id key.
If you are using jenkins jobs to automate processes, you can do following:
1) create testrun using testrail API
2) get ID of this test run
3) run the tests with --tr-run-id=$TEST_RUN_ID
I used these docs:
http://docs.gurock.com/testrail-api2/bindings-python
http://docs.gurock.com/testrail-api2/reference-runs
from testrail import *
import sys
client = APIClient('URL')
client.user = 'login'
client.password = 'password'
result = client.send_post('add_run/1', {"name": sys.argv[1], "assignedto_id": 1}).get("id")
print(result)
then in jenkins shell
RUN_ID=`python3 testrail_run.py $BUILD_TAG`
and then
python3 -m pytest -n 3 --testrail --tr-run-id=$RUN_ID --tr-config=testrail.cfg ...
I have .net core 2.1 project build pipeline as shown below.
I have 2 simple unit test cases which executed successfully. “.NET Core test” command executes successfully and publishes thereport.
2018-09-06T22:05:41.8494077Z
2018-09-06T22:05:42.0138714Z Passed Login_Test
2018-09-06T22:05:42.0139277Z Passed Logoff_Test
2018-09-06T22:05:42.0495295Z Results File: P:\*******2018-09-06_17_05_41.trx
2018-09-06T22:05:42.0514262Z
2018-09-06T22:05:42.0516990Z Attachments:
2018-09-06T22:05:42.0517972Z P:\*****\TestResults\329a8629-7ec1-454b-97d8-2623d29bbd0e\XXXXX 2018-09-06 17_05_39.coverage
2018-09-06T22:05:42.0518130Z
2018-09-06T22:05:42.0518516Z Total tests: 2. Passed: 2. Failed: 0. Skipped: 0.
2018-09-06T22:05:42.0519312Z Test Run Successful.
2018-09-06T22:05:42.0528123Z Test execution time: 1.9003 Seconds
2018-09-06T22:05:42.1424722Z Publishing test results to test run '90'
2018-09-06T22:05:42.1424881Z Test results remaining: 2. Test run id: 90
2018-09-06T22:05:42.1592889Z ##[section]Async Command Start: Publish test results
2018-09-06T22:05:42.2384075Z Published Test Run : http://******/_TestManagement/Runs#runId=90&_a=runCharts
2018-09-06T22:05:42.2384433Z ##[section]Async Command End: Publish test results
2018-09-06T22:05:42.2385471Z ##[section]Finishing: VsTest - testAssemblies
But I don’t see any “Test Results” under dashboard.
I included "Visual Studio Test" too, but still dont see the result in the dashboard. What am I missing here?