I'm trying to run my selenium automation scripts in CI pipeline using Azure DevOps. I have configured Visual Studio test task to run my automation scripts by selecting test plan option.
Now it runs all my automation scripts which associates with test cases. But it does not take the order define in the test case.
How do we define the order to run test cases? Currently it is not running according to test cases order.
For Example:
I have test cases: Test A, test B, test C.
I want to run test in order B,C,A.
You may create a main test and use any order, something similar was considered here: Controlling execution order of unit tests in Visual Studio
try this
use TestNG annotations and run with Priority tag.
Related
I have a TeamCity build configuration that builds a C# project, runs some unit tests, and then does some extra things. My question is: Can I get information about my unit test run stored into build configuration variables (i.e. how many tests were run, how many were successful, how many failed, how many were skipped) so that I can then check these variables in a PowerShell script in later build steps and perform different actions depending on how many tests have passed?
AFAIK the best way is to ask these information directly to teamcity server using its REST API (pay attention, maybe the build locator could be a little be tricly to be found, if the build is still running).
By other hand, you can parse your NUnit test result file (or files if you run more than one NUnit test runner step in your build) inside your build agent machine.
What I'm aiming for is closing the loop on test coverage.
My goal would be:
Define test cases with PM and list them out under a Test Plan in Azure DevOps.
Associate Selenium tests in Visual Studio with specific test case(s).
When the release pipeline runs the UI test, I would expect to see on the dashboard the % of test cases that passed/failed or were not even associated.
Is that even possible?
Check the following link:
https://learn.microsoft.com/en-us/azure/devops/test/associate-automated-test-with-test-case?view=azure-devops
Coded UI test, Selenium tests, and unit tests written using Version 1
of the MSTest framework can be associated with a test case.
The process to associate an automated test with a test case is:
Create a test project containing your automated test.
Check your test project into an Azure DevOps repository.
Create a build pipeline for your project, ensuring that it contains the automated test.
Use Visual Studio Enterprise or Professional 2017 or a later version to associate the automated test with a test case. The test case must have been added to a test plan that uses the build you just defined.
Check this extension that I have created https://github.com/JanuszNowak/janono.ado.testcase.associate.cli . It allows associating in automatic manner. So if yours selenium test are using xunit, nunit or mstest you can associate them automatically.
Code sample:
namespace ExampleTestProject
{
[TestClass]
[janono.ado.testcase.associate.Organization("janono-pub")]
public class UnitTest1
{
[TestMethod]
[janono.ado.testcase.associate.TestCase(5)] //<---
public void TestMethod1()
{
//yours test method content
//...
//
}
}
}
Need clarification- Can we build CI CD pipeline using Jenkins where my development code written is in C# and automation code is java eclipse
Requirement :
1. Whenever developer will commit the code then new Build should be generated
2. It should run unit test and smoke suite
3. Once Testing smoke suite passed it should show build passed else build should be failed
Can we achieve it using Jenkins. I have concerns like my development and automation code in two different language. In order to build pipeline do we need to have dev and test code in the same repository
Thanks in advance . Please suggests me
I am in exploration phase.
I need help in configuration. I don't think any code will require for this.
When I run pytest_collect_file to collect all tests (manual/automation) this collector run first and I can't get the ID's of the automation test to remove duplicate tests.
I want to report my manual test via PyTest, so I wrote some code that get the execution status of the tests and now I'm trying to report each test with the status of the execution test via PyTest.
To do that i'm using with pytest_collect_file pytest function, and I collect all manual test.
The problem is that now I have all the tests of the project, the manual and the automation tests. I want to remove the automation test from the collection list to remove duplicate report. I can get the testID of the automation test if pytest collect all the automation test before I'm add the manual via pytest_collect_file but the problem is that this function called first.
my collector base on this code
https://docs.pytest.org/en/latest/example/nonpython.html
I currently have a code base which has NUnit tests organised into suites. We currently have a build task on our CC box which runs all of the tests in one CC project.
Is there a way we can configure CC.Net to allow us to individually run the different test suites from the dashboard.
Something along the lines of a list of all the suites and a button next to each to run those tests.
The only way I can think of doing it now is to set each suite up as its own project and control them all from the top level dashboard, but we have alot of test suites and this complicates adding more.
Thanks in advance.
I'm afraid there is no other solution than creating a separate project for each test suite.
How could CCNET be aware of something like a test suite? What about different implementations of test suites?
For CCNET a project is atomic i.e., a project is the smallest available element you can run and evaluate the results for.