I want to populate test plan with result of tests that are done by our CI pipeline on github. I am thinking of using a release pipeline like this. (Azure DevOps Test Plans - Synchronize JUnit test results in Runs with test cases on Tests Plan)
But as the author was not able to populate test plans with it. I am looking for a better solution.
As I need the project report for the test plan.
Related
I run CI/CD pipeline on Azure DevOps and generate Junit xml files with test results. By the way, I use Cypress framework for my tests.
Is there any way to ingest Junit results into TestRail 7.5 and publish test results in "Test Runs and Results" tab in TestRail?
Actually, what I would like to achieve is to generate new runs on TestRail and push the test results after each run on pipeline.
Can anyone that has done it previously help me, please?
You might want to have a look at Railflow which provides exact functionality you want. It is a commercial tool but it provides free support for JUnit report uploading into TestRail.
P.S. I'm developer at Railflow
I have all my test cases in Azure Devops Test Plans and now I'm executing them manually against a build.
How do I view manual test results by build?
I can view and filter test results by build on the "Runs" page but I would prefer to see test results grouped by build not filtered to a single build.
Any options for this?
I had hoped the the Test Plans - Progress Report would provide this but I don't see a way.
Maybe there is a way to group all my manual tests executions into a Test Run?
I researched and attempted with the related Azure DevOps REST API and Azure CLI, however, I did not find any available way.
It seems that we can only directly filter the test results to the build run level, and cannot to the build pipeline level.
Maybe, you can indirectly get the associated build pipeline via the following way:
Use the REST API "Runs - Query" to list the test runs associated with a specified build pipeline.
Use the REST API "Results - List" in a loop to list the test results of each test run.
Until recently I ran Selenium automated tests from the Azure Test Plans using a YAML pipeline for the Build part and a classic Release Definition for the Release part.
Now, the organization I work in does not allow the use of classic Release Definitions from Azure DevOps.
So I extended my YAML Build Pipeline with stages for running my automated tests on different environments.
But I also want to make use of the Azure Test Plans for running my tests.
My problem is that when I want to run a test from my Test Plan, the only possibility for selecting a Release pipeline is the classic Release Definition I have created.
Is there any possibility to trigger the run of my automated tests from the Azure Test Plans, but to use a YAML pipeline when selecting the Stage.
Thank you!
I tried to edit the settings of my Test Plan, but in order to select a Stage, I have to select a Release Pipeline, and the only options are the ones existing in the Pipelines -> Releases windows of Azure DevOps
Currently we are able to associate only one mstest to a testcase in azure devops. I want to associate more than one test to a testcase. Any ideas please.
How to associate more tests to a testcase in azure devops
For this issue, I think it is not available to associate more than one test to a test case. A test can only be associated with one test case. For example: suppose we associate multiple tests into a test case. When running this test case, some of the tests successes and some of the tests failures, we can't determine whether the test case is successful or not, so this is unreasonable.
If you want to run multiple tests,you need to create multiple test cases.You can also try to create a test Plan or test suite, then add multiple test cases to the test suite or test plan, then run the test suite or test plan.
Please refer to Create test plans and test suites.
Righ now, I am setting a CI/CD pipeline on Azure devops 2019. I have implemented some Autometed test, but I would like to know if there is a way to set a stage in a pipeline with a task that allow me to trigger a manual test created on test plans ?
To do this first we need to Associate automated tests with test cases. Then you can use Visual Studio Test task by selecting tests using Test Plan to run tests from your test plan that have an automated test method associated with it.
A similar question has been answered on the Comments in Azure DevOps Labs.Please refer to it.