I'm currently using cTest to publish test results from google test into azure.
Is there any way i can decorate my test so it comes up already linked to a requirement. without manually having to link it in azure?
Edit:
I'm using gtest recordproperty to add data to the xml output of the test case. I have tried with #1234( as in a commit), 1234 and several combinations. I cannot find anything in the documentation.
Currently, you could refer to this doc: Run automated tests from test plans to automate test cases in your test plans and run them directly from Azure Test Plans. And then open the Test Plans page and select the test plan to see the status of your tests if tests are updated after test execution is complete.
In addition, if you turn on below “Automatically link new work in this build” option in build pipeline page, and then you can link work item to Git commit, the new build queued by this commit will link to this work item when it completes successfully.
Also, you could enable “Create work item on failure” option to create a new work item when this build is failed.
BTW, you could link work items to deployments refer to this doc.
Related
I have all my test cases in Azure Devops Test Plans and now I'm executing them manually against a build.
How do I view manual test results by build?
I can view and filter test results by build on the "Runs" page but I would prefer to see test results grouped by build not filtered to a single build.
Any options for this?
I had hoped the the Test Plans - Progress Report would provide this but I don't see a way.
Maybe there is a way to group all my manual tests executions into a Test Run?
I researched and attempted with the related Azure DevOps REST API and Azure CLI, however, I did not find any available way.
It seems that we can only directly filter the test results to the build run level, and cannot to the build pipeline level.
Maybe, you can indirectly get the associated build pipeline via the following way:
Use the REST API "Runs - Query" to list the test runs associated with a specified build pipeline.
Use the REST API "Results - List" in a loop to list the test results of each test run.
I'm new on DevOps Azure. I'm trying to link my automated tests for Azure DevOps platform, but outcome is always "In progress". I don't get failed or passed for any test. I created my tests in Visual Studio C#.
I created app and CheckIn it in Visual Studio and it's showed in Repos File, also my test folder is shown too. If I do changes in my code it's updated in Repos. So it is fine.
Then, I created Pipeline:TestMirjana-.NET Desktop-CI
and build is successful for this one. Than, I created Release linked for this pipeline:
and it is successful too.
I linked my test case for Visual Studio:
also linked Test Plan Settings for these pipeline and release
but when I run my test case Outcome is always "In progress", no other results shown (not failed or passed)
validation is good
but Outcome is always None
.
So, I'm not sure why it's not working properly.
Maybe I didn't link something properly, can you please support?
Many thanks in advance.
The 'In Progress' result means it has some automation associated but there are no incoming results (having the required test configuration).
Try to run the vstest task in the Release pipeline (instead of the build pipeline) and use "Select tests using=Test plan" and pick the Test plan where you expect to see the results from automated tests. It works for me.
I have a pipeline that runs on pull requests and executes tests , it also produces an xml execution report (it is a node project and tests run via jest) , and coverage report that I use for sonarcloud. However , the developers want to be able to see the tests results on each pr and see what tests failed in particular, so my question is whether it is possible to somehow export and display those results in github or sonarcloud
Please refer to the following two methods to get the test cases metric in sonarcloud:
You can add sonar.testExecutionReportPaths in Additional Properties option in Prepare Analysis Configuration task. About sonar.coverageReportPaths
For SonarQube to pick up and analyse the content of the report file, you need to inform the sonar scanner. About the location of the XML file. To do so, edit the sonarqube.properties file. Please refer to this.
coverage reporting:
sonar.testExecutionReportPaths=reports/xxx/xxx.xml
In addition, you can also refer to this similar case.
I've created CI&CD on Azure repo, and it is doing code deployment on App service.
Now created Automation that opens the browsers and do actions and generates reports for test cases. (C#+Selenium+report package).
Now I need to the setup Test plan, I know, needs to run headless testing, my the questions are
need to do without buying a Test plan package, then how we should do? (I have a subscription but not with test plans)
How to do using Azure Test plans (any recent blog pls, and no MS docs links).
any help appreciated
As far as I know, you can execute Selenium tests directly in the pipeline(Without test plan).
Here are the steps:
Add Visual Studio Test Task to run the Selenium tests and the test result will be output to the Test tab(Build Pipeline or Release Pipeline).
You also could use Publish Test Results task to publish test result.
Here is a blog about the detailed steps: Automating Selenium Tests in Azure Pipelines.
For headless testing,
You could refer to UI testing considerations.
By the way, since you have already created CI/CD, you can also consider integrating UI testing steps into CI/CD.
If you still want to configure test plan, you could refer to this doc.
I have a automation script which built in Java/selenium. I want to run my test cases from VSTS by triggering my script and also want to get the result back in VSTS. Can anybody give me the path how can I make that happen. Also where should I keep by project?
I was doing research on it. But doesn't make sense to me
https://learn.microsoft.com/en-us/azure/devops/pipelines/test/continuous-test-selenium?view=azure-devops
run my test cases from VSTS by triggering my script and also want to
get the result back in VSTS
For this issue ,there is a blog that gives a path to do this , please refer to it to see if it helps .
In this path , you need to do stuff both to your project, and to your vsts build definition. The jest-junit package is needed in the project and then create a build with three tasks(“npm”,“Publish Test Results”, “Publish Code Coverage Results” ).