I've created CI&CD on Azure repo, and it is doing code deployment on App service.
Now created Automation that opens the browsers and do actions and generates reports for test cases. (C#+Selenium+report package).
Now I need to the setup Test plan, I know, needs to run headless testing, my the questions are
need to do without buying a Test plan package, then how we should do? (I have a subscription but not with test plans)
How to do using Azure Test plans (any recent blog pls, and no MS docs links).
any help appreciated
As far as I know, you can execute Selenium tests directly in the pipeline(Without test plan).
Here are the steps:
Add Visual Studio Test Task to run the Selenium tests and the test result will be output to the Test tab(Build Pipeline or Release Pipeline).
You also could use Publish Test Results task to publish test result.
Here is a blog about the detailed steps: Automating Selenium Tests in Azure Pipelines.
For headless testing,
You could refer to UI testing considerations.
By the way, since you have already created CI/CD, you can also consider integrating UI testing steps into CI/CD.
If you still want to configure test plan, you could refer to this doc.
Related
I run CI/CD pipeline on Azure DevOps and generate Junit xml files with test results. By the way, I use Cypress framework for my tests.
Is there any way to ingest Junit results into TestRail 7.5 and publish test results in "Test Runs and Results" tab in TestRail?
Actually, what I would like to achieve is to generate new runs on TestRail and push the test results after each run on pipeline.
Can anyone that has done it previously help me, please?
You might want to have a look at Railflow which provides exact functionality you want. It is a commercial tool but it provides free support for JUnit report uploading into TestRail.
P.S. I'm developer at Railflow
I'm currently using cTest to publish test results from google test into azure.
Is there any way i can decorate my test so it comes up already linked to a requirement. without manually having to link it in azure?
Edit:
I'm using gtest recordproperty to add data to the xml output of the test case. I have tried with #1234( as in a commit), 1234 and several combinations. I cannot find anything in the documentation.
Currently, you could refer to this doc: Run automated tests from test plans to automate test cases in your test plans and run them directly from Azure Test Plans. And then open the Test Plans page and select the test plan to see the status of your tests if tests are updated after test execution is complete.
In addition, if you turn on below “Automatically link new work in this build” option in build pipeline page, and then you can link work item to Git commit, the new build queued by this commit will link to this work item when it completes successfully.
Also, you could enable “Create work item on failure” option to create a new work item when this build is failed.
BTW, you could link work items to deployments refer to this doc.
I'm new to Azure DevOps . Our requirement is to fire the automation test suite which coded in eclipse , resides in a devops repo and linked with a pipeline . There is a manual suite which is related to the automation suite and will trigger the automation suite . I don't use Visual studio . As per my latest knowledge Microsoft is yet to provide support.Is there any workaround ?
No, it's not able to do this without Visual Studio.
Tests can be run in the CI/CD pipeline by choosing the test plan or
test suite in the settings of the Visual Studio Test task. Automated
tests can also be run from Azure Test Plans or the Test hub in Azure
DevOps Server (see Web portal navigation). If you are using XAML
builds you can also run these automated tests by using Microsoft Test
Manager.
The process to associate an automated test with a test case is:
Use Visual Studio Enterprise or Professional 2017 or a later version
to associate the automated test with a test case as shown below. The
test case must have been added to a test plan that uses the build you
just defined.
You may have to do this with Visual Studio as a prerequisite. Details please refer our official doc here -- Associate automated tests with test cases
As noted here:
https://learn.microsoft.com/en-us/azure/devops/pipelines/test/review-continuous-test-results-after-build?view=azure-devops#tests-tab
If you use the Visual Studio Test task to run tests, diagnostic output logged from tests (using any of Console.WriteLine, Trace.WriteLine or TestContext.WriteLine methods), will appear as an attachment for a failed test.
How can I enforce Azure Devops pipelines to publish attachments for successful tests?
Update
riQQ pointed to the another stackoverflow question:Azure DevOps: Console output for green tests is missing
Proposed solution doesn't help me, because they solved using TestContext. We are using xUnit, and there is no TestContext for xUnit.
How can I enforce Azure Devops pipelines to publish attachments for
successful tests?
Sorry but as I know this is not supported behavior for VSTest Task. It's by design that only failed tests should have the log as attachment. And after my check, no extensions in Marketplace extends this kind of option.
So if you do want this option, you may consider using rest api like what riQQ suggests above. Or you can post a new feature request in DC forum, it it gets enough votes, the team would consider it seriously.
Hope it helps :)
Righ now, I am setting a CI/CD pipeline on Azure devops 2019. I have implemented some Autometed test, but I would like to know if there is a way to set a stage in a pipeline with a task that allow me to trigger a manual test created on test plans ?
To do this first we need to Associate automated tests with test cases. Then you can use Visual Studio Test task by selecting tests using Test Plan to run tests from your test plan that have an automated test method associated with it.
A similar question has been answered on the Comments in Azure DevOps Labs.Please refer to it.