As noted here:
https://learn.microsoft.com/en-us/azure/devops/pipelines/test/review-continuous-test-results-after-build?view=azure-devops#tests-tab
If you use the Visual Studio Test task to run tests, diagnostic output logged from tests (using any of Console.WriteLine, Trace.WriteLine or TestContext.WriteLine methods), will appear as an attachment for a failed test.
How can I enforce Azure Devops pipelines to publish attachments for successful tests?
Update
riQQ pointed to the another stackoverflow question:Azure DevOps: Console output for green tests is missing
Proposed solution doesn't help me, because they solved using TestContext. We are using xUnit, and there is no TestContext for xUnit.
How can I enforce Azure Devops pipelines to publish attachments for
successful tests?
Sorry but as I know this is not supported behavior for VSTest Task. It's by design that only failed tests should have the log as attachment. And after my check, no extensions in Marketplace extends this kind of option.
So if you do want this option, you may consider using rest api like what riQQ suggests above. Or you can post a new feature request in DC forum, it it gets enough votes, the team would consider it seriously.
Hope it helps :)
Related
I run CI/CD pipeline on Azure DevOps and generate Junit xml files with test results. By the way, I use Cypress framework for my tests.
Is there any way to ingest Junit results into TestRail 7.5 and publish test results in "Test Runs and Results" tab in TestRail?
Actually, what I would like to achieve is to generate new runs on TestRail and push the test results after each run on pipeline.
Can anyone that has done it previously help me, please?
You might want to have a look at Railflow which provides exact functionality you want. It is a commercial tool but it provides free support for JUnit report uploading into TestRail.
P.S. I'm developer at Railflow
I've created CI&CD on Azure repo, and it is doing code deployment on App service.
Now created Automation that opens the browsers and do actions and generates reports for test cases. (C#+Selenium+report package).
Now I need to the setup Test plan, I know, needs to run headless testing, my the questions are
need to do without buying a Test plan package, then how we should do? (I have a subscription but not with test plans)
How to do using Azure Test plans (any recent blog pls, and no MS docs links).
any help appreciated
As far as I know, you can execute Selenium tests directly in the pipeline(Without test plan).
Here are the steps:
Add Visual Studio Test Task to run the Selenium tests and the test result will be output to the Test tab(Build Pipeline or Release Pipeline).
You also could use Publish Test Results task to publish test result.
Here is a blog about the detailed steps: Automating Selenium Tests in Azure Pipelines.
For headless testing,
You could refer to UI testing considerations.
By the way, since you have already created CI/CD, you can also consider integrating UI testing steps into CI/CD.
If you still want to configure test plan, you could refer to this doc.
I'm using Microsoft Unit Tests to validate my solutions via Azure DevOps Pipelines. I am able to see tests being executed and the % of code coverage
However, I can't see the code coverage results (or they won't render)...
Here is my publish code coverage task - am I missing something to have it render here?
This is a known issue on Azure devops. Now, we could only download the report, and open it with Visual Studio.
Azure devops only support the download link for .coverage files currently. The white page you see is a UI glitch. This scenario is only supposed to render a download link to the coverage file.
Besides, this issue has been submitted in this earlier suggestion ticket linked here:
support vstest .coverage "code coverage" build results tab
This feature request is On Roadmap, I believe it will be released soon, you can follow this thread to know its latest feedback.
In addition, we found a similar case and I have tested via the answer, set the code coverage tool to Cobertura, then I get the code coverage report in the Azure DevOps pipeline.
The company I work on recently purchased SonarQube Enterprise to improve code quality throughout all repositories. I found out that there is a feature that enables SonarQube to comment automatically on PRs targeting a specific branch, and I successfully managed to try that out.
Thing is:
That configuration is not scalable: I would need to manually configure every repo to follow that rule
That configuration needs a build pipeline to be defined "old school" on Azure DevOps to work, and we are moving into Pipeline as Code, starting of course with CI (where this takes place)
Anyone managed to get the PR commenting working in that scenario? Or, at least, solving the #1 problem?
Cheers
You can use REST APIs to do whatever configuration you need to do across your repositories. Refer to the REST API documentation.
Shouldn't matter, although I haven't tested it. The SonarQube tasks aren't aware of whether the build source is YAML or visual designer/classic/JSON builds. The underlying tasks and job running architecture is the same. As long as the build is hooked up to a branch policy, it should still work.
I recently got to know Dockup and while I really love this kind of integration in Github, I am wondering if the same thing can be achieved in Gitlab, for free.
Basically, what I would like to achieve is:
On merge request, build tests and deploy in a dedicated environment which would use the name of the merge request
Send a message to a given slack about the environment or the failure of the build / tests in the pipeline with the related link of the pipeline
It seems that since Gitlab 11.6 it is possible to have Pipelines for merge requests but I don't really see how to get the information of the merge request or even who has submitted the merge request to use it for creating the dedicated deployment environment in my pipeline script?
How can I get that?
Note: It seems only a webhook can provide the information about the user and all the details.
Found the information I wanted in https://docs.gitlab.com/ee/ci/variables
CI_MERGE_REQUEST_TITLE
CI_MERGE_REQUEST_PROJECT_URL
GITLAB_USER_NAME
CI_MERGE_REQUEST_ASSIGNEES
Probably won't even need to create a webservice for the webhook since everything can be found in the environment variables when running the pipeline script, this is pretty awesome!