I'm trying to create a release pipeline in VSTS that runs my xUnit-tests as specified in a Test Plan.
Long story short: I can't get it to work.
What I'm using:
Azure DevOps (formerly VSTS)
Visual Studio Test task (v2.*)
Test project targeting .NET Core 2.1
xunit 2.4 with xunit.runner.visualstudio 2.4
In Azure DevOps I defined a Test Plan that contains a Test Suite which contains a Test that has an Associated Automation which points to my xUnit test.
I had to use the REST API to link the test code to the Test as described here.
I can select that Test in the visual designer for the VSTest task.
When I run the release pipeline the VSTest task fails with the following error message:
DiscoveryMessage : System.IO.FileNotFoundException: Unable to find tests for D:\a\r1\a\Foo.Tests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
The path to the Foo.Tests.dll is correct, all required files are copied as well.
I explicitly specified the framework version in a .runsettings file (as the option Other console options doesn't work when using the Test plan option).
Specified the path to custom test adapters
used Visual Studio 2017 and Installed by Tools Installer options
Added a .NET Core Tool installer to install the correct .NET Core SDK
...and any other combination of settings I could think of.
The error message is still the same.
Any ideas what I might be missing? Your help would be greatly appreciated at this point!
After several more hours we stumbled across a web page that stated that you don't have to copy the binaries of your test project as input for the VSTest task but PUBLISH it instead. That never came to mind as vstest.console.exe runs smoothly when you point it at the binaries on a local machine.
UPDATE: We had to add a Publish Artifact task at the end of our Build Pipeline and make the Release Pipeline pick up the published artifact.
Related
I have a web api (REST) project that is written in .NET and I have written a few webtests (.webtest) that test those apis.
While those tests run fine locally from visual studio, I want to integrate them into my VSTS (Azure Devops) build pipeline, so as to identify and breaking changes that could break any of those APIs.
I am not able to find any task in build pipeline which can run the webtests as part of build. I see option for running unit-tests though.
So, wanted to check what am I missing here.
You might want to find an alternative approach as this link implies it has been deprecated.
Visual Studio web performance test (.webtest file) is tied to the load
test functionality and is deprecated. Some customers have used
.webtest for other purposes such as running API tests, even though it
was not designed for that purpose. Many API testing alternatives are
available in the market. SOAP UI is a free, open source alternative to
consider, and is also available as a commercial option with additional
capabilities.
You could try to use cmd task command line to run MSTest with arguments.
Add Run Command Line step/task to execute MSTest command
Add Publish Test Results step/task
On the other hand, you can do test in Unit Test too, just send the request and check the response, related thread.
Also as Matt mentioned, since Visual Studio web performance tests (.webtest files) are tied to the load test functionality and is also deprecated. You could take a look at this blog here: Cloud-based load testing service end of life
I've got a couple builds in my VSTS build pipeline. A pull request build that builds the solution, the 'Build Solution' step works fine here, everything completes. Then it goes into a CI_Build, then it goes into a CI_Build and this builds the solution to be deployed through Octopus. The CI_Build seems to be set up the same way as the first build, yet I get the following error:
Error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.6" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend.
Anyone know what could cause a successful 'Build Solution' in one build, but not another when it's building the same solution?
We've got some private agent pools set up and I was using the wrong one. The agent pool must include VS2017
I have service fabric project .net core 2.1 project and have created unite test case. I added the VSTest step after the build step and enabled "Code Coverage Enabled" but I do not see any information related code coverage after the build.
You need the test sdk's running on the agent (VS Pro or Enterprise). You can also use the Visual Studio Test Platform Installer task for this, as this will install all the necessary tools.
You need to publish the test results with a task 'Publish test results' if I am correct.
I'm using VS 2017 with the new csproj in a .NET Standard class library, trying to test the library with a .NET Core 1.1 test project using MSTest testing framework with the dotnet test command. Running locally works perfectly fine; when I send the build to continuous integration, I get the error:
No test discoverer is registered to perform discovery of test cases.
How do I get this discoverer registered, and my tests running, in VSTS?
This is my build process on VSTS (detailed on my blog here)
Add a dotnet restore task.
Then a dotnet build task.
Add a dotnet test task with the arguments --no-build --logger "trx;LogFileName=tests-log.trx
Add a Publish test results task with the following settings
Test Result Format = VSTest
Test Result Files = **/tests-log.trx
Merge Test Results = (checked)
In Control Options set Run this task to run even if a previous task has failed
Refer to these steps below:
Create a new .Net Core test project with MSTest testing framework, there are MSTest.TestAdapter and MSTest.TestFramework package references, which is importance
Add .NET Core (Preview) step (Command: restore)
Add .NET Core (Preview) step (Command: build)
Add .NET Core (Preview) step (Command: test)
Queue build with Hosted VS2017 build agent.
What is the best way to integrate nunit tests into TFS 2010? Is it via generic tests or is there a better approach to running them?
Ideally I'd like to have the granularity of one generic test per test assembly and have a way to surface the results in the TFS build report.
As of now (Oct 2011), the easiest way is probably via the NUnit activity that can be found in the Community TFS Build Extensions.
You can run nunit tests from command line and therefore you can automate these tests via your (Workflow) build template.
Since there aren't a lot of custom build activities available for TFS 2010 yet, you could write your own to make sure that it integrates better with the TFS Build Report.
Here you can find some information about writing a custom build activity.
Another approach would be to create unit tests via Visual Studio. These tests will be automatically run when performing a build.
Ian Battersby wrote a great tutorial how to set up NUnit tests for Visual Studio 2010.
Basically, you need to:
download his build workflow template and set to your build definition
install NUnit and NUnitTFS in your tfs machine
add paths to installed directories in build process parameters
set Configurations to Build setting in build process parameters
change your NUnitTfs.exe.config with your collection name