What is the best way to integrate nunit tests into TFS 2010? Is it via generic tests or is there a better approach to running them?
Ideally I'd like to have the granularity of one generic test per test assembly and have a way to surface the results in the TFS build report.
As of now (Oct 2011), the easiest way is probably via the NUnit activity that can be found in the Community TFS Build Extensions.
You can run nunit tests from command line and therefore you can automate these tests via your (Workflow) build template.
Since there aren't a lot of custom build activities available for TFS 2010 yet, you could write your own to make sure that it integrates better with the TFS Build Report.
Here you can find some information about writing a custom build activity.
Another approach would be to create unit tests via Visual Studio. These tests will be automatically run when performing a build.
Ian Battersby wrote a great tutorial how to set up NUnit tests for Visual Studio 2010.
Basically, you need to:
download his build workflow template and set to your build definition
install NUnit and NUnitTFS in your tfs machine
add paths to installed directories in build process parameters
set Configurations to Build setting in build process parameters
change your NUnitTfs.exe.config with your collection name
Related
Please tell me if it's possible to associate Azure DevOps test cases with JUnit automated tests from eclipse. If yes, is there any step by step document I can follow? Please help
As I know it's not supported scenario.
You can find the supported tests from this document:
Coded UI test, Selenium tests, and unit tests written using Version 1 of the MSTest framework can be associated with a test case.
Tests that use MSTest v2, NUnit, and xUnit frameworks can be associated with a test case workitem when using Visual Studio 15.9 Preview 2 or later. However, these tests cannot be run using Microsoft Test Manager and XAML builds.
Tests that use the .NET core framework can be associated with a test case workitem when using Visual Studio 15.9 Preview 2 or later. To run the .NET core tests the appropriate target framework must be specified in a runsettings file. However, these tests cannot be run using Microsoft Test Manager and XAML builds.
Tests that use other test frameworks such as Chutzpah (for JavaScript tests such as Mocha or QUnit), or Jest cannot be associated with a test case.
Associating generic tests may work, but running these tests is not supported.
So we recommend using Visual Studio to associate the automated tests. And for now the Junit framework is not supported in such scenario.
I have a web api (REST) project that is written in .NET and I have written a few webtests (.webtest) that test those apis.
While those tests run fine locally from visual studio, I want to integrate them into my VSTS (Azure Devops) build pipeline, so as to identify and breaking changes that could break any of those APIs.
I am not able to find any task in build pipeline which can run the webtests as part of build. I see option for running unit-tests though.
So, wanted to check what am I missing here.
You might want to find an alternative approach as this link implies it has been deprecated.
Visual Studio web performance test (.webtest file) is tied to the load
test functionality and is deprecated. Some customers have used
.webtest for other purposes such as running API tests, even though it
was not designed for that purpose. Many API testing alternatives are
available in the market. SOAP UI is a free, open source alternative to
consider, and is also available as a commercial option with additional
capabilities.
You could try to use cmd task command line to run MSTest with arguments.
Add Run Command Line step/task to execute MSTest command
Add Publish Test Results step/task
On the other hand, you can do test in Unit Test too, just send the request and check the response, related thread.
Also as Matt mentioned, since Visual Studio web performance tests (.webtest files) are tied to the load test functionality and is also deprecated. You could take a look at this blog here: Cloud-based load testing service end of life
I'm trying to create a release pipeline in VSTS that runs my xUnit-tests as specified in a Test Plan.
Long story short: I can't get it to work.
What I'm using:
Azure DevOps (formerly VSTS)
Visual Studio Test task (v2.*)
Test project targeting .NET Core 2.1
xunit 2.4 with xunit.runner.visualstudio 2.4
In Azure DevOps I defined a Test Plan that contains a Test Suite which contains a Test that has an Associated Automation which points to my xUnit test.
I had to use the REST API to link the test code to the Test as described here.
I can select that Test in the visual designer for the VSTest task.
When I run the release pipeline the VSTest task fails with the following error message:
DiscoveryMessage : System.IO.FileNotFoundException: Unable to find tests for D:\a\r1\a\Foo.Tests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
The path to the Foo.Tests.dll is correct, all required files are copied as well.
I explicitly specified the framework version in a .runsettings file (as the option Other console options doesn't work when using the Test plan option).
Specified the path to custom test adapters
used Visual Studio 2017 and Installed by Tools Installer options
Added a .NET Core Tool installer to install the correct .NET Core SDK
...and any other combination of settings I could think of.
The error message is still the same.
Any ideas what I might be missing? Your help would be greatly appreciated at this point!
After several more hours we stumbled across a web page that stated that you don't have to copy the binaries of your test project as input for the VSTest task but PUBLISH it instead. That never came to mind as vstest.console.exe runs smoothly when you point it at the binaries on a local machine.
UPDATE: We had to add a Publish Artifact task at the end of our Build Pipeline and make the Release Pipeline pick up the published artifact.
I have a winForm application. I want to install it on the server every time the CI/CD happens. Later, after I install it(i don't know how to automate installation as well.), I want to run UI tests on it. What task should I add to my release management?
I only found web deployment when I researched. Since, mine is a desktop application, I need different build/release task. I could do it from vs2017(by right click project + publish)
I want similar one from TFS 2017.
You can try to build with the publish profile. See Publish profile.
In TFS using VS Build step with the MSBuild Arguments something like below:
/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.StagingDirectory)\\" /p:PublishProfile="YourPbulishProfile"
You can reference this article : Using Web Deploy in Visual Studio Team Services Release Management
To run coded UI tests from TFS.
Firstly, you need to run your agent as an interactive process. Regarding how to deploy an agent on windows, please refer to this link.
Secondly, add your coded UI tests project into TFS version control.
Last, create a build definition to use Visual Studio Test Agent Deployment task and Run Functional Tests task to run the tests.
Reference below articles:
Continuous integration, test, and deployment tutorial
Deploy a Desktop App from TFS to VM for Coded UI Test.
I found a way to publish my desktop application on to the server. below is the article which explains the build tasks needed.
Click here
I've switched to using Visual Studio Code and and am missing the ability to right-click and publish files.
I read here that a task runner can be used to build/deploy files...
Publish Web Deploy using VS Code
That answer only mentions Grunt and I cannot find any articles on how to do this with Gulp which I am already using. I've got as far as finding an article on how to hook up Gulp and MSbuild which I assume must be the first step.
However, I'm unsure about the deployment step as I don't know what the "publish" button in standard VS does. It seems to be using something called MSDeploy or "web deploy" and requires a bunch of publish settings, so presumably this can be done from gulp?
I know I must sound a bit clueless but I'm not really a Microsoft guy I'm a front-ender, just happen to be working with other people's .net projects occasionally. Any help would really be appreciated.
WebDeploy is the technology that VisualStudio uses for web deployments. WebDeploy uses MSDeploy behind the scenes. MSDeploy is capable of far more than just web app deployments. We use it for deploying Windows Services, Scheduled Tasks and SQL databases as well.
Here are details on you to use the MSDeploy commandline. The sync command is used most commonly for deployments.
You can use the gulp-mswebdeploy-package plugin to create a web deploy package as part of you gulp build task. you can the use the msdeploy command line to deploy the site.