I am automating functional test cases of a application where major part of it is Web UI but at some steps I have a desktop application in Main Frames ,which I automated using TestStack.White. I am able to run these test case in Visual Studio on my computer.
We have planned to put these automated test cases in CICD pipeline in VSTS.
Will I able to run test step where desktop application is involve in CICD pipeline?If yes do how can I do it?
Please suggest.
You can run UI test through Visual Studio Test task (check Test mix contains UI tests option).
Microsoft Hosted agents are pre-configured for UI test for both web apps and desktop apps.
For Self-hosted agent, you need to run it as an interactive process with auto-logon enabled (When configuring agents, select 'No' when prompted to run as a service)
More information: UI testing considerations
Related
As far as I know we can Run automated tests from test plans on Windows agent by using "vstest". Do we have any support/ way to run automated tests from test plans on Linux boxes?
As per azure doc: To run automated tests from test plans, we would need "vstest" to be installed on agent box. However, since we are using Linux box - do we have any alternative?
You can see this doc:
The reason for why you are facing the error after run the Visual Studio Test Agent Deployment task, is the OS of the agent you are using does not meet its requirement.
Until now, we only support the Windows-based machines for this task:
Also, we have hardcode this requirement into our task definition. This means there's no any work around can for you use if the agent you want to use is Ubuntu. Because we has script to firstly check the environment, to see whether it meets the requirements of the task.
You can raise a suggestion request on our official forum to expand this feature into Azure Devops. Our PM would review it and kindly to consider to take it into our develop roadmap.
I have a web api (REST) project that is written in .NET and I have written a few webtests (.webtest) that test those apis.
While those tests run fine locally from visual studio, I want to integrate them into my VSTS (Azure Devops) build pipeline, so as to identify and breaking changes that could break any of those APIs.
I am not able to find any task in build pipeline which can run the webtests as part of build. I see option for running unit-tests though.
So, wanted to check what am I missing here.
You might want to find an alternative approach as this link implies it has been deprecated.
Visual Studio web performance test (.webtest file) is tied to the load
test functionality and is deprecated. Some customers have used
.webtest for other purposes such as running API tests, even though it
was not designed for that purpose. Many API testing alternatives are
available in the market. SOAP UI is a free, open source alternative to
consider, and is also available as a commercial option with additional
capabilities.
You could try to use cmd task command line to run MSTest with arguments.
Add Run Command Line step/task to execute MSTest command
Add Publish Test Results step/task
On the other hand, you can do test in Unit Test too, just send the request and check the response, related thread.
Also as Matt mentioned, since Visual Studio web performance tests (.webtest files) are tied to the load test functionality and is also deprecated. You could take a look at this blog here: Cloud-based load testing service end of life
As part of Integrating the TOSCA (Automation tool) in the release pipeline, we are unable to kickoff the IE browser.
If you are using hosted agents then it's not supported, you have to run the agents in interactive mode for the UI related things.
So just try to deploy an private agent and run the agent in interactive mode, please see Deploy an agent on Windows for details.
I have a winForm application. I want to install it on the server every time the CI/CD happens. Later, after I install it(i don't know how to automate installation as well.), I want to run UI tests on it. What task should I add to my release management?
I only found web deployment when I researched. Since, mine is a desktop application, I need different build/release task. I could do it from vs2017(by right click project + publish)
I want similar one from TFS 2017.
You can try to build with the publish profile. See Publish profile.
In TFS using VS Build step with the MSBuild Arguments something like below:
/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.StagingDirectory)\\" /p:PublishProfile="YourPbulishProfile"
You can reference this article : Using Web Deploy in Visual Studio Team Services Release Management
To run coded UI tests from TFS.
Firstly, you need to run your agent as an interactive process. Regarding how to deploy an agent on windows, please refer to this link.
Secondly, add your coded UI tests project into TFS version control.
Last, create a build definition to use Visual Studio Test Agent Deployment task and Run Functional Tests task to run the tests.
Reference below articles:
Continuous integration, test, and deployment tutorial
Deploy a Desktop App from TFS to VM for Coded UI Test.
I found a way to publish my desktop application on to the server. below is the article which explains the build tasks needed.
Click here
I've setup Continuous Integration in my project hosted at Visual Studio Online.
Among other things, I deploy my web application to a test slot and test database and I run integration tests.
However, I don't need the test slot and database to be running all the time. I would like to start them both for the integration tests to run and stop them after my integration tests are done, all automated.
I played around with the Azure PowerShell vNext task, but I couldn't figure out how to accomplish this. Any help would be appreciated. More than the actual answer, I would appreciate a link with more information, if possible.
For the web app slot, you can add AzurePowerShell step in your build defition can create two power-shell script: one start the website via Start-AzureWebsite command before the build start and another one stop the website via Stop-AzureWebsite after the testing is finished.
For the database, you cannot stop it. Refer to this question for details: How do I stop and start a SQL Azure database?