Whenever I try to run the coded UI test in On-premise agent machine with Azure build-release pipeline my tests are getting aborted since yesterday. I have changed nothing on the release pipeline and build definition.
Below is the error message I am getting in the release pipeline test run console :-
2019-04-24T06:48:27.4475294Z test settings id : 1026822
2019-04-24T06:48:27.4475397Z Build location: C:\agent\_work\r1\a
2019-04-24T06:48:27.4475515Z Build Id: 5133
2019-04-24T06:48:28.0903047Z Test run with Id 1038940 associated
2019-04-24T06:48:37.6929302Z Received the command : Start
2019-04-24T06:48:37.6944608Z TestExecutionHost.ProcessCommand. Start Command handled
2019-04-24T06:48:58.8010958Z Received the command : Stop
2019-04-24T06:48:58.8011508Z TestExecutionHost.ProcessCommand. Stop Command handled
2019-04-24T06:48:58.8011845Z SliceFetch Aborted. Moving to the TestHostEnd phase
2019-04-24T06:48:58.9585180Z Please use this link to analyze the test run : 'test run URL'
2019-04-24T06:48:58.9585816Z Test run '1038940' is in 'Aborted' state with 'Total Tests' : 3 and 'Passed Tests' : 0.
2019-04-24T06:48:58.9604537Z ##[error]Test run is aborted. Logging details of the run logs.
2019-04-24T06:48:58.9606187Z ##[error]System.Exception: The test run was aborted, failing the task.
2019-04-24T06:48:59.0826921Z ##########################################################################
2019-04-24T06:48:59.1608855Z ##[section]Finishing: Test run for Test plans
I am running those test from past 6 months and I have not changed anything on the build or release pipeline and suddenly I am getting the above error on azure release pipeline test run console while triggering the run.
Note: this happened since yesterday(April 24, 2019). Till Monday(April 22, 2019) everything was working fine.
I believe there might be some changes from Microsoft end but I am not sure about that.
I am running the test on Windows 10 environment.
Related
We are using devops to build our .net 4.7.2 application. As part of that, we are running the unit tests which are using the nunit framework and test runner.
It has been running fine for about 18 months, but has just stopped working in the last day :(
It's using the standard template for running the tests and looks like:
- task: VSTest#2
displayName: "Running tests"
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\*test*.dll
!**\*TestAdapter.dll
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
However, now it is failing the step with the following logs:
NUnit Adapter 4.2.0.0: Test execution started
Running all tests in D:\a\1\s\Configuration.Tests\bin\Release\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
Running all tests in D:\a\1\s\Configuration.Tests\bin\Release\testcentric.engine.metadata.dll
NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
Running all tests in D:\a\1\s\Api.Tests\bin\Release\testcentric.engine.metadata.dll
NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
Running all tests in D:\a\1\s\CommunicationTests\bin\Release\testcentric.engine.metadata.dll
NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
Running all tests in D:\a\1\s\Domain.Tests\bin\Release\testcentric.engine.metadata.dll
NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
Running all tests in D:\a\1\s\packages\NUnit3TestAdapter.4.2.1\build\net35\testcentric.engine.metadata.dll
NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run
NUnit Adapter 4.2.0.0: Test execution complete
No test is available in D:\a\1\s\Configuration.Tests\bin\Release\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll D:\a\1\s\Configuration.Tests\bin\Release\testcentric.engine.metadata.dll D:\a\1\s\Api.Tests\bin\Release\testcentric.engine.metadata.dll D:\a\1\s\CommunicationTests\bin\Release\testcentric.engine.metadata.dll D:\a\1\s\Domain.Tests\bin\Release\testcentric.engine.metadata.dll D:\a\1\s\packages\NUnit3TestAdapter.4.2.1\build\net35\testcentric.engine.metadata.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
##[error]Could not find testhost
Results File: D:\a_temp\TestResults\VssAdministrator_WIN-FVJ4KUK6IFI_2022-08-18_12_38_44.trx
##[error]Test Run Aborted.
Total tests: Unknown
Passed: 110
Total time: 16.7203 Seconds
Vstest.console.exe exited with code 1.
**************** Completed test execution *********************
Test results files: D:\a_temp\TestResults\VssAdministrator_WIN-FVJ4KUK6IFI_2022-08-18_12_38_44.trx
Created test run: 1080
Publishing test results: 112
Publishing test results to test run '1080'.
TestResults To Publish 112, Test run id:1080
Test results publishing 112, remaining: 0. Test run id: 1080
Published test results: 112
Publishing Attachments: 1
Execution Result Code 1 is non zero, checking for failed results
Completed TestExecution Model...
##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
##[error]Error: The process 'D:\a_tasks\VSTest_ef087383-ee5e-42c7-9a53-
ab56c98420f9\2.205.0\Modules\DTAExecutionHost.exe' failed with exit code 1
##[error]Vstest failed with error. Check logs for failures. There might be failed tests.
Finishing: Running tests
Looking through this log, it seems that the nunit tests have run successfully, but it might be trying to run mstests? It is frustrating when devops gets an update and it breaks working pipelines.
We have the similar situation.
The unit tests are run with xUnit.
/TestAdapterPath:"D:\a\1\s" Starting test execution, please wait... A
total of 36 test files matched the specified pattern.
2.4828
##[error]Could not find testhost
Data collector 'Code Coverage' message: No code coverage data
available. Profiler was not initialized..
2.0273
##[error]Could not find testhost
Data collector 'Code Coverage' message: No code coverage data
available. Profiler was not initialized..
2.3746
##[error]Could not find testhost
Data collector 'Code Coverage' message: No code coverage data
available. Profiler was not initialized..
1.992
##[error]Could not find testhost
Data collector 'Code Coverage' message: No code coverage data
available. Profiler was not initialized..
4.8409
##[error]Could not find testhost
Data collector 'Code Coverage' message: No code coverage data
available. Profiler was not initialized..
2.1874
##[error]Could not find testhost
I compared the output of the successful run and the failed run and found the different version of the test platform. If you don't specify a version, the default version will be the latest and probably a preview one. So I add something in YAML to specify a workable version.
- task: VisualStudioTestPlatformInstaller#1
inputs:
versionSelector: 'SpecificVersion'
testPlatformVersion: '17.2.0'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
codeCoverageEnabled: True
vsTestVersion: 'toolsInstaller'
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/test/vstest?view=azure-devops
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/tool/vstest-platform-tool-installer?view=azure-devops
I am running my UI Tests on Github CI and the tests are flaky. I don't understand how I can fix it. The animations are disabled and I am running the tests on a iPhone 13 plus. A lot of tests are running green, but some are not working. Locally, I got everything working.
These are some logs:
2022-06-21T13:42:23.2627250Z t = 63.34s Tap Cell
2022-06-21T13:42:23.2707530Z t = 63.34s Wait for com.project.project to idle
2022-06-21T13:42:23.2733620Z t = 63.41s Unable to monitor event loop
2022-06-21T13:42:23.2734250Z t = 63.41s Unable to monitor animations
2022-06-21T13:42:23.2734800Z t = 63.42s Find the Cell
2022-06-21T13:42:24.1158670Z t = 64.45s Find the Cell (retry 1)
2022-06-21T13:42:24.1287900Z t = 64.45s Collecting extra data to assist test failure triage
2022-06-21T13:42:24.2022460Z /Users/runner/work/project/UITestCase.swift:665: error: -[project.UserTagTest testTapInTextView] : Failed to get matching snapshot: Lost connection to the application (pid 12676). (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service created from an endpoint was invalidated: failed to check-in, peer may have been unloaded: mach_error=10000003.)
It can not find the cell because of these logs:
Unable to monitor event loop
Unable to monitor animations
I know this because I sometimes get different errors than the error above, which says that the connection to the application is lost, right below the Unable to monitor... error logging.
Is there anything I can try? I don't have a reproduction project. This is the command that is executed:
xcodebuild test -project project.xcodeproj -scheme project-iosUITests -destination 'platform=iOS Simulator,name=iPhone 13 Pro,OS=15.5'
The CI runs 35 tests and 5 fails randomly with the Unable to errors. Is there any suggestion to fix this problem?
I use Specflow with SpecRunner+ I am using the Deafult.srprofile to to re-run failed tests 3 times in visual studio it shows 2passed 1 failed but the status of the test is a failure, the same goes for azure devops if a re-ran test passes the outcome of the run is a failure. The Failures are sometimes caused by locator timeouts or server timeouts not often but saw it happen few time thats why we decided to implement a re-run.
Could anyone help on this?
022-02-09T12:40:13.8607507Z Test Run Failed.
2022-02-09T12:40:13.8608607Z Total tests: 37
2022-02-09T12:40:13.8609271Z Passed: 36
2022-02-09T12:40:13.8609858Z Failed: 1
2022-02-09T12:40:13.8617476Z Total time: 7.4559 Minutes
2022-02-09T12:40:13.9226929Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
2022-02-09T12:40:14.0075402Z ##[error]Error: The process 'D:\Microsoft_Visual_Studio\2019\Common7\IDE\Extensions\TestPlatform\vstest.console.exe' failed with exit code 1
2022-02-09T12:40:14.8164576Z ##[error]VsTest task failed.
But then the report states that it was retried 3 times which 2 of the retries were seccusefull but still a failure status on the azure devops run.
The behavior of the report is the correct one and sadly this can't be configured to be changed.
What you can do is to adjust how the results are reported back to Azure DevOps.
You can configure it via the VSTest element in the srProfile- File.
This example means, that at least one retry has to be passing:
<VSTest testRetryResults="Unified" passRateAbsolute="1"/>
Docs: https://docs.specflow.org/projects/specflow-runner/en/latest/Profile/VSTest.html
Be aware that we have stopped the development of the SpecFlow+ Runner. More details here: https://specflow.org/using-specflow/the-retirement-of-specflow-runner/
I tried to run Azure DevOps tests in a build pipeline.
Tests are executed on a new agent, i got the following error.
Setup Azure DevOps
##[error]The slice of type 'Discovery' is 'Aborted' because of the error : System.Exception: NUnit Adapter 4.0.0.0: Test discovery complete
Received the command : Stop
TestExecutionHost.ProcessCommand. Stop Command handled
SliceFetch Aborted. Moving to the TestHostEnd phase
Test run '1007278' is in 'Aborted' state.
##[error]Test run is aborted. Logging details of the run logs.
##[error]System.Exception: The test run was aborted, failing the task.
The Problem is that after slice process the test case filter isn't working, solution was the rename to “TestCategory”, before it was “Category“ for TestCaseFilter
I am using msdeploy -presync to backup the current deployment of a website in IIS before the -postsync deploys it, however I recently had a situation where the -presync failed (raised a warning due to a missing dll) and the -postsync continued and overwrote the code.
Both the presync and postsync run batch files.
Obviously this is bad as the backup failed so there is no backout route if the deployment has bugs or fails.
Is there anyway to stop the postsync if the presync raises warnings with msdeploy?
Perhaps the issue here is that the presync failure was raised as a warning not an error.
Supply successReturnCodes parameter set to 0 (success return code convention) to presync option such as:
-preSync:runCommand="your script",successReturnCodes=0
More info at: http://technet.microsoft.com/en-us/library/ee619740(v=ws.10).aspx