I've inherited a jacked VSTS build process and I'm attempting to integrate SonarQube into the mix. Jacked, in that it isn't a clean install and I know the previous owner has done some strange and mysterious things with it that I haven't been able to track down yet.
Anyways, the problem I'm having is that the "Visual Studio Test" task step is deleting the .trx file after publishing the results. I need the .trx file for SonarQube. I can see the trx file is being deleted and after a screen recording of the folder and the active console logging of the VSTS build process pinpointed the publish process where the .trx file disappears. With system.debug = true here are a subset of the logs (the .trx file disappears somewhere in here):
Test results remaining: 165. Test run id: 33.
Leaving UpdateTestResultsAsync
Updated test results: 165
Publishing Attachments: 2
Entering CreateTestRunAttachmentsAsync
Leaving CreateTestRunAttachmentsAsync
Uploading test run attachements individually
Completed PublishTestResult
Completed Publish Test Results: 0
I've tried a number of things, the latest of which was to install the latest VS 2017 Enterprise IDE to the build machine and am using these settings in the task:
Not sure if this helps but here is an approximation of the vstest call the above makes as shown in the console:
vstest.console.exe /TestCaseFilter:"FullyQualifiedName=<all the dlls in my solution>"
"C:\agent\_work\2\s\Source\Sln\Project1\bin\Debug\net461\Project1.Tests.dll"
"C:\agent\_work\2\s\Source\Sln\Project2\bin\Debug\net461\Project2.Tests.dll"
"C:\agent\_work\2\s\Source\Sln\Project3\bin\Debug\net461\Project3.Tests.dll"
/EnableCodeCoverage
/InIsolation
/logger:"trx"
/TestAdapterPath:"C:\agent\_work\2\s\Source\Sln"
/diag:"C:\agent\_work\_temp\4acf64a0-4b51-11e8-9bf5-3b4a52af383b.txt"
-#############################
If I run vstest.console.exe from the command line, the results aren't published and the .trx file isn't deleted.
-#############################
I'd like the Visual Studio Test task step to publish but not delete the .trx file.
Any help would be appreciated. Thanks!
-###################
Update 1: Additional logging of the final "Completed Test Execution" section:
**************** Completed test execution *********************
Current Phase: MS.VS.TestService.VstestConsoleAdapter.ExecuteVsTestPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.ExecutionVsTestPhaseResults
PERF WARNING: Running the phase MS.VS.TestService.VstestConsoleAdapter.ExecuteVsTestPhase: took 5414.7701 ms
Current phase: MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhase
Starting Publish Test Results: TIA: False
Starting PublishTestResult
Test results files: C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27.trx
Updating Test results: 35
Entering ParseTestResultFiles
runContext.Platform is null
runContext.Configuration is null
runContext.ReleaseURI is null
runContext.ReleaseEnvironmentUri is null
Reading test results from file 'C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27.trx'.
Setting run start and finish times.
Attachment location: C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27\In.
Adding run level attachment: C:\agent\_work\2\s\TestResults\SERVERNAME$_SERVERNAME_2018-04-30_07_43_27\In\SERVERNAME\SYSTEM_SERVERNAME 2018-04-30 07_43_21.coverage.
Total test results: 168.
Leaving ParseTestResultFiles
pdating test results: 168
Entering UpdateTestResultsAsync
Publishing test results to test run '35'.
PERF: GetTestCaseResults: took 333.1965 ms
PERF WARNING: GetTestCaseResults: took 333.1965 ms
Test results remaining: 165. Test run id: 35.
Leaving UpdateTestResultsAsync
Updated test results: 165
Publishing Attachments: 2
Entering CreateTestRunAttachmentsAsync
Leaving CreateTestRunAttachmentsAsync
Uploading test run attachements individually
Completed PublishTestResult
Completed Publish Test Results: 0
Current Phase: MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhaseResults
PERF WARNING: Running the phase MS.VS.TestService.VstestConsoleAdapter.PublishTestResultPhase: took 1719.7748 ms
Current phase: MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhase
Started RerunValidationPhase for 35
Maximum number of attempts running the failed tests reached: 3
Processed: ##vso[task.logissue type=warning;]Maximum number of attempts running the failed tests reached: 3
Current Phase: MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhaseResults
PERF: Running the phase MS.VS.TestService.VstestConsoleAdapter.RerunValidationPhase: took 1.1836 ms
Current phase: MS.VS.TestService.VstestConsoleAdapter.CleanupPhase
Started CleanupPhase 35
Test run needs to be marked as complete.
Entering UpdateTestRunAsync
Leaving UpdateTestRunAsync
Completed CleanupPhase 35
Current Phase: MS.VS.TestService.VstestConsoleAdapter.CleanupPhase Phase Result: MS.VS.TestService.VstestConsoleAdapter.CleanupPhaseResults
PERF: Running the phase MS.VS.TestService.VstestConsoleAdapter.CleanupPhase: took 259.2514 ms
Completed the ExecutionStateModel
Completed TestExecution Model...
rc:0
success:true
File exists. Size: 1224675 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.txt
Agent.Version=2.133.3
Agent.TempDirectory=C:\agent\_work\_temp
defaultRoot: 'C:\agent\_work\_temp'
findOptions.followSpecifiedSymbolicLink: 'true'
findOptions.followSymbolicLinks: 'true'
matchOptions.debug: 'false'
matchOptions.nobrace: 'true'
matchOptions.noglobstar: 'false'
matchOptions.dot: 'true'
matchOptions.noext: 'false'
matchOptions.nocase: 'true'
matchOptions.nonull: 'false'
matchOptions.matchBase: 'false'
matchOptions.nocomment: 'false'
matchOptions.nonegate: 'false'
matchOptions.flipNegate: 'false'
pattern: '*host.*.txt'
findPath: 'C:\agent\_work\_temp'
statOnly: 'false'
findPath: 'C:\agent\_work\_temp'
findOptions.followSpecifiedSymbolicLink: 'true'
findOptions.followSymbolicLinks: 'true'
C:\agent\_work\_temp (directory)
C:\agent\_work\_temp\.taskkey (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-43-19_00381_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-25_11343_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-35_20720_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-42_87908_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-43-22_36319_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-25_98843_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-36_03532_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-43_70721_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b061-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b063-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\proxy (file)
14 results
found 14 paths
applying include pattern
adjustedPattern: 'C:\agent\_work\_temp\*host.*.txt'
4 matches
pattern: '*datacollector.*.txt'
findPath: 'C:\agent\_work\_temp'
statOnly: 'false'
findPath: 'C:\agent\_work\_temp'
findOptions.followSpecifiedSymbolicLink: 'true'
findOptions.followSymbolicLinks: 'true'
C:\agent\_work\_temp (directory)
C:\agent\_work\_temp\.taskkey (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-43-19_00381_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-25_11343_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-35_20720_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-42_87908_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-43-22_36319_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-25_98843_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-36_03532_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-43_70721_1.txt (file)
C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b061-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\cc80b063-4c84-11e8-87c2-ab8f332e19bd.txt (file)
C:\agent\_work\_temp\proxy (file)
14 results
found 14 paths
applying include pattern
adjustedPattern: 'C:\agent\_work\_temp\*datacollector.*.txt'
4 matches
8 final results
File exists. Size: 11672 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-43-19_00381_1.txt
File exists. Size: 11672 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-25_11343_1.txt
File exists. Size: 11671 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-35_20720_1.txt
File exists. Size: 11582 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.datacollector.18-04-30_07-45-42_87908_1.txt
File exists. Size: 413384 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-43-22_36319_1.txt
File exists. Size: 45500 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-25_98843_1.txt
File exists. Size: 45350 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-36_03532_1.txt
File exists. Size: 45500 Bytes
Processed: ##vso[task.uploadfile]C:\agent\_work\_temp\cc80b060-4c84-11e8-87c2-ab8f332e19bd.host.18-04-30_07-45-43_70721_1.txt
task result: Succeeded
Processed: ##vso[task.complete result=Succeeded;]VsTest task succeeded.
Agent.Version=2.133.3
Release.ReleaseUri=undefined
Release.ReleaseId=undefined
Build.BuildUri=vstfs:///Build/Build/745
Build.Buildid=745
Agent.Version=2.133.3
telemetry area: TestExecution feature: TestExecutionTask data: {"builduri":"vstfs:///Build/Build/745","buildid":"745","codeCoverageEnabled":true,"overrideTestrunParameters":"false","pipeline":"build","runTestsInIsolation":true,"task":"VsTestConsoleFlow","runInParallel":false,"result":"Succeeded","settingsType":"none","testSelection":"testAssemblies","tiaEnabled":false,"vsTestVersion":"15.0.27428","consoleOptionsEnabled":"","rerunEnabled":true,"rerunType":"basedOnTestFailurePercentage"}
Processed: ##vso[telemetry.publish area=TestExecution;feature=TestExecutionTask;]{"builduri":"vstfs:///Build/Build/745","buildid":"745","codeCoverageEnabled":true,"overrideTestrunParameters":"false","pipeline":"build","runTestsInIsolation":true,"task":"VsTestConsoleFlow","runInParallel":false,"result":"Succeeded","settingsType":"none","testSelection":"testAssemblies","tiaEnabled":false,"vsTestVersion":"15.0.27428","consoleOptionsEnabled":"","rerunEnabled":true,"rerunType":"basedOnTestFailurePercentage"}
I can also reproduce the result.
For VS Test task with the version 2.*, it will delete the .trx file after executing VS Test task.
If you need to get the .trx file after VS Test task, you can use VS Test task with the version 1.*. It will keep the .trx file in the source directory.
A possible solution is here
https://developercommunity.visualstudio.com/content/problem/747717/vstest-not-creating-trx-in-folder-testresults.html
Open
Build Details -> Tests -> Filter in Passed Tests -> Open Test Run -> Attachments
The Files are already on the server, but not as artifacts, but under Test Runs
I know this is a old post, but people looking for a solution would be benefited by it working with the latest version of the task. You need to add the path to keep results in a different folder than the "$(Agent.TempDirectory)/TestResults" which will be removed at the end of the pipeline run.
From the information on "Test results folder" of the task...
Folder to store test results. When this input is not specified, results are stored in $(Agent.TempDirectory)/TestResults by default, which is cleaned at the end of a pipeline run. The results directory will always be cleaned up at the start of the vstest task before the tests are run. Relative folder path if provided will be considered relative to $(Agent.TempDirectory)
I have solved it by having the task variable, "Test results folder", pointing to a folder other than the temp directory. With this I could retain trx files after the task is run. In my case, I need trx files for further processing.
Related
I have a test pipeline on concourse with one job that runs a set of luigi tasks. My problem is: failures in the luigi tasks do not rise up to the concourse job. In other words, if a luigi task fails, concourse will not register that failure and states that the concourse job completed successfully. I will first post the code I am running, then the solutions I have tried.
luigi-tasks.py
class Pipeline1(luigi.WrapperTask):
def requires(self):
yield Task1()
yield Task2()
yield Task3()
tasks.py
class Task1(luigi.Task):
def requires(self):
return None
def output(self):
return luigi.LocalTarget('stuff/task1.csv')
def run(self):
#uncomment line below to generate task failure
#assert(True==False)
print('task 1 complete...')
t = pd.DataFrame()
with self.output().open('w') as outtie:
outtie.write('complete')
# Tasks 2 and 3 are duplicates of this, but with 1s replaced with 2s or 3s.
config file
[retcode]
# codes are in increasing level of severity (for most applications)
already_running=10
missing_data=20
not_run=25
task_failed=30
scheduling_error=35
unhandled_exception=40
begin.sh
#!/bin/sh
set -e
export PYTHONPATH='.'
luigi --module luigi-tasks Pipeline1 --local-scheduler
echo $?
pipeline.yml
# <resources, resource types, and docker image build job defined here>
#job of interest
- name: run-docker-image
plan:
- get: timer
trigger: true
- get: docker-image-ecr
passed: [build-docker-image]
- get: run-git
- task: run-script
image: docker-image-ecr
config:
inputs:
- name: run-git
platform: linux
run:
dir: ./run-git
path: /bin/bash
args: ["begin.sh"]
I've introduced errors in a few ways: assertions/raising an exception (ValueError) within an individual task's run() method and within the wrapper, and sys.exit(luigi.retcodes.retcode().unhandled_exception). I also tried failing all tasks. I did this in case the error needed to be generated in a specific manner/location. Though they all produced a failed task, none of them produced an error in the concourse server.
At first, I thought concourse just gives a success if it can run the file or command tasked to it. I'm not sure it's that simple, though. Interestingly, when I run the pipeline on my local computer (luigi --modules luigi-tasks Pipeline1 --local-scheduler) I get an appropriate return code (e.g. 30), but when I run the pipeline within the concourse server, I get a return code of 0 after the luigi tasks complete (from echo $? in the bash script).
Would appreciate any insight into this problem.
My suspicion is that luigi doesn't see your config file with return codes. Its default behavior is to return 0, whether tasks fail or succeed.
This experiment should help to debug that:
Force a failed job: add an exit 1 at the end of begin.sh
Hijack the job: fly -t <target> i -j <pipeline>/<job> -> select run-script
cd ./run-git; /bin/bash begin.sh
Ensure the luigi config is present and named appropriately, e.g. luigi.cfg
Re-run the command: LUIGI_CONFIG_PATH=luigi.cfg bash ./begin.sh
Check output: echo $?
I have a release pipeline with a QA/Smoke Test stage, that generates XML files containing test results.
If I run this manually on my machine, obviously I have access to the XML files and I can see the details but on the agent I cannot since we don't have access to those Microsoft hosted agents to view the files.
Is there a way to pipe the files "out" in the task for viewing? maybe there's a third marketplace task that can achieve that?
Here's the deployment result:
2021-06-06T23:34:19.1260519Z Results File: D:\a\r1\a\qa-automation\TestResults\CurrentReport\Logs\junit.xml
2021-06-06T23:34:19.2448029Z Results File: D:\a\r1\a\qa-automation\TestResults\.\CurrentReport\Logs\detailedLogs.xml
2021-06-06T23:34:19.2533810Z
2021-06-06T23:34:19.2596243Z Failed! - Failed: 22, Passed: 2, Skipped: 0, Total: 24, Duration: 52 m 11 s - EED.dll (netcoreapp3.1)
Here's the stage YAML:
steps:
- script: |
git clone https://.../qa-automation.git -b master
cd qa-automation
testrun.bat --cat "EDSmoke" --env dev
displayName: 'Clone qa-automation repo'
Is there a way to pipe the files "out" in the task for viewing? maybe there's a 3rd marketplace task that can achieve that?
You can try with following task:
Write-host "##vso[task.uploadfile]<PathOfTheFiles>\<filename>"
Like:
Write-host "##vso[task.uploadfile]$(System.DefaultWorkingDirectory)\qa-automation\TestResults\CurrentReport\Logs\junit.xml"
View and download attachments associated with releases
Would you like to upload additional logs or diagnostics or images when
running tasks in a release? This feature enables users to upload
additional files during deployments. To upload a new file, use the
following agent command in your script:
Write-host "##vso[task.uploadfile]"
The file is then available as part of the release logs. When you
download all the logs associated with the release, you will be able to
retrieve this file as well.
You can also add a powershell script task in your release definition to read the smoke test output and output it to the console. Then you will be see the content of the log files from "Logs" tab powershell script step. And you can also click "Download all logs as zip" to download the smoke test result files.
My Azure DevOps pipeline tasks successfully complete without issues except for the final deployment step:
Job Issues - 1 Error
The job running on agent XXXX ran longer than the maximum time of 00:05:00 minutes. For more information, see https://go.microsoft.com/fwlink/?linkid=2077134
The build logs state the operation was canceled:
021-03-02T20:50:00.4223027Z Folders: 695
2021-03-02T20:50:00.4223319Z Files: 10645
2021-03-02T20:50:00.4223589Z Size: 672611102
2021-03-02T20:50:00.4223851Z Compressed: 249144045
2021-03-02T20:50:03.6023001Z ##[warning]Unable to apply transformation for the given package. Verify the following.
2021-03-02T20:50:03.6032907Z ##[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the <DependentUpon> tag for each config in the csproj file and rebuild.
2021-03-02T20:50:03.6034584Z ##[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
2021-03-02T20:50:04.5268038Z Initiated variable substitution in config file : C:\azagent\A2\_work\_temp\temp_web_package_3012195912183888\Areas\Admin\sitemap.config
2021-03-02T20:50:04.5552027Z Skipped Updating file: C:\azagent\A2\_work\_temp\temp_web_package_3012195912183888\Areas\Admin\sitemap.config
2021-03-02T20:50:04.5553082Z Initiated variable substitution in config file : C:\azagent\A2\_work\_temp\temp_web_package_3012195912183888\web.config
2021-03-02T20:50:04.5642868Z Skipped Updating file: C:\azagent\A2\_work\_temp\temp_web_package_3012195912183888\web.config
2021-03-02T20:50:04.5643366Z XML variable substitution applied successfully.
2021-03-02T20:51:00.8934630Z ##[error]The operation was canceled.
2021-03-02T20:51:00.8938641Z ##[section]Finishing: Deploy IIS Website/App:
When I examine the deployment states, I notice one of my tasks takes quite a while for what should be a fairly simple operation:
The file transform portion takes over half of the allotted 5 minutes? Could this be the issue?
steps:
- task: FileTransform#1
displayName: 'File Transform: '
inputs:
folderPath: '$(System.DefaultWorkingDirectory)/_site.com/drop/Release/Nop.Web.zip'
fileType: json
targetFiles: '**/dataSettings.json'
It may be inefficient but FileTransform log shows a significant amount of time spent after the variable has been substituted. Not sure what's causing the long delay, but the logs don't account for the time after the variable has been successfully substituted:
2021-03-02T23:04:44.3796910Z Folders: 695
2021-03-02T23:04:44.3797285Z Files: 10645
2021-03-02T23:04:44.3797619Z Size: 672611002
2021-03-02T23:04:44.3797916Z Compressed: 249143976
2021-03-02T23:04:44.3970596Z Applying JSON variable substitution for **/App_Data/dataSettings.json
2021-03-02T23:04:45.2396016Z Applying JSON variable substitution for C:\azagent\A2\_work\_temp\temp_web_package_0182869515217865\App_Data\dataSettings.json
2021-03-02T23:04:45.2399264Z Substituting value on key DataConnectionString with (string) value: ***
**2021-03-02T23:04:45.2446986Z JSON variable substitution applied successfully.
2021-03-02T23:07:25.4881687Z ##[section]Finishing: File Transform:**
The job running on agent XXXX ran longer than the maximum time of 00:05:00 minutes.
Based on the error message, the cause of this issue is that the running time of the Agent Job has reached the maximum value set.
If you are using the release pipeline, you could set the timeout in Agent Job -> Execution plan ->timeout.
If you are using the build pipeline, you could set the timeout in Agent Job -> Agent Job -> Execution plan -> timeout (For An Agent Job) and Options -> Build job -> Build job timeout in minutes (For whole Build pipeline).
The file transform portion takes over half of the allotted 5 minutes
From the task log, the zip package contains many files and folders. Therefore, transform task will take more time to traverse the file to find the target file.
I have .net core 2.1 project build pipeline as shown below.
I have 2 simple unit test cases which executed successfully. “.NET Core test” command executes successfully and publishes thereport.
2018-09-06T22:05:41.8494077Z
2018-09-06T22:05:42.0138714Z Passed Login_Test
2018-09-06T22:05:42.0139277Z Passed Logoff_Test
2018-09-06T22:05:42.0495295Z Results File: P:\*******2018-09-06_17_05_41.trx
2018-09-06T22:05:42.0514262Z
2018-09-06T22:05:42.0516990Z Attachments:
2018-09-06T22:05:42.0517972Z P:\*****\TestResults\329a8629-7ec1-454b-97d8-2623d29bbd0e\XXXXX 2018-09-06 17_05_39.coverage
2018-09-06T22:05:42.0518130Z
2018-09-06T22:05:42.0518516Z Total tests: 2. Passed: 2. Failed: 0. Skipped: 0.
2018-09-06T22:05:42.0519312Z Test Run Successful.
2018-09-06T22:05:42.0528123Z Test execution time: 1.9003 Seconds
2018-09-06T22:05:42.1424722Z Publishing test results to test run '90'
2018-09-06T22:05:42.1424881Z Test results remaining: 2. Test run id: 90
2018-09-06T22:05:42.1592889Z ##[section]Async Command Start: Publish test results
2018-09-06T22:05:42.2384075Z Published Test Run : http://******/_TestManagement/Runs#runId=90&_a=runCharts
2018-09-06T22:05:42.2384433Z ##[section]Async Command End: Publish test results
2018-09-06T22:05:42.2385471Z ##[section]Finishing: VsTest - testAssemblies
But I don’t see any “Test Results” under dashboard.
I included "Visual Studio Test" too, but still dont see the result in the dashboard. What am I missing here?
So I am trying to run a powershell script that is triggered by TeamCity to run specific unit tests based on the names of the files that were changed on each github commit.
Here is how I am running it from the command line:
C:\MyFolder\bin\NUnit.ConsoleRunner.3.4.1\tools\nunit3-console.exe "C:\MyFolder\Bin\UnitTesting.dll" --test="MyFolder.QuickTests.DaoTests.ProductDaoTests.ProductBasicTest"
But I keep getting this, it runs it just never runs any tests:
NUnit Console Runner 3.4.1
Copyright (C) 2016 Charlie Poole
Runtime Environment
OS Version: Microsoft Windows NT 10.0.14393.0
CLR Version: 4.0.30319.42000
Test Files
MyFolder\Bin\UnitTesting.dll
Test Filters
Test: MyFolder.QuickTests.DaoTests.ProductDaoTests.ProductBasicTest
Run Settings
WorkDirectory: C:\Users\Me
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.0
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 2
Test Run Summary
Overall result: Passed
Test Count: 0, Passed: 0, Failed: 0, Inconclusive: 0, Skipped: 0
Start time: 2016-10-17 20:28:43Z
End time: 2016-10-17 20:28:43Z
Duration: 0.303 seconds
Results (nunit3) saved as TestResult.xml
Now when I run it without the --test command like this:
C:\MyFolder\bin\NUnit.ConsoleRunner.3.4.1\tools\nunit3-console.exe "C:\MyFolder\Bin\UnitTesting.dll"
It runs every unit-test that we have, but I don't want to run them all, I want to run specific quick ones, and only run the large ones when we go to staging/production servers so our developers don't have to wait 15 to 20 minutes every time they commit something.
Some additional info:
-My namespace that I am using for this is
MyFolder.QuickTests.DaoTests.ProductDaoTests
The Class I am calling is:
ProductBasicTest
Some of the names like the folder directories were changed because they are %teamcity% placeholders for file directories.
What am I doing wrong to not be able to run specific tests?
For some reason my nunit-console is not recognizing the /run command or /fixture or --test=.
EDIT:
I upgraded to 3.5.0 and am still getting the same issues, I am not able to use --test.
C:\MyFolder\bin\NUnit.ConsoleRunner.3.5.0\tools\nunit3-console.exe "C:\MyFolder\Bin\UnitTesting.dll" --test="MyFolder.QuickTests.DaoTests.ProductDaoTests.ProductBasicTest"
That is the new location, and getting the same issue.
When I do --where for MyFolder it crashes Powershell but doesn't actually run anything.
When I do --explore it does the same as --where for MyFolder and does nothing for MyFolder.QuickTests .
EDIT EDIT:
Thanks to Rob I found the docs here and looked at the --where function with --where "name=ProductBasicTest" which will run all the files in that Test-Suite!
So thanks to Rob one of the issues that I was running into, is it was not recognizing my namespace correctly with QuickTests. So whenever I ran the function it was not running correctly.
To fix this you can go to the Test xml file output and see what names it was running tests under.
To run these individually you can run it by the name with the command:
"nunit3-console.exe C:\PathToDll.dll --where "name = NameOfTest"