I have a release pipeline with a QA/Smoke Test stage, that generates XML files containing test results.
If I run this manually on my machine, obviously I have access to the XML files and I can see the details but on the agent I cannot since we don't have access to those Microsoft hosted agents to view the files.
Is there a way to pipe the files "out" in the task for viewing? maybe there's a third marketplace task that can achieve that?
Here's the deployment result:
2021-06-06T23:34:19.1260519Z Results File: D:\a\r1\a\qa-automation\TestResults\CurrentReport\Logs\junit.xml
2021-06-06T23:34:19.2448029Z Results File: D:\a\r1\a\qa-automation\TestResults\.\CurrentReport\Logs\detailedLogs.xml
2021-06-06T23:34:19.2533810Z
2021-06-06T23:34:19.2596243Z Failed! - Failed: 22, Passed: 2, Skipped: 0, Total: 24, Duration: 52 m 11 s - EED.dll (netcoreapp3.1)
Here's the stage YAML:
steps:
- script: |
git clone https://.../qa-automation.git -b master
cd qa-automation
testrun.bat --cat "EDSmoke" --env dev
displayName: 'Clone qa-automation repo'
Is there a way to pipe the files "out" in the task for viewing? maybe there's a 3rd marketplace task that can achieve that?
You can try with following task:
Write-host "##vso[task.uploadfile]<PathOfTheFiles>\<filename>"
Like:
Write-host "##vso[task.uploadfile]$(System.DefaultWorkingDirectory)\qa-automation\TestResults\CurrentReport\Logs\junit.xml"
View and download attachments associated with releases
Would you like to upload additional logs or diagnostics or images when
running tasks in a release? This feature enables users to upload
additional files during deployments. To upload a new file, use the
following agent command in your script:
Write-host "##vso[task.uploadfile]"
The file is then available as part of the release logs. When you
download all the logs associated with the release, you will be able to
retrieve this file as well.
You can also add a powershell script task in your release definition to read the smoke test output and output it to the console. Then you will be see the content of the log files from "Logs" tab powershell script step. And you can also click "Download all logs as zip" to download the smoke test result files.
I build a rubygem on Azure DevOps. I used that step to use RSpec:
script: bundle exec rspec spec --format RspecJunitFormatter --out test_results/TEST-rspec.xml
displayName: 'rake spec'
While execution i'm getting that issue: https://dev.azure.com/saigkill/hoe-manns/_build?definitionId=3
So it returns '1' without any error before. Maybe i have missed something?
As written there a system.debug = true helped me, to identify the problem. One of my tests failed.
I'm trying to run grunt tasks through a batch script and am calling grunt as follows:
call npm install
call npm install grunt
However, if this returns an error, then the VSTS build on the hosted build agent still shows up as successful (even with a logged error in the script). Does anyone have any good examples of how to get it to return an error to the build?
I've been looking at using powershell, but without any luck so far, with code as follows:
In gruntfile.js:
grunt.initConfig({
shell: {
ps: {
options: {
stdout: true
},
command: 'powershell ../../errors.ps1'
}
}
});
grunt.registerTask('default', function() {
try {
grunt.task.run([
'less:desktop',
'less:tablet',
'less:smartphone',
'less:homepage_desktop',
'less:homepage_tablet']);
} catch(e) {
grunt.task.run([
'shell:ps']);
throw e;
}
});
In errors.ps1:
$URL_Format_Error = [string]"Error found in running grunt. Please investigate grunt logs"
Write-Error $URL_Format_Error -ErrorAction:Stop
return
The code run in the exception handler never gets called, and a warning is output with a compilation error in the .less file, but the powershell is never run. Is there a way I can hook into the warning perhaps, and run my powershell then?
As an alternative, when I try to add a grunt task to the VSTS build definition after a batch script to run the NPM install, I just keep getting the following error (even after seeing a successful NPM installation in the batch script):
Fatal error: Unable to find local grunt
Hence, I'm not sure if I can run the grunt task in a separate task the VSTS build definition, if I'm using a hosted build agent. I'm inclined to think that would only work if I had my own build server.
Just try to use Write-Error in combination with an exit 1 in your script.
Write-Error ("Some error")
exit 1
Reference this thread : How to fail the build from a PowerShell task in TFS 2015
I have a Jenkins build pipeline created using workflow plugin. At the beginning the pipeline runs a gulp build inside of the docker container and then archives test results using the following code
step([$class: 'JUnitResultArchiver', testResults: 'build/test-results/*.xml'])
In the following steps I package up the artifacts and ship them to the binary repository.
When unit tests are not passing Jenkins understands the build is unstable and marks it yellow. However it still continues with subsequent steps in the pipeline. Is there any way make the pipeline stop when unit tests are failing?
the JUnitResultArchiver will cause this condition to be true when the build is unstable:
currentBuild.result != null.
If I remember correctly it sets it to UNSTABLE, but it is enough to check that is different than null.
So you could do something like
step([$class: 'JUnitResultArchiver', testResults: 'build/test-results/*.xml'])
if (currentBuild.result == null) {
//contintue with your pipeline
} else {
//notify that the build is unstable. //or just do nothing
}
There is nothing to do at Jenkins side but at Gulp side. The call to gulp CLI needs to return a proper error value to have the sh step failing correctly.
Jenkins just interprets what the shell is returning, so you juts need to make Gulp to return a fail when tests fail (see this blog post, it seems to achieve exactly that).
I am getting the error "Test reports were found but none of them are new. Did tests run?" when trying to send unit test results by email. The reason is that I have a dedicated Jenkins job that imports the artifacts from a test job to itself, and sends the test results by email. The reason why I am doing this is because I don't want Jenkins to send all the developers email during the night :) so I am "post-poning" the email sending since Jenkins itself does not support delayed email notifications (sadly).
However, by the time the "send test results by email" job executes, the tests are hours old and I get the error as specified in the question title. Any ideas on how to get around this problem?
You could try updating the timestamps of the test reports as a build step ("Execute shell script"). E.g.
cd path/to/test/reports
touch *.xml
mvn clean test
via terminal or jenkins. This generates new tests reports.
The other answer that says cd path/to/test/reports touch *.xml didn't work for me, but mvn clean test yes.
Updating the last modified date can also be achieved in gradle itself is desired:
task jenkinsTest{
inputs.files test.outputs.files
doLast{
def timestamp = System.currentTimeMillis()
test.testResultsDir.eachFile { it.lastModified = timestamp }
}
}
build.dependsOn(jenkinsTest)
As mentioned here: http://www.practicalgradle.org/blog/2011/06/incremental-tests-with-jenkins/
Here's an updated version for Jenkinsfile (Declarative Pipeline):
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'make build'
}
}
stage('Test') {
steps {
sh 'make test'
script {
def testResults = findFiles(glob: 'build/reports/**/*.xml')
for(xml in testResults) {
touch xml.getPath()
}
}
}
}
}
post {
always {
archiveArtifacts artifacts: 'build/libs/**/*.jar', fingerprint: true
junit 'build/reports/**/*.xml'
}
}
}
Because gradle caches results from previous builds I ran into the same problem.
I fixed it by adding this line to my publish stage:
sh 'find . -name "TEST-*.xml" -exec touch {} \\;'
So my file is like this:
....
stage('Unit Tests') {
sh './gradlew test'
}
stage('Publish Results') {
// Fool Jenkins into thinking the tests results are new
sh 'find . -name "TEST-*.xml" -exec touch {} \\;'
junit '**/build/test-results/test/TEST-*.xml'
}
Had same issue for jobs running repeatedly (every 30 mins).
For the job, go to Configure, Build, Advanced and within the Switches section add:
--stacktrace
--continue
--rerun-tasks
This worked for me
Navigate to report directory cd /report_directory
Delete all older report rm *.xml
Add junit report_directory/*.xml in pipeline
Rerun the test script , navigate to Build Number → Test Result
Make sure you have one successful build without any failure, only after this you can able to see the reports
Make sure that you have mentioned the correct path against "Test report XMLs" under jenkins configuration, such as "target/surefire-reports/*.xml"
There is no need to touch *.xml as jenkins won't complain even though test results xml file does not change.
if you use Windows slave, you can 'touch' results using groovy pipeline stage with powershell:
powershell 'ls "junitreports\\*.*" | foreach-object { $_.LastWriteTime = Get-Date }'
It happens if you are using a test report which is not modified by that job in that run.
In case for test purpose if you are testing with already created file then, add below command inside jenkins job under Build > Execute Shell
chmod -R 775 /root/.jenkins/workspace/JmeterTest/output.xml
echo " " >> /root/.jenkins/workspace/JmeterTest/output.xml
Above command changes timestamp of file hence error wont display.
Note: To achieve same in Execute Shell instead of above, do not try renaming file using move mv command etc. it won't work , append and delete same for change file timestamp only works.
For me commands like chmod -R 775 test-results.xml or touch test-results.xml does not work due to permission error. As work around use is to set new file in test report settings and command to copy old xml report file to new file.
you can add following shell command to your "Pre Steps" section when configure your job on Jenkins
mvn clean test
this will clean the test
Here's an updated version of the gradle task that touch each test result files.
From Jenkins pipeline script, just call "testAndTouchTestResult" task instead of "test" task.
The code below is with Kotlin syntax:
tasks {
register("testAndTouchTestResult") {
setGroup("verification")
setDescription("touch Test Results for Jenkins")
inputs.files(test.get().outputs)
doLast {
val timestamp = System.currentTimeMillis()
fileTree(test.get().reports.junitXml.destination).forEach { f ->
f.setLastModified(timestamp)
}
}
}
}
The solution for me was delete node_modules and change node version (from 7.1 to 8.4) on jenkins. That's it.